Quarry
On this page:
Introduction
Quarry provides a general-purpose Unix computing environment for academic and instructional use. IU's newest supercomputer, Quarry is an IBM HS21 Bladeserver cluster running Red Hat Linux, with TORQUE (also called PBS) and Moab for job management and SoftEnv to simplify application and environment configuration. Quarry is a seven teraflop system built from Intel processors.
The user environment is generally similar to AVIDD's, except that
Quarry uses SoftEnv. When you log into Quarry for the first
time, a file named .soft will be created for you,
defining system defaults, such as Intel's C/C++ and Fortran
compilers. You can add more packages using the Nano editor (similar to
Pico); for a list of the possibilities, enter the command
softenv.
If you have questions about the system, email the High Performance Systems group. For help with migrating applications or optimizing code, email the High Performance Applications team.
Hardware configuration
Quarry consists of 140 IBM HS21 Blade servers, each containing two Intel Xeon 5335 quad-core processors, 8 or 16GB of memory, a 36 or 73GB locally attached SAS disk for local scratch space, and gigabit Ethernet for system interconnects. The Blade configuration provides electrical power and cooling advantages that make Quarry much more cost effective than AVIDD, while occupying much less space.
Following is information about Quarry's cluster nodes:
-
Compute Nodes (140, HS21 Bladeserver):
- 2 x 2.0GHz quad-core Intel Xeon 5335 processors
- 8 or 16GB 667MHz DDR2 ECC
- 36 or 73GB locally attached SAS disk
- 250TB GPFS global scratch
- 340TB Lustre global scratch (Data Capacitor)
- 340TB Lustre global scratch (Data Capacitor - WAN)
- 2 x 1GB Ethernet card
-
Image Servers (2, E1350 X3650):
- 2 x 2.5GHz dual-core PowerPC 970MP processors
- 8GB 533MHz DDR2 SDRAM
- 73GB SAS disk (67GB in
/scratch) - 1 x Myricom M3S-PCIXD-2-I (Lanai XP)
-
Management nodes (3, E1350 X3650):
- 2 x 1.65GHz dual-core Power5+ processor
- 8GB 533MHz DDR2 SDRAM
- 73GB SAS disk
- 2 x Emulex LP10000 PCI-X/133MHz FC adapters
Networking
Each Blade chassis has six gigabit-Ethernet connections to a Force10 E1200 switch that connects to the other research systems. Fourteen HS21 Blades, each with gigabit-Ethernet adapters, share those six connections.
Storage
The disk for Quarry's 266TB GPFS volume is hosted on 10 DataDirect Networks S2A 9500 storage controllers, each dual-pathed to 5 SAF 4248 chassis. Physical disks are aggregated in an 8+2 RAID configuration; combined with the dual-pathed controllers and active/active GPFS storage hosts, this provides multiple levels of redundancy for Quarry's storage space. Home directories are available via NFSv3 over gigabit Ethernet.
Quarry currently shares the GPFS volume with Big Red and the Research Database Complex (RDC).
Home directory disk space is allocated on the IBM N5500 NAS storage device. The default 10GB disk quota is shared with Big Red and the RDC if you have accounts on those systems.
More information
For more information about Quarry, see:
