324-node/100 Teraflops System provides a Bridge to Greater Science Productivity

By Jan Zverina

The National Science Foundation (NSF) has awarded the San Diego Supercomputer Center (SDSC) at the University of California, San Diego, $2.8 million to build and deploy a new high performance computer system called Trestles.

Designed to increase productivity for a broad spectrum of researchers, Trestles will have 10,368 processor cores, a peak speed of 100 teraflop/s, and 38 terabytes of flash memory. It will begin operations before the end of 2010 and remain in use for three years under the NSF award. The system will work with and span the deployments of SDSC’s recently introduced Dash system and its larger Gordon data-intensive system, to become operational in mid-2011.

Like Dash, Trestles will be available to users of the TeraGrid, the nation’s largest open-access scientific discovery infrastructure.

“Trestles is appropriately named because it will serve as a bridge between SDSC’s unique, data-intensive resources available to TeraGrid users now and into the future,” said Mike Norman, SDSC’s interim director. “The overarching goal of Trestles is to enable as much productive science as possible in this data-intensive era.”

To be configured by SDSC and Appro, Trestles is based on quad-socket, 8-core AMD Magny-Cours compute nodes connected via a QDR InfiniBand fabric. Each of the 324 nodes will have 32 cores, 64 GB of DDR3 memory, and 120 GB of flash memory. With a total of 10,368 cores, Trestles will offer significant compute capability for meeting the science objectives of a large number of users.

In addition, SDSC is taking an innovative approach of complementing the existing capability of TeraGrid systems that favor large users and high system utilization, with a system that has the capacity to deliver science impact for large numbers of TeraGrid users.

"TeraGrid user data show that scientific impact occurs across all allocation levels and that many well-known computational science usage models do not require large-scale jobs or huge allocations,” said Richard Moore, deputy director of SDSC and a co-PI on the project.

However, a fair number of computational science approaches require resources with scheduling flexibility and rapid turnaround. “By focusing on core counts of 1,024 or less, Trestles is designed to serve a much larger number of users while simultaneously improving their productivity as measured by turnaround and the number of jobs completed,” said Allan Snavely, associate director of SDSC and also a co-PI for the new system.

SDSC’s approach with Trestles with regard to its size, allocation range and scheduling practiceswill also benefit the emerging Science Gateway paradigm for high-performance computing (HPC) system access. Science gateways is a relatively recent phenomenon in supercomputing. Currently led by Nancy Wilkins-Diehr of SDSC, the TeraGrid Gateway program began in 2004 as web portals designed and used by scientists. The program extends the analysis capabilities of these community-designed interfaces through the use of supercomputers, yet insulates users from supercomputing complexities. While gateway usage still represents a small fraction of total TeraGrid usage, about 25 percent of TeraGrid users now access resources via gateways rather than through single-user accounts, according to TeraGrid data.

To ensure that productivity on Trestles remains high, SDSC will adjust allocation policies, queuing structures, user documentation, and training based on a quarterly review of usage and user satisfaction data. SDSC’s Dash and Triton Resource clusters use a matrixed pool of expertise in system administration and user support, as well as the SDSC-developed Rocks cluster management software. SDSC will also run the Inca monitoring framework to ensure the system meets all service-level expectations. SDSC’s Advanced User Support will establish key benchmarks to accelerate user applications, and subsequently will assist users in tuning and optimizing applications for Trestles.

SDSC announced last November that it won a five-year, $20 million grant from the NSF to build and operate Gordon, the first high-performance supercomputer to employ a vast amount of flash memory to help speed solutions now hamstrung by slower spinning disk technology. Dash, a smaller prototype of Gordon, was deployed in April 2010. Both systems are being integrated by Appro and use a similar design philosophy of combining commodity parts in innovative ways to achieve high-performance architectures.

Trestles is also a world famous surf break in San Diego County and one of the largest remaining undeveloped stretches of Southern California coastline. It is named for the railway bridge that crosses San Mateo Creek, home to a highly diverse and fragile community of plants, birdlife, mammals and reptiles, making it an ideal area to study biodiversity and climate change.

Source: National Science Foundation