Pittsburgh Picks Big Ben

The Pittsburgh Supercomputing Center (PSC) has chosen a Cray Inc. (Nasdaq: CRAY) supercomputer to replace an existing Hewlett-Packard Co. (NYSE: HPQ) processor-based cluster. The new Cray supports PSC's TeraGrid undertaking (see Big Ben Comes to Pittsburgh).

TeraGrid, which is backed by the National Science Foundation, is a project to harness the processing power of eight supercomputing sites across the U.S. for a range of research. These include PSC, Argonne National Laboratory, and Oak Ridge National Laboratory (see Supercomputers Boost Grid Computing and TACC Receives NSF Reward)

For the last few years, PSCs cluster, named LeMieux after Pittsburgh Penguins’ hockey legend Mario LeMieux, has been at the forefront of the center's TeraGrid work. But Jim Kasdorf, PSC’s director of special projects, told NDCF that the cluster, which was installed in 2001, is “getting a bit long in the tooth.” [Ed. note: not unlike its namesake.]

LeMieux (the cluster, not the hockey star) uses 3,000 HP Alpha processors spread across 178 cabinets and offers a peak performance of six teraflops (trillions of calculations per second). But the new supercomputer, which is named Big Ben after the much younger Pittsburgh Steelers

quarterback Ben Roethlisberger and some Benjamin Franklin guy, offers both higher performance and a smaller footprint.

Completed in February, Big Ben uses 2,090 Advanced Micro Devices (NYSE: AMD) Opteron processors in just 22 cabinets, and has a peak performance of 10 teraflops. “It’s a much, much, smaller footprint, and purpose-designed for high-performance computing,” says Kasdorf.To illustrate his point, Kasdorf says that if every person on Earth held a calculator and did one calculation per second, they would still be 1,500 times slower than Big Ben. [Ed. note: So, we won't try that then.]

Big Ben also offers more storage than its predecessor, with 245 terabytes of local disk from DataDirect Networks Inc., compared to Lemieux’s 72 terabytes of HP storage. However, both systems have access to 2.4 petabytes of archived tape storage from Storage Technology Corp. (StorageTek) (NYSE: STK), according to Kasdorf.

A number of organizations are currently deploying clusters to deal with complex, high-performance computing work, and the U.S Department of Energy’s Sandia National Labs recently chose a blade cluster to replace a supercomputer. Typically, the reasons cited for deploying cluster technology are flexibility and cost (see Sandia Blasts Off Blade Cluster, Luebeck Looks to Clusters, and Statoil Builds Dell Cluster).

But, despite all the cluster hype, Kasdorf believes that Big Ben is still the best fit for PSC’s TeraGrid work. This is thanks, he says, to the Opteron processors’ faster clock speeds and, notably, Cray’s purpose-built SeaStar interconnect technology. “That gives us improved bandwidth and latency,” he says. “For the applications that need to run 2,000 processors all at once, very tightly synchronized, supercomputing is better.”

LeMieux, on the other hand, is more of an “ad hoc” system, according to Kasdorf, relying on processors with lower clock speeds and interconnect technology from Quadrics. A spokesman for Quadrics told NDCF that LeMieux uses the original version of its QsNet technology, which has since had a major overhaul. Another upgrade is planned for next year, he adds.Despite deciding to switch to a supercomputer, Kasdorf still plans to use LeMieux for the foreseeable future. The director adds that there is “prize-winning research” still underway on the cluster, such as a Carnegie Mellon University project on earthquake simulations. “It’s stable and productive,” he says. “It just sits there and cranks work out.”

But, over time, key applications will be migrated from LeMieux to Big Ben, such as a University of Oklahoma research project for predicting severe storms, he adds.

Initially, LeMieux and Big Ben will run in tandem on the TeraGrid, although the cluster will eventually be phased out. Kasdorf is not sure exactly when this will be, but he adds that it is likely to be within the next two years.

But nothing stands still for long in high performance computing, and Kasdorf hopes to make Big Ben even bigger. ”My guess would be that we will probably upgrade to dual-core Opterons -- that gives us more processing power."

Funding, however, will dictate how far Kasdorf and his team can go. “We have dreams of 50 to 100 teraflops, but this all takes money."Currently, around 1,000 scientists across the U.S use the TeraGrid, although this number is expected to leap to 4,000 by 2009, as well as up to 10,000 students and educators.

HP was unavailable for comment on this article.

— James Rogers, Site Editor, Next-Gen Data Center Forum