BYU Flashes Its Blades

A large U.S. university has rolled out a massive cluster of blade servers to ease the pressure on systems and storage in its primary research center.

Quinn Snell, director of the Fulton supercomputing lab at Brigham Young University, told Byte and Switch that three SGI Origin machines and a 128-node cluster of IBM servers were simply being overwhelmed” by the volume of work the university was doing. “We had a major backlog on the other machines,” he admits. “There were too many jobs.”

The new supercomputer, named Mary Lou after the donor who paid for it, uses 630 Dell PowerEdge 1855 servers in four separate clusters. The system sports a total of 630 nodes and 1,260 Intel Xeon processors. Boasting a peak performance of nine Teraflops, Mary Lou is split across four separate locations on BYU’s campus in Provo, Utah. It came into service earlier this year.

Though Snell is pleased with the performance of his SGI and IBM machines, he was keen to try out the blade servers. “We had a very tight power and heat budget -- blades consolidate power and reduce heat,” he explains. This, he adds, made it “quite advantageous” to go with blades.

Users are currently sending out mixed messages about the value of blade servers. On the one hand, some IT managers see the technology as “costly and immature,” whereas others are wowed by the ability to fit a large number of servers into a tight space. (See Are Blades Cutting It?, Study Highlights Blade Disappointment, and Blades for Buffalo .)That said, Snell is not the only IT manager to choose blade servers over more traditional supercomputing kit. The U.S. Department of Energy’s Sandia National Labs, for example, recently chose a blade cluster to replace a supercomputer. Typically, the reasons cited for deploying cluster technology are flexibility and cost. (See Sandia Blasts Off Blade Cluster, Luebeck Looks to Clusters, and Statoil Builds Dell Cluster.)

BYU takes a unique approach to its supercomputing facilities, which also prompted the decision to deploy blades. Whereas most supercomputing sites are used for graduate research, this is not the case in Provo, where undergraduates also get their hands on high-performance kit. “Everybody and anybody can have access to [the supercomputers],” explains Snell. High demand only underlined the need to build a new supercomputer. “This being a bigger machine means that we are able to get more jobs through at faster speeds -- researchers are not standing around waiting.”

Although he doesn't offer specific figures, Snell says Mary Lou is “significantly cutting down the compute time” for a range of projects. “We have several chemistry, physics, biology, and computer science professors using Mary Lou." More than 200 people currently do research on the new clusters.

But, with all this compute power suddenly available, Snell and his team also had to figure out how they were going to store all this data. Ultimately, BYU opted for 15 Terabytes of disk storage from DataDirect Networks and a file system from Ibrix.

BYU is also using a mixture of interconnects to tie the different parts of Mary Lou together. The largest cluster, says Snell, which is 210 nodes, is connected via InfiniBand. “That’s for some of our research that needs high-speed communications. We have got some chemistry professors that run some very communication-intensive work.”The other three clusters that make up Mary Lou rely on Gigabit Ethernet, although 10-Gbit/s Ethernet is used to link everything up. “We needed something with a higher bandwidth to connect all the clusters together."

Snell admits that Mary Lou may be expanded at some point in the future, particularly with his IBM cluster nearing the end of its life. ”We’re looking at that right now. One thought is that rather than buying another cluster, we may just add to this cluster.”

— James Rogers, Senior Editor, Byte and Switch

Organizations mentioned in this article: