NASA Upgrades Supercomputer

Even supercomputers need upgrades to stay that way, and the NASA Ames Research Center Center pumped more than $3 million over the last few months to add the capacity and bandwidth necessary to keep its Columbia supercomputer humming along.

The project included adding 600 Tbytes of disk connected to DataDirect S2A9550 SANs, 20 Sun StorageTek T10000 tape libraries, and 320 ports of 4-Gbit/s Brocade Fibre Channel switches. (See DataDirect Intros Solutions and Sun Launches Tape Libraries.) Columbia is now connected to 1.1 Pbytes of storage, including a 440-Tbytes SGI InfiniteStorage SAN installed when it launched in 2004. (See SGI, Intel Build NASA Supercomputer.)

The computer includes 20 SGI Altix clusters with 512 processors apiece and 10,240 Intel processors, and it has a peak processing power of 62 trillion floating point operations per second (teraflops).

"When we deployed it, we thought 440 Tbytes of disk would be enough," says Bill Thigpen, Columbia Project lead at NASA. "It turned out we needed to significantly increase our cache fronting the tape, and we needed more disk and higher bandwidth all over the place. That's why we did the upgrade."

Named in memory of the fallen Space Shuttle astronauts, Columbia supports all of NASA's space launches and research.Project consultant Alan Powers from NASA contractor Computer Sciences Corp. (CSC) says the team chose DataDirect's new S2A9550 SAN because it was the fastest in benchmark tests against InfiniteStorage and other systems. He says DataDirect's architecture lets NASA Ames connect twice as many devices as other systems.

"We picked DataDirect mainly for the speed," he says. "Its performance basically doubled the others we measured. And to get the same amount of bandwidth from the others we'd have to double the ports, and that would double the price of the system and the HBAs and switches."

Powers says they changed switches to upgrade from 2 Gbit/s and also to save space at the Moffett Field, Calif., data center. NASA replaced two 128-port Brocade switches with four 64-port Silkworm 4900 2u directors, and added two 32-port Silkworm 4100s. The Silkworm 4900 is less than 3.5-inches high, while Brocade's 384-port Silkworm 48000 directors are 24 inches high.

"Our facility is full with equipment now," Powers says. "The rack we had didn't have enough space to put more switches. The size of the 4900 fit perfectly where we needed to fit the equipment."Although Ames uses Cisco for Ethernet switches, he never considered switching vendors for SAN switches. "We've used Brocade since the 1-gig days, and I can remember one port problem during all that time."

Powers says the upgrade isn't finished yet. Only eight of the T10000 tape libraries are in production; the rest are still being tested. The center is also upgrading from Altix 3700 to 4700 systems. He says they are also evaluating InfiniBand interface cards in the DataDirect system."We're always evaluating our options," he says. "It's an ongoing thing."

Dave Raffo, News Editor, Byte and Switch