Ethernet Enlarges Supercomputing

Gigabit Ethernet is now the network of choice in more than half of the world's top supercomputing sites, and InfiniBand's popularity is rising.

The list of the world's Top 500 supercomputers was unveiled today at the 21st International Supercomputer Conference in Dresden, Germany. It shows that gigabit Ethernet is the interconnect for 256 (51.2 percent) of the world's top supercomputing sites, up from 212 (42.4 percent) one year ago. (See High-Speed Links Favor Ethernet.)

Installations of InfiniBand, still the third-favorite Top 500 interconnect, have doubled since last June. Thirty-five (7 percent) of the Top 500 now use it, compared with 16 (3.2 percent) last year.

Meanwhile, installations of Myrinet, a packet-switching technology sold by Myricom Inc., have plummeted. While still the second-place choice, only 54 (10.8 percent) of the Top 500 sites use Myrinet, compared with 141 sites, or 28.2 percent, last year. (More on that momentarily.)

Table 1: Top 500 Supercomputer Interconnects

While commercial applications don't have the same requirements as those for supercomputing, there's value in examining how the HPC capabilities could turn up in lower-end products that support enterprise storage and computing. (See Future SANs Stir Debate.) Suppliers are clearly looking to bridge that gap. IBM, for example, recently announced a 10-Gbit/s Ethernet switch blade for its BladeCenter chassis in an effort to get ahead of similar blade offerings from HP and anticipated ones from Sun. (See IBM Flashes 10-Gig Switch Blade, HP Brandishes Blades, and Sun Glints Off Blades.)

There's evidence of market momentum behind InfiniBand. Cisco just announced 20-Gbit/s Double Data Rate (DDR) InfiniBand switches. Mellanox, Silverstorm, and Voltaire already have them. (See Voltaire Doubles Up and Mellanox Doubles InfiniBand.) InfiniBand is also part of IBM's and HP's blade servers.

But the CEO of Myricom, Chuck Seitz, isn't waiting for next year's list to shift his allegiance to the winning team. This week in Dresden, Myricom demonstrated its Myrinet technology running over standard Ethernet LANs, without Myrinet's switches.

Myrinet has created software called Myrinet Express (MX/Ethernet) that runs with the vendor's 10-Gbit/s Myri-10G NICs, announced last year as part of Myrinet's move toward gigabit Ethernet. (See Myricom Sees Ethernet Light and Microsoft Accelerates iSCSI Support.) This week's demo includes an eight-node cluster of Intel-based servers equipped with the Myri-10G NICs linked via Fujitsu switches.

Seitz claims that running Myrinet over Ethernet provides five to ten times lower latency than is possible with TCP/IP over Ethernet. "What we're doing with MX/Ethernet is creating an open solution that brings the best of HPC to Ethernet with no compromises in terms of standards, interoperability, or software compatibility," Seitz said in a prepared statement. "I'd like to think Bob Metcalfe would be proud.Seitz can't offer any customers who've adopted MX/Ethernet, which is available with bundles of NICs, so it's too soon to tell whether Myricom's new technique will take hold. Supercomputing users of Myrinet include the University of Illinois, Indiana University, the University of Southern California, Vanderbilt University, and Los Alamos National Laboratory.

Bob Wheeler of the Linley Group consultancy believes Myricom will have to get more active in promoting its new technique. "They have the presence in the HPC space, but a lot depends on their ability to promote this as a true multivendor, interoperable method," he notes. While Myrinet already publishes its protocol, he thinks it may take an industry group to achieve widespread support.

Other facts about the Top 500 Supercomputer list include:

The next Top 500 list will be released in November.

— Mary Jander, Site Editor, Byte and SwitchOrganizations mentioned in this article: