Sun Guns Midtier Box

Sun Microsystems Inc. (Nasdaq: SUNW) is back at the benchmark trough less than a month after being forced to withdraw test results on its high-end storage array because of a flawed benchmark -- but this time, Sun is claiming that its newest midrange storage system kicks butt (see Sun Tests Out StorEdge 6320).

The company says benchmark results for its StorEdge 6320 system, tested using the Storage Performance Council (SPC)'s SPC-1 benchmark, shows it offers better performance in terms of I/O operations per second (IOPS) than other midrange storage systems, including the Hewlett-Packard Co. (NYSE: HPQ) Enterprise Virtual Array (EVA) model 2C12D. Sun started shipping the StorEdge 6320 last month (see Sun Thickens Up in the Middle).

In May, a bug in the SPC benchmark test negated results that would have crowned Sun's StorEdge 9980 -- a rebadged version of Hitachi Data Systems (HDS) Lightning 9980V storage array -- the fastest storage system in the world. Sun vowed to redo the test, promising it would show performance as good as, if not better than, the first one (see Sun Bit by Benchmark Bug).

But first, it's touting the test results of the 6320. The company says the latest test, which was done in conjunction with a Sun Fire 4800 server, had been slated for this month all along. While the company has already received a patch for the bug on the previous 9980 test, Steve Guzowski, group marketing manager for Sun, says it's taking time to get the entire $1.5 million configuration back together for testing.

"That's a fairly large piece of equipment," says Guzowski. "We borrow it for tests, but it's been shipped back to another division... It's fully our intention to have the 9980 [test results] up there [on the SPC site]."That may well be, but after what had appeared to be record-breaking results of the bungled 9980 benchmark, the 6320 may seem like a bit of a letdown.

The 6320, which delivered 44,805.74 IOPS, didn't actually turn in top results on any aspects of the test. At the high end, the 22-controller StorEdge 6320 system cluster failed to beat the eight-node 3PAR Inc. S800 configuration's 47,001 IOPS. And the 6320's cost of $15.56 per IOPS didn't come close to the $4.48 per IOPS delivered by Dell Computer Corp.'s (Nasdaq: DELL) low-end SCSI array.

Guzowski, however, insists that the 6320 offers the best overall solution among the systems in its class. Compared with HP's EVA, for example, the 6320 fares well on the SPC-1 test. The HP EVA delivered 20,096.97 IOPS and $23.88 per IOPS -- which means the Sun 6320 is more than twice as fast and offers 35 percent lower cost-per-IOPS.

While 3PAR's performance may be better, its system was nearly $10 more per IOPS than Sun, he points out. And while Dell's price/performance metric may seem intimidating, its performance of 7,650.4 IOPS certainly doesn't, says Guzowski. "It's hard to beat Dell [on price]," he says, "but look at their performance."

More important, Guzowski says, is that the fully loaded 22-controller 6320 system shows near linear scaleability, compared with its 12-controller configuration. At 25,340.29 IOPS, the latter system delivers roughly half the performance of the larger system, but its price per IOPS of $15.40 is almost identical."This is representative of a trend toward clustered control systems," says Evaluator Group analyst Randy Kerns, pointing out that until recently most companies didn't cluster more than two controllers together. "This allows for more linear scaling."

But the question of how much these tests actually play into customers' buying decisions remains. For example, EMC Corp. (NYSE: EMC), one of the test's starkest opponents, has long insisted that the SPC benchmark is useless, because, it claims, the test doesn't reflect real-world customer configurations.

While some customers claim to disregard the benchmark for that very reason, others say it's an important part of the evaluation process. Peter Lilienthal, the CTO of systems integrator and consultancy Strong Link Data, says he generally only considers vendors that submit to the benchmark.

"Benchmarks help level the playing field," says Lilienthal. "They give us an impression of different platforms... If you want the fastest platform, go with the benchmarked companies." Strong Link Data, based in Orlando, Fla., recently advised the U.S. Army to purchase the StorEdge 6320 for a joint simulation systems project.

Eugénie Larson, Reporter, Byte and Switch