Panasas Leads Charge to Parallel NFS

Panasas is so obsessed with an emerging standard for managing NFS file access that it will launch free client source-code this summer. (See Panasas Accelerates pNFS Adoption.)

The question is, Who'll glom on?

The standard is Parallel NFS (pNFS), an extension of the next release of NFS (Network File System), which is working its way through the IETF approval process. As an adjunct to the upcoming NFS 4.1, pNFS describes a way for the NFS protocol to process file requests to multiple servers or storage devices at once, instead of handling the requests serially.

"You could say NFS was invented by Bill Joy at Sun back in 1983, and the thing hasn't had a major performance upgrade in two decades," says Panasas VP of marketing Larry Jones. The I/O processing involved in retrieving stored NFS files is still mostly serial, he notes. It's time to bust the bottleneck by making it parallel.

Clustered file systems are growing in popularity, but not all of them have parallel I/O for NFS (though some claim performance improvements via their own file systems). Panasas's claimed differentiator has been a clustered storage client called DirectFlow that deploys parallel I/O for NFS.One long-time Panasas customer says DirectFlow has met his particular processing requirements. "We use Panasas DirectFlow because NFS will not do this many-processes-to-one-file scenario that large HPC sites have," states Gary Grider, group leader of the High Performance Computing Systems Integration Group at Los Alamos National Laboratory, in an email to Byte and Switch.

"We don't use Panasas for NFS, we use it as a parallel file system with the direct client," Grider asserts. "For commercial sites, good performance with low client overhead is very important and pNFS certainly has advantages over normal NFS, but HPC siteshave this need to allow hundreds or thousands of processes to write into a single file. If you do this on a conventional NFS system (clustered or not), you would have to serialize the write operations and thus the performance would be tens of megabytes/sec. With pNFS,these operations can be done in parallel and thus could scale to gigabytes/sec and beyond."

Panasas has sought a wider audience for DirectFlow for years. Garth Gibson, founder and CTO of Panasas, joined with engineers from NetApp to get IETF work underway on a parallel file system for NAS back in 2004. The effort became pNFS and was undertaken by the IETF as part of its NFSv4 working group.

Panasas DirectFlow is a direct precursor to pNFS, according to Jones and Gibson. When the IETF approves the new standard, which is anticipated by year's end, Panasas will have a significant first-mover advantage.

Not a bad spot for Panasas, which claims to be doing a booming business in high-performance computing (HPC), with revenues growing more than 105 percent in 2006. But with about 100 customers, the company is still a niche play, even though it claims that at least 25 percent of its revenues come from a growing roster of enterprise customers, as opposed to scientific researchers.Being first out with pNFS could push that ratio higher, analysts say. "I think we'll see pretty much all the big players adopt pNFS," says Robin Harris, senior analyst at the Data Mobility Group. Everyone interested in HPC and big computing clusters, he notes, including EMC, IBM, NetApp, and Sun, are likely to support it.

IBM, Ibrix, and EMC already have their own clustered storage systems with parallel I/O. Moving to a standard seems a natural way to enhance the approach.

But who will really make the move? At least one analyst thinks enterprises don't really need pNFS to improve the performance of clustered systems. "All of the clustered file system NAS vendors have at some fundamental level data coming in over Ethernet that's served by different nodes," says Arun Taneja of the Taneja Group consultancy. "They all do it differently. Panasas does it in a very different way, and I'd call them the odd duck of the group."

But Taneja acknowledges that, if large storage players get behind pNFS, the power of standardization could take over. Then, vendors like BlueArc, Exanet, Isilon, Polyserve (through its alliance with HP), and others would probably look to support it, he says.

While Panasas hopes to widen its appeal through pNFS, another expert says that, for now, pNFS solves problems very specific to HPC environments. "It's not just for big files, it's for multiple, time-sensitive computations," says analyst Mike Karp of Enterprise Management Associates. Where calculations are independent and require lots of instantaneous processing, pNFS could serve a big need.This doesn't mean only high-end research labs, Karp notes. "There are a whole slew of commercial applications that can take advantage of it. Wall Street analysts who rely on financial math involving sub-cent forecasts come to mind.

Gary Grider of Los Alamos sees a broader future of pNFS, even though as an HPC user he fits Panasas's main customer profile. "[T]he overhead for data movement should be lower for pNFS. Over time, the pNFS client may supplant the need for custom file system clients for parallel file systems."

Mary Jander, Site Editor, Byte and Switch