Storage

07:00 AM
David Woolf
David Woolf
Commentary
50%
50%

Testing Fuels The NVMe Revolution

Testing procedures and tools developed at places like UNH's InterOperability Laboratory help technologies like NVMe advance.

Game-changing technologies are exciting. In a recent blog, Jim O’Reilly wrote that Non-Volatile Memory express (NVMe) was about to "revolutionize" the solid-state drive (SSD) industry in as radical a change as introduced by Fibre Channel in the 90s and SCSI in the 80s.

Revolutions, however, can be tricky to manage. To benefit from new technologies, an industry needs to harness their power. In the case of a new protocol such as NVMe, the interested parties first must collaborate on writing it. Then comes testing and further documentation. Here are five ways the University of New Hampshire Interoperability Lab (UNH-IOL) is supporting NVMe.

Integrators list: With any disruptive technology, it helps to have a single record for products to interoperate across platforms and conform to the new specification or standard. That is the role of the UNH-IOL’s NVMe Integrators List.

How do the UNH-IOL and the NVM Express Organization vet products for this list? First, we collect requirements, often involving company feedback, issues from previous tests, or interest in new features. We discuss those requirements, and if approved, write up and test the procedures. Then the UNH-IOL debuts those procedures at plugfests. 

Test procedures: A documented and publicly available test program ensures transparency for NVMe. The program involves an interop and conformance component.

The interop procedure tries to mimic the use of a drive in the real world. It is first installed in a system, which is booted from a non-NVMe boot drive. The visibility of the NVMe drive in the OS is verified, and the drive is formatted. Then a cross-platform tool is used to perform a series of tests, carried out across a series of reboot and shutdown cycles across a variety of OS and hardware platforms. A number of optional or informational tests can also be performed.

The conformance tests are aimed at validating that the SSD is conformant to the NVMe specification. To that end, we use a test tool that acts as an NVMe host to exercise and stimulate the NVMe SSD.

Common test tools: To ensure uniformity in the testing performed at a company’s internal lab, UNH-IOL, and at any plugfest, a common set of tools is provided through UNH-IOL’s NVMe Consortium. We use two primary test tools to perform the tests.

The first is the IOL Interact PC Edition tool, based on the open source tNVMe tool that the UNH-IOL administers. It can be run using any PC with an Intel PCIe chipset. The second is the IOL Interact Teledyne-Lecroy Edition. This tool is a set of UNH-IOL authored scripts and automation that uses the Summit series of PCIe tools from Teledyne-LeCroy. It can report exactly what is happening on the PCIe bus, and report on NVMe at the packet level. 

Plugfests: Industry plugfests serve as a public marker of the progress that new technologies make. NVMe has held two plugfests so far, both hosted by the UNH-IOL. 

At the plugfest, SSD implementations are cycled through tests against a variety of host platforms, operating systems, and drivers. Company representatives are present, making debugging easier. The plugfests also serve as a kickoff point for new test procedures and requirements. After the event, the UNH-IOL reviews the results to determine which products have qualified for the Integrators List. 

Testing services: Testing that occurs at the plugfest gets a lot of attention, and rightly so. But NVMe test services continue at the UNH-IOL. Some companies may not have been ready for the plugfest, or the event didn’t line up with their product release schedule. The opportunity to test and place products on the NVMe Integrators List is still open through testing at the UNH-IOL throughout the year. 

Results from these plugfests are promising, and as O’Reilly noted in his column, "justify claims that NVMe has arrived." Recent weeks have yielded additional developments signaling increased support for NVMe, including product announcements, OS support, and support from the test community. Testing can be tedious work; but without it, technological breakthroughs such as NVMe would be less beneficial to the industry at large.

David Woolf leads several efforts in the areas of storage and mobile technology at the University of New Hampshire InterOperability Laboratory (UNH-IOL). He is an active participant in a number of industry forums and committees that address conformance and interoperability ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Susan Fogarty
50%
50%
Susan Fogarty,
User Rank: Strategist
8/10/2014 | 7:40:34 AM
Re: Work at the IOL
LOL, @mikemoree, I have toured the IOL and it seems like a fabulous place to work. It's also only about a half hour from my house. Very tempting!
Susan Fogarty
50%
50%
Susan Fogarty,
User Rank: Strategist
6/24/2014 | 1:23:38 PM
Re: Work at the IOL
Hi David, thanks so much for your response and the additional details. 15 companies is a lot -- I would say that's a large number of early adopters. We're looking forward to hearing more about NVMe as it develops.
David Woolf
50%
50%
David Woolf,
User Rank: Apprentice
6/23/2014 | 2:11:14 PM
Re: Work at the IOL
Hi Susan, today UNH-IOL is working with 15 companies in the NVMe space, generally early adopters. Additionally there are 2 test and measurement companies involved. we expect the membership to continue to expand as NVMe adoption ramps up.
MarciaNWC
50%
50%
MarciaNWC,
User Rank: Strategist
6/20/2014 | 6:55:02 PM
Re: Work at the IOL
I'm curious too about the consortium and how many storage vendors participate. It seems that the kind of coordination UNH-IOL does must be challenging but the payoff is invaluable to the industry.
Susan Fogarty
50%
50%
Susan Fogarty,
User Rank: Strategist
6/18/2014 | 10:34:01 AM
Work at the IOL
David, thanks for this post. It's really interesting to get a glimpse into the details of how the InterOperability Lab does its work. About how many different storage vendors does the lab collaborate with on conformance testing for NVMe? (I'm wondering if so far it is only a few early adopters, or if most of the industry is participating.)
Cartoon
Hot Topics
6
The Rise Of White-Box Storage
Jim O'Reilly, Consultant,  8/27/2014
4
Cloud Storage Security: AWS Vs. Azure
Anna Andreeva, security quality assurance engineer, A1QA,  8/26/2014
4
Cloud Storage: How To Pick A Provider
Crystal Bedell, freelance writer,  8/26/2014
White Papers
Register for Network Computing Newsletters
Current Issue
Video
Slideshows
Twitter Feed