Ethernet Interfaces Transform Object Storage

The idea of direct Ethernet drive interfaces dates back to at least 2001, although very little interest was generated among the very conservative storage clientele. A product platform using this technology finally arrived just a few months ago, and it may end up being one of the profound game-changers in the industry. 

At its simplest, the new interface means a lower-cost solution for small network-attached storage and backup appliances in the SMB sector of the market. This will alleviate the need for controllers and multi-drive boxes, likely replacing them with single external drives that connect directly to the LAN. With Ethernet comes a simple object interface called Kinetic that acknowledges that disk drives generally follow a write-once operation mode, and don't erase data often. (Seagate Technology launched the Kinetic Open Storage Platform in October.)

Those savvy in the storage game can get RAID-like protection by plugging two drives into the LAN and using host software, even with the object interface -- though this might be called replication in object store parlance.

But life gets interesting when you assemble a larger configuration. One drawback of enterprise-grade object stores is the need for many nodes, each with a relatively small amount of storage. That can get expensive. With an Ethernet switch replacing the SAS expander typically used, a JBOD (just a box of disks) can deliver 24 or 64 data stores, greatly simplifying the storage configuration.

Several notable vendors have projects underway to build storage appliances using JBODs and object storage code running on a server core, distributing the data blocks and replicas onto the JBODs. It's quite possible that the server core could be a virtual engine on an x86 server, pointing to some very slim storage systems. 

The servers handle deduplication and compression, so there's no loss of functionality. In time, this code will probably migrate to every server in the cluster, allowing data to be moved directly to the data stores. Compression saves on storage space, but host-based compression has the added advantage of reducing network bandwidth. It's like having 40 Gbit/s Ethernet for the price of 10 Gbit/s!

Direct Ethernet connection to the drives removes a layer of hardware, and all the associated latencies, from the storage stack. In addition, the object-oriented interface on the drive removes the layers of indirect data paths we've built into traditional SCSI-based block layer protocols.

For example, block IO completes up to four address translations before reaching the SCSI interface, and then a typical IO goes through LUN address translation, drive SCSI address translation, and drive physical address translation. They all take time, especially if a data lookup is involved. 

In an object storage system, the whole stack right up to the top layer in the host can be replaced by the object store code, which can figure the placement of the required data using the same algorithm that placed it in a data store in the first place. A native interface for object store drives would be much thinner and more efficient than the current approach. SSD and all-flash arrays would benefit from the new interface as well.

There are still some challenges to overcome. Most object stores spread data over a set of nodes. This helps overall efficiency of storage and prevents hotspots, but the algorithm typically breaks data into 64-KB chunks that are too small for today's storage, reducing system performance. Larger chunk sizes will improve the situation.

Advanced data protection systems, such as erasure coding (where data is striped over many drives), also face issues. This is easier when the physical drives are owned by a single controller such as a RAID card, rather than a globally shared resource as in the new system.

Security concerns could emerge, given that third parties could access the Ethernet "storage fabric." This is no more of problem than we already face in iSCSI and FCoE networks, however.

Ethernet object storage looks very promising, especially when you think about the interfaces in storage converging on Ethernet as a single solution in the longer term. We can expect alternatives to Kinetic to appear as the compute power of drives increases. This could be the beginning of something as big as the advent of SCSI, which changed drives forever.

Jim O'Reilly is a former IT executive and currently a consultant focused on storage and cloud computing.