Ask, "what's the best way to connect to my storage"? and you're sure to get a wide variety of answers. A few years ago, the answer you'd get would be largely dependent on the type of vendor that you were working with, since most only supported one method each. Now though most vendors support at least two and a few support just about all storage connectivity options. They will now often answer "Whatever one you want," which doesn't solve your problem.
The selection process has been made even more complex by increasing options. The number of connectivity options to storage has continued to increase over recent years. In addition to the old standbys of network attached storage (NAS), fibre channel (FC), and iSCSI, we have added fiber channel over Ethernet (FCoE) and ATA over Ethernet (AoE). In fact, as we describe in our "SAS Primer White Paper," even serial attached SCSI (SAS) can be shared to a degree. Finally there is the resurgence of direct attached storage (DAS), thanks to smarter applications like Exchange 2010.
The truth is that when selecting a storage connection method, most administrators are going to select one they are already familiar with in some way. If you already have a SAN in place and it is based on FC, than you will likely continue down that path. If you don't have FC experience, more than likely you're going to look at one of the four Ethernet options (NAS, iSCSI, FCoE, and AoE).
As much as I like FC and as dominant as it is in the data center today, making the case to move to fibre if you don't already have some implemented is increasingly hard. 10-Gbps Ethernet is more than fast enough for most environments to leverage one of the IP options without concern for the overhead of IP. Servers, in most cases, have CPU overhead to spare, so dealing with the additional IP overhead is even less of an issue.
If maximum Ethernet storage performance is required, the off-load capabilities of iSCSI and FCoE takes the concern about IP overhead away. FCoE is basically fibre using Ethernet cables to connect, so this seems like an ideal option for FC sites that want to leverage Ethernet. As we discussed in our article "Taking A Holistic Approach to Storage Performance," AoE also leverages an inexpensive host bus adapter that provides the connectivity. It is a raw Ethernet transfer not doing any IP-to-storage conversions.
There are exceptions to every situation, for example 16-Gb FC is on the horizon, and in some extreme cases, especially where SSD is involved, it may be a better option for maximum performance. Ethernet though is not standing still: 40GbE is also right around the corner, so the performance advantage that FC has may be short lived.
If you go with Ethernet, the debate becomes more interesting since there are so many viable choices. Not to be left out of the conversation is the resurgence of DAS. In an upcoming article I'll cover when and if you should consider DAS instead of shared storage for your next project.
The short answer to the question "How should you connect your storage" is really going to depend first on what you know and then on what you need to accomplish. Performance eventually becomes an issue in almost every data center. Our advice is to select the fastest performing storage connectivity method with the most headroom for growth that is going to have the smallest learning curve based on your background. Then test that decision out and compare it to at least some alternatives.
Follow Storage Switzerland on Twitter