Click to enlarge in another window
With the jumbo frames, ensure all the devices on your iSCSI network--including switches, initiators and targets--are configured to use the same maximum frame size. There is no standard, maximum jumbo-frame size, and we've seen equipment supporting frame sizes from 9,000 bytes to 16 KB.
If your servers or disk arrays are set to a larger maximum frame size than your switches, your iSCSI system will appear to be working perfectly until you start doing large data transfers that exceed the switch's maximum frame--then disk I/O errors will start cropping up.
Stick Your TOE in the Water
Although most current operating systems support iSCSI software initiators that let you use any Ethernet card to connect your servers to an iSCSI disk array, don't just use any old Gigabit Ethernet card for your iSCSI connection. Ethernet cards designed for workstations, for example, use a 32-bit PCI bus with just more than a gigabit of bandwidth that gets shared with other devices on the bus. A server Ethernet card uses the much faster PCI-X or PCI-Express bus and performs onboard TCP/IP checksum/off-load, which reduces the CPU's iSCSI traffic processing. Broadcom's NetXtreme controller chips, which come with most server motherboards, also do checksum/off-load.