In a move planned since the turn of the century, Western Digital's latest Caviar Green drives use 4K data sectors rather than the 512 (and occasionally 520) byte sectors that have been the norm. The larger sector size both boosts the formatted capacity and enables better ECC that can correct larger errors than today's drives could. Since today's operating systems and RAID controllers are expecting 512 byte blocks, the Caviar Green does 512 byte emulation, but as most of us would suspect, there are a few pitfalls awaiting blind installation of a 4K drive in a system that's not ready for it.
While the jump from 512 byte to 4K sectors is the first change to disk sector size in a couple of decades, using larger sector sizes to reduce the overhead of sector headers and trailers is an old technique. Back in the '80s, I made a few dollars writing BIOS hacks for CP/M machines that allowed them to format their floppies with 1024 byte sectors for a 20 percent capacity boost over the default 128 or 256 byte sectors. Back then, larger sectors had the drawback of increasing the minimum allocation block size wasting disk space in breakage as the small files of the time had to be allocated in 1K blocks. A 1200 byte letter written in WordStar would use 2048 bytes on disk formatted for 1K sectors but just 1280 bytes on a disk with 256 byte sectors.
While the capacity boost from bigger sectors is nice, WD gets about 11 percent more data on the drive from using 4K sectors they've dubbed Advanced Formatting, the driving force to 4K sectors is the improved error correction. As drive vendors cram more bits into the same space, the strength of the electrical signal in the drive's heads decreases. Unfortunately the background noise level remains the same, so picking out data from amongst the noise becomes harder in each generation of drive. Some report that the bit error rate of data for today's drives is around one in a million (10^6), so drives are already highly dependent on ECC to deliver their expected 10^14 error rate. Smaller bits also means that a surface defect, even one just a micron in size, wallops more bits, so the ECC needs to be more robust.
Vendors could have added extended ECC to the 512 data block in each sector, but that would reduce the available space for user data. Since ECC algorithms are more efficient for larger data blocks, a drive with 4K sectors could use 100 bytes per sector of ECC and be able to correct errors up to 1000 bits long, while a drive with 80 bytes of ECC per 512 byte data sector, which is twice what today's drives use, could only correct an 800 bit error. Bigger sectors means better ECC and more user data. Who could complain about that? Okay, some have complained that WD hasn't boosted the spec sheet BER to reflect the better ECC, but from where I sit the spec sheet is just marketing anyway.
The other good news is that 4K turns out to be something of a magic number for today's computers. X86 processors work in 4K memory pages, and most file systems, including NTFS and HFS+, use 4K allocation clusters and database applications from Oracle to Exchange to read and write data to the disk in pages that are 4K or a multiple of 4K bytes. Once we clear the startup phase, and servers are actually reading and writing their disks in native 4K blocks, we could get additional marginal efficiency by managing resources in 4K chunks.
