Special Coverage Series

Network Computing

Special Coverage Series

Commentary

Howard Marks
Howard Marks Network Computing Blogger

Flash Write-Back Caching Limitations

As flash-based caching systems add write-back capabilities, be aware that performance gains can hit a wall.

While caching has been around at least as long as the PC itself, the current generation of caching systems that use flash rather than main memory as their cache medium are a different kettle of fish. As the first of these server-side caching products add write-back caching to their bags of tricks, it's important that you, dear reader, understand how write caches really work.

To begin, we have to note that a write cache--regardless of whether it's using NVRAM in an array controller, flash in a hybrid system or flash in a server--can really only accelerate bursts of write traffic. After all, the whole idea of a cache is that it’s a place data can land quickly until it can be written to the backing store.

More Insights

Webcasts

More >>

White Papers

More >>

Reports

More >>

Just a few years ago, when the write cache on a high-end disk array was just a few gigabytes, cache exhaustion was commonplace. I remember Microsoft's top Exchange guys saying that array or RAID controller caches just didn't matter; a heavy-duty application could run its storage system out of cache in just a few minutes.

Of course, once the write cache is full, any new data has to either be written directly to the backing store, bypassing the cache, or wait for the caching engine to flush some other data to make room. Either way, performance falls back to the speed of the backing store, which is hopefully fast enough so users don’t bring out their torches and pitchforks.

Today, we can easily use a terabyte or more of flash on a single server. Virident (acquired earlier this month by Western Digital subsidiary HGST) this week announced a 4.6-Tbyte half-length, half-height PCIe flash card. These huge caches can accelerate bursts of traffic that last not just minutes but hours. However, it's important to remember that the data eventually has to be written to your backing store.

[Read why Howard Marks doesn't think the limited write endurance of flash-based SSDs is a cause for significant concern in "SSDs And The Write Endurance Bogeyman."]

On some of my pessimistic days, I imagine a team of IT administrators getting a rude surprise a few months after they fire up their latest hybrid storage system. Their applications write just a little more data each day than their SATA disk-based backing store can absorb. During the day, they write 58 Tbytes of new data, and overnight the system flushes 55 Tbytes to the disk.

Everything seems fine until the time comes when there isn't any more free space in the cache. Performance will then drop from the 100,000 IOPS the flash cache was providing to the 1,000 IOPS the RAID-6 SATA backing store can deliver.

The good news is that at least some of the producers of hybrid systems and server side write-back caching software recognize the problem and have built mechanisms into their products that apply back-pressure, delaying each IO just a tiny bit as the cache fills so users will see a more gradual drop in performance.

Users of write-back caching products, regardless of whether they run in a storage system or the server, should watch the amount of data in their cache and act to speed up their backing store if the cache doesn't periodically empty completely. While I can imagine a system that only empties its cache over the weekend working perfectly well, I wouldn’t sleep well at night if my systems didn’t get all their data to the disk within a few hours of when my applications wrote it.

[Get deep insight into how flash-based solid state disks (SSDs) work and how they can be deployed in Howard Marks' session, "SSDs In The Data Center" at Interop New York Sept. 30-Oct. 4.]



Related Reading



Network Computing encourages readers to engage in spirited, healthy debate, including taking us to task. However, Network Computing moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing/SPAM. Network Computing further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | Please read our commenting policy.
 

Editor's Choice

Research: 2014 State of Server Technology

Research: 2014 State of Server Technology

Buying power and influence are rapidly shifting to service providers. Where does that leave enterprise IT? Not at the cutting edge, thatís for sure: Only 19% are increasing both the number and capability of servers, budgets are level or down for 60% and just 12% are using new micro technology.
Get full survey results now! »

Vendor Turf Wars

Vendor Turf Wars

The enterprise tech market used to be an orderly place, where vendors had clearly defined markets. No more. Driven both by increasing complexity and Wall Street demands for growth, big vendors are duking it out for primacy -- and refusing to work together for IT's benefit. Must we now pick a side, or is neutrality an option?
Get the Digital Issue »

WEBCAST: Software Defined Networking (SDN) First Steps

WEBCAST: Software Defined Networking (SDN) First Steps


Software defined networking encompasses several emerging technologies that bring programmable interfaces to data center networks and promise to make networks more observable and automated, as well as better suited to the specific needs of large virtualized data centers. Attend this webcast to learn the overall concept of SDN and its benefits, describe the different conceptual approaches to SDN, and examine the various technologies, both proprietary and open source, that are emerging.
Register Today »

Related Content

From Our Sponsor

How Data Center Infrastructure Management Software Improves Planning and Cuts Operational Cost

How Data Center Infrastructure Management Software Improves Planning and Cuts Operational Cost

Business executives are challenging their IT staffs to convert data centers from cost centers into producers of business value. Data centers can make a significant impact to the bottom line by enabling the business to respond more quickly to market demands. This paper demonstrates, through a series of examples, how data center infrastructure management software tools can simplify operational processes, cut costs, and speed up information delivery.

Impact of Hot and Cold Aisle Containment on Data Center Temperature and Efficiency

Impact of Hot and Cold Aisle Containment on Data Center Temperature and Efficiency

Both hot-air and cold-air containment can improve the predictability and efficiency of traditional data center cooling systems. While both approaches minimize the mixing of hot and cold air, there are practical differences in implementation and operation that have significant consequences on work environment conditions, PUE, and economizer mode hours. The choice of hot-aisle containment over cold-aisle containment can save 43% in annual cooling system energy cost, corresponding to a 15% reduction in annualized PUE. This paper examines both methodologies and highlights the reasons why hot-aisle containment emerges as the preferred best practice for new data centers.

Monitoring Physical Threats in the Data Center

Monitoring Physical Threats in the Data Center

Traditional methodologies for monitoring the data center environment are no longer sufficient. With technologies such as blade servers driving up cooling demands and regulations such as Sarbanes-Oxley driving up data security requirements, the physical environment in the data center must be watched more closely. While well understood protocols exist for monitoring physical devices such as UPS systems, computer room air conditioners, and fire suppression systems, there is a class of distributed monitoring points that is often ignored. This paper describes this class of threats, suggests approaches to deploying monitoring devices, and provides best practices in leveraging the collected data to reduce downtime.

Cooling Strategies for Ultra-High Density Racks and Blade Servers

Cooling Strategies for Ultra-High Density Racks and Blade Servers

Rack power of 10 kW per rack or more can result from the deployment of high density information technology equipment such as blade servers. This creates difficult cooling challenges in a data center environment where the industry average rack power consumption is under 2 kW. Five strategies for deploying ultra-high power racks are described, covering practical solutions for both new and existing data centers.

Power and Cooling Capacity Management for Data Centers

Power and Cooling Capacity Management for Data Centers

High density IT equipment stresses the power density capability of modern data centers. Installation and unmanaged proliferation of this equipment can lead to unexpected problems with power and cooling infrastructure including overheating, overloads, and loss of redundancy. The ability to measure and predict power and cooling capability at the rack enclosure level is required to ensure predictable performance and optimize use of the physical infrastructure resource. This paper describes the principles for achieving power and cooling capacity management.