Learn how to maximize your existing data storage investments.
So you have a bunch of EMC RAID arrays and a couple of Dell iSCSI SAN boxes, topped with a NetApp filer or two. What do you say to the CEO who reads my articles and knows enough to ask about solid-state drives, all-flash appliances, hyperconverged infrastructure, and all the other new innovations in storage? “Er, er, we should start over” doesn’t go over too well! Thankfully, there are some clever -- and generally inexpensive -- ways to answer the question, keep your job, and even get a pat on the back.
SSD and flash are game-changers, so they need to be incorporated into your storage infrastructure. SSDs are better than enterprise-class hard drives from a cost perspective because they will speed up your workload and reduce the number of storage appliances and servers needed. It’s even better if your servers support NVMe, since the interface is becoming ubiquitous and will replace both SAS and (a bit later) SATA, simply because it’s much faster and lower overhead.
As far as RAID arrays, we have to face up to the harsh reality that RAID controllers can only keep up with a few SSDs. The answer is either an all-flash array and keeping the RAID arrays for cool or cold secondary storage usage, or a move to a new architecture based on either hyperconverged appliances or compact storage boxes tailored for SSDs.
All-flash arrays become a fast storage tier, today usually Tier 1 storage in a system. They are designed to bolt onto an existing SAN and require minimal change in configuration files to function. Typically, all-flash boxes have smaller capacities than the RAID arrays, since they have enough I/O cycles to do near-real-time compression coupled with the ability to down-tier (compress) data to the old RAID arrays.
With an all-flash array, which isn’t outrageously expensive, you can boast to the CEO about 10-fold boosts in I/O speed, much lower latency , and as a bonus a combination of flash and secondary storage that usually has 5X effective capacity due to compression. Just tell the CEO how many RAID arrays and drives you didn’t buy. That’s worth a hero badge!
The idea of a flash front-end works for desktops, too. Use a small flash drive for the OS (C-drive) and store colder data on those 3.5” HDDs. Your desktop will boot really quickly, especially with Windows 10 and program loads will be a snap.
Within servers, the challenge is to make the CPU, rather than the rest of the system, the bottleneck. Adding SSDs as primary drives makes sense, with HDDs in older arrays doing duty as bulk secondary storage, just as with all-flash solutions, This idea has fleshed out into the hyperconverged infrastructure (HCI) concept where the drives in each node are shared with other servers in lieu of dedicated storage boxes. While HCI is a major philosophical change, the effort to get there isn’t that huge.
For the savvy storage admin, RAID arrays and iSCSI storage can both be turned into powerful object storage systems. Both support a JBOD (just a bunch of drives) mode, and if the JBODs are attached across a set of server nodes running “free” Ceph or Scality Ring software, the result is a decent object-storage solution, especially if compression and global deduplication are supported.
Likely by now, you are using public clouds for backup. Consider “perpetual “storage using a snapshot tool or continuous backup software to reduce your RPO and RTO. Use multi-zone operations in the public cloud to converge DR onto the perpetual storage setup, as part of a cloud-based DR process. Going to the cloud for backup should save a lot of capital expense money.
On the software front, the world of IT is migrating to a services-centric software-defined storage (SDS), which allows scaling and chaining of data services via a virtualized microservice concept. Even older SANs and server drives can be pulled into the methodology, with software making all legacy boxes in a data center operate as a single pool of storage. This simplifies storage management and makes data center storage more flexible.
Encryption ought to be added to any networked storage or backup. If this prevents even one hacker from reading your files in the next five years, you’ll look good! If you are running into a space crunch and the budget is tight, separate out your cold data, apply one of the “Zip” programs and choose the encrypted file option. This saves a lot of space and gives you encryption!
Let’s take a closer look at what you can do to transform your existing storage infrastructure and extend its life.
(Image: Production Perig/Shutterstock)
Add all-flash arrays
Providing both a performance and capacity boost, all-flash arrays are a must-have for SAN solutions. Originally a small, fast cache for HDD SANs, the all-flash concept has grown into top-tier storage housing the relatively small amount of hot data in the business, with slow RAID arrays hosting all the old, cold data.
The performance boost would be enough for many users, but all-flash boxes have such high bandwidth that it is feasible to run compression as a background job, which typically multiplies the effective capacity of both the all flash array and the old RAID storage by 5X.
Imagine going to the boss and saying that the storage pool now runs 10 times faster and you increased total storage from 100TB of older RAID to 700TB by adding a 40TB (effectively 200TB) all-flash array!
All-flash arrays may be able to create a single storage pool from all your storage, too, including iSCSI and even from different vendors or models. Pooling SMB or NFS filers requires an all-flash unit with those protocols supported.
NVMe-based all-flash arrays promise even faster operation, but require new Ethernet server NICs.
(Image: Timofeev Vladimir/Shutterstock)
Build hyperconverged infrastructure
If you have a bunch of surplus servers lying around, even older models, consider building a hyperconverged cluster. Today’s HCI unit is a pre-configured appliance, but Nutanix, a top hypercovergence supplier, is moving to an all-software model, so the stack will be available and will run on almost all COTS boxes. The drives from RAID arrays can usually be made to fit the COTS boxes. Nutanix supports iSCSI and Fibre Channel over Ethernet; the alternative of coupling Fibre Channel-based RAID to Nutanix nodes appears to work, but isn’t formally supported. Nutanix uses Fibre Channel for file migration.
This provides a way to build out an HCI test cluster, for example, or a way to provide cold secondary storage to a new HCI cluster. Depending on your use cases, you might even go to production with these HCI units, perhaps supplementing other purpose-built HCI gear.
Create an object store
If you have a set of RAID arrays that you're pulling from Tier 1 storage, consider repurposing them to an object store. If you convert the arrays to JBODs, each drive can be independently mounted to a server, which could be an inexpensive 1U rack server connected to maybe six or 10 drives. These become object store drives (OSDs) and objects can be distributed across them.
If you use erasure coding and ensure that data is distributed across several of the servers, this is a robust configuration with decent performance. You’ll need software, but Ceph is free, although getting a support license from Red Hat may be a good idea. Alternatively, Scality’s Ring product is well featured.
Compression and global deduplication are good features with this type of configuration, expanding capacity and really speeding up traffic.
Consider perpetual storage
With the advent of cheap, high capacity SSDs and HDDs, and even more so the cloud, backup, disaster recovery, and protection against hackers are all focusing on some method of "perpetual" storage. The idea is that you can always go back to any point in time, such as before the flood or data breach, and continue operations.
To convert to a perpetual mode of operation, you can either utilize a snapshot tool within the operating system on your servers or offered with the storage units, or alternatively, use a continuous backup software package from a third-party vendor. Covering al use cases from active storage to DR and backup is more complex, since offsite copies must be kept. Replication or erasure coding capabilities from public cloud providers can help.
If you set up the full gamut of services, you can handle accidental or deliberate file erasure, data breaches, and loss of a local data center or cloud setup in stride, with minimal downtime and additional cost (presuming you're doing backups anyway).
Software-defined storage (SDS)
This is a cutting-edge technology in storage, but one type of storage management tool, including products offered by IBM Tivoli and Red Hat, allows all of the storage in a data center to be pooled into a single storage entity. This allows easier management and better capacity utilization. Better still, it means that a forklift storage upgrade isn’t necessary as appliances are replaced.
SDS will enable many more capabilities over the next few years, including global deduplication and compression, automated tiering and archiving to the cloud, all of which protect that legacy storage for a while longer.
(Image: Timofeev Vladimir/Shutterstock)
Secure data with encryption
If you want to keep your data safe, encrypt it at the server and NEVER give data keys to a third party. Using drive-based encryption is risky. Encryption prevents unauthorized reading of data from your files, which is the main goal of most hackers today. It doesn’t prevent deletion or overwriting of data, although the latter should be detectable, at least.