Decision Maker

Is Flash Memory in the Future of Your Storage Network?

• By Don Jones
• 10/01/2011

Solid-state drives (SSDs) based on flash memory are becoming pretty common these days, especially in ultraportable notebooks like the Apple MacBook Air. What are still less common are SSDs used as a mid-tier storage cache—although you're likely to see a significant increase in this use case, especially in server applications. Let's gaze into the crystal ball for a look at what's in the very near future.

First of all, we can probably all agree that SSDs are too expensive to use as primary storage in a Storage Area Network (SAN). That'll eventually change, but for the foreseeable future, spinning platters will continue to have a value advantage, meaning you'll be able to get bigger and bigger hard disks for less and less money. That's good, because it doesn't look like our storage needs are going to do anything but skyrocket over the next decade.

Most major SAN manufacturers already offer SSD-based caching, which usually takes the form of an array of enterprise-grade SSDs (what some are calling enterprise flash drives, or EFDs) that sit between your servers and the actual SAN. The idea is that the SSD array pre-fetches data from the SAN based on usage patterns, making it available for faster transfer to the server. These SAN add-ons are typically pretty costly, but they can deliver a significant performance improvement.

We're seeing something similar, albeit smaller in scale, at the desktop level. The newest chipsets from Intel feature the ability to use an on-board SSD as a high-speed cache for the computer's primary storage. Once the technology is fully developed, this per-machine cache will actually have the opportunity to be more effective than a SAN-level cache.

Enter Software-Based Caching
A breed of software-based cache utility is starting to enter the marketplace. The software sits between your server applications and your primary storage, "shimming" the NTFS file system, for example. It uses on-board EFDs in the 150GB to 300GB range, either connected via Serial-Attached Storage (SAS) or directly to the server's PCIe bus (which actually offers higher throughput than SAS). Properly tuned, this caching software can effect some marvelous performance improvements. In fact, my company recently conducted some benchmarks with a customer using SQL Server and saw triple the performance using SAS EFDs and a newly released cache-management utility.

Packed with Potential
Impressive as that is, it's not as impressive as where I predict this kind of technology will go. Remember Stacker, the utility that provided whole-disk compression for your MS-DOS machines? That capability (after some legal wrangling) eventually became the "compress" checkbox in Windows NT, a built-in feature that we all pretty much take for granted these days. That's where SSD-based caching is headed: into the OS. I can easily see a day when some version of Windows automatically detects and uses a locally installed EFD (or an EFD mirror, even) and uses it to speed up access to physical storage. In fact, these caches can already speed up access to both local and SAN-based storage, helping to overcome the latency present in even the fastest SAN connection, as well as the inevitable access waits for the SAN's spinning platters.

The more such a cache could overcome the slowness of spinning platters, the better a performance boost we'd see.

Eventually, we'll probably all be relying on SSDs for primary storage. At that point, the mid-tier SSD cache might go away—or it might simply evolve into faster, more expensive SSDs, still doing the job of speeding access to primary storage. Either way, this is definitely an area for smart decision makers to keep an eye on. The price-to-performance ratio makes this a set of techniques and technologies that matter.