In-Depth
Storage Disruptors: How Flash Drives and Cloud-Based Services Are Transforming IT Storage Plans
Microsoft is among a number of new providers upsetting the status quo by offering software defined storage, cloud-based data protection, flash-based arrays and converged Web scale infrastructure.
Advances in enterprise storage for years were largely relegated to higher-capacity drives and arrays and occasional boosts in throughput. Established players including EMC Corp., Hewlett-Packard Co., Hitachi Data Systems, IBM Corp. and NetApp had set the agenda for high-end storage with numerous others serving the mid-range and low end of the market and remain formidable forces. But look who's now putting the squeeze on these players. They're facing competition from companies such as Fusion-io Inc., GridStore Inc., Nasuni Corp., Nimble Storage Inc., Nutanix, Pure Storage Inc., SimpliVity Corp., SolidFire, Unitrends Inc. and Violin Memory Inc., plus many others that only a few years ago had yet to release their first products.
Even Microsoft has become a key player. Having acquired storage array supplier StorSimple two years ago and InMage last month, the company took a major step forward toward trying to disrupt the status quo of how enterprises store and protect their data. In addition, Microsoft continues to advance its Storage Spaces technology in Windows Server that virtualizes commodity disks. VMware Inc. has competing software for its virtualization suite called Virtual SAN (vSAN).
These are the storage disruptors because these players are reshaping how IT decision makers are planning their storage architectures and roadÂmaps. Moreover, they're changing the economics of storage as data growth reaches an all-time high. Many of the newer storage providers have respectable revenues and customer bases and have attracted huge interest from venture capital investors. Some have gone public or have become M&A targets.
SanDisk Corp. in June agreed to acquire flash storage vendor Fusion-io, led by CEO and former HP CTO Shane Robson, for $1.1 billion. Over the past year, Nimble and Violin have gone public though both of their shares declined sharply before starting to bounce back more recently. Pure Storage in April raised $225 million bringing its total funding to $470 million and a stated market value of $3 billion. Perhaps seeing how Nimble and Violin have struggled out of the gate, CEO Scott Dietzen recently was quoted as saying Pure Storage isn't ready to go public.
There are three main catalysts driving these changes. First is the growing presence of flash-based solid-state drive (SSD) arrays in datacenters providing application performance unthinkable a few years ago, as noted in last fall's cover story ("Flash Invasion," November 2013). Most of the players have both pure flash arrays and hybrid systems that accommodate both hard disk drives (HDDs) and SSDs. Established players including EMC and NetApp also now offer flash in their portfolios. EMC last month bolstered its XtremeIO arrays with the ability to instantly create in-memory snapshots for petabyte-scale applications that require real-time performance. The company also optimized the flash option on its high-end VMAX platform. And in June NetApp released its largest scale flash array to date, the FAS8080 EX, which it says scales to approximately 4 million IOPS.
The second driver is cloud storage, which is emerging as a tier that in many cases is replacing tape or secondary locations with drive arrays. Almost every storage hardware and software vendor is enabling the cloud as a storage tier or a target for backup and recovery and long-term archiving. Third is the vision every hardware and software infrastructure provider is evolving toward -- software-defined datacenters (SDDCs). Components of the SDDC are software-defined networking (SDN) and software-defined storage (SDS). SDS also powers the growing crop of converged systems, combining compute, storage and networking into a single system. Cisco, Dell Inc. and HP offer converged systems along with newer players Nutanix and SimpliVity.
Software-Defined Storage
It's early days for SDS, which uses software to decouple the storage functions from specific physical hardware, in some ways like server virtualization has done for creating virtual compute infrastructure and orchestration. Experts say there are many ways to look at SDS. One is around disaggregating the traditional SAN and storage arrays around system-level storage such as Microsoft Storage Spaces in Windows Server or VMware vSAN.
Others describe SDS as software bundled with hardware that can orchestrate and intelligently manage the tiers using APIs that leverage applications and plug-ins to OSes and virtual machines (VMs). Because SDS is still emerging, like any technology, there's still plenty of hype around it, experts warn. There are also a lot of nuanced interpretations.
Underscoring that point, analyst Anil Vasudeva, president and founder of IMEX Research, compared SDS to server virtualization during a recent webinar panel discussion presented by Gigaom Research. "Software-defined storage is a hypervisor for storage," Vasudeva says. "What a hypervisor is to virtualization for servers, SDS is going to do it for storage. [Of] all the benefits of virtualization, the reason why it took off was basically to create the volume-driven economics of the different parts of storage, servers and networks under the control of the hypervisor."
Prominent storage expert and fellow panelist Marc Staimer, president and chief dragon slayer of Dragon Slayer Consulting, had a somewhat different view. "In general, server virtualization was a way to get higher utilization out of x86-hardware," counters Staimer. "The concept of a hypervisor, which originally came about with storage virtualization, didn't take off because of what happened with storage virtualization [and] the wonderful storage systems that were being commoditized underneath a storage virtualization layer. What you're seeing today is commoditizing the hardware with software-defined storage."
Siddhartha Roy, principal group program manager for Microsoft (which sponsored the Gigaom Research webinar), says it's early days for SDS, especially among enterprises. "Enterprises will be a lot more cautious for the right reasons, for geopolitical or compliance reasons. It's a journey," Roy says. "For service providers who are looking at cutting costs, they will be more assertive and aggressive in adopting SDS. You'll see patterns vary in terms of percentages but the rough pattern kind of sticks."
SDS deployments may be in their early stages today, but analyst Vasudeva says it's going to define how organizations evolve their storage infrastructures. "Software-defined storage is a key turning point," he says. "It may not appear today, but it's going to become a very massive change in our IT and datacenters and in embracing the cloud."
Both analysts agree that the earliest adopters of SDS in cloud environments, besides service providers, will be small and midsize businesses. For Microsoft, its Storage Spaces technology in Windows Server is a core component of its SDS architecture. Storage Spaces lets administrators virtualize storage by grouping commodity drives into standard Server Message Block 3.0 pools that become virtual disks exposed and remoted to an application cluster.
"That end to end gives you a complete software-defined stack, which really gives you the benefit of a SAN array," Roy says. "We were very intentional about the software-defined storage stack when we started designing this from the ground up."
Flash Expansion
Less than a year after our cover story on flash, its presence in the datacenter appears to be extending rapidly. While flash arrays are far from pervasive, they're not a novelty in the datacenter, either. Even a growing number of midsize organizations are deploying flash arrays. Take Lifescript, a Web publisher focused on women's health. The Mission Viejo, Calif.-based content producer generates terabytes of storage and because of the volume of e-mail newsletters it produces and distributes, performance is critical.
Lifescript is an early adopter of flash drives in its datacenter. A longtime EMC shop, Lifescript started using 3PAR storage about six years ago, when it was an early upstart that was able to provide Web scale performance with its hybrid array of a small amount of flash and Fibre Channel drives. Lifescript Chief Technology Officer Jack Hogan found the datagrowth from all of the content its properties produce was starting to require more capacity and greater performance. Knowing that some all-flash SSD arrays were emerging, Hogan decided to consider his options.
At the time, 3PAR storage wasn't available in a pure flash array configuration suited for Lifescript's requirements, though Hogan says it has since come out with one. Lifescript ended up deploying Pure Storage's FA-420 all-flash array, equipped with 22TB of raw storage and with compression and deduplication it can store close to 100TB, according to Hogan.
"We are now running 100 percent of our production storage on Pure Storage all-flash SAN arrays," Hogan says. "Ultimately, we were able to pay for the implementation of that by consolidating our datacenter [and] removing the 3PAR storage on the floor. It took up a lot of power and space and to add more throughput, we needed more spindles, which meant we would need more space and power."
It has markedly improved the throughput of its Exchange Server and business intelligence (BI) applications based on SQL Server Enterprise Edition, according to Hogan. For example, BI jobs that took six hours to process in the past now complete in 20 minutes. "We have really been able to throw some heavy, heavy workloads at these all-flash arrays, and by every measure it has far exceeded our expectations," he says. With the SSD capacity Lifescript acquired, the company is utilizing about 30 percent, Hogan notes.
"What we figured out when we started the company five years ago was that by coupling low-cost flash with data-reduction technologies, you can actually make it a viable mainstream storage technology," says Matt Kixmoeller, a vice president at Pure Storage.
Many storage vendors are extending their APIs to enable SDS or more automation of storage, such as provisioning specific tiers based on specific policies, conditions and applications. Pure Storage has APIs for OpenStack-based cloud infrastructure and supports VMware APIs. While it doesn't support Microsoft Storage Spaces to date, Kixmoeller says it supports Microsoft Volume Shadow Copy Service (VSS) for integration with applications such as Exchange, SharePoint and SQL Server. It also supports Microsoft Multipath I/O (MPIO), where it can automate via CLI. "But there's no official plug-in to the Microsoft management stack yet," Kixmoeller says.
One flash array vendor that does boast tight integration with Windows Server and Hyper-V environments is Violin. Microsoft and Violin forged a close technical partnership two years ago, where the team in Redmond helped co-develop the Windows Flash Server. Microsoft wrote custom code in Windows Server 2012 R2 and Windows Storage Server 2012 R2 that interfaces with the Violin Windows Flash Array, launched in April.
The Windows Flash Array, designed to ensure latencies of less than 500 microseconds, comes with an OEM version of Windows Storage Server. "Customers do not need to buy Windows Storage Server, they do not need to buy blade servers, nor do they need to buy the RDMA 10GB embedded NICs," says Eric Herzog, CMO and senior vice president of business development. "Those all come prepackaged in the array ready to go and we do Level 1 and Level 2 support on Windows Server 2012 R2," Herzog says.
The April launch of the Violin Windows Flash Server carried a 64TB configuration with a street price of $395,000. Violin last month added a new entry-level 16TB system, which starts at $140,000. "We've added all of these capacities, which lets us go after smaller companies and departmental-level deployments," Herzog says. "It can easily fit two SQL Server databases, maybe three, depending on the size of the database. You can pay as you grow. You don't have to add new hardware, just purchase a license key." The entry-level system can scale in capacity to 64TB.
While Pure Storage and Violin emphasize pure flash arrays, most others -- including those that offer traditional HDD-based systems -- are offering hybrid solutions that have a mix of spinning disks and flash-based SSDs. Companies are largely competing on their software IP. Yet, as the economics of flash continue to become more appealing, expect suppliers to tip the balance of their hybrid arrays toward SSDs.
Seeing increased demand for flash, Nimble Storage, a fast-growing storage provider that offers hybrid arrays, in June launched its CS700 array available with an all-flash shelf. The company says this new system, which incorporates the Nimble Cache Accelerated Sequential Layout architecture and a cloud management platform it calls InfoSight, is suited for VDI and high-transactional databases. Each node supports 16TB and it can scale to four nodes, or 64TB. With the full array, it can scale to a petabyte of capacity.
Nimble claims the new array supports 500,000 IOPS. "It's not just about the specs, it's how we get there," says Ajay Singh, vice president of product management at Nimble. "Candidly, not that many workloads need 500,000 IOPS and tens of terabytes of capacity. In a typical environment you might have a small percentage of your data that needs that. Another big chunk that needs a balance of IOPS and capacity, and then another big chunk that just ends capacity, so one way we're different is with the same architecture -- we let you get all of those workloads in one system."
Web Scale and Converged Infrastructure
Also driving SDS and changing the enterprise storage scene is converged infrastructure, which consists of network, server and storage in a combined system. Cisco made a big push into converged infrastructure with its UCS platform adding server blades to its network gear. Cisco's push into the server market wasn't taken kindly by Dell and especially onetime partner HP, which both followed suit with their own converged infrastructure offerings.
Dell has made its share of acquisitions of storage and networking companies over the years but recently has turned to Nutanix to develop the Dell XC Series of Web-scale Converged Appliances set for release in the fourth quarter. Nutanix had a large presence at the Microsoft TechEd conference in Houston back in May where it showcased its Virtual Computing Platform. Powered by the Nutanix OS (NOS), the Web-scale appliances have a control fabric with multi-cluster management that's the basis of the converged compute, storage and networking system.
"We have eliminated the storage tier all together," says Laura Padilla, Nutanix director of strategic alliances. "There's no SAN and there's no external NAS. As customers go to more virtualized workloads, the whole concept of networked storage is somewhat of a mismatch of virtualization. The reason being is in a networked storage model where there's an external network tier, the hardware can become a bottleneck to performance and security."
Earlier this year, Nutanix added support for Windows Server 2012 R2 and Hyper-V in its offering. It already had supported VMware and KVM-based hypervisors. Gridstore is another up-and-coming provider of Web-scale storage infrastructure that supports Hyper-V. In fact, the company earlier this year decided to focus exclusively on the Microsoft datacenter platform. "We fit into Windows as a device driver, it's very clean and easy to fit in that level," says Kelly Murphy, founder and CTO of Gridstore. "With VMware and ESX, it is not easy and clean, so it was that technical aspect, as well."
Like Nutanix, Gridstore describes its infrastructure as Web scale, though it hasn't adopted a converged systems approach. Gridstore hardware is pure storage arrays that combine both HDDs and SDDs. At the Microsoft Worldwide Partner Conference (WPC) last month in Washington, D.C., Gridstore launched a larger 48TB node that can scale up to 250 systems, or 12PB. The company also says its hardware will integrate with Microsoft System Center and its Cloud OS platform.
Cloud As a Target
As the price of cloud storage declines almost monthly, more and more companies are using the cloud as a second or third tier of their storage infrastructures. All of the major storage software and backup and recovery providers support cloud storage in some way, though most have preferred providers or methods of connectivity.
Unitrends Inc., perhaps the most established of the disruptors, is among a number of vendors that offer storage appliances linked to the public cloud for disaster recovery. Its current offering uses the company's own cloud service. Chief Technology Officer Mark Campbell hints that Unitrends will offer connectivity to other clouds down the road. While Campbell says with its own cloud it can provide 24x7 "white glove" service, many customers will want lower-cost alternatives, especially for less-critical systems and data.
"The cost advantage we perceive will continue to get better with the public cloud with services such as Azure," Campbell says. So we want to make sure we're offering different levels of SLAs and different levels of providing for our customers going forward."
Whether it's SDS, flash, the growth of Web scale and converged systems or the cloud, few will argue that storage in the enterprise isn't going through a transition. Who will win and lose and set the agenda for next-generation storage is very much in play, but it's clear that unconventional players like Microsoft are moving aggressively into storage (see "Microsoft Makes Aggressive Push into Storage"), hoping to rearrange the deck. As some say traditional SAN and NAS are becoming legacy systems, others are giving them a new lease on life.
"A few years ago, storage was as boring as heck," says Enterprise Strategy Group analyst Mark Peters. "Hearing about new developments was like the release of new tires for cars. Now we have all kinds of fun things going on."