The Advent Of Enterprise and All-flash Storage

Storage technology has, for decades, been on a slow, steady path of progress, increasing data density in disk platters and incorporating increasingly large amounts of flash storage into individual drives to decrease access times. However, the advent of all-flash storage and the past few years’ progress have rocketed storage tech forward astonishingly. 2013 was a great year for flash storage, but 2014 will, without a doubt, be even better.

We’ll see both better solid state hardware, more efficient controllers and more elegant software. Add to that the fact that enterprise companies have started adopting solid state storage arrays and you have the right recipe for massive changes in business storage architecture, which will assuredly trickle down to consumer-level storage devices, making the world a better, faster place.

An all-flash storage array isn’t just a bunch of solid state drives shoved in a box and hooked up to a server. That’s part of the truth, but far too simplistic an approach to work on a large scale. Instead, racks of solid state drives on blades are hooked up via high-speed, redundant networking ports to a host controller, which is where the real tech advances are. Sure, the speed of solid state memory is increasing and the data density is rising dramatically, but none of that would matter if you couldn’t efficiently use the drives.

The host controller is responsible for receiving data from servers on its network and deciding what to do with it. A good host controller will not only direct data to drives evenly, keeping wearing even and preventing a single drive from being overloaded with data, but compress, deduplicate and perform parity calculations and storage for data. This saves space and provides the ability to rebuild data when a drive fails. Often, you can add multiple host controllers for redundancy and expand to new solid state drive racks when you start running low on space.

Everyone’s been talking about big data and its impact on business, but too little attention is being paid to the storage methods for big data. If you stick a multi-terabyte database on a hard disk array, it’ll take hours (perhaps even days) to run reports, far too long to be of any real use. Instead, we’re going to see enterprise companies start using flash arrays for big data storage, leveraging the speed and reliability of flash to speed up data analysis.

2014 won’t spell the end for hard disk storage (or any other storage technology, really), but it will see its prominence in enterprise storage fade significantly. When you’re running a major corporation that collects information about thousands or hundreds of thousands of customers, you don’t want mechanical storage that gets slower the more data you stuff it with. Instead, you want a storage solution that will offer steady access times no matter how much data is on the device.

Flash storage has come a long way since the first semiconductor memory modules in the 1970s. Before, the technology had low memory capacity and an extremely high cost. Although raw memory capacity is still lower than hard disks and the cost is still higher, efficiencies in host controllers have seen the prices drop to a point comparable to hard disk storage through bringing logical capacity to roughly the same levels (with deduplication and compression) and by advances in manufacturing that have dropped the cost of production significantly. 2014 will see these trends continue, helping solid state storage supplant hard disk storage as the monarch in enterprise data storage architecture.