All-flash arrays are changing the way we assemble storage for most applications in large businesses. Even though...
the price per terabyte is high compared to hard drives, the dramatically higher performance -- measured as IOPS -- means that all-flash arrays contribute differently to the way a virtual data center operates.
The purpose of dropping an all-flash array into configuration storage area network (SAN) is to make frequently used data (known as hot data) available quickly. In a typical virtual cluster, the solution to disk performance problems was to add more drives to the SAN. With each drive contributing just 150 IOPS, modern systems are still IO-starved and the result has been unnecessarily large primary tier storage capacity. Hot data is generally small compared with the capacity of these primary hard drive tiers, and the all-flash arrays are a good option for storing this limited data on super-fast storage.
The latest all-flash array units can compress data, which multiplies their effective capacities by factors of three to six times. This gives plenty of room for growth. Even so, the need to stage cooler data down to a secondary storage tier is paramount. This tier can take compressed data from the all-flash array, so it gets the same 3X to 6X capacity multiplier. With compression, it’s possible to repurpose existing primary or secondary storage and give it a life extension, obviating secondary tier purchases for a couple of years.
All-flash arrays impact more than storage capacity and performance in a virtual cluster. Virtualization has always created IO problems. With disk storage spread thin over many virtual instances, the IO rate per instance can get very low in some cases. Running LAMP stacks usually means low IO per instance, and the creation of several instances can create a boot storm.
We usually see boot storms with virtual desktop configurations, so it’s no surprise that many companies are looking to all-flash arrays for their virtual desktop infrastructure. An all-flash array provides so much more performance that boot storms become network congestion rather than IOPS issues. In many cases, this means they are no longer a problem.
At mid-level loading, all-flash array IO takes much less time to complete than hard drive arrays could achieve. Read or write operations take around 50 microseconds, versus the minimum 3 milliseconds on a disk array. This makes applications run faster, with a corresponding reduction in server resources needed for a given job. This reduction is affected by the amount of waiting for IO completion in the application, so it varies considerably.
At the extreme, high speeds make in-memory database possible, with performance gains of as much as 100X. While it is possible to store changes into local solid state drives (SSDs), data integrity and availability goals usually demand a networked copy before the data is considered properly committed. The low latency of all-flash arrays makes this possible, and is a direct alternative to distributed storage in the database cluster.
Is the all-flash array here to stay?
The all-flash array does have some serious competition in its role within a virtual cluster. The boot storm is an artifact of creating virtual instances containing their own copy of the desktop OS and apps. This wastes DRAM space and creates boot storms. A better approach would be to migrate to a containers model instead of traditional virtualization. With containers, there is only one copy of the image in any physical server, and this is shared on a read-only basis by all the instances. This saves much of the DRAM space and allows more instances per server, saving on server cost.
We are also in the early days of local instance stores in virtualized servers. These SSDs are shared by the instances and provide fast storage for both images and data. While they have the big disadvantage of not being backed up to the network in real time, big data applications, can get a real boost. Extending this type of storage to be a virtual SAN raises the possibility of creating a sort of distributed all-flash array, with the SSDs being accessible to any server in the virtual cluster. The jury is still out on whether this is a better approach than an all-flash array.
The all-flash array took off rapidly because it could be dropped into existing SANs with low risk and instantly boost performance. It has a lot of momentum behind it, and doesn’t typically require large changes. Containers and vSANs, on the other hand, represent a radical change, which means all-flash arrays will be around a while.
Clustered virtual host environments: The pros and cons
Achieve high availability with virtual server cluster