Software-defined storage is still in the early stages of its evolution. Many products fail to meet the goals of...
the software-defined storage approach and, although they are still a valuable step forward, can create future problems as the software vendor base expands and interoperability becomes more important.
Let's address the goals of software-defined storage (SDS). A successful SDS product removes much of the software from the storage nodes and reimplements services as distributable, scalable virtualized functions living anywhere within the virtual instance pool on the cluster. Within this thought process, services can be created or deleted depending on workload demand, while functions can be chained together to achieve complex operations.
A key goal is that services from many vendors can interoperate, allowing ad-hoc mash-ups and a pay-as-you-go option on the -aaS model. This open approach will increase competition and speed the evolution of new tools and improved performance.
The rising tide of hyper-converged systems
The current reality of SDS is that much of the thought process in SDS has been conflated with the rising wave of hyper-converged systems. In many ways, this is appropriate, since the service virtualization can easily be supported in such systems, but there are alternatives, as we'll see later in this article.
Hyper-converged systems stem from the evolving view that the storage appliance of the future is compact with relatively few drives and uses Ethernet and commercial off-the-shelf (COTS) CPUs. That's the same description one would apply to a server, and so we get the hyper-converged offering. Generally, this is a good starting point for entering SDS, as most traditional vendors have a turnkey SDS product.
These turnkey offerings are fixed configurations pre-certified to work with the SDS code installed and usually have a compliment of management tools as well as a virtual SAN software package, commonly Nutanix.
With Hobson's choice on hardware platforms and software builds from any given vendor, the real decision on the part of the buyer is which vendor to choose. This situation may last another year, but eventually the code will be freely available on all COTS systems and the platform vendor base will expand, causing prices to drop. Expect software vendors to open up to a broader set of hardware suppliers in the short term and to allow their code to run on whitebox systems within a few months.
Despite all of this, hyper-converged software remains closed. This brings us to alternative ways into SDS. The most useful is based on the very popular Ceph universal storage stack. Ceph is designed to be virtualized and has been demonstrated running with just a bunch of disks and even farms of Ethernet drives. Clearly, this is a fuller separation of services from the storage devices and opens up a breadth of alternative configurations.
Exploring alternative configurations in SDS
Being able to consider alternatives has become important for SDS. The overarching goal of any SDS product is to deliver data as quickly as possible, and there are some signs that hyper-converged offerings still struggle with bottlenecks, resulting in less than stellar performance. Networking is a major issue, for example, as all-flash arrays typically have 40 Gigabit Ethernet (GbE) links or better, compared to a shared 10 GbE link on a hyper-converged appliance.
Rethinking data flows and adding Remote Disk Memory Access to the Ethernet system certainly helps, as do additional Ethernet links, but you absolutely need good caching approaches and careful data management, which are still missing from the SDS infrastructure.
We are moving to an all-flash universe in the next two years. Seagate Technology has recently announced that 15,000 RPM Package Manager enterprise hard drives will no longer be developed. Any SDS product needs to address life with fast solid-state drives (SSD) in the storage node or hyper-converged appliance. That's a roadmap question that vendors should respond to with a detailed explanation as to how they will achieve the performance potential of these fast SSDs. Remember that at this stage in SDS evolution, vendor lock-in still exists with many offerings.
A final comment: SDS can't exist in a vacuum. Software-defined networking (SDN), its networking equivalent, is much further along in overall hardware and software ecosystems, but a good SDS product must interact with the SDN environment and allow for the automated orchestration of both in concert. That's another question to ask your SDS vendor candidates and, again, expect a detailed answer.
The software-defined data center is the inevitable future of IT. It's a dynamic and rapidly evolving area, and now is definitely the time to get your feet wet, if you haven't yet done so. Within a couple of years, the agility and cost benefits will far outweigh any other concerns, to the point that it will likely be a key strategy of every corporation.
Uncovering the hidden cost of SDS
What are the pros and cons of SDS management?
Overcome common SDS hurdles