There is a lot of emphasis on virtual machine resiliency and fault tolerance, but what happens when the power goes...
out? The largest enterprises may be able to keep VMs running using generators or quickly fail over to a remote data center, but what about companies that depend on server battery backup? Let's consider strategies that you can use to reduce power consumption and improve the odds that your most critical VMs will remain online throughout the outage.
When the power goes out and you are forced to run on batteries, your goal must be to keep the systems running for as long as you can and also know when to initiate a shutdown so that your servers have time to power off gracefully before the battery is exhausted. While this concept seems simple enough, battery power management is an art.
Planning your response to a power failure
The first point you need to consider is how long your batteries will last. There is usually no clear answer. Still, it is important to have an estimate of how long you can expect the batteries to hold out while they are supporting a normal workload. Only then can you begin planning your response to a power failure.
Although a host server battery backup is undeniably important, you can't focus solely on server power. Your virtualization hosts are tied into other hardware. For example, are your host servers connected to a storage area network or a Cluster Shared Volume? If so, is your external storage also protected by battery, and how long will that battery power last?
The same consideration must also be made for networking hardware such as switches. A host server cluster can only function if the cluster nodes are able to communicate with one another. As such, it is imperative that your batteries be able to power your network switches for the duration of the power failure (or until your virtualization hosts have been shut down).
Of course, those are basic considerations that are just as valid in a physical data center as they are in a virtual data center. However, server virtualization presents some challenges of its own.
Virtualization-specific challenges to server battery backup
Earlier I mentioned that it was tough to determine how long your virtualization hosts would be able to run on battery power. One of the things that makes this a hard answer is that the amount of time a battery can power a server depends heavily on how hard the server is working. A heavily used server simply consumes more power than an idle one.
That being the case, you should look for ways to reduce the server's power consumption when it is running on battery. One thing that you might be able to do is to initiate a shutdown of low-priority VMs. Perhaps, for example, you have some redundant virtualized domain controllers that you can shut down. Taking a step like this will help accomplish two things.
First, shutting down low-priority VMs can help to reduce power consumption (although there will be a power spike during the shutdown process due to increased storage I/O). Shutting down low-priority VMs will also reduce the load on the hosts, thereby decreasing the amount of time that it takes to shut down the host servers if necessary.
Another point to consider is whether multiple host servers share a battery. In a perfect world, each host server would have its own battery. However, in the age of endless IT budget cuts, administrators may be forced to share batteries.
If multiple hosts are sharing a battery, can you live-migrate VMs off of some hosts and then shut down the empty hosts? If so, then you can save a significant amount of battery power. However, keep in mind that a minimum number of hosts must remain online in order for your host server cluster to continue to function.
You must also consider what you will do when you start running out of power. Should you put VMs into a saved state or shut them down? It is usually faster to put a VM into a saved state, but there are some applications that have trouble resuming from a saved state. For example, if a VM is running a database application, then it might be better to shut down that VM rather than forcing it into a saved state.
One last question that is worth considering is how you can automate your response to a power failure. If the power fails and your hosts are running on battery, you don't want to have to manually migrate VMs or shut down unnecessary hardware one piece at a time. Thankfully, you may be able to use IT automation software to automate the process.
Power outages are never fun for IT professionals, and server virtualization can add to the challenge of dealing with an outage. The key is having a plan and the correct policies in place to deal with those challenges quickly to keep critical VMs running as long as possible.