In this series on patching virtual environments, I've covered the benefits of a server patch management strategy, patch management best practices and patching methods for virtual machine (VM) updates. In this tip, I discuss how virtual machine host monitoring affect VM performance as well as how to develop sound patching policies to maintain virtual environment efficiency.
Virtual machine host monitoring and hardware architecture
It would be nice if we could treat every VM the same way and apply similar tolerances for downtime. But because this is not possible with physical hardware, it makes sense that it doesn't apply to virtual hardware either. In fact, when multiple VMs compete for the same virtual machine host resources, virtual hardware can be more complicated.
Previously, I explained that VM patching methods are similar to physical server updates. The difference, however, is how patches are deployed to every VM in your environment.
When multiple VMs on the same host try to do something simultaneously, there is a chance that the virtual machine host resources could become saturated, generally resulting in a degradation of available CPU and disk I/O resources. In most cases, the hosts will finally complete all the workload requirements, but the downtime may be much longer than expected because each VM tries to complete its task first.
In my environment, I can have more than 30 VMs on a single host. If these VMs are patched and attempt to simultaneously reboot, each VM residing on the host would experience performance degradation.
The lesson here is that when you establish a virtual machine patching schedule, you need to monitor host resources and account for virtual host hardware architecture. To ensure VM stability, you should correctly size up virtual host hardware and potentially reduce the number of VMs on a host to accommodate for peak resource usage.
Designing virtual environment patching policies
Regular security patches are the most common disruptions to VM workloads, but numerous supporting software updates can affect VM performance.
While it's critical to monitor host resources, this process becomes problematic when you perform antivirus scans during peak loads. For the most part, antivirus scans are desktop-centric and generally architected to use as many resources as possible so they can finish scans quickly when the system is idle. Unfortunately, antivirus software doesn't account for the host's resource load. If I have 20 VMs running on an eight-core CPU virtual host, with each VM using a CPU, for example, I have a 2.5:1 CPU-to-core ratio. In everyday processing, this works fine. But if an antivirus scan, patch distribution or even a disk defragmentation occurs on each VM simultaneously while these VMs think they are idle, I have 20 VMs trying to use 100% of their CPU or disk I/O resources. In this scenario, VM performance and processing can come to a crawl because the host is saturated, slowed down or interfering with VM workloads.
Again, similar to architecting hardware, reviewing policies for overall patching and trying to distribute the load is very important in shared-resource environments. With this approach, your environment may take longer to configure, and you might have to get creative, but if the goal is to maximize the VM-to-host ratio, then this critical planning will ensure your environment runs efficiently.
Undoubtedly, patching VMs is a necessity. With any sharing environment, however, all of the peak VM workload requirements need to be carefully managed to ensure the overall health of your virtual environment. Making sure all the patches that optimize and secure VMs are installed is only the first step. Coordinating the deployment, updating, scanning and other everyday functions -- based on the available virtual machine host resources -- is where you find the real talent of a virtual server architect.
Have you had any issues with monitoring host resources or VM performance issues because of a patch? Keep the conversation going here.
About the expert
Rob McShinsky is a senior systems engineer at Dartmouth Hitchcock Medical Center in Lebanon, N.H., and has more than 12 years of experience in the industry -- including a focus on server virtualization since 2004. He has been closely involved with Microsoft as an early adopter of Hyper-V and System Center Virtual Machine Manager 2008, as well as a customer reference. In addition, he blogs at VirtuallyAware.com, writing tips and documenting experiences with various virtualization products.