KVM is a popular virtualization platform that's easy to manage at low densities, but administrators need to be deliberate with installation and configuration to manage KVM at scale.
Basic tools, such as the command line or a GUI, enable relatively simple management of a small number of KVMs. On a larger scale that involves hundreds or thousands of machines, administrators should employ KVM best practices that prioritize deliberate installation, careful configuration and continuous optimization.
To manage KVM at scale, administrators first need to plan at scale. Haphazard, unchecked growth can create nightmare scenarios that damage the quality of a KVM deployment. Administrators should have a standardized deployment for host hardware and KVM host builds.
Standardization and commitment to KVM best practices ensure there are few surprises during the addition of new hosts, which means fewer unexpected issues. Similarly, administrators should have standard host software for build environments. Standardization simplifies patching.
Build the environment with scale in mind. Automate the build process to save time and limit manual interaction. Several platforms, such as Red Hat Virtualization and Proxmox, support fairly simple live migration, which often requires shared storage.
The ability to live migrate and limit downtime caused by maintenance and hardware failure is essential, even though it can increase infrastructure costs. Use thin provisioning with shared storage, but only at one end. Don't thin provision on both the storage array side and the KVM disk side.
Manage KVM installations
Administers must have virtio drivers installed when building KVM images. By default, most modern Linux distributions support KVM out of the box and have the required drivers built into the kernel.
Windows support is a different story. To successfully install Windows, administrators need to install the requisite drivers. Without them, the system disks can't be seen during installation and setup. The simple way around this problem is to have two virtual CD drives attached to the VM -- one with virtio drivers and the other with the OS. This method works, but it's inefficient.
Include the virtio drivers on the Windows installation CD to make a Windows image deployment easier. In low-memory situations, install the balloon memory driver to recover memory.
Build templates to save time. Templates promote easy deployment techniques, as well as consistent control over versioned builds.
Also, don't install useless items. For example, servers don't often require sound, so don't bother with the virtual sound card.
Choose a management platform
Other aspects of managing at scale include the care, monitoring and cultivation of a well-groomed deployment. It can sometimes be challenging to manage KVM day to day.
Some vendors, such as Red Hat, include an extensive management GUI. There are many different vendors that provide management GUIs, and administrators must choose one that is appropriate for their use cases.
KVM management is often disjointed. Employing the best platform in the right context is critical. There are many to choose from that are free or paid, SaaS or on premises.
Beyond the GUI, scripting knowledge is necessary to manage KVM at scale. Frequently performed tasks should be automated.
Experiment to optimize KVM best practices
Build this management foundation and then work toward other methods. For example, when installing Windows on a KVM host, it defaults to an integrated drive electronics (IDE) disk. This configuration isn't optimal, but it works. Replace the IDE configuration with a faster Serial Advanced Technology Attachment interface to see a noticeable speed improvement.
The same is true of other drivers, including the network. Use the paravirtual network stack rather than the default card to perform faster network actions. This will also place less stress on the KVM hosts. Paravirtualized network cards communicate directly with the KVM host network card, which reduces the time spent going up and down the stack.
Design the base images incrementally to enable slow and steady maturation of the deployed guest. Don't attempt to throw everything into the build; a good deployment will have a small number of base images.
Building KVM at scale is simple, insofar as the general rules of virtualization design still apply, and complex due to some unique considerations. KVM can do anything other hypervisors can do, but it requires mastery of KVM best practices to make those capabilities efficient and useful. KVM is as robust and stable as any of its competition if administrators manage KVM deliberately.