There are several KVM storage options, including virtual disk files, file system-based storage and device-based storage.
To manage Kernel-based Virtual Machine (KVM) storage, you can use Logical Volume Manager and create storage pools. When you create a KVM virtual machine (VM), it by default uses a virtual disk file for back-end storage. With this setup, the VM thinks it sees an actual hard disk, but it instead sees the virtual disk file that represents the hard disk. This extra file-system layer can slow down your system.
Still, virtual hard disks based on a disk image file are not all bad, because disk files are easily made available to other KVM virtualization hosts. But if you want optimal KVM virtualization performance, there are other KVM storage options to consider.
File system-based KVM storage
When setting up a KVM host, you can choose the file system directory (dir) or formatted block storage (fs ) to get started with KVM storage. The default selection is dir, where you choose a directory in the local file system where you'll create a disk image file.
The fs option lets you provide the name of a formatted file system that you're going to dedicate to storing disk image files. The main difference between this KVM storage option and the dir option is that with fs, the file system isn't mounted on a specific location.
With either option, you can work on a local file system or on a file system that’s physically hosted on a SAN. The physically hosted file system has some benefits, because a SAN can be accessed by multiple hosts simultaneously, which isn't the case for a local disk or file system.
Another file-based disk storage method is netfs, which lets you specify the name of a network file system such as a Samba mount. Using this method for KVM storage is convenient, because it’s easy to access a file system on another server, which also allows you to access the disk files from multiple hosts.
All these file system-based KVM storage methods have a disadvantage, though: the file system itself. That’s because the VM’s hard disk doesn't write directly to the KVM storage device, but to the file system on the host OS. This means there’s an unnecessary layer when accessing and writing files, which often weakens performance. So if you're looking for optimal performance settings with KVM virtualization, it’s better to go for device-based storage.
Device-based KVM storage
Another way to handle KVM storage is to take a device-based approach. There are four different methods that allow you to address physical storage directly: disk, iSCSI, SCSI and logical. Disk lets you write to a disk device directly. The iSCSI and SCSI methods are an alternative but comparable approach in which you address the disk device by using its SCSI or iSCSI address directly. The benefit of this KVM storage method is that you use persistent naming that doesn't depend on the order that the host OS discovers the device.
These methods for addressing disks have a disadvantage as well though: inflexibility. There’s no easy way to change the size of a virtual hard disk or use snapshots with device-based KVM storage.
For optimal KVM storage flexibility, use Logical Volume Manager (LVM). One benefit is that LVM allows you to use snapshots, which are otherwise not available as a KVM virtualization feature.
LVM also allows you to put storage in a volume group, from which you can easily create a logical volume. The volume group is the abstraction of physical disk devices. So if you run out of available disk space, you can just add a new device to the volume group, which makes the added storage space directly available to the logical volumes as well. Using LVM makes disk-space allocation more flexible, and it's easy to add or remove storage.
Finally, LVM works well in both single-host and multiple-host scenarios. With multiple hosts, you just create the logical volumes on the SAN. And if you're using Cluster LVM, you can easily configure the hosts for simultaneous access to the logical volumes.
Working with KVM storage pools
To make KVM storage management even easier, you can create storage pools. By creating a storage pool at the host level, it’s easier to manage access to KVM storage devices. Working with KVM storage pools also makes it possible to offer storage that is prepared beforehand. This tactic is useful in larger environments, where the storage manager is often not the same person as the manager who creates VMs. So it's a good idea to create KVM storage pools even before you add the first VM.
As you dive into KVM virtualization, start by setting up a KVM storage pool at the host level and offer LVM logical volumes in that pool. You won’t regret the added benefit of using snapshots with LVM, and this KVM storage method allows for the greatest flexibility.