Enabling huge pages is a small step that can make a big difference

Setting up a KVM virtual machine can be quick and easy, but taking the extra step to enable huge pages can enhance the VM's performance.

It's easy to set up a KVM virtual machine: Launch your favorite KVM installer and walk through the setup procedure. It's an easy way to create a virtual machine, but the simplicity doesn't give you the best possible performance.

On a KVM host, many additional features are available, including huge page support. By working with huge pages, memory allocation becomes a lot more efficient, resulting in better overall performance of your virtual machines (VMs), and setting them up isn't too difficult.

Huge memory pages allow the Linux kernel to address memory in a more efficient way. By default, Linux deals with memory in 4,096-byte memory pages. That means that a VM with 1 GB of RAM will have 262,144 memory pages, which creates significant administrative overhead.

The Linux kernel huge pages feature allows you to define large memory pages with a default size of 2 MB. On a VM with 1 GB of RAM, using 2-MB huge pages reduces the amount of memory pages that the Linux kernel has to manage to just 512, which significantly lessens administrative overhead.

Before beginning to allocate huge pages, you should know that memory reserved for huge pages cannot be used by anything else. Therefore, you need to calculate how much memory you want to reserve for the host operating system and how much memory you want to allocate to the VMs.

Setting up the hugetlbfs file system

The Linux kernel can work with two different types of huge pages: transparent huge pages and the hugetlbfs file system. For setting up a host with KVM VMs, it's more efficient to take the hugetlbfs file system approach. This involves two steps:

  • You need to create a mount point. Use the command mkdir /huge
  • The hugetlbfs file system needs to be mounted from /etc/fstab. To do this, include a command like the following in /etc/fstab:

hugetlbfs  /huge  hugetlbfs  defaults 0 0

Before making these settings permanent, you should check your distribution settings. Modern distributions, like my OpenSUSE 13.1 test system, use hugetlbfs by default and mount it on /dev/hugepages.

Next, you need to make sure that the amount of huge pages you need for your VMs is actually reserved. This goes through the /proc/sys file system. In order for your kernel to know it should reserve 512 huge pages on next reboot, include the following command in /etc/sysctl.conf:

vm.nr_hugepages = 512

To make sure that comes up as expected, reboot your computer. After restart, use the following commands to verify the huge pages system has been properly initialized:

mount | grep huge
grep Huge /proc/meminfo

If the last command gives the following result, huge pages support has been set up properly:

msh:~ # grep Huge /proc/meminfo
AnonHugePages:       30720 kB
HugePages_Total:     512
HugePages_Free: 512
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize:       2048 kB

At this point your system is ready and you can start your VMs with huge page support. Use for instance the qemu-x86_64 command with the --mem-path /huge option to tell the VM that it has to be started with huge page support. Do you notice an improved performance? Then you can start setting up huge page support as a default in the KVM configuration files.

Using huge pages makes memory management on the host operating system more efficient. Setting up huge pages isn't difficult, but make sure to test it properly before making it a permanent solution.

Dig Deeper on Virtual machine performance management