How Xen works and stacks up against other virtualization technologies

While VMware has controlled the i386-based virtualization market for years, open source Xen is rapidly gaining popularity. This is part one of a series on Zen virtualization, which explores how Zen works and how it compares with other virtualization technologies.

Today's virtualization market is filled with different virtualization solutions. While VMware has controlled the i386-based virtualization market for years, Xen, the open source solution, is rapidly gaining popularity. This series will explain what Xen is, how it compares to other kinds of virtualization and how it works.

To understand Xen's virtualization approach, you first need to understand what virtualization is about. In the early days of computing, virtualization didn't really exist. Instead, emulation was used. In emulation, the behavior of a computer (hardware and software) is copied to a software program. The emulation layer talks to an operating system which talks to the computer hardware. The operating system that you want to install in an emulation layer doesn't see that it is used in an emulated environment, and you can install it as you usually would. Two popular open source emulators are QEMU and Bochs.

One of the most important properties of emulation is that all of the hardware is emulated including the CPU. Advantages include being able to run an operating system that was developed for another architecture on your own architecture. But there is also a disadvantage: Virtualizing a complete CPU comes with a heavy performance price.

In the next generation, virtualization was taken to a higher level. Between the emulation layer (responsible for interpreting instructions from the virtualized machines) and the hardware, no host operating system was required to run a virtual machines on the hardware. Instead, the virtual machine monitor, also known as the hypervisor, was introduced to run directly on the hardware. Because of this new architecture, virtualization became much more efficient. VMware, for example, was very succesful with this approach as implemented in VMware ESX.

In one approach with hypervisor-based virtualization, all instructions generated by the virtualized machine need to be translated to the appropriate format for the CPU, which involves a lot of work for the hypervisor. Another approach, used by VMware ESX Server, uses direct execution for most guest CPU instructions, running instructions directly on the host physical CPU with little performance overhead.

In the approach used by Xen, there is no translation. This is accomplished in one of two ways. Option number one is to use a CPU that understands the unmodified instructions that are generated by the virtualized operating system and interprets them (full virtualization). Option number two is to modify the operating system so that it generates instructions that are optimized for use in a virtualized environment (paravirtualization).

Full virtualization versus paravirtualization
Full virtualization is one way virtualizing a machine. Using this method, the virtual machine talks to a component called the virtual machine monitor (VMM), and the VMM communicates with the hardware platform. To use full virtualization in a Xen environment, you need a CPU that understands unmodified instructions that are generated by the virtualized operating system. Without this special CPU feature, it's not possible to use full virtualization in Xen. This is because in the Xen approach not every instruction that is generated by the virtualized operating system is translated to a format that every CPU understands, because this is very resource intensive. Instead, the virtualization feature that is implemented in modern CPU's helps the virtualized operating system in a way that it can send out unmodified instructions. The main advantage of full virtualization is that an unmodified operating system is installed. This means that virtually every operating system that runs on the same architecture can be virtualized.

The most efficient approach in virtualization is paravirtualization. In paravirtualization, the guest operating system uses a specialized API to talk to the VMM which is responsible for handling the virtualization requests and putting them to the hardware. Because of this special API, the VMM doesn't need to do a resource-intensive translation of instructions before the instructions are passed to the hardware. Also, when using the paravirtualization API, the virtualized operating system is capable of generating much more efficients instructions. A disadvantage however is that you do need a modified operating system that includes this specific API, and for certain operating systems (mainly Windows), this is an important disadvantage because that kind of API is not available.

For optimal performance paravirtualization is currently the way to go because instructions that are generated by the virtualized operating system don't need to be translated. Unfortunately, in some operating systems it is not possible to use complete paravirtualization, as it requires a specialized version of the operating system. To ensure good performance in such environments, paravirtualization can be applied for individual devices. This means that, for example, the instructions that are generated by the CPU are handled by using hardware virtualization, but the instructions that are generated by specific devices such as network boards or graphical interface cards can be modified before they leave the virtualized machine by using paravirtualized drivers. Some vendors offer paravirtualized driver packages for specific operating systems. These are frequently available as a separate purchase. It often pays to use these specialized driver sets, as the performance of devices like the network board and the hard disk can increase considerably.

The Xen approach to virtualization
In a Xen environment there are two major components. First, there is the virtual machine monitor or VMM, also known as the hypervisor. This is the layer that sits between the hardware and the virtual machines, and it is the first layer that must be loaded on to the hardware, before anything else is started. After loading the hypervisor, the virtual machines - called "domains" in the Xen world - can be deployed. Of these, one virtual machine plays an important role. This is the domain0 machine, a privileged domain which typically is the operating system that is installed before any other virtual machine.

The domain0 machine has some specific responsibilities. Since the hypervisor doesn't include any drivers to talk to the hardware, or an interface to talk to the administrator, these are offered by the domain0. From this domain, the administrator uses the Xen tools that allow him to create other virtual machines ("domainU's" in Xen terminology). These domainU's are also known as the unprivileged domains. This is because in the i386-based CPU architecture, they can never get the highest priorities. Only the Domain0 can do that.

Also in the domain0, the "xend" process is loaded. This process manages all other virtual machines and provides access to their consoles. When creating virtual machines, the administrator uses configuration utilities that talk to the domain0 directly. (In the second part of this series, I will discuss how to do this.)

Xen and other open source projects
When working with Xen, the differences between open source projects and others sometimes causes confusion. The roots of Xen come from the Computer Laboratory of the University of Cambridge in the UK, which is where the open source Xen project was founded. (This project included the virtual machine monitor, which is just the core component of the Xen environment. Apart from scientists from the University of Cambridge, important parties from the IT world participate in the Xen open source project as well. These include IBM, AMD, HP, Red Hat and Novell.)

Since the Xen approach was a huge step forward in virtualization, the founders of the open source project started their own company XenSource, which was recently purchased by Citrix. The purpose of this company was to deliver a complete virtualization solution, based on the Xen hypervisor, to compete with other products such as VMware ESX. Other parties also integrated the Xen hypervisor in their own products. Linux vendors Red Hat and Novell, for example, both have their own version of Xen included in their operating system. Since most components of a Xen solution are open source, there solutions are rather similar. In the next articles in this series, I'll focus on the open source parts of the Xen virtualization solution.

About the author: Sander van Vugt is an author and independent technical trainer, specializing in Linux since 1994. Vugt is also a technical consultant for high-availability (HA) clustering and performance optimization, as well as an expert on SLED 10 administration. 

Dig Deeper on Citrix XenServer