Analyzing performance in virtualized environments

Want a better way to monitor the performance of your virtual machines? In this tip, Justin Korelc and Ed Tittel take you step-by-step through the most common monitoring tools, complete with screen shots.

Measuring and analyzing performance in virtual environments is trickier than you might think. In this article, we'll discuss the inherent challenges of and suggest tools for monitoring virtual machine performance.

In large part, the biggest problem in monitoring virtual machine performance stems from the way that virtual machines (VMs) perceive time. Each VM recognizes the passage of time only when it is instantiated and running and not when it is in a suspected state.

Performance measurement tools are sometimes not able to recognize the time that the VM loses when it's in a suspended state. The built-in Windows performance monitor (aka perfmon.exe), for example, measures only what happens when its container VM is active.

Virtual environments need a tool that runs at the hypervisor level, outside the individual operating system contexts (and their respective performance monitoring tools) within any VM instance. Applications that report performance within specific operating systems may need to be reworked for specific hypervisors in order to maintain a sense of time and activity outside specific VM.

When working with newly reworked applications, it's important to set thresholds for acceptable application response times. This allows you to find out how many VMs can be active on a specific server platform at any given time while still maintaining adequate performance. This also lets you monitor the performance impact of the VM environment's processing requirements (the VM overhead).

How one virtual machine performance monitoring tool works Within Windows Virtual Server, managing the VMs requires an interesting combination of tools. Virtual Server sets up a management Web site, as part of the installation process, that is accessed through the loopback IP address ( on the host running Virtual Server. As Figure 1 illustrates, the management Web site shows the status for all the virtual machines on a specific host, including the base OS.

(Figure 1: The Master Status screen for the current host shows which VMs are active and provides a list of status messages. This also shows a Windows XP VM in the process of being removed.)

On the management Web page, the Virtual Server Manager lets you view and control CPU resource allocations on a per-VM basis, as shown in Figure 2. You can use this to allocate CPU resources by relative weight (column 1, which assigns how much of the total CPU the VM can use as compared to other active VMs) or you can set minimum CPU levels (Reserved Capacity, column 2) and maximum CPU availability (Maximum Capacity, column 3), while also viewing actual consumption levels in columns 4 and 5, along with a utilization graph at the far right. This provides a rough-and-ready mechanism for allocating CPU resources among VMs running under Virtual Server.

(Figure 2: The CPU Resource Allocation screen lets you control how CPU resources are apportioned across active (and inactive) VMs.)

For each VM, you can also drill into status information on a per VM basis. Figure 3 shows the detailed status information for the VM listed in the two preceding screenshots as "Windows Server 2003 (Base)". This display provides information about the VM status, running time, average CPU utilization in the past 60 seconds, heartbeat status (a measure of how many of the allocated cycles that it actually received), disk activity and configuration data.

(Figure 3: The Status screen lets you observe status information on a per VM basis.)

When it comes to setting alerts and thresholds on VMs, however, this is still handled through the Performance tool (aka Performance Monitor, perfmon.exe). When Virtual Sever is installed, it also registers objects named Virtual Machine and Virtual Processor to the collection of objects under perfmon.exe's control. You work through the standard interface to set alerts, capture counters and write to trace logs.

Suggested performance tools

The details for proper instrumentation and reporting vary from one VM environment to another. Here are some of the tools available:

  • VWware Workstation has performance counters that let users collect performance data from VMs running on a Windows machine.
  • In ESX Server, you can use vcpu at the command line to get information about VM activities.
  • Microsoft Virtual Server Management Pack provides tools and information on monitoring VMs inside that platform.
  • The Xensource.com sites lists several performance monitoring tools, including Ganglia.
  • The SunPerf.com Web site documents numerous freeware, built-in and commercial performance monitoring tools for Solaris, nearly all of which now include VM capabilities.

Justin Korelc is a longtime Linux hacker and system administrator who concentrates on hardware and software security, virtualization and high-performance Linux systems. Ed Tittel is a full-time freelance writer based in Austin, Tex., who specializes in markup languages, information security, networking and IT certification. Justin and Ed have contributed to books on Home Theater PCs and the Linux-based MythTV environment, and they write regularly about Linux for various TomsHardware sites.

Dig Deeper on Virtual machine performance management