KVM supports live migration without shared storage

A new Red Hat utility could solve long-standing problems with live migration of virtual machines and even enable the move to public clouds. But users remain cautious.

Red Hat will soon offer a new utility for live migration of virtual machines that doesn't require shared storage. The protocol could drive virtualization into new environments such as public cloud computing, where shared storage is not always available, observers said.

This week Red Hat Inc. confirmed that ,in January, its new utility was added to the open source Kernel-based Virtual Machine (KVM). Further, a Red Hat represenative confirmed that graphical user interface support will be added to the forthcoming Red Hat Enterprise Virtualization (RHEV) 3.0.

While the technology has been billed as a way to solve long-standing issues with live migration, such as costly SAN requirements as well as the difficulty of migrating VMs over long distances, details on how the utility actually works, users say, are unclear.

"It looks promising, said Greg Scott, a CTO at Infra Support Etc. Corp., a firewall maker. "But so many times we read a press release only to get excited about an offering that turns out to be vaporware."

How the utility works
The Reservoir project led by IBM has claimed credit for developing and open-sourcing the code underlying the feature, according to a blog posted Aug. 26 by Dr. Yaron Wolfsthal, a senior manager of system technologies and services for IBM Research.

So many times, we read a press release -- only to get excited about an offering that turns out to be vaporware.

Greg Scott,
CTOInfra Support Etc. Corp.

The utility works by assessing whether the destination hypervisor has access to the source VM's virtual disks, Wolfsthal wrote in an email. If the destination hypervisor doesn't have access to the source VM's virtual disks, the "virtual disks are copied by a special protocol between source and destination hypervisors," Wolfsthal wrote.

"This protocol combines push-and-pull techniques to achieve efficient copying." If the source and destination hypervisor have access to the same virtual disk, only deltas are copied during the migration process.

But Wolfsthal and IBM have not provided further technical detail on what this "special protocol" is, what's meant by "push-and-pull techniques," and how the utility achieves efficient copying.

Users show interest, but want more detail
Jeff Boles, the director of validation services at the Hopkinton, Mass.-based Taneja Group, said the lack of publicly available details on how the utility works is worrisome.

"Sounds great, but given the level of details released, it smells like horse hockey," said Boles. "The trickiest stuff is all in the push-pull and the on-disk state synchronization. You could use a number of mechanisms."

Without more information to go on, Boles said, the utility could boil down to something as simple as a virtualization-aware orchestration layer added to rsync, a familiar open source utility that uses a specialized protocol to detect whether a master system image exists and that copies delta changes over distance between two machines.

Today, rsync typically uses a two-pass process to minimize downtime during migration, but requires at least brief interruption. IBM said that the new utility can perform live migrations without interrupting a running virtual machine.

But even with caveats that they need to know more, users say they can see benefits of performing live migrations without shared storage. "This may form the basis for physical-to-virtual migrations in the libvirt universe: a key piece that needs to be addressed soon," said Scott. "And it could be KVM's answer to VMware's Storage vMotion, and on a bigger scale, if this tool can scale across different management domains, it could make public clouds more palatable.

The cost of shared storage, typically in the form of a SAN, is the value IBM has emphasized with the new utility, but there are ways to use cheap local storage in a virtualization environment while maintaining the ability to do live migration. Examples include virtual storage appliances that make local storage appear shared, such as Hewlett-Packard Co.'s P4000 Virtual Storage Appliance (VSA) for VMware, and VM6 Software's VMex product for Hyper-V.

Instead of a way to save on on-premise hardware, said Todd Mitchell, general manager of cloud computing for service provider The Planet, a utility like Red Hat's is more likely to be used where users don't have control over the storage systems that host virtualization : that is, public clouds. "One inherent problem is that the cloud today only scales as far as what your service provider has racked and stacked," he said. "This could allow the use of resources between several providers."

Gregory Rosenberg, the CTO for Red Hat value-added reseller RICIS Inc., suggested the utility could be also used to keep local copies of data synchronized with an offsite cloud service, in case the Internet connection to the offsite cloud service fails.

VMware to play catchup?
While VMware's live migration feature, vMotion, requires shared storage today, VMware users say they expect their vendor to have something similar to the KVM utility soon. "Live migration without shared storage will probably come to vSphere soon. It's essentially just a mix of vMotion and Storage vMotion," said Eric Siebert, a senior systems administrator at Boston Market Corp.

But while VMware has demonstrated a concept known as "distance VMotion" for the past year or so, performing live migrations over distance without shared storage is a thorny problem to solve. EMC Corp. has taken a stab at this with its specialized VPlex , which uses new mechanisms for metadata sharing to allow active-active stretched clusters of storage.

If the open source utility within KVM works as advertised, it could be an alternative means of solving the same puzzle -- and a potential feather in the cap of VMware competitors when it comes to the innovation wars.

Here, experts are divided on the market impact of such a development, even if it becomes available through standard Linux distributions or within vSphere soon.

"I don't see enterprise clients circling a date on their calendars and waiting for that feature," said Chris Wolf, research vice president for Gartner Inc.

But while observers remain skeptical about the utility, the idea of live migrations without shared storage has become increasingly more relevant. "Two of the most critical areas in the development of the cloud are data access separate from physical presence and data movement," said Taneja Group's Boles. "The need to move workloads is crucial to that."

Beth Pariseau is a senior news writer for SearchServerVirtualization.com. Write to her at [email protected].

Dig Deeper on Open source virtualization