Administrators want the VM live migration process to complete as quickly as possible. Windows Server 2008 R2 and Windows Server 2012 did not offer a way to optimize the live migration process, but thankfully this has changed in Windows Server 2012 R2.
You must first understand how live migration works to understand how to optimize it. When an administrator initiates a live migration, Hyper-V establishes a connection to the destination host and creates an empty VM. The hypervisor then allocates hardware to the newly created VM. This hardware allocation matches that of the source VM.
Next, Hyper-V copies the VM's memory across the wire to the newly created VM. The memory is copied as a series of 4 KB blocks. The VM's working set memory is continuously modified throughout the copy process because the VM continues to run. As such, Hyper-V keeps track of which blocks are modified and copies modified blocks on an as-needed basis. This involves several different memory copy passes. Once the memory contents have been completely transferred, control of the underlying storage is transferred to the destination host and the VM is brought online on the destination host. Finally, network traffic that is destined to the VM gets rerouted to the destination host's network adapter.
VM live migration speed
Two primary factors affect live migration speed.
First, the number of changes to the VM's memory will affect live migration performance. The full memory contents must be replicated to the destination host, which means that the more memory modifications that occur during the replication process, the more memory replication operations that will have to occur after the initial copy operation completes. The more changes to the VM's memory, the longer the live migration process will take to complete.
Second, how quickly the Hyper-V hosts can transfer memory contents across the network affects the speed of a live migration. The faster this copy process occurs, the more quickly the live migration process completes. This concept forms the basis of the new live-migration performance options.
You can access the Windows Server 2012 R2 Hyper-V live-migration performance options by opening the Hyper-V Manager, right-clicking on your host server and selecting the Hyper-V Settings option from the resulting shortcut menu. When the Hyper-V Settings dialog box displays, expand the Live Migrations tab and click on Advanced Features, as shown in Figure A.
Windows Server 2008 R2 and Windows Server 2012 Hyper-V used a basic TCP/IP connection to facilitate the live migration process. As you can see in Figure A, TCP/IP migration is still an option, but it is not the default option.
By default, Hyper-V uses compression, a new feature in Windows Server 2012 R2, for live migration. This feature compresses the VM's memory prior to transmitting it to the target host. Compression reduces bandwidth consumption and decreases the amount of time required to livemigrate the VM.
The downside to compression, however, is that the process consumes CPU resources. Fortunately, Hyper-V hosts are rarely bound to CPUs. Physical memory is usually the most limiting resource. Even so, Hyper-V checks the CPU load prior to performing a live migration. If sufficient CPU resources are not available, then Hyper-V automatically reverts to using a standard TCP/IP-based live migration without compression.
Aside from compression, you can also use SMB (Server Message Block) for live migration. This migration technique comes from the idea of hardware offloading. The memory of the VM being livemigrated is transmitted to the destination host over an SMB connection. SMB offers better live migration performance, but is not the default option because it only works if the network adapters on both the sending and receiving hosts have Remote Direct Memory Access (RDMA) support. As of October 2013, commodity network adapters do not offer RDMA.
Microsoft has made strides toward improving live migration performance. If you have RDMA-compatible network adapters on all your Hyper-V hosts, then you will receive the best performance by enabling the SMB option. If not, you should use the compression option -- unless your Hyper-V hosts are CPU bound.
This was first published in October 2013