A disaster recovery strategy (DR) is fairly simple if you use enterprise-level virtualization platforms such as VMware vSphere or Citrix Systems XenServer. But if you run an open source Xen hypervisor, things get more difficult.
Enterprise-class platforms include complete backup and disaster recovery solutions in their default management tools, but with open source Xen, you have to collect the tools yourself. Open source solutions lack automated backup tools, so admins need to manually back up the important components.
To get your virtual infrastructure up and running quickly after a disaster, a disaster recovery strategy with open source Xen includes two important best practices: backing up a virtual machine (VM) configuration file and the disk storage back end. The Linux command line can help perform and troubleshoot these processes.
Backing up the disk storage back end
A good place to start with a disaster recovery strategy is backing up the disk storage back end, which is usually found on the storage area network (SAN).
The storage back end comes in two different forms: a disk image file or a raw device. You don't need to back up the entire VM with these devices, just the complete storage back end. But you do need to make sure that the image file is stable. That means freezing its state using snapshot technology, then making a backup of the snapshot.
To create a snapshot, use your SAN's snapshot feature. If you use a logical volume as the storage back end, you can use open source snapshot technology, such as Logical Volume Manager. As long as the disk back end is undamaged, you can use it to boot the failed VM after a disaster.
Backing up the configuration file
Another disaster recovery best practice for open source Xen is to back up the VM configuration file. Improve your disaster recovery strategy by simply making a copy of this file. Store both copies safely so you can restore them in case of a disaster.
When disaster strikes, you need to restore access to affected VMs as fast as possible. If the disk file and its configuration are not corrupt, just copy them to another location, such as a new host or disaster recovery site. If the configuration file refers to the location of the storage back end, it won't be difficult to restore access.
Disaster recovery obstacles
There are a few problems, however, that can limit the success of your disaster recovery strategy with open source Xen. If the configuration file gets lost, for example, you'll have to recreate the file.
The virt-manager tool allows you to import an existing VM and recreate the configuration file. Importing an existing VM is similar to creating a new VM. The difference is that you won't start an installation, you'll just start the VM itself. Virt-manager also lets you specify the hardware settings, including the virtual disk that you want to use.
A more difficult scenario arises when the disk file is damaged. In this case, the disaster recovery strategy becomes more complex, but it's possible to recover the file from the Linux command line. Mount the disk file and copy all relevant contents to a new, temporary location. After you copy the valuable data, you'll have to recreate the VM from scratch.
Recovering the configuration file
First, enter losetup-a to make a connection between the loop device and the image file that you want to access. (The -a stands for the arguments used in the command.) You need this command because, with Linux, you can mount devices, but not files. The command provides an overview of all loop devices that are currently allocated.
In most cases, you won't see loop devices yet, so use /dev/loop0 to indicate which loop device to connect the image file to. Assuming the name of the image file is /var/lib/xen/images/vm1/disk0, for example, use the following command to create a loop file for mounting the image file:
losetup /dev/loop0 /var/lib/xen/images/vm1/disk0
If you already use a few loop devices, ensure that you assign the new device a different name, such as /dev/loop1. (Like other Linux devices, the names are numbered sequentially.)
Next, enter losetup-a again, and you'll see that a loop device has been created and is connected with the image file. Then use the following command, including your loop device name, to analyze which partitions exist in the image file:
fdisk -l /dev/loop0
This command provides a list of all the partitions in the file and their current sizes. Based on this information, you can probably guess where the root file system is located. Before you can access this file system, though, make sure the related device files are created. If you have the multipath-tools package installed, the following command will do that for you:
kpartx -a /dev/loop0
The image file uses the loop0 device, so the new device files will be named /dev/mapper/loop0p1, /dev/mapper/loop0p2 and so on. Then use these files to mount the virtual operating system's file system. No matter which OS you use, you can just mount the virtual partitions from the Linux host.
You now have complete access to its files and can ensure that all important data is copied to a safe location. Finally, clean up the temporary device files using the following commands:
kpartx -d /dev/loop0
losetup -d /dev/loop0
With open source Xen, disaster recovery best practices are different from those with enterprise-level platforms. Data centers usually implement Xen on a smaller scale, so the limitations of a disaster recovery strategy with open source Xen are not a problem in many cases. As long as you back up the VM configuration file and the disk storage back-end, your disaster recovery strategy will help you survive any disaster that comes your way.
About the expert
Sander van Vugt is an independent trainer and consultant based in the Netherlands. Van Vugt is an expert in Linux high availability, virtualization and performance and has completed several projects that implement all three. He is also the writer of various Linux-related books, such as Beginning the Linux Command Line, Beginning Ubuntu Server Administration and Pro Ubuntu Server Administration.