Problem solve Get help with specific problems with your technologies, process and projects.

OpenVZ -- Getting started with Virtuozzo's parent, part two

So how do you run a host operating system on OpenVZ on the hardware node? Find out from contributor Scott Dowdle in part two of this tip.

As you should recall from the previous installment, OpenVZ is made up of three components: 1) kernel, 2) utility programs, and 3) operating system templates. In this tip, we'll delve into running the host operating system (OS) on the hardware node.

A host OS runs on the hardware node. The OpenVZ software requirements say to use Red Hat Enterprise Linux, Fedora Core or CentOS for the host OS. I use CentOS because it is built from the sources of a "prominent North American Enterprise Linux vendor" and it also includes Fedora Core's yum package manager which will allow us to use OpenVZ yum repository. The minimum hardware requirements are pretty basic, so any PC built within the last three years should do.

CentOS Install
I'm not going to cover CentOS installation other than to say that you want to pick the custom installation type, the minimum package set, and disable the firewall and SELinux. Don't forget to yum update your system. Feel free to consult the fine CentOS online documentation if needed at While you could do a heavier install, you really want to save as many resources to your VPSes (virtual private servers) as possible. The root user on the Host OS has full access to the VPSes, so for security reasons, run only the necessary server applications.

Installing the OpenVZ kernel
Since our example uses CentOS, we can use the OpenVZ yum repository. Login as "root" and do the following:

cd /etc/yum.repos.d

yum will now include the OpenVZ repository. To install the OpenVZ kernel, login as root and issue the following commands:

yum install ovzkernel

If on an SMP system:

yum install ovzkernel-smp

yum will install the latest stable OpenVZ kernel. To make the OpenVZ kernel the default, edit the /etc/grub.conf file as needed or be prepared to manually pick the OpenVZ kernel from the grub boot menu.

Before booting the OpenVZ kernel, some modifications need to be made to /etc/sysctl.conf. The recommended template follows:

# On Hardware Node we generally need
# packet forwarding enabled and proxy arp disabled
net.ipv4.ip_forward = 1
net.ipv4.conf.default.proxy_arp = 0
# Enables source route verification
net.ipv4.conf.all.rp_filter = 1
# Enables the magic-sysrq key
kernel.sysrq = 1
# TCP Explict Congestion Notification
# net.ipv4.tcp_ecn = 0
# we do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0

Kernel installation and configuration is done so reboot to the OpenVZ kernel.

Installing the OpenVZ utility programs
Once rebooted the next step is to install the various OpenVZ utility programs. Login as "root" and do the following:

yum install vzctl vzquota

The next step is to start the vz service:

service vz start

The vzctl package installs the /etc/init.d/vz control script and by default it is set to start up in run levels 2 through 5.

Installing the OpenVZ OS template management tools
As "root" do the following:

yum install vzpkg vzyum vzrpm43-python vzrpm44-python

With the OS Template tools installed, next we want to install the template metadata for CentOS 4. As root, do the following:

yum install vztmpl-centos-4

Before attempting to create an OS template, let's examine how an OS template is defined and how to modify the package set. Use the rpm command to get a list of files that were part of the install:

rpm -ql vztmpl-centos-4

Expected command output:


All files are located in /vz/template/centos/4/i386/. The default.list and minimal.list text files contain lists of of packages that make up a default and minimal CentOS VPS installs. You can edit these files to add additional packages and I recommend adding the following:


Building an OS Template Cache
The vzpkgcache command is used to download all of the software packages from which a CentOS OS template cache is created. Issue the following command:

vzpkgcache -f centos-4

yum will produce several screens of output as it goes through all of the dependencies and if you are using a slower machine, this may take a while. Once yum has downloaded all of the packages, it will install them to a temporary location and create one big .tar.gz in the /vz/template/cache directory. Use ls to see the final product:

ls -lh /vz/template/cache

In this case you should see something like:

-rw-r--r--  1 root root 112M Dec 5 15:52 centos-4-i386-default.tar.gz

With an OS Template Cache you can create a VPS. To keep your OS Template current as CentOS updates come out, you'll want to periodically rerun vzpkgcache to have it look for updates.

Pre-build OS Templates - If desired, you can download some pre-built templates rather than building your own. Pre-built templates are available at

Creating your first VPS
In the example commands below I use IP addresses from the private .168. block. You'll want to modify the commands for the desired IP addresses (host and DNS) on your LAN. The vzctl command is used for VPS creation, configuration, and operation. vzctl can take multiple parameters on a single command line but for clarity I do each operation on a separate command line. To create the VPS, issue the following command:

vzctl create 101 --ostemplate centos-4 --config vps.basic

This vzctl command uses the create keyword which will extract the centos-4-i386-default.tar.gz file to /vz/private/101/. It will also create a config file named 101.conf in the /etc/vz/conf/ directory. OpenVZ uses numbers to identify VPSes and in this case, I used 101. The OpenVZ documentation says that numbers 1-100 are reserved and since the IP address of the VPS ends in 101, numbering the VPS 101 isn't a bad idea.

The following vzctl commands use the set keyword. If the VPS happens to be running, it will communicate the config change dynamically to the running VPS. The --save on the end will write the config change to the VPS config file. As you issue the following vzctl set commands, their meaning should be fairly straight forward:

vzctl set 101 --ipadd --save
vzctl set 101 --hostname vps101 --save
vzctl set 101 --nameserver --nameserver --save
vzctl set 101 --userpasswd root:areallycoolpassword

An IP address is assigned, then a hostname is given, the nameservers are defined (put into /vz/root/101/etc/resolv.conf) and a root password is set. Of course you'll want to use a different root password. That is enough information to define a unique VPS although a slew of additional options with defaults are listed in the /etc/vz/conf/101.conf file. Examine 101.conf for additional parameters and their defaults.

The following vzctl set command can be issued to make this VPS start up automatically whenever the Host OS boots:

vzctl set 101 --onboot yes –save

Staring your VPS for the first time - Now that VPS 101 is defined, you can start it using the vzctl command with the start keyword:

vzctl start 101

You should see a message saying your VPS was started successfully. You can use the vzlist command to get a summary of your VPSes. To access your new VPS, either issue the vzctl command with the enter keyword or ssh into your VPS as root:

vzctl enter 101


ssh root@

Once in, you can use your new VPS as if it were a stand alone machine. Have a look around and notice that it just like a stand alone machine... with its own startup scripts, configurations, user accounts, home directories, etc. The VPS root user has almost complete control without access to the Host OS nor any other VPS. The VPS root user can not repartition the hard drive nor change the network card configuration. There are some signs that you are within a VPS. Look at the device names for the hard disk and the network card to see what I mean.

VPS resource management
Each VPS has a special file named /proc/user_beancounters which offers an accounting of the assigned resources and the resources used. /proc/user_beancounters also counts resource access failures. Issue the following command as root within the VPS:

cat /proc/user_beancounters

Below is an example:

uid  resource           held    maxheld    barrier      limit    failcnt
101: kmemsize        1492229    1833592    2752512    2936012          0
     lockedpages           0          0         32         32          0
     privvmpages        8032      15759      98304     107150          0
     shmpages             39         39       8192       8192          0
     dummy                 0          0          0          0          0
     numproc              19         25         65         65          0
     physpages          2164       6388          0 2147483647          0
     vmguarpages           0          0       6144 2147483647          0
     oomguarpages       2164       6388       6144 2147483647          0
     numtcpsock            3          7         80         80          0
     numflock              3          9        100        110          0
     numpty                1          1         16         16          0
     numsiginfo            0          2        256        256          0
     tcpsndbuf             0       6708     319488     524288          0
     tcprcvbuf             0     136396     319488     524288          0
     othersockbuf       2236       9800     132096     336896          0
     dgramrcvbuf           0       8380     132096     132096          0
     numothersock          5          9         80         80          0
     dcachesize            0          0    1048576    1097728          0
     numfile             402        510       2048       2048          0
     dummy                 0          0          0          0          0
     dummy                 0          0          0          0          0
     dummy                 0          0          0          0          0
     numiptent            10         10        128        128          0

The last column, named failcnt, holds incremental values of resource access failures. If a failcnt value is not 0, then your VPS had a resource failure. For example, the numproc resource above has a barrier / limit of 65 which means that the VPS is restricted to no more than 65 concurrent processes and if you tried to start up a 66th process, that process would fail to start and the failcnt value for numproc would be increment by 1. It is very important for the root user of the Host OS to monitor the failcnts and adjust VPS resource values as needed. An easy way for the Host OS root user to monitor the resource usage of all VPSes is with the following command line:

less /vz/root/*/proc/user_beancounters

That will print out all of the user_beancounters for all of the VPSes on the host node. The vzctl command with the set keyword is used to adjust various resources. For example, to adjust the barrier / limit for privvmpages, one could:

vzctl set 101 --privvmpages 

and above should be replaced with actual values. Again, note that if the VPS is running, the resources are adjusted dynamically / immediately and the --save flag saves the setting to the config 101.conf. All you need to know is that if something has a failcnt, you increase the value a little and keep monitoring until you reach a level where there are no more failcnts for that resource. It may take a few days of monitoring to tweak a VPS's resources to avoid resource failures unless one wants to set the barrier and limit values overly high. Note that these values are actually pooled or shared between the host OS and all of the VPSes, unlike most other virtualization products which only offer static resource allocation. I don't have room here to document what all of the resources are, so refer to the OpenVZ User's Guide and Wiki for more details.

One resource you won't find listed in /proc/user_beancounters is the diskspace setting which has a default value of 1GB. If you are going to need more than 1GB of diskspace for VPS 101, you need to set it. The following command will dynamically set the disk quota to 30GB and update the 101.conf file.

vzctl set 101 --diskspace 31457280:31517280 –save

Note that this does not actually reserve physical disk space for VPS 101. It just assures that VPS 101 can use no more than the stated value as disk space is also a pooled or shared resource. This means you can over commit disk space if desired, which isn't necessarily a bad thing.

Cloning a VPS
Let's say you have a VPS setup just the way you want it and you'd like to make another VPS exactly like it. An easy way is to manually do what the vzctl create command would do:

1) Copy the /vz/root/
   to /vz/root/
2) Copy /etc/vz/conf/
    .conf to /etc/vz/conf/

Of course you'd replace and with real values. Before copying, stop the source VPS long enough to do the copy (vzctl stop ). You can use the cp command with the -a flag, but I prefer to use rsync for copying more complex directory structures. After the copy operation is done, you can start the source VPS back up. Before starting the new VPS, be sure to modify its configuration by using vzctl set or by manually editing the .conf file as you should change the IP address and hostname to avoid a conflict.

Backing up a VPS
A VPS is represented on your filesystem by a configuration file in /etc/vz/conf/ and by its files under the /vz/root/ / directory. To make a backup of a VPS, it is very similar to cloning except the destination directory would be wherever you store backups.

Managing a VPS
Remember that a VPS is functionally a separate computer. It may even be a completely different Linux distribution than the host OS and as a result, one must take care to apply relevant security updates as they are released. Having root access to the host OS means you can enter all VPSes using a vzctl enter command. You can also execute commands on a VPS from the host OS by using the vzctl command with the exec keyword. The vzctl command has a ton of options so read the vzctl man page for a complete list.

Guides galore
I've covered OpenVZ installation, VPS creation, and basic VPS operation but I have only scratched the surface. Play around and have fun. OpenVZ has some top-notch documentation, and I recommend printing out the User's Guide. Note that this document was written prior to the addition of live migration features and the creation of the OpenVZ yum repository. There is also a wiki-based installation guide.

If you can't find needed information in this tip or these documents, use the OpenVZ Forums. The OpenVZ Development Team is very active in the OpenVZ Forums, and they also have a Bug Tracker as well. Enjoy your OpenVZ journey.

Dig Deeper on Oracle VM and other virtualization technologies

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.