Virtualization Strategies for the CIO
As you should recall from the previous installment, OpenVZ is made up of three components: 1) kernel, 2) utility programs, and 3) operating system templates. In this tip, we'll delve into running
Requires Free Membership to View
the host operating system (OS) on the hardware node.
A host OS runs on the hardware node. The OpenVZ software requirements say to use Red Hat Enterprise Linux, Fedora Core or CentOS for the host OS. I use CentOS because it is built from the sources of a "prominent North American Enterprise Linux vendor" and it also includes Fedora Core's yum package manager which will allow us to use OpenVZ yum repository. The minimum hardware requirements are pretty basic, so any PC built within the last three years should do.
CentOS Install
I'm not going to cover CentOS installation other than to say that you want to pick the custom installation type, the minimum package set, and disable the firewall and SELinux. Don't forget to yum update your system. Feel free to consult the fine CentOS online documentation if needed at CentOS.org. While you could do a heavier install, you really want to save as many resources to your VPSes (virtual private servers) as possible. The root user on the Host OS has full access to the VPSes, so for security reasons, run only the necessary server applications.
Installing the OpenVZ kernel
Since our example uses CentOS, we can use the OpenVZ yum repository. Login as "root" and do the following:
cd /etc/yum.repos.d wget http://download.openvz.org/openvz.repo
yum will now include the OpenVZ repository. To install the OpenVZ kernel, login as root and issue the following commands:
yum install ovzkernel
If on an SMP system:
yum install ovzkernel-smp
yum will install the latest stable OpenVZ kernel. To make the OpenVZ kernel the default, edit the /etc/grub.conf file as needed or be prepared to manually pick the OpenVZ kernel from the grub boot menu.
Before booting the OpenVZ kernel, some modifications need to be made to /etc/sysctl.conf. The recommended template follows:
# On Hardware Node we generally need # packet forwarding enabled and proxy arp disabled net.ipv4.ip_forward = 1 net.ipv4.conf.default.proxy_arp = 0 # Enables source route verification net.ipv4.conf.all.rp_filter = 1 # Enables the magic-sysrq key kernel.sysrq = 1 # TCP Explict Congestion Notification # net.ipv4.tcp_ecn = 0 # we do not want all our interfaces to send redirects net.ipv4.conf.default.send_redirects = 1 net.ipv4.conf.all.send_redirects = 0
Kernel installation and configuration is done so reboot to the OpenVZ kernel.
Installing the OpenVZ utility programs
Once rebooted the next step is to install the various OpenVZ utility programs. Login as "root" and do the following:
yum install vzctl vzquota
The next step is to start the vz service:
service vz start
The vzctl package installs the /etc/init.d/vz control script and by default it is set to start up in run levels 2 through 5.
Installing the OpenVZ OS template management tools
As "root" do the following:
yum install vzpkg vzyum vzrpm43-python vzrpm44-python
With the OS Template tools installed, next we want to install the template metadata for CentOS 4. As root, do the following:
yum install vztmpl-centos-4
Before attempting to create an OS template, let's examine how an OS template is defined and how to modify the package set. Use the rpm command to get a list of files that were part of the install:
rpm -ql vztmpl-centos-4
Expected command output:
/vz/template/centos/4/i386/config /vz/template/centos/4/i386/config/.rpmmacros /vz/template/centos/4/i386/config/default.list /vz/template/centos/4/i386/config/gpgkeys /vz/template/centos/4/i386/config/gpgkeys/RPM-GPG-KEY /vz/template/centos/4/i386/config/gpgkeys/RPM-GPG-KEY-centos4 /vz/template/centos/4/i386/config/install-post /vz/template/centos/4/i386/config/install-pre /vz/template/centos/4/i386/config/minimal.list /vz/template/centos/4/i386/config/rpm /vz/template/centos/4/i386/config/update-post /vz/template/centos/4/i386/config/yum.conf /vz/template/centos/4/i386/vz-addons /vz/template/centos/4/i386/vz-addons/MAKEDEV-3.3.13-1.3.swsoft.i386.rpm /vz/template/centos/4/i386/vz-addons/dummy-centos-4-1.0-3.swsoft.i386.rpm /vz/template/centos/4/i386/vz-addons/repodata /vz/template/centos/4/i386/vz-addons/repodata/filelists.xml.gz /vz/template/centos/4/i386/vz-addons/repodata/other.xml.gz /vz/template/centos/4/i386/vz-addons/repodata/primary.xml.gz /vz/template/centos/4/i386/vz-addons/repodata/repomd.xml /vz/template/centos/4/i386/vz-addons/vzdev-3.3.13-1.3.swsoft.i386.rpm /vz/template/centos/4/i386/yum-cache
All files are located in /vz/template/centos/4/i386/. The default.list and minimal.list text files contain lists of of packages that make up a default and minimal CentOS VPS installs. You can edit these files to add additional packages and I recommend adding the following:
yum mc links system-config-date nano
Building an OS Template Cache
The vzpkgcache command is used to download all of the software packages from which a CentOS OS template cache is created. Issue the following command:
vzpkgcache -f centos-4
yum will produce several screens of output as it goes through all of the dependencies and if you are using a slower machine, this may take a while. Once yum has downloaded all of the packages, it will install them to a temporary location and create one big .tar.gz in the /vz/template/cache directory. Use ls to see the final product:
ls -lh /vz/template/cache
In this case you should see something like:
-rw-r--r-- 1 root root 112M Dec 5 15:52 centos-4-i386-default.tar.gz
With an OS Template Cache you can create a VPS. To keep your OS Template current as CentOS updates come out, you'll want to periodically rerun vzpkgcache to have it look for updates.
Pre-build OS Templates - If desired, you can download some pre-built templates rather than building your own. Pre-built templates are available at OpenVZ.org.
Creating your first VPS
In the example commands below I use IP addresses from the private
.168. block. You'll want to modify the commands for the desired IP addresses (host and DNS) on your LAN. The vzctl command is used for VPS creation, configuration, and operation. vzctl can take multiple parameters on a single command line but for clarity I do each operation on a separate command line. To create the VPS, issue the following command:
vzctl create 101 --ostemplate centos-4 --config vps.basic
This vzctl command uses the create keyword which will extract the centos-4-i386-default.tar.gz file to /vz/private/101/. It will also create a config file named 101.conf in the /etc/vz/conf/ directory. OpenVZ uses numbers to identify VPSes and in this case, I used 101. The OpenVZ documentation says that numbers 1-100 are reserved and since the IP address of the VPS ends in 101, numbering the VPS 101 isn't a bad idea.
The following vzctl commands use the set keyword. If the VPS happens to be running, it will communicate the config change dynamically to the running VPS. The --save on the end will write the config change to the VPS config file. As you issue the following vzctl set commands, their meaning should be fairly straight forward:
vzctl set 101 --ipadd 192.168.0.101 --save vzctl set 101 --hostname vps101 --save vzctl set 101 --nameserver 192.168.0.1 --nameserver 192.168.0.2 --save vzctl set 101 --userpasswd root:areallycoolpassword
An IP address is assigned, then a hostname is given, the nameservers are defined (put into /vz/root/101/etc/resolv.conf) and a root password is set. Of course you'll want to use a different root password. That is enough information to define a unique VPS although a slew of additional options with defaults are listed in the /etc/vz/conf/101.conf file. Examine 101.conf for additional parameters and their defaults.
The following vzctl set command can be issued to make this VPS start up automatically whenever the Host OS boots:
vzctl set 101 --onboot yes –save
Staring your VPS for the first time - Now that VPS 101 is defined, you can start it using the vzctl command with the start keyword:
vzctl start 101
You should see a message saying your VPS was started successfully. You can use the vzlist command to get a summary of your VPSes. To access your new VPS, either issue the vzctl command with the enter keyword or ssh into your VPS as root:
vzctl enter 101
or
ssh root@192.168.0.101
Once in, you can use your new VPS as if it were a stand alone machine. Have a look around and notice that it just like a stand alone machine... with its own startup scripts, configurations, user accounts, home directories, etc. The VPS root user has almost complete control without access to the Host OS nor any other VPS. The VPS root user can not repartition the hard drive nor change the network card configuration. There are some signs that you are within a VPS. Look at the device names for the hard disk and the network card to see what I mean.
VPS resource management
Each VPS has a special file named /proc/user_beancounters which offers an accounting of the assigned resources and the resources used. /proc/user_beancounters also counts resource access failures. Issue the following command as root within the VPS:
cat /proc/user_beancounters
Below is an example:
uid resource held maxheld barrier limit failcnt
101: kmemsize 1492229 1833592 2752512 2936012 0
lockedpages 0 0 32 32 0
privvmpages 8032 15759 98304 107150 0
shmpages 39 39 8192 8192 0
dummy 0 0 0 0 0
numproc 19 25 65 65 0
physpages 2164 6388 0 2147483647 0
vmguarpages 0 0 6144 2147483647 0
oomguarpages 2164 6388 6144 2147483647 0
numtcpsock 3 7 80 80 0
numflock 3 9 100 110 0
numpty 1 1 16 16 0
numsiginfo 0 2 256 256 0
tcpsndbuf 0 6708 319488 524288 0
tcprcvbuf 0 136396 319488 524288 0
othersockbuf 2236 9800 132096 336896 0
dgramrcvbuf 0 8380 132096 132096 0
numothersock 5 9 80 80 0
dcachesize 0 0 1048576 1097728 0
numfile 402 510 2048 2048 0
dummy 0 0 0 0 0
dummy 0 0 0 0 0
dummy 0 0 0 0 0
numiptent 10 10 128 128 0
The last column, named failcnt, holds incremental values of resource access failures. If a failcnt value is not 0, then your VPS had a resource failure. For example, the numproc resource above has a barrier / limit of 65 which means that the VPS is restricted to no more than 65 concurrent processes and if you tried to start up a 66th process, that process would fail to start and the failcnt value for numproc would be increment by 1. It is very important for the root user of the Host OS to monitor the failcnts and adjust VPS resource values as needed. An easy way for the Host OS root user to monitor the resource usage of all VPSes is with the following command line:
less /vz/root/*/proc/user_beancounters
That will print out all of the user_beancounters for all of the VPSes on the host node. The vzctl command with the set keyword is used to adjust various resources. For example, to adjust the barrier / limit for privvmpages, one could:
vzctl set 101 --privvmpages: --save
One resource you won't find listed in /proc/user_beancounters is the diskspace setting which has a default value of 1GB. If you are going to need more than 1GB of diskspace for VPS 101, you need to set it. The following command will dynamically set the disk quota to 30GB and update the 101.conf file.
Note that this does not actually reserve physical disk space for VPS 101. It just assures that VPS 101 can use no more than the stated value as disk space is also a pooled or shared resource. This means you can over commit disk space if desired, which isn't necessarily a bad thing.
Cloning a VPS Of course you'd replace Backing up a VPS Managing a VPS Guides galore If you can't find needed information in this tip or these documents, use the OpenVZ Forums. The OpenVZ Development Team is very active in the OpenVZ Forums, and they also have a Bug Tracker as well. Enjoy your OpenVZ journey. This was first published in January 2007vzctl set 101 --diskspace 31457280:31517280 –save
Let's say you have a VPS setup just the way you want it and you'd like to make another VPS exactly like it. An easy way is to manually do what the vzctl create command would do:
1) Copy the /vz/root/
A VPS is represented on your filesystem by a configuration file in /etc/vz/conf/ and by its files under the /vz/root/
Remember that a VPS is functionally a separate computer. It may even be a completely different Linux distribution than the host OS and as a result, one must take care to apply relevant security updates as they are released. Having root access to the host OS means you can enter all VPSes using a vzctl enter
I've covered OpenVZ installation, VPS creation, and basic VPS operation but I have only scratched the surface. Play around and have fun. OpenVZ has some top-notch documentation, and I recommend printing out the User's Guide. Note that this document was written prior to the addition of live migration features and the creation of the OpenVZ yum repository. There is also a wiki-based installation guide.
Join the conversationComment
Share
Comments
Results
Contribute to the conversation