Problem solve Get help with specific problems with your technologies, process and projects.

How to install and configure Windows Server 2003 for VMware

VMware Server: Some assembly required. Never fear, expert Andrew Kutz is here -- read this eight-part tip and be the IT pro in the know on getting VMware up and running.

Installing Microsoft Windows Server 2003 is simple enough, but getting it set up right with VMware Server can pose some special challenges. We'll discuss the installation and configuration basics in this part of our series on getting started with VMware Server on Windows.

Part one of this series offered basic guidelines and a brief look at how VMware Server works. Part two reviewed the important components and services within VMware Server and how to prepare the host severs. Let's look at installing Windows next.

Installing Windows and its components

Installing Microsoft Windows Server 2003 is straightforward enough, and most IT departments already have their own standard methods for doing so. The one exception may be disk partitioning. Create separate partitions for the OS and the data (create these on different RAID containers if that option is available). This configuration is conducive to speed and security.

Once the installation is completed and the OS is booted, please go ahead and format the DATA partition. The remainder of this guide will assume that the SYSTEM drive is the c: drive and the DATA drive is the e: drive.

Now that both the SYSTEM and DATA drives are available, it is time to configure the system's page file. Set the SYSTEM drive's page file to a static value of 768 and set the DATA drive's page file to a static value of twice the amount of RAM in the machine or the largest value allowed. Apply the changes and reboot.

Configure the system's network settings. If the machine has multiple NICs, make sure to assign valid network settings to all of them. Hopefully one of the NICs could be placed on a private network; this will increase the security of the box.

IIS, SMTP and Network Tools

After the reboot, there are still some additional Windows components needed -- IIS, SMTP, and Network Tools. IIS is needed for the VMware Server MUI. A mail server is always good to have.

Network Tools is not a commonly installed component, but it contains the very handy Net Monitor -- a tool that is quite useful to have when debugging at the network protocol level. (Because several VMs will be sharing a common network interface, it may be necessary to closely examine that interface's traffic at some point.)

Configuration tips 

You'll need to tweak a few settings for optimal security and performance.


Generally the first step to securing Windows is patching the server. Why not make this the first step after installation, you ask? Well, not all the patches would take since the additional components were not yet installed. Since the server is off the network, it does not hurt to go ahead and install those additional components prior to installing the patches.

After Windows and its additional components have been installed, install the latest service packs and patches. Because the server is currently not on the network, this step will require downloading all the latest service packs and patches from Microsoft Windows Update ( from the Windows Update Catalog) on another computer that is known to be secure and free of viruses, burning them to a CD or putting them on a flash drive and installing them from the portable media.

Unless the flash drive has a physical read-only switch that can be set after the files have been copied to it, a CD is the safest transport medium.


If the server has two or more network interfaces, it is time to declare one of them the dedicated management interface.

Open the Network Connections folder. If a NIC was placed on a private network, select this NIC; otherwise, choose any NIC. Rename this connection "Private." Rename the other connection "Public"; for more than one connection, append 01, 02, 03 and so forth to the end of "Public."

Remote Desktop

The Remote Desktop service should listen for incoming connections on the dedicated management interface. Click on the "Start" button and then the "Run" entry. Type "tscc.msc /s" and strike return. On the left, click on the folder labeled "Connections." On the right, right-click on the connection labeled "RDP-Tcp" and click "Properties."

This should have brought up a new window. Click on the tab labeled "Network Adapter." There should be a drop-down menu labeled "Network adapter." The option selected should read "All network adapters configured with this protocol." This means that all of the NICs on the machine are listening for incoming RDP client connections on port 3389 (the default RDP port).

This is not the desired behavior. Restrict RDP to listen to incoming connections on a dedicated NIC.

Select the NIC that was declared the private management interface.

If this server is registered in DNS, the DNS name of the server will not respond to RDP connections unless the DNS entry of the server points to the IP address of the NIC that has been designated the management interface.

You need to register the DNS entry with the IP address assigned to the management interface, create a second DNS entry with the IP address assigned to the management interface or have the server administrators create a RDP shortcut on their clients that points directly to the IP address assigned to the management interface. The last choice is the most secure because an attacker querying a DNS server for the names of servers to attack will not be returned an entry that expose the RDP port of this server.

Disk caching

Enabling disk caching on the e: drive may improve the performance of VMware Server because the virtual disk images will be stored there (p. 153 of VMware Server Admin Manual). This option should be enabled already, but to check it click the "Start" button, click the "Run" entry, and then type "compmgmt.msc" and strike return.

On the left-hand side click on "Storage" and then "Disk Management." Right-click on the disk that contains the e: drive (probably disk 1) and click on "Properties."

A new window will appear labeled "Local Disk (E:) Properties". Click on the tab labeled "Hardware." Select the first disk drive with the type "Disk drives". Click the button labeled "Properties." A new window will appear. Click the tab labeled "Policies."

Verify that the option labeled "Optimize for performance" is selected and click the "OK" button. Click "OK" again. It is now okay to go ahead and close the "Computer Management" application.

Disk defragmentation

Fragmentation can lead to a severe degradation in performance with virtual disks, because of the size of the files the virtual disks are stored in. VMware recommends running a defragmentation utility in order to reduce fragmentation. Microsoft Windows Server 2003 ships with a defragmentation tool, but there is a much better one available. Although not free, it is simply impossible to beat the O&O Defrag utility for keeping disks defragmented.

At the time of this writing O&O Defrag V8.5 Server is available for $219.00 for a single-user license. There is also an available 30-day trial. This is seriously, hands-down, no questions asked, the best defrag software that this author has ever used.

Part four will look at some security issues and go over installation of IIS.

Go back to part two Go to part four


Table of Contents

Dig Deeper on VMware virtualization