Years ago the notion of system configuration meant that IT shops made sure they had appropriate hardware characteristics, operating system versions and patch levels. Today’s proliferation of virtual machines (VMs) has dramatically expanded the concept to include every server element, including hardware setups, hypervisor patching, and the makeup of every VM image.
In many cases, every machine must share a common, well documented configuration to meet disaster recovery and business continuance needs as well as to demonstrate regulatory compliance. But the sheer number of VMs in a modern data center makes manual configuration almost impossible. So administrators routinely use configuration management tools to exercise fast and accurate control over an increasingly dynamic computing environment.
Configuration management is the broad practice of knowing and controlling how systems are set up throughout their lifecycle. Configuration management traditionally starts at the hardware level and tracks system devices through purchase, deployment, maintenance and decommissioning.
Hardware is tangible and can easily be tracked through asset management or inventory control systems. From a hardware standpoint, configuration management would include such concerns as the number of processors, the amount of memory, local disk storage and network interfaces.
Getting a handle on today’s configuration management
Configuration management has also evolved today to embrace all types of software, starting with operating systems and drivers and extending all the way to the virtualization platform/hypervisor, VM images and applications residing on each VM. Administrators must see that each software element is updated and patched to an appropriate level—which is often a more complicated and convoluted process than tracking hardware.
Further complicating configuration management is that system setups can differ between groups. “Maintaining identical configuration within each group is imperative,” said Rand Morimoto, president of Convergent Computing, a solutions provider in Oakland, Calif.
It’s important to note that the emergence of virtualization has diminished the importance of hardware among some administrators tasked with configuration management. “In our virtual environment, the configuration of [hardware in] the environment makes a difference, but it is a lesser variable whether that machine is going to be working appropriately,” said Chris Steffen, principal technical architect at Kroll Factual Data, a provider of business information in Loveland, Colo. This idea is easy to understand. Because a VM is intended to run on any physical machine with adequate resources, the state of the hardware itself can play a lesser role when compared to the state of a VM’s “golden image.”
Regardless of the emphasis on hardware and software, proper configuration management can bring consistency to an organization, especially a virtualized enterprise with a proliferation of VMs running their own instances of operating systems and applications. Such consistency can be critical when creating— or “standing up”—a new VM. It’s a much faster process, it ensures that the VM is fully compliant with required patch levels, and it goes a long way in maintaining an adequate regulatory compliance posture in data centers.
Skilled configuration management can also support troubleshooting. Morimoto described a recent client who experienced stability problems with Windows Active Directory, causing new employees and passwords to randomly disappear. “The problem was their Active Directory domain controllers spanned seven different versions of Windows that had known incompatibilities with one another in Active Directory replication,” Morimoto said. “While a change on one domain controller took place, another domain controller would overwrite or not accept the change, and the propagation of information caused the inconsistencies.” Issues like this are not uncommon but can be prevented with solid configuration management.
Generally speaking, the goals of configuration management within a virtualized and non-virtualized environment are identical with some differences. First, virtualization simplifies the use of “cold spare” systems. It’s easy to create a new VM from an existing VM image, but Morimoto said that changes in any working VM must be accounted for in its image file. Careful configuration management policies can ensure timely image updates and avoid the trouble and time needed to correct outdated images. This issue can usually be addressed by keeping the image file patched and is easily handled using tools such as Microsoft’s Offline Virtual Machine Servicing Tool 2.0.1. By injecting patches into an offline Hyper-V image, administrators can keep an image updated and ready to spin-up with the proper software/patch versions. But the patched image should still be tested and verified before placing it into service.
Virtualization without configuration management can also expose systems to greater security risks. Traditional application servers are straightforward to secure because network traffic is easily regulated at the switch. When multiple VMs share a system, the communication between them is harder to monitor, which increases the possible compromise of a weak or poorly configured VM and threatens the other VMs on that system. “Without some sort of tools to manage configurations, virtual environments can miss crucial patches or updates and then become security risks that may affect other business-critical applications,” said Pete Sclafani, vice president of strategy at 6connect Inc., a managed service provider in San Jose, Calif.
Selecting configuration management tools
Aside from acquisition, deployment and ongoing licensing costs, it’s important to focus on the actual configuration management attributes of the tool that are appropriate for different environments. Trying to choose a universal tool that performs a myriad of unrelated functions will often result in wasted money and effort.
Morimoto cited a client with 97% Windows and 3% Mac systems. The client chose a third-party configuration management tool that natively supported both platforms, even though its Microsoft System Center Configuration Manager 2007 would support both with add-on agents for Mac support. “Because it wasn’t native and out-of-the-box, this organization chose a completely different management tool that took the organization two to three times longer to implement and did a terrible job managing the Windows systems,” Morimoto said.
This example ties directly into the importance of heterogeneity. A prospective configuration management tool should integrate seamlessly with each environment and support all of the hardware, operating systems and hypervisors involved in its configuration management initiative. Also consider the range of configuration-related features such as patch control. Although it is unlikely that an off-the-shelf tool will support everything, examine the availability of add-ins for future growth and expansion of the management effort.
Before making a final decision, be sure to test the prospective tool in a lab environment or limited evaluation deployment and run some comprehensive performance tests to measure any impact that configuration management agents may have on the managed VMs. Remember to enable the desired features and pay particular attention to agent memory utilization and the corresponding impact on paging and other application performance attributes.
Another part of your evaluation should include asset or change tracking—see that the configuration management tool can locate VMs as they move between physical servers. Applications like VMware’s VMotion can migrate a VM between two servers in the data center or between servers across a continent, so the configuration management product should provide some accounting of the virtual assets located on—and moved among—your various servers. Such tracking and auditing functionality can help assure control over the virtual environment, which in turn can improve the organization’s regulatory compliance posture.
Configuration management tools are far more powerful and intelligent today than they have been in years past. Current tools can do things like sniff the infrastructure to evaluate the current environment, check the Internet for update information and make update recommendations with a high degree of autonomy.
Administrators should understand that configuration management tools are not a panacea—they are not help desk management tools, nor are they monitoring and reporting tools. Administrators should be able to correlate other performance and monitoring information back to configuration data. For example, they may be prompted to use configuration management tools when performance monitoring reveals that a database has hit an IOPS limit or is low in available RAM.
A deployment plan should consider the security implications of the configuration management server itself. “We have worked with customers that focus on open source environments but leverage an off-the-shelf Windows installation for their configuration management application,” said Sclafani, adding that regular updates and patches of the configuration management system could easily be overlooked, potentially opening security holes that would allow attackers to exploit uncorrected vulnerabilities on the configuration management platform.
Getting the most from configuration management
Generally speaking, the actual deployment of configuration management tools in a virtual setting differs very little from deployment in a traditional non virtualized environment. As with most tools of this type, administrators should expect to install the product as its own server—often a VM would work fine— and provide access to an extensive back end database to retain and access configuration data compiled by the tool.
Organizations without suitable databases will need to deploy one before actually rolling out their configuration management tools. However, administrators should perform a careful post-deployment performance analysis of their chosen configuration management tools to ensure that any agents or traffic do not affect the server or network unexpectedly. “When we first deployed Ops Manager, we were very surprised at the overhead it needed,” Steffen said. “Look at your resources and make certain that it isn’t sucking the lifeblood out of your network and your systems.”
Another thing to remember is to pay attention to storage utilization. Configuration management can increase storage use when tracking configuration data, patching and handling server images. Excess storage usage can be addressed with a planned storage upgrade, but the configuration management tool itself may provide data reduction features.
It’s most efficient to combine systems into the same configuration, but experts warn against trying to make every single system 100% identical. Instead, create groups with acceptable configurations. It probably isn’t wise to “force-fit” all of the accounting systems into the same configuration as application development, so plan configuration groups that are suited to your particular organization.
Finally, document the implementation, setup and practices that are required for a configuration management platform. This takes time up front but can often pay dividends in the future. “One of the biggest issues is having to deal with personnel turnover,” Sclafani said. “Imagine someone coming in fresh and having to figure out the implementation.”
Next- Generation configuration management
An ever-increasing suite of features and functionality is being incorporated into configuration management tools, such as monitoring, paging, alerting and offline patching. Although this trend can add value to tools, it also muddies the waters during product selection and implementation—making it much harder for administrators to choose the best product for their environments. Expect to see configuration management capabilities incorporated into operating systems or hardware devices.
Tools are also getting more independent, with the ability to manage groups and autonomous actions to keep the network running. The tools also have more analytical features, but this sophistication takes more resources and affects performance—especially if the tool is deployed as an agent. “For example, Hyperic has some great analytics and monitoring that is agent based, but minimum RAM is 4 GB. It also requires Java,” Sclafani said.
Morimoto pointed to the Desired Configuration Management (DCM) capability in Microsoft’s System Center Configuration Manager 2007 as an example of intelligence and autonomy in configuration management. DCM allows the creation of standard profiles that can configure systems within the given group.
About the Author:
Stephen J. Bigelow, a senior technology writer in the Data Center and Virtualization Group at TechTarget Inc., has more than 15 years of technical writing experience in the PC/technology industry. He holds a bachelor of science in electrical engineering, along with CompTIA A+, Network+, Security+ and Server+ certifications, and has written hundreds of articles and more than 15 feature books on computer troubleshooting, including Bigelow’s PC Hardware Desk Reference and Bigelow’s PC Hardware Annoyances. Contact him at firstname.lastname@example.org.