Managing change in dynamic virtual environments

Virtualization brings flexibility and dexterity, enabling rapid change. But change management processes are at odds with that flexibility. How can you reconcile the two?

Let’s face it: Whether your IT environment is large or small, managing and tracking change is never easy. And when you add the new dimension of virtualization technologies, you only augment the hurdles in an already onerous process. While virtualization has inherent benefits, natural integration with change management systems or processes is by no means one of them.

In this sense, server virtualization represents a true paradigm shift in terms of tracking the various items in an environment. In a pre-virtualized universe, you might have had a single business process that required a single application. That application may have resided on a single server, and that server had a single disk storage system (local or storage area network). But now, that physical server is a logical server, and that logical server runs on a physical server with lots of other logical servers. All the logical servers connect to logical networks and logical storage that all map with physical storage. Managing these myriad relationships and server configurations— as well as their relationship to all items in a configuration management database (CMDB)—requires a true change in thinking.

In this article, we explore the challenges of managing change in virtual environments and some of the technologies that offer solutions to these challenges. We also ask whether the new crop of systems management tools on the market has mitigated the hurdles of tracking change in a dynamic environment.

The benefits of virtualization

Virtualization eases administrative burdens, provides flexibility and offers several management benefits:

  • Rapid provisioning of new virtual guest operating systems allows you to create new virtual machines (VMs) in seconds.
  • New VMs can be placed dynamically on any host system according to the load on the host’s systems at that time.
  • Once virtual machines are created, they can be dynamically load-balanced across host machines and moved as needed, without the knowledge of an administrator.
  • New host systems can be added to the pool of virtualization hosts in less than an hour and take on the load of anywhere between 10 and 50 or more virtual guests.
  • New virtual networks and data stores can be added in seconds.
  • Virtual machines can be downloaded from the Internet and brought into a virtual infrastructure. These downloaded VMs have a full OS, preconfigured applications and, in many cases, an unknown level of security patches.

But the flexibility of virtualization is also the key reason that it causes problems for change management. The dynamism and flexibility of virtualization don’t take the rigidity of change management into consideration. For example, with VMware Inc.’s VMotion and Distributed Resource Scheduler (DRS)—tools that together enable live migration of virtual machines as well as resource allocation among these VMs— virtual machines can be added automatically to any server with free resources when a new VM is created. With VMware’s Virtual Desktop Infrastructure, pools of virtual desktops can be configured. Those desktops can then “float” between any physical server that has the lowest load. But because a desktop can be dynamically created without administrator knowledge and placed on whichever server has available resources, these changes can occur without being accounted for.

The benefits of change management

Organizations don’t universally deploy CMDBs. According to a 2007 Data Center Purchasing Intentions Survey, 50% of responding companies had deployed configuration management databases or planned to implement one over the following year.

Those without a CMDB have only a mishmash of various disparate configuration and asset databases: a network management application, a database management application, PC management, performance management, server management and service-level agreements in a Word document as well as business processes and application dependencies that aren’t documented anywhere. If you try to run a single report of all assets and whether configuration changes have occurred as well as the kinds of changes, you won’t be able to do so—or at least not in a timely manner. Some shops without a traditional CMDB may use management tools such as VMware Lifecycle Manager or Stage Manager, which exclusively address change tracking and lifecycle management of virtual systems.

Obviously, this is no way to run a large and dynamic IT department, and CMDBs attempt to corral this dispersed information. A CMDB contains IT information about server, storage and network devices but data isn’t limited to these areas. It also contains logical elements such as service-level agreements, business services and application dependencies. From this database, you have a reference for all IT asset visualization and reporting. A CMDB reporting tool, for example, provides reports such as change tracking and impact analysis.

And for those IT shops that have implemented the IT Service Management (ITSM) branch of the IT Infrastructure Library (ITIL), CMDBs have become increasingly important. For these shops, IT business processes such as IT service management, change management, asset management and server configuration management comprise the overarching structure that enables these large organizations to manage the demands placed on them and to improve operations with governed, consistent processes.

With ITIL, incidents initiate a chain of processes. As an incident works through this chain, a CMDB is the central repository for tracking an incident. And all these IT business processes must be driven from a single source—a configuration management database—and must reference a “single point of truth.” When a database is a single source of truth, it is the sole repository of data. No duplicate records can appear in the database, records must be relationally structured and business processes must define how data is processed. In sum, a CMDB depicts physical and virtual assets and the relationships between them; by accounting for the relationships between these elements, companies can make changes without causing unnecessary disruption.

How does virtualization affect CMDB Systems?

Now that we’ve discussed how CMDBs depict physical and virtual assets, we’ll discuss the specific areas of IT service level management that are affected by virtualization.

Service-level management: With server consolidation using virtualization, physical machines are consolidated into virtual machines. Each one of these “machines,” or servers, has an associated business process, and a certain service level must be met for that business process. While some features of virtualization (such as distributed resource scheduling and high availability) may improve based on the level of service provided, virtualization may also undermine the level of service provided because additional applications and processes use fewer hardware resources.

Capacity management: When using virtualization for server consolidation, the capacity of your server systems changes dramatically, where many more applications vie for the resources of a single physical server. Once physical servers are consolidated, how is application performance affected? How will you balance the load? How will you prioritize CPU, RAM, disk and network resources relative to the most critical business applications? Once servers are consolidated, you need a robust system for load balancing, resource scheduling/prioritization, and performance reporting to succeed at capacity management.

While this may sound like a virtualization problem, it isn’t really. As in any situation with shared resources, you have these same capacity questions and issues and require the same robust features. For example, when you implement a storage area network and share it between multiple servers, you still need to ensure that resources aren’t over utilized by a server, that the load is balanced across multiple connections and that robust reporting is available to provide trend information.

Availability management: How does virtualization affect availability of business processes and applications? How will it affect reliability, maintainability and security? With server consolidation, there is a greater concern about all aspects of availability. On the other hand, when it’s done properly, virtualization eases the administrative burden associated with availability.

When servers are consolidated, for example, if a physical server goes down, all virtual guest servers on that host system also go down. That could translate to some 10 to 50 virtual guest systems. Thus, when servers are consolidated, server, storage and network reliability is of much greater importance. So too with security. If one physical server is compromised, all virtual guests on that server are compromised. On the other hand, server virtualization allows you to more easily secure multiple servers, to more quickly provide high availability and to spend less time maintaining servers. Features such as VMware’s High Availability, DRS, VMotion and Update Manager can provide a virtualization admin the ability to keep virtual servers running 24 hours a day, regardless of which physical servers go down because of hardware failure, security patches or misbehaving applications.

Service desk and incident management: With server consolidation, service desk and incident management are affected: You augment risk somewhat by putting more eggs in one basket. Thus if an outage occurs that cannot be resolved with availability features, it may affect servers that would have been unaffected had those servers not been consolidated.
While server consolidation with virtualization typically provides consolidated incident management for virtualized guests, you have to connect virtualization management with your traditional incident management application for your traditional servers to have a centralized database (i.e., a CMDB).

Consider what it would take to manually manage the volume of changes that occur in a dynamically changing virtualized environment. In a medium or large company, can you imagine the pain of keeping up with these changes? It would be an attempt to document chaos. The only true antidote to the pain of a manual system is an automated one.

While virtualization makes it easier to add new configuration items to a CMDB in the form of virtual machines, server virtualization won’t automatically populate or integrate with a CMDB. Now that virtualization has earned its place in data centers, the integration of these two systems needs to be undertaken. And now CMDB developers and virtualization vendors are hard at work to connect the two.
Several ITSM and CMDB applications have been released or updated to support virtualization. For example, the Opalis Integration pack for Vmware Infrastructure 3 integrates service desk, CMDB and network tools with Vmware Infrastructure to automate provisioning and to integrate incident/problem management, change and configuration management, maintenance procedures, and disaster recovery. With this feature set, for example, you can move from manually provisioning new VMs and manually updating a CMDB to complete automation of the process (and that is just one of the features provided).

Five ways to balance flexibility and rigidity

Here are five ways to balance the two opposing forces of CMDBs and virtualization and, potentially, to solve the problems that arise from managing change in a dynamic virtual infrastructure.

1. Synergy: Create synergy between the group administering ITSM and a CMDB and the group administering the virtualized infrastructure by meeting to discuss the new challenges that virtualization brings. New ideas can be formed and implemented (such as the ideas suggested in this list) to allow these two seemingly incompatible pieces of infrastructure to work together. For example, if a CMDB is already in place and virtualization is brought into an infrastructure, you need coordination to determine the best way to integrate these two systems. At minimum, every time a new virtual machine or virtual server is added, the virtualization group and the ITSM/CMDB group need to communicate “manually” to ensure that the database stays up to date.

2. Open standards: Pressure vendors to use and move toward open standards for virtualization management. If more virtualization vendors fully adopt systems management standards for their virtualization technologies, the integration between ITSM/CMDB solutions and virtualization will become more transparent.

VMware developed the new Open Virtual Machine Disk Format (OVF), for example, which opens its virtual machine storage containers to enable administrators to more easily download and bring virtual machines into a virtualized infrastructure and allows other software applications to gain insight into those virtual disks. Virtual machines that use the OVF format can be downloaded from the VMware Virtual Appliance Marketplace. Additionally, the specification allows ITSM and CMDB developers to better integrate change and capacity management applications with VMware virtual machines. Open standards such as OVF, for example, could allow a third-party ITSM application to identify a new virtual machine (or group of virtual machines) that will be added to the virtualized infrastructure and use the metadata in an OVF container to make decisions about provisioning.

3. Analysis: Analyze your centralized systems management approach and how it relates to virtualization. Remember that virtualization is not just a tool for IT but a part of your business processes. Consider how integrating virtualization management processes into ITSM processes can save your organization time. Also consider how systems management will change. With VMware’s VMotion and DRS, for example, virtual servers can move dynamically from physical server to server without your approval. These kinds of virtualization concepts require a centralized systems management redesign. For example, you may not know about the integration and automation that is possible until you analyze the extent to which your current systems management technologies can integrate with your virtualization management technologies. From there, spend time researching your possible options.

4. Documentation: Keep in mind that existing system management documentation will have to be rewritten. Other than the aspects of ITSM and CMDB that we have covered, other critical server management processes such as performance management, maintenance, server reboot, patching, remote access for administrators and others have to be redesigned. Once servers are virtualized, for example, recovery procedures for a server experiencing downtime will be quite different from what they were prior to server consolidation.

5. Mapping: You need to map CMDB objects with virtualization objects by defining where each of the managed objects in a virtualized infrastructure will fit and how they are related. For more information, see these resources:

  • VMware’s Proven Practice: Virtualization Configuration Management
  • BMC Software’s white paper on how Atrium CMDB integrates with virtualization
  • Opalis Integration pack for VMware

Once you perform some analysis, it is likely that you will see value in mapping virtualization objects with CMDB objects. This mapping facilitates your desired end result: automation, which enables time savings for systems administrators and cost savings for the company.

As the future ushers in greater demands for virtual environments, it becomes critical to integrate ITSM and CMDBs with a virtual infrastructure. And the more dynamic the organization, the more ITSM is needed. When virtualization is thrown into the mix, there are inherent benefits for IT administrators and for companies as well. But if you use ITSM and a CMDB, you must integrate and map your virtualized infrastructure and a CMDB. Fortunately, several ITSM/CMDB vendors support virtualization already, and VMware has documented how to best map your virtualized infrastructure with your CMDB.

About the Author

David Davis is a Certified Information Systems Security Professional, a VMware Certified Professional and Microsoft Certified Systems Engineer and has worked in the IT industry for 15 years. Currently, he serves as the director of infrastructure at Train Signal Inc. ( He has written hundreds of articles and created six video training courses, including the Train Signal VMware ESX Server video training series. His websites are Happy and

Dig Deeper on Application virtualization