Infrastructure provisioning challenges: Linking physical and virtual

Infrastructure provisioning tools often fail to bridge the management gap between physical and virtual. Is vendor standardization the answer?

Virtual infrastructure provisioning usually involves a graphical user interface or scripting tools, both of which ease management of a large number of virtual machines and the applications inside those VMs. Decoupling operating systems from hardware has played a huge role in enabling easier management. But all virtual components get their resources from a physical component; no matter how clever your hypervisor is, it still needs physical resources.

Many organizations deployed virtualization for the capital expense savings, but found they save on operational expenses as well. The operational expense savings results from improved manageability and the ability to use automation to work with large numbers of VMs with minimal manual effort. Keeping these savings in mind, how could you mimic the improvements in the physical parts of your infrastructure? Should hardware vendors step up to allow open standards for uniform management of their products?

In every virtualization implementation, you still need to manage, monitor and make configuration changes to physical hardware. While you can easily use a script to create a new VM or virtual disk, you need to manually create and present a new LUN to the hypervisor.

Admins must perform many routine physical provisioning activities to support a virtualized infrastructure: From LUN and virtual LAN (VLAN) provisioning to storage fabric-zoning and virtualization host-building, there are many moving parts underneath a hypervisor. Each one of these moving parts is essential to the hypervisor’s performance of its job. Managing these physical components is often more technically demanding and detailed than managing VMs. IT pros often use a different vendor for servers, storage and networking. But even if they use a single vendor to source all components, truly universal provisioning and management tools are rare.

Single-pane-of-glass management tools

Every vendor offering a management tool will tell you the tool enables complete infrastructure provisioning through a single pane of glass. In reality, the tools only manage a portion of your environment through that pane.

Your server hardware vendor will manage the hardware and firmware in those servers, but it is unlikely to manage your storage array or network devices well. Similarly, a virtualization vendor will probably not be able to manage the physical servers, storage or networking, let alone the operating system inside the VM and the applications running on top of that.

Without true single-pane management, how can you get a clear picture of the factors contributing to an application performance or stability issue?

Customers want a single tool that covers end-to-end provisioning and ongoing management of everything from new storage and network devices through host builds. There will still be a need for people with screwdrivers to rack and stack physical hardware, but the instant they hand over the hardware, scripts or APIs should take over the rest of the process.

Managing the physical components of a virtual infrastructure

Virtualization hardware and software innovation

Implementing converged infrastructure hardware

Virtualization hardware: Horizontal vs. vertical scaling

The hypervisor and the hardware

The hypervisor is still tightly coupled to the hardware. While we have all but eliminated driver conflicts and hardware instability inside VMs, those problems persist between the hypervisor and the hardware. Many admins experience hypervisor stability issues with cutting-edge hardware, due to drivers that don’t always work well under load. The ability to manage the physical hardware and the attached drivers is even more critical for a virtualized environment; a single driver failure can hurt a large number of VMs all at once.

Relating hypervisor actions to problems within VMs or problems at the physical hardware level is often difficult. A faulty network interface card (NIC) in a physical machine is always visible inside that computer. A faulty physical NIC in a virtualization host, however, may affect different VMs at different times as they move between hosts, but may only affect some of the VMs on a particular host. Without understanding the relationships between physical and virtual components, you may have a difficult time locating the cause of a problem.

Physical and virtual monitoring practices

Most monitoring tools have adapters and connectors that allow them to ingest data from all sources, but the effort required to obtain useful information from the tools may be more work than it's worth. Targeted tools look at only one area of your environment, while Swiss army knife tools cover everything but require a lot of development to work with the exact collection of components in your infrastructure. A standardization of data formats and communication would help integrate various specialist monitoring tools.

How vendors could change infrastructure provisioning

Vendors should perform the physical tasks and tightly control what hardware goes in the racks before and after they are delivered. This tight control makes it easier to write the applications that glue the components together, which is a selling point of converged infrastructure.

In a converged infrastructure environment, admins have tight control over hardware with a dedicated software layer for management. This model is particularly useful for larger organizations that want to focus on what's inside the VM, rather than the plumbing that lies beneath it.

Letting customers choose their own preferred compute, network and storage hardware, and subsequently building a software layer that automates the different parts, would be more difficult than vendor-controlled hardware. Furthermore, a third-party cloud automation vendor usually glues together the disparate pieces.

Having open standards would make a big difference here. If hardware vendors wrote their automation products to a standard API that cloud automation vendors could rely on, then those vendors would have an easier time gluing. A standardized layer would provide a common subset of functions, but it would lack the secret sauce that hardware vendors use to add value to their own products.

Managing a complex virtualized environment is a complex job and, at the end of the day, there is no substitute for smart people accomplishing difficult tasks. Good management tools should make it possible to reduce the amount of manual work, but the pace of change that is inherent in today’s IT landscape makes it difficult for tools to deliver the full service that customers want. A collection of tools that do their own jobs very well may be a better solution than a universal tool making too many compromises.

Dig Deeper on Server hardware and virtualization