Today, virtualization technology can be deployed across most, if not all, of an enterprise -- from storage to servers, into the network and all the way to each user's endpoints. Virtualization problems can arise in any of these areas.
Although server virtualization technologies are now quite stable, administrators must carefully consider workload balancing. Each application demands some of the server's computing power, so workload balancing is often preferable, because it distributes workloads across a number of servers so that computing demands complement each other.
The same holds true for memory and I/O-intensive applications. Incorrect workload balancing can lead to virtualization problems around application performance and stability, which can take down all of the VMs on an afflicted server. Advance testing of each application before deployment can identify resource needs and help administrators make the best workload decisions.
Workload balancing best practices
Avoid the temptation to load servers to 100% utilization, and don't over-commit server resources. Server consolidation is the principal appeal of virtualization, but leaving some computing resources in reserve allows for changes in application needs as well as room for VM failover when trouble strikes.
If every physical server is loaded to 100% of its computing resources, there is no place to fail over VMs when a server fails. Experts generally recommend loading a server from 50% to 80% utilization. Some resource-intensive or mission-critical applications may be the only VM on a physical server.
Also remember that virtualization problems, whether caused by a VM crash or physical hardware fault, can affect all of the VMs running on that server. There is considerably more risk to production. Although most IT professionals already know this, they often do not consider the consequences of downtime and restoration.
Think about a virtualized server with 30 VMs. If the server fails, all 30 VMs will be affected, and most administrators are unaccustomed to dealing with 30 simultaneous server failures. Failover tactics can be used to move VMs from the problematic server or to restart VMs from storage onto other server hardware that have adequate computing resources. You'll have to manually restart or restore any VMs not protected with failover when the server is fixed.
Fixing virtualization problems: A slow climb
Every VM restored to a physical server demands computing resources, which in turn reduces the remaining computing resources and slows the restoration of subsequent VMs -- a factor that can turn a well-planned restoration into an all-day affair.
Todd Erickson, chief operations officer and senior vice president of First Flight Federal Credit Union, a financial institution in Havelock, N.C., recounted a recent controller failure that needed a restoration.
"Everything was perfect," he said. "All my data was there. I had backups. I had snapshots. It was all good. But what we didn't realize at the time was that it took us almost 22 hours to restore all the images. I lost a full business day of some applications."
Restoration issues should be tested and measured and accounted for before real trouble occurs. Network issues are often a lesser factor in virtualization problems, but it's important to deploy an architecture that provides the necessary amount of bandwidth and resiliency for a virtual infrastructure. For example, an iSCSI SAN may be entirely inappropriate for demanding VMs like SQL Server or Exchange Server.
Similarly, traffic-intensive applications like a transactional order entry system may interfere with the performance of virtual endpoints by delaying keyboard/mouse input or causing choppy graphic updates. Any virtualization deployment should be preceded by a thorough evaluation of network needs.
Virtualization can also complicate any troubleshooting process. For example, a LUN created from virtualized storage may comprise storage devices across several physical RAID groups or even storage subsystems. Similarly, you can easily migrate a VM between physical servers. Flexibility is the key to success in virtualization technologies, but administrators must rely on expertise and solid management practices to help them isolate and correct problems quickly.
"The ability to manage those misbehaving VMs comes into play," said Ty Hacker, director of technical services at I-Business Network LLC, a financial/accounting Software as a Service provider located in Marietta, Ga. Hacker said management tools definitely would be one area he would like to concentrate on because that one bad VM can wreak havoc on the entire host.
About the author
Stephen J. Bigelow, a senior technology writer in the Data Center and Virtualization Media Group at TechTarget Inc., has more than 15 years of technical writing experience in the PC/technology industry. He holds a bachelor of science in electrical engineering, along with CompTIA A+, Network+, Security+ and Server+ certifications, and has written hundreds of articles and more than 15 feature books on computer troubleshooting, including Bigelow's PC Hardware Desk Reference and Bigelow's PC Hardware Annoyances. Contact him at firstname.lastname@example.org.