With the release of vSphere 4.1, VMware made it official that ESXi will be the future default form of its hypervisor. Users have seen the writing on the wall, and most say they will make the switch-- but not without trepidation about losing the familiar ESX Linux console for management.
ESX included a standard distribution of a Linux operating system that could be automated and queried using Linux commands and scripts already familiar to server admins. ESXi strips out the service console and is managed remotely. This makes the hypervisor's footprint substantially smaller, and potentially more efficient, than a full OS. The lack of a full OS also offers less of an "attack surface" for hackers. With the more compact ESXi, a spinning disk is no longer required; virtualization environments can be created using a thumb drive, and ESXi can also be embedded into server hardware before it leaves the factory.
With the 4.1 release, VMware made some concessions to help users make the switch. Previously, using a command-line interface to manage ESXi hosts required going into "Tech Support Mode," which was widely known as "unsupported mode" prior to 4.1. VMware says that "unsupported mode" was a misnomer but acknowledged through a spokesperson that as of 4.1, "Tech Support Mode is now fully supported for use by the end customer on their own." VMware also says it will continue to support users running ESX on releases of vSphere up to and including 4.1, but will no longer develop new features for the ESX version going forward.A long-standing discussion
At this point, some users view the conversion to ESXi as old news and not worth debating. "It's really not that big a deal," said Michael LaForge, a network administrator for Columbia Memorial Hospital. "You just have a couple of virtual appliances out there for secure PowerShell" rather than a console on each host. "It's not really that different. Instead of connecting directly into each host, you use the appliance to connect into multiple hosts."
Users also say they understand the reasons for the switch, even if they haven't made it yet. Eric Parson, a senior systems support engineer at U.S. Cellular, said his team is setting up a "sandbox" environment to kick the tires on ESXi. "I do understand where [VMware] is trying to go -- they don't want two operating systems to maintain and [with ESXi] there's no backdoor hack into the Linux kernel where you can get console access."
"In the grand scheme of things, they're separating duties the way they should for efficiency and security," said Tom Becchetti, who manages both servers and storage for a large manufacturing company. "People I know that would complain are usually very deep in the Linux programming world, and because of that they're looking to move to KVM [Kernel-based Virtual Machine virtualization]. But to me, that's not real world. From a corporate perspective, it's easier for [VMware] to develop [ESXi] and lock it all down. The ultimate goal is a more stable, secure product."Dipping a toe in the ESXi waters
Still, old habits die hard, and users who have tested ESXi or made the switch say it wasn't without sticking points. "It took me a while to get used to ESXi," said Matt Liebowitz, a solutions architect at VMware enterprise partner Kraft & Kennedy Inc. Even though Liebowitz says he's come around. "I've used ESX since [version] 1.5. It was difficult for me to accept at first."
When it came to managing disk subsystems or physical hard drives on a host, for example, Liebowitz said it was much easier for him to use standard Linux tools like fdisk to create partitions than to use VMware's version of PowerShell, which is dubbed vSphere PowerCLI. But with an ESXi host set up in his lab for experimentation and training, "I'm going to the console less and less," he said.
While the tools VMware has supplied with ESXi replace most of the console's functions for management, users say they miss certain features of the console. "We decided that when we migrated to ESX 4.0 that we'd take the jump from "full fat" to installable ESXi, mainly for the reduced patching requirements," said Chris Dearden, a senior hosting center engineer at a large accountancy and professional services firms. "That migration isn't quite complete yet, so we still have a few service consoles out there. We've yet to hit a serious problem requiring a service console agent, although a way to remotely restart management services would be handy on the rare occasions they need a kick."
Depending on the environment, doing the upgrade will be easier said than done. "A lot of stuff we're running is still on [version] 3.5," said Christian Metz, a systems administrator at a Fortune 300 company. "Getting them all migrated over will be a huge task." "The conversion for us looks to be more of a pain than a help," Metz said.
An embedded hypervisor is a convenient idea in theory, Metz said, but "it might not ship right, which is what happened to me. Wwe ended up installing ESX anyway or shipping the server back. The embedded hypervisor doesn't require spinning disk, but if it doesn't ship correctly, it just causes more problems than it's worth."
As an aside, Metz added that he still associates ESXi with its original incarnation: a "dumbed down" free version of VMware's hypervisor; VMware has tried to change that perception by renaming its free hypervisor product without the ESXi moniker, but the impression that ESXi is an entry-level product persists.
Once the conversion has been made, the lack of familiarity with ESXi's command set becomes the next problem. "The concern is all our logic is built around being able to address the physical host. My team is Unix-centric, and they understand what the Unix commands are reporting back to us and where logs are likely to be," said U.S. Cellular's Parson.
Parson added that from a security perspective, there are ways VMware's PowerCLI could create bad habits; it's possible to generate a new password key each time scripts are run, but "the lifespan of that key is five minutes, which makes people who cook scripts bury passwords in their documents. … Remote CLI commands end up sitting around in BAT scripts with the ESX root password clear-texted in, which, from a security standpoint, drives me nuts," Parson said."VMware could do a better job"
Users who have already made the move say it wasn't a pleasant experience, and they call on VMware to make things easier for customers. "I had some angst with the move from 3.5 to 4.0, leaving the service console behind," said Israel Lawson, the director of virtualization technology at a large healthcare administration software company. "VMware needs to do a good analysis of the everyday functions used by their community, and start pre-writing and delivering the API [application programming interface] tools to do that stuff easily."
As it stands today, Lawson said, "You can do [the same things with PowerCLI], but it's not easy. It almost takes an internal development shop to do it. And you have to maintain it through new releases, and as bugs are found. VMware just needs to do a better job with it."
But like most of his peers, Lawson said despite some of the discomfort, his company has been swallowing the medicine and slowly getting used to ESXi. "If you spend the time up front, then the capabilities are certainly there, and all you're losing … is access to a familiar command line for the Linux box that you get with ESX," he said. "The writing's been on the wall the last three years. And at some point, any shop out there that runs VMware is just going to have to pick a date and a time and just do it. It's going to be painful, and you're going to just have to live with it."Beth Pariseau is a Senior News Writer for SearchServerVirtualization.com. Write to her at