VMware has been pushing its virtualization security vision over the last month, which is enabled by its vShield family of products. But recent revelations about the architecture of VMware vShield Manager have raised eyebrows among potential users, who appear to be putting deployments on hold.
According to a blog post last week by Scott Drummonds, an EMC Corp. vSpecialist and former VMware technical marketing director, vShield Manager can become a single point of failure if it is not configured properly. Specifically, “improper use [of vShield products] can disable the network connection of vCenter virtual machines. Fixing this problem is not intuitive.” Nor, for that matter, is regaining access to the virtualized infrastructure if the vCenter management console is inaccessible.
“The general problem is that installing vShield App or Zones on an ESX host running vCenter introduces a circular dependency that can make installation, operation, or uninstallation fail,” Drummonds said. If vShield Manager shares a network with a virtualized instance of vCenter VM , a failure within vShield may cause the vCenter VM to lose its connectivity to the rest of the network, making management difficult.
Users, partners wary of potential single point of failure
Some virtualization pros see the problem as more acute than Drummonds describes. “The overall concept people need to realize is that if the vShield [Manager] appliance is down, so is any virtual network protected by the vShield appliance,” said one systems integrator, speaking on condition of anonymity. “In some environments this would theoretically be all of production.”
Some users do realize this, and it’s putting the brakes on their vShield evaluations. “While vShield Zones sounds good in theory, it introduces a VM through which all the protected traffic is funneled,” said a data center supervisor working in the higher education field. “I worry about congestion and [vShield Manager] becoming a single point of failure.”
The VMware Knowledge Base at least touches on the issue, advising that users “ensure that the vShield Manager and other vShield appliances meet…hardware requirements” to avoid the problem. In his blog post, Drummonds also recommended that users run a separate ESX cluster for management devices, including vCenter and vShield Manager, to avoid the circular dependency problem.
But beefing up vShield Manager hardware or deploying a separate management cluster can run up against the cost and consolidation objectives for virtualization projects, especially for small companies, said Chris Dearden, a systems integrator based in the UK.
“A separate management cluster is a good design for a larger shop, but a small one might not have the option to dedicate another couple of hosts for VMware management,” Dearden said.
VShield design flaw or feature?
The problem described by Drummonds isn’t unique to vShield or even to virtual firewalls, pointed out Edward Haletky, CEO and analyst with the Virtualization Practice LLC. In fact, the way vShield Manager locks down the infrastructure upon failure is in keeping with longstanding security best practices.
An electronic door lock on a room or building should not unlock if it loses power; it should “fail closed,” not “fail open,” Haletky explained. Likewise, a virtual firewall, particularly one that puts a wrapper around an entire virtual data center, should follow the same principle, he said.
Also, just as a physical key can serve as backup access for an electronic lock in the event of a power failure, users should design fallback methods for accessing the virtual infrastructure. “It’s possible to trust virtual appliances [for security], but know the limits. If it fails, have a plan to use a different secure access path,” Haletky said.
VMware could also do its part to make vShield ‘fail-safe,’ by re-routing access through a fallback method automatically rather than locking down entirely, for instance. In fact, some virtual firewall vendors, such as Trend Micro, already offer ‘fail-safe’ virtual firewalls, Haletky said.
Still, Haletky warns, there is no one magic bullet or product for deploying virtualization security, and thus no avoiding some level of complexity in such deployments. High-availability products for the vShield Manager appliance, as well as monitoring tools, are just two of the management categories potential vShield users need to consider.
“To fully understand what the product does, what other tools it’s well integrated with, how exactly it works, and know how to fix it when it does…there’s no one tool for all of that,” Haletky said.
Experts: ‘The devil you know’ may not be there forever
If all of this is sounding familiar, it should, according to Illuminata analyst Jonathan Eunice, who said vShield’s issues are typical of 1.0 products. “A lot of these products do not operate in…the warm rosy glow of application land, but they also operate at failure time, when other things are going wrong. It’s hard stuff to write, it’s hard stuff to debug,” Eunice said. “Security products or any products that operate when things are failing…even after the product is ‘done,’ still take a lot of field testing to get these things right.” But Eunice predicted the resistance will dissipate with time.
Virtualization itself met similar resistance at first too, and circular dependencies were also an issue in the early days of virtual networking, recalled virtualization evangelist Jason Boche, who manages virtual infrastructure in a large enterprise environment.
“This isn’t anything new,” Boche said. “The virtualized data center, if you think about it as a concept, extends beyond just making virtual machines. It extends to infrastructure at several layers…x86 computing, storage, networking, the application layer…but there are benefits to virtual infrastructure beyond saving power and cooling and real estate…there’s an opportunity to do things differently.”
Beth Pariseau is a senior news writer for SearchServerVirtualization.com. Write to her at firstname.lastname@example.org.
Dig deeper on Virtualization security and patch management