Virtualization Security: Where's The Innovation?

Virtualization is standard operating procedure. It also breaks conventional defense mechanisms by hindering visibility and control, creating new attack avenues, increasing complexity, and blurring administrative roles between network and server teams. Our 2012 InformationWeek State of the Data Center Survey shows there's no going back, even if we wanted to: Half of 256 respondents will have at least 50% of their production servers virtualized by the end of next year; 26% will have 75% or more. So it's unfortunate that innovation in the virtualization security market is stalled. The holdup is twofold: First, the lack of a publicized breach targeting the hypervisor has made IT complacent. And second, there's an unwillingness among vendors to take on VMware; it owns most of the market and controls the APIs, a big deal given the scant enterprise adoption of rival server hypervisors.

That leaves us with a limited number of major products for hypervisor network security. Two of them, VMware's own vShield and Juniper's vGW (Virtual Gateway, acquired from Altor), use the APIs provided under VMware's VMsafe security program. Cisco, the other big player in this market, bases its technology around the proprietary Nexus 1000V virtual switch, which was developed in cooperation with VMware but isn't dependent on VMsafe. Cisco hasn't completely hitched itself to VMware's wagon; it has hinted that the technology will be usable with other hypervisors.

If you run a non-VMware hypervisor, you should be looking at Vyatta's Network OS product, which works with Citrix XenServer and Red Hat KVM, and, like VMware's vShield Edge, includes NAT and DHCP servers. Vyatta also adds a sophisticated routing engine with support for IPv4 and IPv6 dynamic routing protocols like BGP, OSPF, and RIP.

Granted, the non-VMware cadre is small for now, as some version of VMware is the primary hypervisor platform for 90% of respondents to our latest InformationWeek Virtualization Management Survey. But the market could get more dynamic should open source cloud systems like OpenStack (which uses KVM) and CloudStack (which uses Xen) gain traction. Microsoft has made some storage and migration enhancements to Hyper-V in a bid to appeal to enterprises but doesn't yet have anything comparable to VMsafe for network security, although third parties are starting to fill the gap. And don't count out startups, like Bromium, led by former Xen architect Simon Crosby, that are focused on virtualization and cloud security. A radically new platform could raise the competitive bar by making secure virtualization a table-stakes feature. Crosby hints at the opportunities for Bromium when he says he believes that in five years, most IT workloads will be in the cloud, whether public or private, and that the hypervisor's "sole value will be security."

Still, for now, VMware's vShield line sets the standard for the VM security market. More important, it effectively defines three segments that align with logical network and virtual machine boundaries--intra-VM (Layer 2 within a virtual switch); inter-VM (Layer 3, between physical hosts in a private cloud); and guest OS (application control within the VM). We delve into each layer in our full report, but this structure is a great baseline for IT teams to plan their security strategies.

Fundamentals: Next-Generation VM Security

Unintended Consequences
Our full report on next-generation VM security is available free with registration.

This report includes 16 pages of action-oriented analysis. What you'll find:

Get This And All Our Reports

On Your Own, For Now

Effective security requires specialized engineering expertise, and therefore few believe that open source projects on their own will provide acceptable security for either KVM or Xen. Microsoft has the resources and talent to develop something like vShield for Hyper-V, but it has yet to do so. "Microsoft admits they're not networking people," says Christofer Hoff, chief security architect at Juniper, adding that he expects Redmond to foster a security ecosystem around Hyper-V, much like that coalescing around the VMsafe partner program.

One problem with all virtual security software is the near impossibility of extending a company's own security policies into the public cloud. The easiest option for VMware shops is adopting VMware's cloud management service, vCloud--something VMware must see as a strategic advantage. However, for companies using Amazon or Rackspace cloud services, your virtualization security policies go out the window when you go to the public cloud. Hoff sees developing a consistent set of high-level security APIs that work across platforms and providers as the next big challenge for virtualization security. But he admits the industry is a long way from converging on such a standardized, interchangeable set of security protocols.

So what can IT do until then?

Plan to combine traditional and virtualized defenses, with a bias toward increased use of virtualization. The security software you pick will be dictated by which virtualization platforms you're using, but make your vendors aware that support for a diverse set of hypervisors is a selling point. That's especially true if you plan to implement desktop virtualization, which is a much more wide-open market in terms of viable vendors. Virtual desktop infrastructure can rein in chaotic PC environments, making it easier to keep devices securely configured and consistently patched. If you're using VDI, move endpoint protection from the guest OS into the hypervisor, replacing standalone, agent-based client anti-malware with an endpoint virtual security appliance. The gains in performance and manageability are significant.

Make sure that server and network virtualization is part of your security team's charter and project plans--not a one-off implemented by VM administrators. Don't underestimate the potential for turf wars, either. As Crosby points out, virtual security appliances expose thorny control issues, especially as the VM management platform now handles vSwitches and logical volumes and is thus rapidly subsuming various categories of data center labor, including security, network, and storage configuration. Server admins inheriting these new roles may be ill-prepared to handle the subtleties.

Don't expect to shed security layers. VM security supplements but doesn't replace other elements of a defense-in-depth strategy, like perimeter hardware firewalls, intrusion prevention appliances, and content filters.

Tie virtualization into your overall security reporting framework. Even virtualized network devices like vSwitches and vNICs need to be monitored and audited, but you don't want another SEIM, network, or intrusion monitoring and management platform. That means virtualized security products must be integrated into the existing network management and reporting infrastructure, not treated as special cases. No silos.

Finally, talk to your vendors. Over the past few years, several have announced and even demonstrated products designed to provide network security for VMs, only to redirect their strategies, either in the face of competition from big boys like Cisco, Juniper, and VMware or after realizing the technical complexity of the task. "Over time, as VMware has added capabilities, other players like Catbird and Reflex have transitioned away from doing enforcement into compliance," says Hoff, referring to all-in-one VM management platforms providing monitoring, policy compliance, and auditing rather than Layer 2 or Layer 3 virtual network security.

Our take is that we're likely to see VMware and other major IT vendors, including CA, Hewlett-Packard, IBM, and Microsoft, encroach on this niche by adding VM management features to their comprehensive infrastructure management suites.

InformationWeek: June 25, 2010 Issue

Download a free PDF of InformationWeek magazine
(registration required)