Virtual Machine Technology

The key component in building this abstraction layer is commonly referred to as a VMM (virtual machine monitor) or, sometimes, a hypervisor. This software is responsible for sharing the computer's physical resources among the many VMs that could be running. The VMM is not an easy piece of software to get right because it must trick the guest OS into thinking it has control of the real hardware. To accomplish this, the VMM runs at processor privilege level Ring 0. The guest OS runs up a level, at Ring 1. Most modern OSs run user applications at Ring 3, where applications are prevented from trampling on or otherwise adversely affecting one another. Running the OS in Ring 1 lets the VMM trap some of the operations the guest OS is attempting (like accessing memory) and take corrective measures.

Another component to creating a VM is abstracting the hardware layer. The VM software must create virtual hardware devices, such as the IDE chip set and network and SCSI cards, to be consumed by the guest OS. Each vendor has specific devices its products will emulate. The software then translates these emulated devices to a device that is present on the physical hardware. By creating these virtual hardware devices, a guest OS can be copied to another computer running different hardware and still work. The VMM is responsible for redirecting virtual devices to physical devices. For example, the Microsoft products emulate an Intel 21141 network card; the VMware products emulate an AMD PCnet card.

Putting VMs to WorkA popular use of virtualization is setting up virtual duplicates of the hardware found on production systems to create a test environment. The number of VMs on one machine depends on the amount of RAM and the size of the OS. For example, Novell NetWare 6.5 requires a minimum of 512 MB of RAM. On a workstation with 1 GB of RAM, you can create more than one VM of NetWare 6.5, you just can't have both running simultaneously.

Virtualization also delivers cost savings. Many organizations, whether due to vendor edict or concern over undesirable interactions, run only one service or application--like Web, DNS, DHCP, e-mail or proxy servers--per computer, making for crowded machine rooms. Many of these services don't need a large amount of CPU horsepower, but best practices dictate they run on fully redundant hardware to get maximum uptime. Instead of purchasing individual computers to run each service, you can buy a couple of beefy servers and run multiple VMs on them. Even with redundant hardware running your VMs, you'll still reduce hardware costs in the long run and realize savings in associated areas--you'll use less electricity and improve heat dissipation in the machine room.

In addition, most of us design our infrastructures to handle peak load even if it occurs only a few times per year--say, at the end of a quarter. By using virtualization, administrators can turn on spare machines and spread VMs across a large set of computers.

The Future of Virtualization

Some of the problems inherent in emulating a complete computer have roots in the x86 instruction set. For a VMM to work properly, it must trap the virtual OS and prevent it from accessing the physical hardware. However, a few x86 instructions can't be trapped. Help is on the way. Future chips from Intel (code-named Vanderpool) and AMD (Pacifica) will have additional assembly-level instructions for virtualization.Also, Microsoft and VMware will soon have competition thanks to the University of Cambridge's Xen project. Xen is an open-source initiative that brings a VMM to Linux. Microsoft and VMware needn't sweat just yet because Xen currently provides VMs only for Linux, but the project's Web site hints that Windows may be a supported guest OS once the new chips from Intel and AMD arrive. Novell also is looking at the Xen project: At BrainShare in March, Novell demonstrated its efforts to make the NetWare kernel work with Xen. In one demo, the NetWare server running under Xen was paused, moved to a different physical computer and resumed as if nothing happened.

But that's all still to come. Intel says it plans to start shipping chips with virtualization support later this year; AMD's chips will ship in 2006.

We installed the VMware Workstation and Microsoft Virtual PC software on a Windows XP Professional workstation with 1 GB of RAM in our University of Wisconsin-Madison partner lab. We installed Microsoft Virtual Server and VMware GSX Server on a whitebox server with 1 GB of RAM and 3-GHz single-processor P4 machines running Windows 2003 Enterprise Server. The VMware ESX Server went on a Hewlett-Packard DL580 quad-processor server. We then created a number of VMs and installed a variety of guest OSs, including Windows XP, Windows 2003 Server, SuSE Linux Enterprise Server and NetWare 6.5, where supported.

We tested three VMware products: VMware Workstation, GSX Server and ESX Server. As you might have guessed, the Workstation product is geared for use on individual desktops with development-type workloads, while the two server products are for VMs that are constantly running in development and production environments. There's quite a bit of overlap in features, but there are some significant differences as well. Microsoft sent us two products: Virtual PC 2004 and Virtual Server 2005. Virtual PC and Virtual Server are similar to VMware Workstation and VMware GSX Server, respectively, with Virtual PC targeted to individual users and Virtual Server meant for a shared environment or production use.

Microsoft Virtual PC 2004
Virtual PC is a good choice for Windows-only shops and for developing VMs to be used on Virtual Server. The first thing we noticed after start-up was that the interface is basic compared with VMware Workstation's. For example, to create a new VM, we wanted to install the OS from an ISO image, but you cannot specify that the CD-ROM use the ISO file instead of the real hardware. It can be done, but we had to start the VM before we could capture the CD-ROM device to an ISO image.The other notable difference is that our choice of virtual hardware was more limited than with VMware Workstation. We could create only IDE disk drives and could not emulate a SCSI-based disk. For many, this won't be a big deal, but those who must use SCSI devices, such as tape drives, will be out of luck with Virtual PC.

To help guest OSs play nicely with the host OS, Microsoft provides VMA (Virtual Machine Additions), which enable better video-driver optimization and host-time synchronization. However, Virtual PC VMAs are available only for Windows guest OSs.

As with the VMware products, the keyboard and mouse were locked into a VM window when it had the focus. We didn't have a lock in the focus when we were using DOS screens, only when we went to a graphical screen, such as the XP Installer. To release the mouse, we used the right key. Once we installed the VMAs, the mouse cursor no longer locked in the VM window.

Virtual PC's "undoable disks" function is similar to VMware's snapshots. With this option enabled on our virtual disk, when the VM was powered down, we were prompted as to whether the changes should be saved. We also liked that in Virtual PC, our VM machine files, the virtual disk file (.vhd) and the machine's definition file could be copied over and used by Virtual Server. This is an advantage over VMware Workstation--if we were careful about how we set up the VM, it could be used by ESX or GSX Server, though we'd lose some functionality, like multiple snapshots. This is because the VMware disk files have changed as the products have moved forward, and the newest format is not backward-compatible.Virtual PC does not provide for multiple save points, like VMware Workstation does. One work-around is to create a "differencing" virtual disk, which will point to a base virtual hard drive, but all updates/changes will be written to a new virtual hard drive file on the host OS. Of course, for each save point desired, you must create a new VM.

Virtual PC 2004. $129. Microsoft, (425) 882-8080. www.microsoft.com

VMware Workstation 5.0
VMware Workstation can be run on top of Linux or Microsoft Windows; we used Windows XP Professional as our host OS.

Once we installed VMware Workstation, creating new VMs was simple. A wizard took us through the process. First, we selected the guest OS to use. Based on our selection, the wizard suggested virtual hardware configurations. For example, when creating a VM for a Windows XP install, it created a VM with an IDE hard drive and 256 MB of RAM. But when we created a VM for NetWare 6.5, the wizard changed the hard drive to a SCSI-based device and 512 MB of RAM. These settings are customizable by the admin.

We could add or change devices after VM creation, but if you change the disk subsystem after installing the OS in the VM, it may no longer function properly because the guest OS won't have the right drivers or will continue to look for a nonexistent device to boot from.VMware Workstation provides extensive vendor support for the largest set of guest OSs of all the products we evaluated. It's comforting to be able to call about difficulties you may encounter when trying to run a particular OS under the virtualization product. But just because an OS isn't officially supported, that doesn't mean it won't work. Even DOS or older versions of NetWare may run fine, just don't call VMware if you have problems!

All the VMware products let you choose whether to use the physical CD-ROM drive on the VM's computer or to assign the virtual CD-ROM drive to an ISO image file. For many OSs, admins can download the ISO of the install media; then, instead of having to burn the ISO to a CD-ROM, you can just point the virtual device to the ISO file. We used both the ISO file method and the physical CD-ROM to get various guest OSs, including Red Hat Fedora, Windows 2000 Professional and NetWare 6.5, installed and running in a VM.

After installing the OS, we installed VMware tools that let the guest OS and the host OS better share resources; we used the installer provided for each supported OS to install that OS' version of the tools. For example, for Windows, this toolset provided a better set of display drivers that let our guest OS run in higher video resolutions and let the mouse move freely between the guest VM and the host OS. Without these drivers, when VMware Workstation had the current window focus, the mouse was "locked" into the VM until we released it with the key combination .

One of the best features in VMware Workstation is its snapshot capability, which let us take a point-in-time image of the system, whether it was running or not. After taking the snapshot, we continued to use the VM; then in those rare cases where we made a mess of things, we simply reverted back to the snapshot. If a snapshot is taken while the OS is running, it includes the system's memory and state--a feature that will save developers lots of time. All the VMware products provide this snapshot capability, but it's most feature-rich in the Workstation version. For example, Workstation is the only VMware product that lets administrators have multiple snapshot points. We took a snapshot after installing Windows XP, then a second one after installing a few applications. With the other VMware and Microsoft products, we could have only a single save point.VMware Workstation also lets you create "teams" with virtual network connections and shared repositories of VMs, or "clones." The ESX and GSX versions don't have these features. The team concept let us gather a set of VMs, then manage the set as a group. We assigned three VMs to a team, for example, then specified the boot order and delay before starting up the next VM in the team. Within VMware teams, we also could create virtual network connections between the various VMs. To simulate VM1 talking over an ISDN line to VM2, for instance, we created a team LAN segment that limited the bandwidth between the two machines to 64 Kbps. We liked this feature because it let us set up multiple VMs in a configuration that more closely emulates our production network.

Cloning is another feature unique to VMware Workstation. In the process of creating a clone, we were given the option to create a "linked" or a "full" clone. The full clone is a complete copy of the original VM. This option essentially automates the process. The linked-clone option doesn't copy over the entire virtual disk file. Instead, the new copy just points to the original source. Changes to the virtual disk are saved in a separate file. We really liked this feature. It gave us the ability to create a library of base OS installs on a shared disk and have our developers all point to the base set. When we created a clone from a VM that had multiple snapshots, we could choose from which snapshot point we wanted to work as the base for the new clone.

VMware Workstation 5.0, $189 for download, $199 for packaged distribution. VMware, (877) 486-9273, (650) 475-5000. www.vmware.com

Microsoft Virtual Server 2005
Microsoft Virtual Server takes the feature set found in the Virtual PC product and brings it into a server environment, much like the VMware GSX Server does the VMware Workstation product. At $999 for the Enterprise Edition, which supports up to 32 processors, compared with VMware GSX Server's cost of $1,400 for two processors and $2,400 for unlimited processors, Microsoft's product is a low-cost choice for shops that support only Windows OSs.

Before we could install Virtual Server, we had to install IIS on the host OS. This is necessary because Virtual Server administration is done over the Web--even remote control of the guest OS can be done over the Web--a cool feature. The downside is we were locked into using Internet Explorer for administration because remote control uses an ActiveX plug-in. There is a standalone program available that let us take control of the guest OSs, similar to the VMC provided by VMware. But we couldn't start/pause/stop or reconfigure the VMs from this standalone tool. The Web interface is required for these tasks.

The Virtual Server Web control interface has some noteworthy improvements over its VMware equivalent. For each VM on the system, a thumbnail image of the console is displayed. Although these are small images, they make it easy to see if there are any error message boxes needing attention. We also liked that the admin Web interface provides a CPU utilization graph of each running VM. In contrast, the VMware Web page provided only a bar graph showing an average of the past five minutes. We preferred the more detailed graphs.

Another nice touch with the Virtual Server Web console is the display of recent events. We could easily track the five most recent events, like a VM being powered on or off, remote control sessions or configuration changes. The number of entries displayed on the main page is configurable, and the full list is accessible in the event viewer.

Virtual Server also gave us a virtual SCSI adapter that we could use for hard drives, so we weren't limited to IDE. If you must attach a larger number of virtual drives to a machine, this option is just the ticket. However, if the VM images are to be shared between Virtual Server and Virtual PC, the SCSI adapter can't be used.

Virtual Server fixes the problem of assigning an ISO image to a CD-ROM device in the machine's configuration, something we couldn't do with Virtual PC. For fun, we installed a nonsupported OS, SuSE Enterprise Linux Server 9. When installing the OS via the GUI install, the mouse was sluggish and unresponsive. We restarted the VM and tried again using the text mode option, which worked quite well. Even though SuSE is an unsupported OS, it appeared to work just fine in our testing.Virtual Server 2005. $499, standard edition (up to four processors), $999 for enterprise edition (up to 32 processors). Microsoft, (425) 882-8080. www.microsoft.com

VMware GSX Server 3.2
While VMware Workstation is targeted to the individual user, GSX Server is a shared resource. Like Workstation, GSX Server let us use Windows or Linux as the host OS. For our evaluation, we chose Windows 2003 Server Enterprise.

GSX Server works similarly to Workstation, but lets you administer VMs over a Web browser or the VMware VMC (Virtual Machine Console), a standalone program available for both Windows and Linux. But to use the VM console of the running OS, you must use the VMC.

The VMC is similar to the VMware Workstation interface, with the exception that all screen display and mouse-keyboard activity is redirected over the network to the GSX Server. We had to authenticate to the GSX Server before gaining access to the VMs; once authenticated, we could start and stop VMs as well as edit or create new VMs. Starting and stopping VMs also can be done with the GSX Server Web administration pages.Next, we set out to get an overall status of our running VMs on the GSX Server using the Web administrator. The admin Web interface showed us what VMs are running, how much RAM they are actively using and an average of the last five-minute CPU load. The host OS memory and CPU utilization are also displayed, a nice addition--if an administrator must track down why a VM is performing poorly, having information about the host OS is helpful.

The Web interface also gave us detailed information about each VM, albeit not as much as we had hoped for. Although processor and memory utilization are good things to watch, we were disappointed that we couldn't also keep track of the network utilization in each VM. VMware addresses this with an add-on product, Virtual Center, that let us monitor all our GSX and ESX Servers from a single console. It also tracks CPU, memory, disk and network usage by host OS and all the running VMs. If you have more than one GSX and/or ESX Server, consider using Virtual Center.

VMware GSX Server 3.2, $1,400 for two-processor system, $2,400 for unlimited processors. VMware, (877) 486-9273, (650) 475-5000. www.vmware.com

VMware ESX Server 2.1
ESX Server is an intriguing product. It lacks some of the features of Workstation and GSX while offering others not available elsewhere. For example, where GSX and Workstation require a working host OS, ESX provides its own host OS. This host OS is based on Linux, using the 2.4 kernel, but it's been customized to provide better support for running VMs. There are no other options for a host OS with this product, but given the customization advantages, that's just fine by us. ESX is targeted at running VMs that require a bit more CPU horsepower. The custom host OS provided with ESX Server provides better resource sharing to the running guest OSs and alleviates the need to share resources with the host OS, as in GSX Server.

ESX Server provides the same base set of VM functionality as its kin but has a more limited set of supported guest OSs (see our features chart, pages 55 and 57). However, ESX Server is the only product we evaluated that let some guest OSs see more than a single CPU. For all the other products, even if the host computer had more than one CPU, the guest OS was allowed to see only one. ESX can expose as many as two CPUs to Windows 2000/2003 Server and a couple of Linux versions. Other supported guest OSs are limited to just a single processor, however.ESX Server also offers unique memory management. ESX scanned the virtual memory used by each of the guest OSs we had running and found pages of memory that were identical among the VMs. Then, instead of using real memory for both copies of the data, ESX used a single copy for both VMs. If all the VMs on a box are running the same OS, say Windows 2003 Server, odds are that some of the memory used by DLLs and other programs common to the OS will be duplicated. When VMware finds these duplicates, it makes them shared. Should a shared page change in one of the VMs, it will be duplicated and unshared. This advanced memory management lets admins run more VMs on the same box.

In our tests, we saw a large memory savings from this feature. On our ESX Server, we had multiple Windows 2003 Server VMs running, two of which were allocated to use 1 GB of RAM. When we first started up all the VMs, the amount of shared memory was virtually nothing. But over time, the two VMs had more than 700 MB marked as shared. With our five VMs running on the box, the Web console reported 2.1 GB of memory saved.

Given that ESX Server is widely used for running resource-demanding VMs, VMware has provided an alternate network card driver (vmxnet) that allows for better performance if the VM will see heavy network usage. In addition, ESX Server let us create virtual switches to group VMs' network access. This also let us assign specific VMs to use specific physical cards for network access. This is a useful addition that can help keep network-hungry VMs from competing for the same physical resource. ESX Server also supports VLANs. For shops with high performance requirements from applications that may be running in a VM, ESX is a better choice than GSX. Just be aware that ESX Server has stricter hardware requirements, such as not being able to use IDE/SATA drives for VM storage. A SCSI, Fibre Channel or SAN disk is needed for that.

VMware ESX Server 2.1, $3,750 for two-processor system; VMware ESX Server SMP support add-on option, $1,250 for two-processor system. VMware, (877) 486-9273, (650) 475-5000. www.vmware.com

James E. Drews is a network administrator for the CAE Center of the University of Wisconsin-Madison. Write to him at [email protected].The virtual machine software market for servers and PCs is getting interesting. Although only VMware and Microsoft offer VM software that can emulate an x86 machine, including the BIOS, the Xen project's open-source approach to server virtualization is supported by many key vendors, including AMD, IBM, Intel, Novell, Red Hat, Sun Microsystems and Unisys. Microsoft is working toward a 2008 launch of its hypervisor, or software that enables multiple OSs to run on a single computer; and upcoming chips from both Intel (code-named Vanderpool) and AMD (Pacifica) will sport additional assembly-level virtualization instructions.

Meanwhile, we decided to evaluate two VM workstations, Microsoft Virtual PC 2004 and VMware Workstation 5.0, and three virtual servers, VMware GSX Server 3.2, VMware ESX Server 2.1 and Microsoft Virtual Server 2005. We structured this as a set of individual reviews rather than a comparative. We like that VMware let us use either Windows or Linux as the host OS, but Microsoft's offerings are less expensive. Our features chart lists supported hosts, guests, VM hardware, networking options and scripting support.