Enterprise private cloud infrastructure is inevitable. I never thought I'd be writing that statement, but think about it. Virtualization by itself is so very 20th century. I was getting into email flame wars about whether virtualization was a fad back in the '90s! (I still have an email from a well-known analyst who insisted that virtualization would never be used in disaster recovery scenarios. We all know how that one turned out.)
Why is private cloud infrastructure inevitable for today's enterprise IT shops? First, it's where the puck is going, not where the puck was. Successful, innovative companies like Amazon and Zynga have clearly proven that this is the future of computing. Second, the benefits are all there. The agility that private IaaS offers is unparalleled, and while it does introduce complexity, it also introduces abstraction and management that helps to deal with that complexity. Some organizations are going whole hog into public cloud, but the largest organizations are almost certainly going to want some "owned" infrastructure, if only to cut costs. Based on my experience of pricing non-burstable (constant-on) infrastructure versus public cloud, the ROI of private cloud makes sense in some instances.
There are tangible benefits to having "servers as software." What do I mean by that? Servers should be destructible and rebuildable via automation, not by some dude who is bringing up a virtual machine and clicking next-next-next. At a talk that I gave last year, I did a demo to show how quickly I could bring up a fully configured cloud server, versus how fast someone in the audience could bring up a virtual machine. In an irony not lost on me, the person who was supposed to bring the media and license keys for the demo did not, and the install-to-virtual demo was a flop. But this experience actually made the point: Virtualization without cloud automation must rely on humans, who aren't all that reliable. Yes, I know that appliances are available, but most of them still need some configuration to work in your environment. A scripted cloud instance mostly does not.
Server destructibility leads to the Two Big Benefits of enterprise private cloud infrastructure. The first is that, while you do tend to invest more in storage with cloud infrastructure (it's not uncommon to configure servers with mirrored storage so that you can get as many "reads" out of your storage as possible, to make up for the delays introduced by abstraction of storage), the destructibility of servers makes folks think differently about, for example, running batch servers 24 by 7. For example, at my organization, we run some custom reports every week on a stand-alone server. These, and other reports, bog down the server so much that it really has to run on its own. Sure, we run other reports, but the server itself is in use perhaps 20 hours a week. While one can script hypervisors to bring machines up, wouldn't it be better to destroy the server (and release all of that storage) when the server's not in use? Of course.
Second, there is nothing that I or other CIOs would love to see more than the death of the "fragile artifact" in the enterprise. You know what I mean--the server that only Jonathan is familiar with. The server that, if it fails, nobody but Jonathan knows how to rebuild. That server, and others like it in your infrastructure, are ticking time bombs. One day they're going to go belly up, and Jonathan is going to be on vacation or otherwise occupied. Cloud infrastructure, when deployed right, makes it so that there are no more fragile artifacts or non-repeatable builds.
Still, there are real-world problems associated with moving to private cloud. As we used to experience with SAN storage, there are still vendors in the stone age that won't support a given platform on a given hypervisor. So, if you decide to use CloudStack, which doesn't support Hyper-V, you'd best invest in VMWare because the chances of your legacy Win32 app vendor supporting Xen or KVM are pretty low.