• 11/20/2013
    12:06 PM
  • Rating: 
    0 votes
    Vote up!
    Vote down!

OpenStack Wins Developers' Hearts, But Not IT's Minds

Nebula Chief Strategy Officer Chris Kemp says IT must develop private clouds for the next generation of applications. Otherwise, it risks irrelevancy.

I write this post after spending a week at the OpenStack summit in Hong Kong. My company, Nebula, has not yet launched in Asia, so I took the opportunity to participate in the sessions and talk to the developers, leaders, and users in the OpenStack community. It was the perfect chance to reflect on where we are today, and the future of the project that I helped start just over three years ago.

My conclusion? OpenStack has captured the hearts of developers, but not the minds of enterprise IT.

As the CTO of NASA and CIO at Ames Research Center, I had the opportunity to deeply immerse myself in an organization where thousands of old applications ran on tens of thousands of servers across thousands of networks in hundreds of datacenters.

While NASA may have a larger and more complex IT footprint than many organizations, all large enterprises seek to run all of these old applications in an environment which looks and acts just like the original computers and networks they were designed to run on.

[Want to learn more about OpenStack? See Google App Engine Swings But OpenStack Is King.]

As servers continue to get bigger and faster while software stays much the same, we have seen servers get virtualized, then the storage. Once we virtualize the network, we will finally be able to faithfully simulate the tangled mess of physical infrastructure that is today's enterprise datacenter. At that point, most software will be able to run on a single, homogenous system. As processors, storage, and networks continue to get exponentially faster and denser, it is conceivable that the contents of an entire datacenter could be virtualized and run on a single computer.

In short, virtualization maximizes the efficiency of running yesterday's PC-era-inspired software on today's PC-era-derived hardware.

This is a wonderful thing, and will keep most of the world's software running without intervention or modification for many decades to come. But this model has very little to do with OpenStack, Nebula, Amazon Web Services, or cloud computing in general.

OpenStack is an open-source reference implementation for infrastructure-as-a-service. OpenStack's community of developers is defining how physical computing, networking, and storage infrastructure are mapped to a set of logical services in a way that will form a new open foundation for a new generation of software that runs on service-driven, scale-out infrastructure.

The first enterprises to adopt OpenStack -- Internet companies like Yahoo and eBay; research institutions like Xerox PARC and CERN; service providers like AT&T and Comcast; government agencies like NASA and NSA -- retain some of the most talented computer scientists and engineers in the world. In most cases, these organizations are using OpenStack to power new, highly strategic, and often very large applications. 

Efficiently building large-scale systems is becoming increasingly critical for nearly every business (or government) that extracts value from better understanding all of our web logs, GPS location data, social media graphs, financial transactions, retail transactions, stock market transactions, electronic health records, genomic data, photographs, videos, satellite imagery, and of course the data from all of the sensors in our mobile phones, wrist bands, watches, televisions, cars, and so forth.

At Nebula, I have the opportunity to speak to thousands of organizations about our product, and it is clear that the chasm between "enterprise IT" and "mission" organizations at most enterprises is growing larger and larger. Business units that operate computing infrastructure outside of "corporate IT" are often referred to as "shadow IT" in older enterprises. At tech companies here in Silicon Valley, that kind of "shadow IT" is referred to as "technical operations," or TechOps.

At top Internet companies, TechOps is home to some of the most talented (and well compensated) engineers in the world. These teams operate very differently from corporate IT. They do not manage servers, VMs, or software -- at least, not the way that most CIOs think of it. They often do not (or ever intend to) virtualize anything. In TechOps, very small teams deploy new software on fleets of hundreds or thousands of servers, often several times a day. Working closely with software engineers, these teams strive to increase the velocity at which new features can be deployed, and often ensure all features are tested at scale.

A new generation of infrastructure that powers new mobile and web applications and puts large amounts of data to good use (in most cases, at least) is being developed. Today, most of that development takes place on public clouds like Amazon Web Services, Google Cloud, and Microsoft Azure. 

This new generation of cloud applications and the public clouds they are being built on have inspired the hearts of developers, but the mind of enterprise IT is still focused on providing "reasonable accommodation" for old applications.

Enterprise IT must either watch as their most strategic and critical applications are built on public clouds, or they must immediately invest in real, standards-based, API-driven private clouds.

The longer enterprise IT waits before providing a true private cloud, the larger the chasm grows between where the business has been and where it's going, and the greater the risk that IT will lose the hearts and minds of the innovators that are essential to the cycle of reinvention and crucial to the success of every enterprise.

Battle lines are forming behind hardware-centric and virtual approaches to software-defined networking. We size up strengths and weaknesses. Also in the SDN Skirmish issue of InformationWeek: Anonymity has a role in business communities. (Free registration required.)


A Future for Open Stack?

I think Open Stack does have a future for those tools and apps which can be hosted on public clouds; but the idea that enterprises will migrate everything to public clouds seems increasingly unlikely.

Conference attendees vote with their feet

We just wrapped up Cloud Connect Chicago last month and our OpenStack Bootcamp was the most attended workshop for our largely enterprise IT-centric audience.  Chris makes a good case in this post and i think it's just a matter of time before we see more IT interest in OpenStack.

OpenStack for new apps; do you buy that?

Open source OpenStack is in an uphill battle against proprietary forces inside the data center, in particular Microsoft and VMware, and public cloud services outside it, primarily Amazon Web Services. It is a two front war and not at all clear that OpenStack will get the upper hand. Kemp, as chief startegy officer of a young OpenStack company, is arguing that new applications will need to use masses of Web site visitor data or masses of customer interaction information and do so on a scale that will benefit the company. VMware and Microsoft have virtualized legacy systems. Use OpenStack for the new kinds of applications that you will need in the future. It's a leap, but I think he's onto something.

Re: OpenStack for new apps; do you buy that?

I think for business IT, it will still take sometime to adopt OpenStack, considering that the future is not very clear for the moment. Many developers try to put in a plug for OpenStack but there are many things to concern beyong technology - legacy applications, private virtualization solutions, etc. I would like to keep my finger crossed and see how the things will go from here.:-)

Re: OpenStack for new apps; do you buy that?

Open stack is still open for discussion, devolopers try and make it easy easy for IT to use any device for any service, but being in IT i dont think its making much differnce as if now, whenever we buy any proprietry server in maximum cases we do get limited duration of services and support for free and meanwhile IT engineers get themselves groomed on same to support after free services from vendor gets over.

Re: OpenStack for new apps; do you buy that?

This is the real world in which we live - you cannot easily get valuable things for free. OpenStack itself is not mature yet and we need more deployment in real life to see how it really works. Especailly, whether or not it can bring concrete business value to IaaS service providers.

Linux was a 10 year adoption in the enterprise

10 years ago would enterprises thought they would have adopted Linux on the scale they are doing today. An emphatic no, but it has been an astonishing success. OpenStack is on the same path.

I work with enterprises and an increasing number are seriously looking at OpenStack. One or two of the bold have already started, though have a strong opensource background and their own developers. The majority are waiting for the easy to install, simple to maintain package, that delivers a hardware agnostic platform with a standard set of open APIs for management of cloud IaaS. It also has to be enterprise ready and offer the manageability and ease of use of VMware. Alongside the existing scale out commodity model.

2014 will be the year when OpenStack breaks into the enterprise. Havana released support for N+1 upgrades and optimised workload scheduling/balancing is in the pipeline. The direction is to provide the ability to manage both older "legacy" services and the new agile born on the cloud apps that dominate our waking hours.

Re: Linux was a 10 year adoption in the enterprise

I agree you steve, Developers can automate access or build tools to manage their resources using the native OpenStack API, but do you account IT for same.

Re: Linux was a 10 year adoption in the enterprise

Agreed OpenStack will only be successful if it can keep both developers and enterprises on board. The challenge for the Foundation is keeping both the opensource purists and the enterprise pragmatists on side. Most of the noise and hot air I see coming is from the purists. But its the pragmatists that will make ensure it is widely deployed.

It was possible with Linux and I hope possible with OpenStack, as there is real benefit for both ends of the spectrum. Essentially they both want the same things, freedom of choice, avoidance of vendor lock in, comprehensive API set.

Re: Linux was a 10-year adoption in the enterprise

Yes, Linux started out slowly in the enterprise, but circumstances have changed. There's much wider recognition of the value of open source code now than when Linus sequestered himself in his bedroom to develop Linux. It turned into one of the largest, sustained software projects of all time. Not clear OpenStack can do that. Linus provided a guiding hand and discriminating filter through wihich additions had to pass. On a project with as many big companies as OpenStack's, often cited as a strength, the elephants dance  but no caller provides overall direction. One outcome: see reservations of Photobucket admins in Photobucket Pictures Its Future On OpenStack.