EMC World 2011 Stresses Both The Cloud And Big Data

IT trade shows or vendor events act as a barometer of the IT industry in two ways. The first is to measure the current state of the health of the industry, and the second is to point out the trends that will affect the future of the industry. Based upon the recently concluded EMC World 2011 in Las Vegas, the IT community is alive and well today and the future is very bright, but with a number of challenges. Hey, IT wouldn’t be worth living if there weren’t ongoing challenges.

Las Vegas is the home of many IT customer-focused vendor events for a very good reason. The city can handle the crowds, and EMC World was hustling and bustling, whether in the Solutions Pavilion (i.e., exhibit hall), the keynotes, the breakout sessions, the training, or even the dining hall and parties. The interest level in what was happening seemed high, which is great, but EMC World also emphasized where the IT world needs to go from here, and that was encompassed by the theme of the conference: Cloud Meets Big Data.

Paul Maritz, CEO of VMware, ably put the future in context in his keynote: “Accelerating the Journey to the Cloud.” He pointed out that there are three layers (which we might view as trends) that are transforming IT simultaneously: 1.) information infrastructure, 2.) applications development and 3.) end user computing.

The first is about “transforming the information infrastructure to cut the tentacles of complexity,” with end goals such as increased system/infrastructure efficiencyand resiliency. Maritz pointed out the need to isolate complexity, encapsulate systems into black boxes and be able to move the black boxes around in the infrastructure. Note that virtualization technologies, such as VMware’s, act as a catalyst for this process. But virtualization is only part of the story. Illustrative of some of the other elements that raise challenges are workload scheduling,interfaces, collecting the right metrics and the ability to analyze them accurately, such as statistical learning models to help determine what is important and whatis not.

Layer 2 is about transforming applications development. One of the driving forces of this trend is the need for new database strategies that may very well employ unconventional, non-relational database structures. There is a great need for increased scale and flexibility, and the role that operating systems play has to be taken into account. Developers should not worry about writing software that optimizes CPU cycles, but rather about writing software that reduces application complexity. As Maritz pointed out, “Let hardware sweat.”The third layer suggested by Maritz focuses on the transformation of end user computing. One manifestation is the need for IT to be able to support the increasing number of computing devices that end users buy and use. "Seven" was the number tossed about during the course of the conference as the eventual number of devices that IT might have to support per user. IT has to learn to deliver capabilities, regardless of the device employed, and will not have a say in what users can use. Note that this also relates to what is meant by the “post-PC era.”

But this transformation is not about device independence so much as it is about how the role and needs of the end user are changing, such as the ability to digest streams of information, much of which may not be IT managed or controlled, and then be able to recombine and redistribute that data to others. In an era of social collaboration, consumer end users take on new hybrid roles, such as architect and programmer.

Maritz pointed out that having three major transformations going on is unprecedented historically; typically, at most, one transformation is going on at a given time. And he is right. To borrow a term from evolutionary theory, we are in the midst of three punctuated equilibria going on simultaneously. That means that IT has to evolve rapidly in three dimensions.

Evolving on even one dimension is a challenge. Now, all three dimensions are interdependent and interlocking to some extent, so mastering them raises the level of difficulty in trying to reach the highest “cloud” where IT-as-a-service reigns. So when someone breathlessly proclaims a major advance on one layer of change, look and see how it fits in with all three layers of change.

EMC, VMware and, to be fair, many other vendors have long pointed out that getting to the cloud requires a journey. Although stated a number of times in the conference, Maritz pointed out that there are three phases to moving to the cloud: the first is the IT productivity phase, where IT can make the decisionsto improve processes and efficiency (such as server virtualization, where consolidating non-critical servers and workloads is involved). The second phase is business productivity, which involves mission-critical applications and where associated issues such as security and multitenancy are addressed. The third and final phase incorporates IT as a service, which fully employs metrics and service catalogs, and where process automation comes into play.

However, there is a problem with this concept: Organizations shouldn’t stop the journey to the cloud simply by adopting current cloud offerings offered bythird-party service providers, but rather should continue on to "true" cloud destinations where IT really runs as a service. Service providers can provide platform as a service (PaaS), infrastructure as a service (IaaS) and software as a service (SaaS) offerings. Does that make a cloud? Well, yes, in the sense that the service provider may be very efficient, may offer online provisioning of services(say, compute or storage resources) and has the necessary resource utilization metrics. But the answer is no in a more important sense--even if an ITorganization uses a cloud provider, this does not mean that IT becomes, by definition, a cloud provider. The reason is that the “true” cloud provides IT as services that users select as they want, need and are willing to pay for them.

To illustrate this point, Sanjay Mirchandani, CIO of EMC, gave a keynote describing EMC’s own internal IT journey to the cloud.

In fact, EMC IT had a big presence at the Solutions Pavilion to demonstrate what it has learned/achieved in its own cloud transformation. Mirchandani discussed the movement of EMC IT through Maritz’s three stages, as it focused on delivering IT as a service.

That movement required EMC to optimize IT production for business consumption, which required changing from a projects-focused mentality to one that provides business capabilities. A lot of tasks are involved, such as moving one of EMC’s two main data centers from Westboro, Mass., to Durham, N.C.

Detailing its progress and challenges is very important, as EMC IT can act as the proverbial canary in the coal mine concerning potential roadblocks (to say that EMC has skin in the game would be an understatement), as well as acting as a reference to show customers how they can overcome hurdles along the way.EMC World’s theme was Cloud Meets Big Data, and the theme was addressed in the ongoing explosion of data, which includes big data in the form of new types of information, such as utility meter readings, RFID transmissions, and the aggregation of individually small but collectively enormous amounts of data gathered on the Internet. It also includes the rising tide of already big data files, such as medical images, video surveillance, broadcasting and entertainment, and genomics.

EMC has made big data a centerpiece with its acquisition of Greenplum and Isilon to complement its existing Atmos solutions. EMC recognizes that when it buys a company it is not only getting intellectual property, it is also getting the brainpower that led to the success of the acquired company in the first place. The first question that I asked Luke Lonergan, the head of Greenplum, in my one-on-one discussion with him was, “Do you feel each morning as if you have died and gone to heaven?” He laughed and said, “Yes, but a heaven with challenges.”

Greenplum’s involvement with Hadoop has obviously attracted a great deal of attention, and with good cause. Lonergan views Hadoop as good, but incomplete. For example, Greenplum can help with making Hadoop more real-time-oriented rather than batch-focused and, by so doing, extend Hadoop’s capabilities, such as performing Twitter sentiment analysis in real-time. To me, the openness to new approaches, such as the non-relational database approach favored by Hadoop, as well as thinking about how social networking data from multiplesources might be fused together to provide real-time insight indicates that Greenplum is one of the brighter drum majors in the big data parade.

But it doesn’t stop there. A new partnership with SAS, a proven leader in analytics, which involves SAS running on EMC Greenplum products, seems to me to be another vindication of the strategy. Apart from supporting opensource, from which wise IT vendors have benefited, seemingly contradictorily, for years, EMC also promoted a summit within a summit with the industry’s first Data Scientists meeting following EMC World 2011. That is important because new ways of thinking about data are going to be increasingly crucial in the era of Big Data.

Unfortunately, there is a limit to any report, and many of my copious notes will remain unused as I can’t get around to going into detail about keynotes, product announcements and private discussions at this time.

However, I wanted to communicate to you three things related to EMC World 2011:

1. The three layers where change is transforming IT--at the infrastructure, applications development and end user computing levels--as communicated by Paul Maritz in an elegant, vendor-neutral (albeit with a VMware flavor)manner, was important in how it provided practical and strategic context to what enterprises will experience and incorporate in planning for the cloud.

2 . There was a reinforcement of the idea that cloud computing has been and remains a journey, and that you can’t just up and buy a cloud today. Instead, IT will have to implement structure and discipline in many places, suchas designing and building service catalogs, and not just by performing virtualization/consolidation alone.

3 . There was the notion that big data is, yes, a big deal. Not only is it a big deal for traditional vertical industries, such as broadcast and entertainment, but also to a broader base of companies, such as those that can benefit from new sources of data, like tracking/analyzing online user sentiments. But, as with the cloud, using big data is an ongoing journey, where figuring out whatdata is needed, collecting it and then analyzing it for insight that can lead to action remain works in progress. Hey, that is a good challenge to have, especially when you consider the potential benefits.

EMC World 2011 covered a lot more ground than these three points. However, it should be enough for one day to understand that there are three IT trends that you have to deal with simultaneously, that getting to the cloud is a journey that will take time and effort, and that you have to know that big data is real so that you can deal with it more effectively.

EMC is currently a client of David Hill and the Mesabi Group.