Special Coverage Series

Network Computing

Special Coverage Series

Commentary

Bill Kleyman
Bill Kleyman Virtualization & Cloud Solutions Architect, MTM Technologies

Big Data Analytics And Why Datacenter Infrastructure Matters

As the amount of data generated continues to skyrocket, organizations will increasingly rely on data management systems, underscoring the importance of the datacenter.

We are witnessing an amazing trend in the IT world. There are now more devices, more connections to the Internet, and more demands from devices than from people. At the same time, IT environments have a new challenge: the data all these devices produce.

The average employee already utilizes, on average, three to five devices to access a corporate datacenter. As more devices come online and connect to the cloud, they will produce more data -- data that will need to be managed and quantified. Let’s consider some numbers:

• According to IBM, the end-user community has, so far, created more than 2.7 zettabytes of data. In fact, so much data has been created so quickly, that 90 percent of the data in the world today has been created in the last two years alone.

• As of 2012, there are more than 240 terabytes of information collected by the Library of Congress in the U.S.

• Facebook handles 300 million photos a day and about 105 terabytes of data every 30 minutes. It also stores 100 petabytes of data on one Hadoop disk cluster, according to published reports.

• In 2012, the Obama administration officially announced the Big Data Research and Development Initiative. Now, there is more than $200 million invested into big data research projects.

• Research firm IDC forecasts the big data market will grow from $3.2 billion in 2010 to $32.4 billion in 2017.

• Finally, the 2012 IDC Digital Universe report predicts that we'll reach 40 zettabytes by 2020, a 50-fold growth from the beginning of 2010.

My questions to readers are: What good is a large amount of data if it can’t be analyzed or quantified? Furthermore, how is it possible to manage such large data sets over a cloud or WAN environment? Hadoop, for example, has become the unofficial standard behind big data management engines. These platforms are very new and require a new type of skillset to deploy and manage.

[Read about a new generation of cloud-based log aggregation and analysis services that promise to help enterprise IT security teams filter gigabytes of log data in "Big Data Analytics, Cloud Services and IT Security."]

Another challenge is that there aren’t too many people who can speak the language of big data. Remember, all of these technologies are still emerging. But, surprisingly, a 2011 Ventana Research survey, done in conjunction with analytics company Karmasphere, showed that the big data uptake was happening fast.

Ventana polled IT managers, developers, and data analysts working with Hadoop in hundreds of companies of various sizes and industries and found:

• More than one-half (54%) of organizations surveyed are using or considering Hadoop for large-scale data processing needs.

• Over 82% of respondents said they benefit from faster analyses and better utilization of computing resources.

• Eighty-seven percent of respondents are performing or planning new types of analyses with large-scale data.

• A whopping 94% of Hadoop users surveyed say they now perform analytics on large volumes of data not possible before; 88% analyze data in greater detail; and 82% report they now retain more of their data.

• More than two-thirds of Hadoop users perform advanced analysis -- data mining or algorithm development and testing.

Now back to reality. With the advancements in cloud computing, IT consumerization, and the increase in data, where will all of this be controlled? The reality is simple: Behind all of these technologies sits the datacenter. High-density computing, advanced types of networking methodologies, and more efficient ways to control a datacenter infrastructure are the only means to meet today’s business demands.



Related Reading


More Insights



Network Computing encourages readers to engage in spirited, healthy debate, including taking us to task. However, Network Computing moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing/SPAM. Network Computing further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | Please read our commenting policy.
 

Editor's Choice

Research: 2014 State of Server Technology

Research: 2014 State of Server Technology

Buying power and influence are rapidly shifting to service providers. Where does that leave enterprise IT? Not at the cutting edge, thatís for sure: Only 19% are increasing both the number and capability of servers, budgets are level or down for 60% and just 12% are using new micro technology.
Get full survey results now! »

Vendor Turf Wars

Vendor Turf Wars

The enterprise tech market used to be an orderly place, where vendors had clearly defined markets. No more. Driven both by increasing complexity and Wall Street demands for growth, big vendors are duking it out for primacy -- and refusing to work together for IT's benefit. Must we now pick a side, or is neutrality an option?
Get the Digital Issue »

WEBCAST: Software Defined Networking (SDN) First Steps

WEBCAST: Software Defined Networking (SDN) First Steps


Software defined networking encompasses several emerging technologies that bring programmable interfaces to data center networks and promise to make networks more observable and automated, as well as better suited to the specific needs of large virtualized data centers. Attend this webcast to learn the overall concept of SDN and its benefits, describe the different conceptual approaches to SDN, and examine the various technologies, both proprietary and open source, that are emerging.
Register Today »

Related Content

From Our Sponsor

How Data Center Infrastructure Management Software Improves Planning and Cuts Operational Cost

How Data Center Infrastructure Management Software Improves Planning and Cuts Operational Cost

Business executives are challenging their IT staffs to convert data centers from cost centers into producers of business value. Data centers can make a significant impact to the bottom line by enabling the business to respond more quickly to market demands. This paper demonstrates, through a series of examples, how data center infrastructure management software tools can simplify operational processes, cut costs, and speed up information delivery.

Impact of Hot and Cold Aisle Containment on Data Center Temperature and Efficiency

Impact of Hot and Cold Aisle Containment on Data Center Temperature and Efficiency

Both hot-air and cold-air containment can improve the predictability and efficiency of traditional data center cooling systems. While both approaches minimize the mixing of hot and cold air, there are practical differences in implementation and operation that have significant consequences on work environment conditions, PUE, and economizer mode hours. The choice of hot-aisle containment over cold-aisle containment can save 43% in annual cooling system energy cost, corresponding to a 15% reduction in annualized PUE. This paper examines both methodologies and highlights the reasons why hot-aisle containment emerges as the preferred best practice for new data centers.

Monitoring Physical Threats in the Data Center

Monitoring Physical Threats in the Data Center

Traditional methodologies for monitoring the data center environment are no longer sufficient. With technologies such as blade servers driving up cooling demands and regulations such as Sarbanes-Oxley driving up data security requirements, the physical environment in the data center must be watched more closely. While well understood protocols exist for monitoring physical devices such as UPS systems, computer room air conditioners, and fire suppression systems, there is a class of distributed monitoring points that is often ignored. This paper describes this class of threats, suggests approaches to deploying monitoring devices, and provides best practices in leveraging the collected data to reduce downtime.

Cooling Strategies for Ultra-High Density Racks and Blade Servers

Cooling Strategies for Ultra-High Density Racks and Blade Servers

Rack power of 10 kW per rack or more can result from the deployment of high density information technology equipment such as blade servers. This creates difficult cooling challenges in a data center environment where the industry average rack power consumption is under 2 kW. Five strategies for deploying ultra-high power racks are described, covering practical solutions for both new and existing data centers.

Power and Cooling Capacity Management for Data Centers

Power and Cooling Capacity Management for Data Centers

High density IT equipment stresses the power density capability of modern data centers. Installation and unmanaged proliferation of this equipment can lead to unexpected problems with power and cooling infrastructure including overheating, overloads, and loss of redundancy. The ability to measure and predict power and cooling capability at the rack enclosure level is required to ensure predictable performance and optimize use of the physical infrastructure resource. This paper describes the principles for achieving power and cooling capacity management.