• 04/26/2012
    6:16 AM
  • Rating: 
    0 votes
    Vote up!
    Vote down!

Tackling The Big Data Challenge (Part 2)

When discussing big data, it’s important to note that not all data is created on the fly; some data sets grow and expand over time and stay in the organizations for a long time. The first steps to managing big data are to focus on both the technical aspects as well as the business needs.
The difference in pricing has to do with capital cost versus operational cost, he says. "Big vendor solutions could cost $1 million to get started, and then might have a recurring cost of $250,000 per year. Open source software is free, but the engineering to support it might cost $500,000 per year, and the engineers might be tough to find."

The survey found that most products are designed to be very scalable, secure and redundant. Some offer an all-in-one appliance using commodity and proprietary hardware; others offer a software-based system that can be used on different hardware; still others use an open source approach.

Topping the list of technologies being used was Oracle’s Exadata all-in-one appliance, among 21 % of respondents. It was followed by Microsoft SQL Server PDW, which Biddick says is "surprising since we have yet to see actual production deployments and market share is limited."

Like Oracle, he says, PDW offers massive scalability to hundreds of terabytes and high performance through massively parallel processing architecture. Coming in third was IBM DB2 Smart Analytics System, an integrated appliance product designed around IBM hardware and software. Biddick also mentions EMC’s Greenplum Data Computing Appliance, a unified big data analytics appliance with a massively parallel processing architecture.

In terms of open source vendors, he says, Cloudera, MapR and Hortonworks develop and support open source Hadoop software distributions and can all be established quickly. Amazon Elastic MapReduce is a Web service that utilizes a hosted Hadoop framework running on the Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3), he says.

Regardless of the approach an organization chooses once it determines its requirements, not addressing the issue of big data in the enterprise today is fraught with peril, Biddick warns. "You’ll be buried in data with no information. Today, you can’t ignore big data—period."

Learn more about Research: The Big Data Management Challenge by subscribing to Network Computing Pro Reports (free, registration required).

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.

Log in or Register to post comments