Big Data: Store Everything And Watch Storage Grow
Posted by
David Greenfield
November 17, 2010
One of the big stories from the Teradata Partners conference that just finished up in San Diego was the huge advantage that retailers gain by the tracking the buying actions of their customers online and the enormous impact that's going to have on your storage assets in the years ahead. Business managers can't predict what questions they want to ask about their customers and can't say which collected data is useful or not. The answer is to store it all.
Companies have always relied on market research to understand how to market and sell to their customers. But the emergence of online data particularly couple with social media is creating a venerable information explosion that will enable another order of magnitude of insight into customer behavior. "As we transition from regression models to all of these analytics that you do around a network or a graph of related things a whole huge amount of discoveries can be made," says Paul Kent, the vice president of research and development at SAS.
Ebay, for example, conducts some 100 different experiments at any one time on the site, involving thousands of customers and resulting in millions of data points, noted Oliver Ratzesberger, Ebay's senior director of architecture and operations. As such Ebay gains powerful insights into how users purchase products on the site. One practical example, noted by Ratzesberger, was in the way Ebay presented dresses. At any one time there were some about 700,000 dresses that women could choose from, far too many for any one person to scroll through. With research done on its site, Ebay found that a new feature allowing users to establish personal profiles detailing their sizes, preferred style, manufacturers etc. would be well received by customers.
Storage requirements will grow further though not because of the need to map customer behavior across one site, but because organizations want to analyze the social graph of their customers. "The real advantage [of the online world] is the ability to track customers longitudely," says Mark Jeffery from Kellog. The World Bank of Canada, for example, has a relationship with Weddingbells.com, noted Jeffrey, that allows them to track consumer interactions across the sites.
But getting to that valuable "stuff" means building up a massive database of interactions, both on an individual site and informed by intelligence from partnering sites. In fact, Ebay is getting to the point where simply tracking customer actions across the site will no longer be sufficient because its pages changes so frequently (about every five minutes.) So in order to understand analyze customer behavior, Ratzesberger thinks they're heading to the point where Ebay will need to store every screen that a customer sees - an enormous amount of data.
Page: 1 | 2 | Next Page »
