Semantic Technology Key To Mastering Data Growth, Analysis

As big data stores continue to grow and require additional management, enterprises are faced with the task of managing their explosive data growth while also trying to find the best way to analyze that data. According to the recent InformationWeek "Database Discontent" report, a top item on IT departments' 2012 to-do list is handling big data in a way that allows for change over time.

According to Joseph P. Raiti Jr., managing director of Blue Slate Solutions and co-author of the report, there is a perfect storm of factors driving the need for data analysis. Changes in technology during the last 15 years have pushed businesses to adopt the Web and field more end user applications, which results in far more transactions and data on those transactions. The increased volume combined with rapidly dropping data storage costs has created an environment that is continuously generating and storing more and more data, he said.

"Given this growing transactional data along with environments that support big data analysis, businesses have now identified an ability to gain value by analyzing the data they are generating in transactional systems and, more recently, data being generated by their own process automation tools," Raiti said. The process and productivity improvements gained through the Web and implementing new and more integrated transactional systems have created a certain level of process entitlement within enterprises, he added.

As the amount of data available in an enterprise continues to grow, it's becoming a chore to track, manage and understand that data. Additionally, data is scattered in different places--on-premise storage, cloud-based storage, virtualized systems, desktop and notebook hard drives, and across a growing number of mobile devices (some owned by the enterprise and others owned by the worker).

The process and productivity improvements gained through the Web and implementing new and more integrated transactional systems have created a certain level of process entitlement within enterprises, he added.

"Knowing where to look in the data is becoming the new focus of analysis," said David Read, CTO of Blue Slate Solutions and co-author of the report. "For years, business subject matter experts [SMEs] have defined reports, cubes, universes and so forth. Understanding the data, where the interesting and potentially profit-driving enlightenment could be found, was defined manually."

Data is no longer managed so simply, though. With the significant depth and breadth of data contained inside and outside the enterprise, in addition to the high volume of transactions that are continually generating more data, there is no reasonable way for people to know where to look when seeking out actionable knowledge, Read said. Predictive analytics will likely outpace reporting and traditional business intelligence efforts in the future, and they will be used to inform SMEs about where to invest their business intelligence efforts, he added.IT departments often use SQL-based systems for performing operations on data of a uniform type, but the analysis breaks down when it comes to unstructured data. Some enterprises have found the answer in NoSQL, but according to the report, that's not always effective because as the average size of enterprise data stores increases, the feasibility of restructuring and reloading each time the business requires a new view of its data will decrease.

As the fundamental way data is structured changes, semantic technology will be the solution. Semantic technology has matured in the last few years, and Read and Raiti said they expect it to become the new gold standard for housing enterprise data.

Analysis tools in general have matured in their capabilities while also dropping in price.

"The analysis tools to get this done have grown in capability and have also reduced in price. So there really is a confluence of mature technology producing large amounts of data in organizations where process improvements have become harder to find and the easier place to gain value is through data analysis," Raiti said.

Learn more about "State of Database Technology" by subscribing to Network Computing Pro Reports (free, registration required).