Issue TOC Print full article Print this page Download as PDF E-Mail this URL Flame the author     In this article Introduction Information Dissemination Products Reviewed How We Tested Data Clean and Normal Report Card

When we set out to test business-intelligence solutions we began searching for large volumes of data. We managed to obtain more than 2 million CRM (customer-relationship management) records from our corporate headquarters and imported them into a Microsoft SQL Server 2000 database in our Green Bay, Wis., Real-World Labs®. The CRM data consisted of subscriber information from a number of CMP publications--with identifying information about the subscriber (e-mail address, telephone and fax number, last name) removed during the export process.

We cleaned up the data, created all the necessary supported tables and then used that data as the basis for our tests. Each BI product was installed on a Dell Optiplex running Microsoft Windows 2000 Server and used our SQL Server installation as the data source. We evaluated each product based on its architecture, its method of accessing remote data sources and its metadata management.

That enabled us to build and run queries and analyses against an extremely wide data set. Each data set had an average of 600 columns and several hundred thousand records. We evaluated the products in terms of the ease with which reports were built and presented, and we performed a loose performance evaluation while reports were run. There was a huge difference between the products that stored metadata and query information locally versus MicroStrategy 7i, the single product that used a remote data store as its metadata repository.

To evaluate report building, we asked each product to attempt to answer a couple of questions:

1 What is the level of interest in Internet/intranet products across industries as it relates to the organization's IT budget and the part of that budget spent on infrastructure products?

2 Is there a relationship between the organization's industry and IT budget and the job title of the subscriber?

We evaluated how the product answered these questions and the ways in which the results were presented--graphs, columnar data and so on--as well as how much dynamic control the business analyst had over the data presentation.