The manner in which reports can be displayed does make a difference. Brio's initial report for our first scenario was an impressive 3-D chart that immediately showed exactly which segment of our subscribers was most interested in Internet/intranet products. While generating a report took longer with the other products we tested, we were also equally impressed with the graphing capabilities of all the products.

Glossary OLAP Online Analytical Processing: Its cousins are ROLAP (relational OLAP) and MOLAP (multidimensional OLAP). OLAP CUBE: The basic structure of information in an OLAP system. The OLAP cube stores multidimensional data (each dimension being a column in a database) in a single cell, and ultimately speeds data retrieval by bypassing the data warehouse.

The Web interfaces of all the products we evaluated rocked. All the tools also let you manipulate reports and data from within a Web browser. Information Builders' WebFocus and Cognos Series 7 have the most complete interfaces; both let you manipulate OLAP (online analytical processing) cubes directly via the Web (see glossary). We could change graphs on the fly from bar charts to pie charts to scatter and plot charts, and we could manipulate the columns displayed in all the products easily.

While Cognos and Information Builders use straight DHTML and scripting to provide a rich Web experience, Brio Intelligence requires ActiveX or native Netscape plug-ins. Microsoft Data Analyzer also requires an ActiveX component, limiting browser support to, of course, Internet Explorer. This is not a huge drawback, as most companies have standardized on Internet Explorer, but we appreciate Cognos' and WebFocus' compatibility with open-source browsers such as Mozilla.

Security and Performance

We also considered the products' security and performance. Because of the difficulties inherent in testing performance for these products, our performance evaluations focused on how long it took for a report to be generated. We didn't clock specific times, but during our tests it was obvious that all the products except MicroStrategy are on an even footing in this category. The other products returned results lickety-split, but MicroStrategy took noticeably longer. Performance should not be a huge factor when deciding on a business-intelligence solution, but any tool that requires users to wait for a report for too long will sit unused. For this reason, we kept an eye on this aspect during our product testing.

Security, on the other hand, is a huge issue--not in the omigod-a-hacker-just-got-onto-our-system way, but in ensuring that data access is limited to authorized employees. MicroStrategy provided the most comprehensive security model, offering RBAC (role-based access control) and ACL (access-control lists) to limit access not only to the data but to specific functionality within the product. Cognos Series 7 and Brio Intelligence do an excellent job of addressing this issue, but do not extend their security models to the feature-use level as MicroStrategy does. By comparison, Microsoft's Data Analyzer relies solely on the security provided by a database--which may or may not be active if the data is aggregated from multiple sources in a warehouse.

Pricing Scenario Click here to enlarge

Pricing

Although pricing was only a small factor in our evaluation--our scenario was based on a small to midsize company, and because package deals were available, cost was less of a factor than it would be in a large organization--it is still important to consider. Microsoft and Cognos use a flat per-user fee. Pricing for the other products depends on the deployment platform of the server as well as the specific number of users and their needs. To put all the participants on even footing, we asked for pricing for a company with 100 users (business analysts), three report developers and one administrator. The products would be deployed on a dual-processor, 1.2-GHz Pentium 4 server running Windows 2000. We also included the vendors' annual maintenance fees, which ranged from 0 (Microsoft) to 25 percent (Cognos).

Depending on your needs, a business-intelligence solution's pricing could range from affordable for even small organizations to prohibitive (and note that IT, not sales or marketing, will pay, because it's an application that requires installation and maintenance). Because most of the vendors license their products on a per-user basis, it's often feasible to start with a small implementation that you can later expand as needed.

After careful consideration, we chose Cognos Series 7 as our winner. Series 7 beat out the competition, with top scores for its distribution, analysis and automation of data as well as its excellent data-access and application-access security.