Unfortunately, Business Objects "doesn't do reviews," and though we tried hard to persuade SAS to participate, the vendor said it didn't want to pit its current version against the other products. SAP asked us to include it at the 11th hour, then bowed out without citing a reason.

So we gathered the remaining players and put them to work analyzing our vast store of market data (see "How We Tested Business-Intelligence Solutions"). We used each product to poke and prod our subscriber data, and we discovered that you, our readers, work for companies that spend money on products we evaluate, and that you don't drink enough caffeinated beverages.

Vendors at a Glance Click here to enlarge

Getting To Know You

Actually, we're kidding about the caffeine. The data we analyzed was from CMP Media's CRM system, but the folks at our corporate offices removed large chunks of information before we saw it, to protect you and your company's privacy. We never saw your names, e-mail addresses or phone numbers, and we certainly don't have a clue about what you drink. Rather, Corporate sent us a lot of interesting data that made little sense to most of us until we ran the business-intelligence software.

We evaluated the products based on their methods of report distribution, formats of the analysis presented, data security and mundane configuration and management issues. We also challenged the products to answer two specific questions: one that pertained to interest in Internet and intranet products based on company size and infrastructure spending, and a second that tried to correlate the subscriber's title, industry and IT budget. All five business-intelligence applications provided the same basic results, but their report-development methods and presentation interfaces differ greatly.

In the first scenario--correlating concrete numbers, such as IT budgets and infrastructure spending, with interest in Internet/ intranet-related products--we sought a relationship that could show whether people interested in these products came from a company with particular spending patterns.

Indeed, each product generated a report that comprised these variables and concluded that readers from companies with huge IT budgets that spend copious amounts of money on infrastructure are not particularly interested in Internet/intranet products. But those of you who work for technology companies with comparatively small IT budgets and very little spending on infrastructure were extremely interested in these products. We haven't told our marketing people this, though, so if they appear to target you, it's not our fault. Really.

Our second scenario was designed to examine the tools' ability to analyze more nebulous concepts. In this case, we tried to determine whether there is any relationship between a subscriber's job title and the size of his or her company's IT budget, ostensibly to help automate and streamline the subscription consideration process.

Every product pulled together the relevant information and came up with the same answer: No such relationship exists. If our IT staff had been considering such an option at the behest of our marketing department, we could have provided hard data that showed that such an undertaking would not be cost-beneficial. In other words, the result of our analysis was cost-avoidance--something that can benefit every company.