Web Analytics Services

Emily says: Omniture's SiteCatalyst system offers something for all analysts, from the stats guru to the novice. The interface is easy to navigate and lets you quickly cross-reference different data points. For instance, in a report of the most popular URLs, you can right-click on any one listed to identify the top referrers to that URL. The dashboards are convenient and can be converted to any imaginable format. The admin interface is much better than any of the other products'. It lets you see who's using the system and what they are doing. One drawback is that reports load slowly on dial-up. The speed is acceptable on a cable modem or DSL.All-around good analytics, outstanding ease of use and strong support helped SiteCatalyst just nudge out WebSideStory's HBX. Part of Omniture's marketing pitch is that it hasn't lost a customer in eight years of business, which is impressive. But Web analytics is evolving quickly and far from mature, so the company should avoid resting on its laurels.

Omniture makes a big deal about support. Each SiteCatalyst account gets a dedicated customer-service rep and implementation engineer, and these folks stay with each account, ensuring continuity of care. Omniture says each rep supports just three accounts. The company showed good responsiveness during our tests, but reviewers generally get more than their share of attention. Our only ding in the support area was during tag implementation: Because of some miscommunication, getting the tags to collect the data in the segments we intended took a few tries. That wouldn't have been a big deal except that, during the various tag redos, the support rep made the CMP developer placing the tags feel like a dolt. Not true, and bad form.

As noted, SiteCatalyst's ability to create dynamic segments, which it terms "Advanced Segmentation," is unique. This wizard-driven process let us create measured segments or partitions against the data warehouse. But be warned: This service is priced per report--and not part of the standard cost. As a safeguard against running a really expensive report, reports are staged for analysis and consultation by the account rep and engineer.

Many of the services we tested offer a useful reporting presentation--a browser overlay view. It shows where visitors go on a given Web page by displaying the activity in a browser and where most people are clicking. This view gives you an at-a-glance look at the popular URLs and can be used to determine design fixes. If a popular link is buried at the bottom of the page, for example, it might make sense to move it up.

SiteCatalyst calls this view ClickMap; it creates a report that overlays the Web page with translucent rectangles that, when moused over, show information about the rank number of clicks each link got. Hovering over a page object launches a small pop-up box that displays the number of clicks and popularity rank for that particular link. You also can select a "Top 5 links" button that shows the five most popular links and their number of clicks. As with Active Viewing, WebSideStory's browser overlay report, ClickMap displays site- and page-report summaries, which helped us place the report in context. Another slick graphic is the "referred to" report--this tree diagram uses line thickness to represent traffic amounts.

The SiteCatalyst Attribute Import and Naming Tool, SAINT, is an Excel-template-based tool for importing and exporting data to and from SiteCatalyst. This wizard-driven process is easy enough to use, but it's not automatic like HBX's Report Builder. Rather, SAINT automates bulk naming of pages for setting up marketing campaigns, for example. This automation obviates naming pages one at a time and correlating those pages to campaign data.

SiteCatalyst lets you compare two report periods--not bad, though ThinkMetrics' InSite can compare four. We selected two periods and the data to display, then defined how to sort the data. The result tabled the two dates and compared percentage and numeric change in one column. Another view displayed date ranges as a trend, showing how much traffic similar URLs got over the period.

SiteCatalyst's was one of the easier interfaces to use. Expandable left-hand menus divided the reports between commerce and traffic, for example. We could navigate through submenus or display an overview page, the Report Navigator, to help us find reports we had viewed.

The Trend view provided a topN line graph that, with top URLs shown, made comparative activity spikes easy to understand. (TopN is a reference to limiting a displayed list to the most frequent or highest volume elements for a particular metric.) In addition, SiteCatalyst offered a point-and-click interface to create calculated metrics.SiteCatalyst supports context-sensitive help and provides a PDF with great search capabilities and index. We appreciated the usage suggestions and explanations of the calculated metrics, including the formulas used. SiteCatalyst explained what it meant to normalize data for a two-period, or A/B, report, for instance, and gave us two brief, specific examples that described scenarios where normalization would enhance analysis, and why. Now that's help we would have been willing to wait on hold for!

Interface response was good during our tests. This isn't to say we never had to wait for reports, but SiteCatalyst displays a progress indicator with outs to e-mail the report or cancel it.

SiteCatalyst provides a full range of user-management features. For example, you can force password changes at login and define a time period for which user IDs are valid. SiteCatalyst also includes initial settings denoting user roles, like Security Manager. Within the Security Manager setting, we could set password aging and characteristics, and IP login and e-mail domain restrictions. We also could set up temporary users; the beauty here is that after their defined stay, they went poof--no cleanup by the admin on Aisle 5 required. SiteCatalyst also has user groups for defining permissions and access control to Report Suite and more. For example, we could choose to include or exclude access for each report suite (commerce, traffic and path). We could also choose specific reports within a suite.

However, delegated access control is impossible with SiteCatalyst. Administrative access is global--anyone with administration rights can access the entire site and cannot be relegated to a site subsection, as is required for delegated access.

SiteCatalyst's group-management and report suites go a along way toward limiting the amount of data presented to a user, which is a form of personalization. We liked that SiteCatalyst limited menu and site choices to only those defined for a particular group. Not displaying irrelevant information cuts down on support and improves ease of use. Speaking of ease of use, end users can create dashboards--small representations of favorite reports. These dashboards can be shared, and default dashboards can be assigned to a user ID as well.SiteCatalyst 10, Omniture, (877) 722-7088, (801) 722-7000. www.omniture.com

Emily says: One of the key product differentiators for HBX is that it can import data from the data warehouse to a customized Excel template using the "Report Builder," which is a Visual Basic Excel plug-in. I'm not partial to VB plug-ins, but it works well, though some actions caused odd results. This feature, though somewhat difficult to master, makes it possible for an organization to customize how it sees the traffic data and can automate the creation of otherwise labor-intensive custom views for business managers. The regular browser interface is well-designed and offers something for everyone. HBX conducts Web-based training classes every Friday that any client can join. The class is an excellent introduction, and I love that it happens regularly. It's great to see what kinds of questions users from other companies are asking.

This 8-year-old company's product should be on anyone's shortlist: HBX has the analytics, the support, the response, the price and the No. 1 biggest sizzle feature of the products we tested, bringing it within two one-hundredths of our Editor's Choice score.

What is the biggest sizzle in Web analytics? HBX's Report Builder. Hands down, this got the most and biggest oohs and ahs from end users. It's brilliant in its approach as an ad hoc report writer because it uses the interface everyone is using: Excel's. Admins can create reports, while users get nifty spreadsheets that are updated automatically with current Web analytics information.

We downloaded and installed the Visual Basic Excel plug-in (a slow 10-minute process for us) and, after running through a couple of provided tutorials, began to have a clue. To create reports, you must have Excel 2002, though Excel versions back to Excel 97 can read reports.We took a couple of short wizard walks through the process of selecting analytics fields using browsing and drop-down menus. Field creation and any associated processing is done on the HBX service and automatically downloaded on demand and/or at prescheduled intervals. The resulting data is manipulated and displayed like any other spreadsheet data in Excel.

The process wasn't without its troubles, though. We frequently had VB errors that didn't seem to have all the error coding necessary to politely announce themselves. Granted, it was often our mistakes that summoned the error demons, but we worry about end users' ability to work trouble-free.

Active Viewing is HBX's browser-overlay tool; it also works as a plug-in and showed us click-stream and page-usage stats using translucent overlays. Active Viewing varies the color intensity in the translucent overlays to show rank or total number of clicks. The data is collected and displayed in near real time, as of the last data collection. We were able to display a 92-day history--handy for trending. Active Viewing also shows reports, such as total site traffic, and specific page statistics. Its click analysis includes the Top 10 clicked links, as well as all objects on the page and their ranks. Like SiteCatalyst's ClickMap, Active Viewing showed us pages referred from and going to, though in table form, whereas SiteCatalyst provided a tree diagram. Active Viewing also has a nice view of expired links in a table below the browser view.

All the vendors attempted to provide funnels, a graphic for the display of path analysis--like the pages leading to a purchase or some other result--but HBX provided more information than most. Each level or Web page in the desired sequence tracks how many users make it to each page in succession. At the second layer of this analysis, HBX includes more detail as well, showing where visitors went when they left the funnel.

We also liked HBX's internal-search and failed-keyword reports, which tell you which keywords visitors searched and which failed to return results. And we appreciated HBX's contextual online help. When we looked at the report for unique visits, for example, the help utility clearly defined a unique visit and told us how it differs from weekly and monthly visits, with an example. Short, sweet and to the point.HBX includes a helpful glossary with mailed reports. When we ran a content report and distributed it as a PDF attached to an e-mail, those on the receiving end could readily determine what the report portrayed, with a glossary describing each column header. HBX wasn't unique here--SiteCatalyst and WebTrends also add helpful descriptions.

WebSideStory support from a live individual was not as forthcoming, though, compared with other services. Initially, we called the toll-free support number listed on the vendor's site and got an answering machine. Several hours later, we called back and discovered the number was to the NOC and for after-hours support only. We got the right number, which took us into the main PBX voice prompt and from there to tech support.

HBX's performance was occasionally spotty, making us wait as long as 20 seconds for reports. At least the interface displayed text indicating that processing was taking place and we should cool our jets.

HBX access-control granularity is divided into three functions: administration, report generation and site access. This is a simple setup, but it provides enough control to be useful. HBX doesn't provides strong-password enforcement or allow temporary users, but does expire passwords after some time.

HBX's lack of group management is a real downer and may be a problem for large organizations. If your user base is of manageable size, the per-user access control is functional, permitting administrative or read-only access on a per-site basis or custom-tailored access to particular reports. We chose to allow just a couple of the general categories of reports, which include campaigns, content, custom metrics, dashboard, pages, referrers, segments and system. Along with this user access, HBX adds personalization in the form of dashboards, as WebTrends and SiteCatalyst do. In addition, user access/personalization can prevent a user from seeing Google refers, for example, while allowing those from Yahoo. We also restricted user access to selected specific report types; for example, in the visitors category we chose time zones, languages, states and cities. This let us define granular report access on a per-user basis. Still, having to define access for each user for each site got tiresome fast. Group access is needed.HBX On-Demand Web Analytics, WebSideStory, (877) 2BUY-HBX, (858) 546-0040. www.websidestory.com

Emily says: This product is one of the best. The support engineers understand analytics and give wonderful advice. Drawbacks include difficulty navigating between data among Web sites and combining data across multiple Web sites. I attended a regional user group session where users get together and talk about their experiences. These sessions are fun and useful, and not opportunities for the consultants who host these sessions to do sales pitches. Service throughout the trial was excellent. This product can be run in-house, which is a great benefit for companies like financial institutions, for whom data security is a top priority.

NetIQ's venerable WebTrends is offered as a service, which may surprise some. NetIQ's history of log analysis has pigeonholed the company and is aggressively used by competitors to brand it as "old" and for "small and simple" installations only. That's just plain inaccurate. NetIQ has branched out to offer small business-centric log analysis, tag-based JavaScript products and the service we tested, WebTrends On Demand. WTOD's maturity shows in its documentation, help and support. The company has a large installed base, active user groups in most cities and well-written documentation that has been polished over the years.

The WebTrends software product, which is used to provide the service, can partition a site, making it possible to analyze traffic from various viewpoints. We could create segments, of course, by tag ID, URL and domain. An interesting segment by URL query teased us into thinking we could create dynamic partitioning to answer ad hoc queries about intersite traffic. No such luck. Still, with WTOD there are other ways to skin the data-analysis cat. Data Source Splitter, for example, creates log files that can be used to create sub- or supersets of files. Another segmentation method is to use profiles, called "parent-child," that let you sort analysis-bound data into subgroups based on URL, domain and tag.

WTOD's ad hoc report writing is good. It's divided into configuration and design. With configuration, we chose pages to track, title and data sources. The design functionality resembles a conventional ad hoc report writer, with content, graphic and access choices. WTOD comes with plenty of canned templates and reports, and all can be copied and modified. Because the access is read/write/delete, users can make templates their own with abandon.SmartView, WTOD's browser-overlay report, is like those in SiteCatalyst and HBX, but took much longer to install and use. Logging on was painfully slow at times, but a bigger problem was the eons it took to install, even though our JVM was there. In addition, we got no handholding during the lengthy install, just the browser progress bar. That's bound to create a few support calls.

SmartView is closely tied to the overall WTOD architecture, so profiles set up to track particular domains can translate to SmartView collections. We configured one and found it useful. However, unlike IBM SurfAid, which displays data collected every few minutes, SmartView displays data from the previous day only.

Unique to WTOD is its Express Viewer desktop Web analytics monitoring application, which looks like an MP3 player, with different skins and display options. Downloading and installing this app was a slow process, but we were rewarded with output configurable to show graphs or a ticker tape. This is a quick, easy way to grab specific tidbits of data.

WTOD's interface requires a little too much back and forth to navigate, making us jump from administrative interface to reports. Once in the reporting section, we could get around and make our changes easily. The interface response for report viewing was comprehensive, and we were spared the aforementioned molasses-on-a-cold-day downloading rate. WTOD provided good granularity for all report types and functions within the interface. Thirteen different areas cover a full range of report creation, generation and account administration options.

WTOD's Profiles let us define Web site hierarchies. To test this, we built profiles, with Network Computing's and Network Magazine's sites as children to a CMP Web site. Then, we could use any one of these profiles as a filter for viewing reports through a simple pull-down list. Nice.WTOD's help is the way all help should be: contextual, summarized and linked to related, detailed information. As with HBX, though, the personal touch was harder to come by: We had a problem getting a user ID to activate and were directed to tech support for help. After waiting on hold for a half hour, we left a voice message. Can you hear us now?

WTOD provided the user-access control we expected, with predefined controls for account, data, reporting and administration, but it was a head-scratcher why we had to go into another part of the interface to edit user rights after creating the user.

As with HBX, WTOD lacks delegated control and group control, so users must be set up within each profile. This isn't that big a deal if your user community is small and static, but if it changes much you're going to be hating life. WTOD does have admin policies for strong-password enforcement and password expiration, but no temporary users.

WebTrends 7 On Demand, NetIQ Corp., WebTrends Business Unit, (800) WEBTRENDS, (503) 294-7025. www.webtrends.com

Emily says: The SurfAid interface includes a query tool that enables creation of custom reports on the fly for stats fiends and several point-and-click reports for novice users. The interface is fast, simply designed and well-organized, but it doesn't cater to users who need guidance for more in-depth analysis. This tool would be great for companies that centralize Web site analytics.Because it comes from Big Blue, we thought SurfAid might be somewhat unwieldy, but we needn't have worried. The service responded nimbly, and we found features and functions without having to plow through Red Books or those eye-drooping architecture documents.

LinkTracker, IBM's browser-overlay view, is new. Like its rivals, LinkTracker showed us a number of clicks, ranked with percentages of total clicks and views. Typical of SurfAid, the defaults for the LinkTracker application are exposed and can be changed easily. For example, we set the ranking based on activity since the previous day, from the start of that day to the most recent five minutes. HBX and SiteCatalyst present real-time data, but not with this granularity. As with the other products, the translucent browser view is accompanied by configurable graphs and tables that show overall site traffic.

We created a couple of LinkTracker apps for different time periods and compared day-to-day traffic. We could have e-mailed these reports to our users, who would only have to open the application to view the metrics--no fuss, no muss, no support calls.

"Tons o' Tools" would be an apt nickname for SurfAid. Its Data Analysis Java applet is serious and most useful. We used it to create ad hoc reports on visits, refers and content data. All collected data is available in table and chart formats; and areas of analysis, such as content category, referral and page views, are used to organize measurements, like unique visitors or page views divided by hits.

We easily selected some metrics and ran a report. It took some time and concentration, however, to read the comprehensive online help files, so this functionality may be beyond some end users.The Pathing from Anchor tool, another Java applet, let us choose a page and date, then look for the paths visitors took from that page up to a specified number of pages. This is useful for figuring out behavior from a particular location.

Yet another handy tool, similar to SiteCatalyst's A/B reporting, is SurfAid's Category Comparison app, which displays a metric from one day compared with an earlier period. We could compare Wednesday's pages with Tuesday's, for example, with results that increased by 30 percent or more displayed in red, between 20 percent and 29 percent in yellow, and less than 20 percent in green.

SurfAid's Real Time Monitor is unique: It shows performance data from different time zones. Metrics that can be monitored include visits, bad data, exit links by visits and repeat visitors. The report can span multiple hours up to two days and can be updated as frequently as every minute in a--you guessed it--Java window. Once historical data is displayed, it can be updated at whatever value you choose. The data also can be exported to Excel and saved as a graph. You can deliver reports as Excel worksheets automatically through the report scheduler.

IBM offers a business-integration module that tracks conversion and related values, but it's not part of the base package. Because we didn't focus on campaign analytics--and it cost more money--we didn't install it.

SurfAid doesn't have contextual online help, but the user guides are easy enough to use, and we found info on everything we needed without much digging. It would be nice, though, if a help button were available, together with screen dialogues providing information about particular tasks. Another nit is that the SurfAid interface is slow to load.SurfAid's simple and straightforward user setup had us on our way quickly. By default, six access roles are predefined. Access roles contain a security-administration role apart from regular administration--a separation of duties that many IT shops require.

Group access administration is effective, offering simple setup and granular control. SurfAid spreads control among areas of analysis, such as visit frequency, traffic type or referrals; measurements, such as hits/visits, page views and visitors per minute; filters, such as limits based on sites or directories; and reports. We created reports, then included or excluded them from specific user views based on group membership. With the group-report control, we specified which reports users could export.

We easily created groups and defined any combination of areas of analysis, measurements, filters and reports. Users cannot be members of more than one group, but it's so easy to create groups and grant access that this isn't a huge deal. Personalization consists mainly of showing available reports and a "favorite report box" to which users can easily link.

SurfAid's report-writer interface is a point-and-shoot affair. Although simple to operate once we got the hang of it, it's a tad more complex than those of rivals. SurfAid groups reports into folders and groups, and can show you all reports run month-to-date, regularly scheduled reports, reports delivered over e-mail and reports run on an ad hoc basis. Admins also get a list showing the cost per report and total cost.

SurfAid's Category tool is a Java app for partitioning Web sites. The hierarchy is divided among five areas: content, referral, traffic, time and visitor information. We divided each into separate collection buckets by naming folders or subcategories within each of the five basic areas; we then defined URL match strings to sort out the data. The process was simple and flexible, and we were able to create nested groupings.IBM SurfAid Analytics, IBM Corp., (817) 962-7873. www.ibm.com/surfaid

Emily says: This is a product to watch. Manticore is soon to be (or already is) a graduate of the Austin Technology Incubator. Nick Walker and others have produced an excellent product that integrates Web traffic analytics, e-mail campaign tracking and ad tracking. The interface boasts good-looking and quick-loading reports. The best function is that system users can rename/correct title tags associated with the URLs within the Virtual Traffic Master interface. This is a useful tool, as title tags often don't describe a particular page accurately, and the traffic analyst may not be able to correct the title tag on the Web page. Manticore's support for our trial was better than any other product's: accommodating, flexible, curious, extremely competent and fun to deal with. Also, the code worked well right out of the box.

Showing real leadership, Manticore has broken out of the mold of just reporting Web usage and taken a step toward correlating traffic. E-mail integration is Manticore's longest and strongest function. We didn't test this area heavily, but we ran a simple test where we sent e-mail that was tracked back to the Web site. A cookie placed on a visitor's workstation is shared between the e-mail application and the Web browser. This lets you ID those visitors who receive an e-mail from you and then go to your site. You receive detailed information on e-mails opened and click-throughs, and can identify visitors by e-mail address.

Manticore claims that even if cookies are not shared, Virtual Traffic Master can track e-mail-open rates from any e-mail client that can render HTML e-mail and is not explicitly blocking images.

Virtual Traffic Master didn't offer any browser overlay reporting. We could compare data over time using the typical date ranges. For example, we could display global traffic between Network Computing's and Network Magazine's sites and see the pages with the most hits.It's easy to navigate and change reports in Virtual Traffic Master. Nice touches include ever-present toolbars at the top of each report and tabbed navigation between major categories of reports, such as visitors, pages and demographics. The other services provide the same tools, but Virtual Traffic Master's interface is snappy, and the info pop-ups give new users some clue about where to start drilling into the data.

Manticore's help is always at hand, and a well-designed interface made it easy to figure out. However, the help isn't context-sensitive.

Logging on to the service could be a very slow process, and once we tried unsuccessfully to log in during the middle of the day--a notice announced that database maintenance was in progress. Oops.

User setup was a breeze, thanks to default user, power user and administrator access privileges. Site-access restrictions are missing, though. Interestingly, Manticore does support restricting access by domain or IP address. For example, if a user has rights to get into the reporting interface, and the firewall is open to his or her home broadband connection, you can restrict viewing if he or she is not logged on to an IP domain within the enterprise network. Virtual Traffic Master doesn't allow temporary users or enforce password creation, but it can expire passwords.

The service also doesn't support group access, but offers a couple of personalization options: "My Manticore," a collection of favorite reports easily navigated to via a top-level tab, and Dashboards, which are miniature versions of reports offering a summary view; when we clicked on one, we jumped to the full report. Dashboards typically are set up by each user but can be shared, with full rights to modify or read-only rights.Virtual Traffic Master 4.0, Manticore Technology, (866) 626-8426, 512-305-0270. www.manticoretechnology.com

Intellitracker Maximiser

Maximiser is a full-featured analytics service that was quoted to us at the unbelievable price of just $27,000 per year. But aggressive pricing couldn't make up for only average reporting, administration and ease of use.

Emily says: This is a good product. A minor but distracting aspect is that within the interface, a summary report is called a "Headline." At CMP, the term headline refers to the title of an article. Most CMP users look at the traffic reports to find the most popular articles, which are generally referred to by their headlines.

Maximiser is a full-featured analytic service that was quoted to us at the unbelievable price of only $27,000 dollars annually. But aggressive pricing couldn't make up for only average reporting, administration and ease of use.Although it's missing some tools offered by rival services, notably a browser overlay, Maximiser provides nice tracking with lots of date options, including a range for the penultimate month, but no categorization for various domains by page views. Maximiser's report interface requires some training but is not especially difficult to use. A few times our reports were missing the graphic, and instead displayed a message for us to wait as licensing was downloaded. A redo of the report seemed to alleviate the hiccup. Scheduling reports for distribution and defining who should receive them was a breeze.

Maximiser does have context-sensitive help, but it's not very detailed. For example, when we sought to learn more about "Activities" in the administration setup area, we got a help window explaining that we could create and edit new Activities ... without telling us what constitutes an Activity. The user guide is a Word document without an index.

User management was simple to set up, but we missed being able to dictate account rule settings, such as password length or how often they should be reset. Strong-password enforcement, password expiration and temporary user IDs are lacking, too. Maximiser does offer preset user permissions for assigned roles, which include administrator, observer, power user and user. The help files didn't specify the roles' default rights, however, and the roles are static.

Personalization is good. We quickly set date, language, graphics and filters. The three filters types are marketing, event and visits, and when applied, limit reports to data meeting those requirements. For grins, we set an event filter for those loading a recent Network Computing article on "Cheap NAS" (gotta track your storage users!). It was a simple process of plugging in the URL and giving the filter a name.

Maximiser, Intellitracker, +44(0)20-7667-6854. www.intellitracker.com or [email protected]ThinkMetrics InSite

It only works on a single site, but at the rock-bottom price of $50 per month per 100,000 page views and with no contractual requirement, ThinkMetrics' InSite basic version is a great way to dip your toe in the analytics water. Of course, you won't get segmentation for multiple domains, sites and directories, and its reporting is limited.

Emily says: This is a good, basic, inexpensive Web analytic tool. It's lacking in path analysis reports and browser data, but it does have good top referrer, top URL and search engine key word reports.

It only works on a single site, but at the rock-bottom price of $50 per month per 100,000 page views with no contractual requirement ThinkMetrics' InSite basic version is a great way to dip your toe in the analytics water. Of course, you won't get segmentation for multiple domains, sites and directories, and this also limits the possible reporting. For example, in trying to determine the top refer for multiple sites, ThinkMetrics suggests placing the same tags on both sites. Now, pricing in this case is per 100,000 page views and all the vendors have volume-driven price sensitivity, so though it's true that it will cost more because of the additional site and the additional traffic that site will drive, this is true for all the services we tested. What is unique here is that multiple sites, even when tagged, will look like one large site.

This doesn't mean that there are no advanced features; in fact, InSite's reporting on visitors with the most activity showed an impressive ability to list four time spans, ranging from seconds to years. However, InSite's help has no context, no index and isn't going to offer much help for those who really need it (read: end users). It's a PDF without bookmarks or index, suitable only for those hardy souls willing to RTFM. Forget about context or quick tips as well.On the bright side, we had no complaints about ThinkMetrics' responsiveness. We enjoyed downright snappy service.

ThinkMetrics doesn't offer an administrative account, so we couldn't set up other users or craft any access controls. We decided to look on the bright side: No administration duties.

InSite, ThinkMetrics, 0044 1727 858 427. www.thinkmetrics.com

WebAbacus 4.1

WebAbacus delivers good analytics with an intuitive interface, but its administration, toolset and cost were only average, hurting its overall score.

Emily says: Initially we had a bit of trouble getting the WebAbacus trial set up due to vendor personal emergencies. I never did get any training on this product, but managed to figure out the interface. Reports load quickly and data is organized logically. The greatest thing about this product is that you can apply a filter of a referring URL to any report. For instance, one could view the top URLs viewed by users referred from a single source, as well as the types of browsers and operating systems the users from that referring site are running. Additionally, it offers reports on pages that garner the most 404 errors. The main drawback of this system is you can't create summary reports that aggregate data for more than one Web site.

WebAbacus delivers good analytics with an intuitive interface, but its administration, tool set and cost were only average, hurting its overall score. For example, WebAbacus lacks browser overlay and report comparison tools.

Report creation in WebAbacus was not intuitive nor similar to the other products tested. Reports are displayed as simply a line of text. An icon on that line can be selected to show graphics, and the text can be edited to change selection criteria.

A PDF without bookmarks or index was our only guide through WebAbacus. Some online help is needed to support users in a larger deployment; otherwise plan to buy a comfortable headset and find a place to prop your feet because you'll have lots of people calling.

On the bright side, the interface was speedy, though occasionally we couldn't get graphics to render in our view; only the tabled data was displayed. WebAbacus supports user access from its site--we were not entrusted with administrative control. It also doesn't support Mozilla, only Navigator 6 and IE 5.5 and above.WebAbacus 4.1, WebAbacus, +44.2073.243070. www.webabacus.com

NetRatings SiteCensus

NetRatings, previously known as Red Sheriff and then as Nielsen//NetRatings, is a joint effort by Nielsen Media and NetRatings. Its SiteCensus offers simple controls and deep analytical reporting but lacks the segmentation capabilities, advanced tools and administrative controls found in rivals. Nielsen is in the process of combining its services since acquiring Red Sheriff, but in the meantime, there is some clunkiness.

Emily says: NetRatings' suite of products offers inexpensive tools for all sizes of Web sites and business organizations. Recently bought by Media Metrix, the company seems to be focusing on meeting the analytic needs of content-driven Web sites. We have had Red Sheriff running on several of our Web sites, and users are very pleased with the service. One of the best features of the product is that it can integrate data collected from surveys with traffic data. However, it's difficult to navigate from one reporting suite to another. Also, the interface is not tailored for an organization that plans to implement the product on 150 Web sites.

NetRatings, previously known as Red Sheriff and then as Nielsen//NetRatings, is a joint effort by Nielsen Media and NetRatings. SiteCensus offers simple controls and deep analytical reporting but lacks the segmentation capabilities, advanced tools and administrative controls provided by rival services.SiteCensus' interface isn't difficult to use, but the reports took a bit of study. For example, we ran top path reports, and the graphical output was good, but we had to know what stories or objects the URLs related to if we hoped to understand the significance of the report. Other products segment site information into areas; IBM and WebTrends do a great job of this, letting you easily rename pages using HTML title tags so they are more meaningful. Nielson is in the process of combining its services since purchasing Red Sheriff, but in the meantime, there is some clunkiness. For example, when logging in from our home page, a subsequent page was presented to choose one of the Nielsen//NetRatings' products, then the user ID and password were evaluated.

Neilsen//NetRatings' interface was sometimes slow, especially when logging in. This could be attributed to the recent purchase of Red Sheriff and the current cobbling together of various products.

SiteCensus' user access is simplistic, with an e-mail confirmation of user additions. However, our new users didn't show up right away, and the system let us continually add the same user ID. We didn't like receiving an e-mail showing the user ID and password in plain text--a little too trusting for us. And passwords are shown in plain text on the administrative site. Only a single user can have admin access, and as you might suspect given the loosey-goosey attitude about security, there are no strong passwords.

Nielsen/NetRatings SiteCensus, NetRatings, (212) 703-5900 (East Coast), (408) 941-2900. www.netratings.com

Watchfire Corp. WebXM 3.0 Analytics ModuleWatchfire, a purveyor of Web site usability, privacy and compliance products, offered its Web analytics service for review. But its lack of multisite analytics, high price ($99,000) and required three-year commitment didn't compare well with rivals, landing it in the basement.

Emily says: This would be a great product to implement on one big Web site. In-depth page error data (including spelling errors on pages) is integrated with the Web traffic data. The problem with implementing this on 150 sites is it would take too long. We had difficulty during the trial getting it set up even after countless conference calls and technical noodling, so I can't imagine we'd ever get it working correctly on all our sites.

Watchfire, a purveyor of Web site usability, privacy and compliance products, offered up its Web analytic service for review. But its lack of multisite analytics, high price ($99,000 dollars) and required three-year commitment didn't compare well with rivals, landing it in the basement on this review. Watchfire did tell us that multisite functionality will be added in November.

Watchfire does offer a browser-overlay view. While similar in form to the other products that provide this functionality, it lacks the cool translucent overlays available in SiteCatalyst and HBX. One nice feature is the "tear-off reports" that float outside of the browser. These icons are helpful for comparing data across multiple reports because they don't take up much space up on the desktop. Watchfire's online help is slightly better than that of ThinkMetrics--it's a PDF with bookmarks and index. If the launching of the help PDF was contextual, opening at the location pertaining to the screen we had questions about, then we could see it being used. But it's not, and it will be a cold day in userville when an inhabitant reads through a PDF when help is only a phone call away.

Watchfire's user management is different from most. Instead of having the typical user name, password, access-type dialogue, it only wants a name. No strong-password enforcement, expiration or temporary users. And browser support for the management/report console is limited to IE 5.5 or greater.WebXM 3.0 Analytics Module, WatchFire Corp., (800) 282-5951, (781) 810-1450. www.watchfire.com

Bruce Boardman, executive editor of Network Computing, tests and writes about network management and systems. He has 12 years' experience managing networks and distributed computing for a financial service provider. Write to him at [email protected].

Our testing involved tracking live visitor traffic on the www.networkcomputing.com and www.networkmagazine.com sites. We ran unique JavaScript tags, one set for each site and a common tag that ran on both sites, giving each site a total of two tags per vendor. The individual tags communicated with each vendor's production service, giving us specific visitor information for the site individually as well as global traffic across both sites. Initially, we were unsure of how much overhead tagging would place on the sites and used Gomez's GPN service to track the tag download time. None of the tags for any of the vendors required much more than a few seconds to download, even over slow dial-up links.

Here's a sample tag:

Each vendor's tags gathered data for one week on our production sites. So, despite the length of our tests--more than 12 weeks--we collected only a week's worth of data for each vendor, reducing our ability to compare trending reports. We did maintain management interfaces on all the products for the entire time to fully explore and compare feature sets.This is the first product analysis Network Computing has undertaken with our parent company, CMP Media LLC, and we made it clear to all participating vendors that CMP may come to different conclusions.

Like any enterprise, CMP has specialized requirements, whereas we at NWC strive to present a complete analysis of technology products to help readers make decisions based on their own particular circumstances. Serendipitously, however, CMP reached the same conclusion we did.

Although WebSideStory's HBX made it onto the company's shortlist, Emily Sunderman and her team, along with select users from all of CMP's business groups, selected Omniture SiteCatalyst, our Editor's Choice. They liked that it's well-suited to a wide range of end users and offers tight administrative control.

We receive about 1 million page views per day by 350,000 unique visitors across 150 Web sites. Growth has been a steady 10 percent annually and is projected to remain so for the next five years. This universe of 150 Web sites comprises seven business groups. Each group needs global information about each site within its group as well as where the entire group is positioned in the universe. We need 150 user logons with distinct permissions. Availability and speed of the service must meet 99.5 percent availability and average load times of Top URL reports of 10 seconds over a 56-Kbps dial-up link. A service-level agreement must be included ensuring these performance thresholds. We are converting from an in-house system and will need six months of log files converted. This is a single file requiring just one conversion script. Log-file conversion is not a deal-breaker, but please indicate whether your price quote includes this conversion. Please specify price below, and which of the following line items are included.

R E V I E W

Web Analytics

Sorry,
your browser
is not Java
enabled

Welcome toNETWORK COMPUTING's Interactive Report Card, v2. To launch it, click on the Interactive Report Card ® icon

above. The program components take a few moments to load.

Once launched, enter your own product feature weights and click the Recalc button. The Interactive Report Card ® will re-sort (and re-grade!) the products based on the new category weights you entered.

Click here for more information about our Interactive Report Card ®.