Lab Tested: Automated E-mail Archivers

The technology is too young, and the number of functions that must be pulled together--including records management, content analysis, indexing, search and retrieval, policy engines, tamperproof storage, security, access controls and a tight integration with the mail store--are too numerous for any vendor to do all of them perfectly. But you can help set reasonable expectations going into the project.

For our tests, we required real-time, active e-mail archiving products that support both Exchange and Domino, let enterprises manage e-mail storage, aid compliance with retention requirements in the United States, and allow for both discovery and retrieval of messages. The archives must provide access mechanisms for both compliance officers and users, and must support retention policies as well as automatic content analysis, including attachment content. The ability to age messages out of the archive based on retention policies is vital. The product's message-classification scheme must allow metadata for each record, but the products also must be capable of classifying messages without metadata. Finally, we asked for as-tested list pricing for both 200 and 1,000 seats, and for maintenance costs as a percentage of the purchase price.

Before you perform pilot testing, work with your constituents to define industry-appropriate e-mail retention policies. After all, the ability to implement your policies is the most critical component of the evaluation. If the product you're evaluating is incompatible with a particular policy--say, one requiring you to automatically purge all e-mail from the archive, mail store and any end-user storage after three years, except for messages that might be subject to court-ordered discovery--look elsewhere, no matter how whiz-bang the user interface. For guidance on establishing a policy, see "The Trouble With E-Mail," page 40. How We Can Help

We asked 10 vendors to supply software that would work with our Microsoft Exchange test bed and support Microsoft's SQL Server as the back-end database (see "How We Tested E-Mail Archiving," page 52). Seven vendors said they were interested, but only products from EMC, Waterford Technologies and Zantaz met our criteria and could go the distance with our rigorous test plan.

Although iLumin Software Services sent its product to our Syracuse University Real-World Labs®, the company decided at the last minute that it didn't have the resources to participate. ZipLip was in the middle of a personnel transition and couldn't meet our deadline for delivering software to the labs. IBM and Vignette didn't have offerings that met all our test criteria. We look forward to testing these products in the future. AXS-One, Feith Systems & Software, and Ixos Software never replied to our invitation.

Our three participants worked closely with us during the review, starting with on-site or WebEx-conferencing installation services and continuing with training and support as we put the products through their paces. We judged the offerings based on configuration, scalability and architecture; multiple user interfaces; price; features; and compliance support. Our test e-mail store included both historical records and a steady stream of live e-mail.

The three products have a lot in common. All use Exchange's built-in message-journaling (Exchange 5.5) or message-archiving feature (Exchange 2000 and 2003) to grab inbound and outbound messages. They also can archive mail stored on users' hard drives in local Outlook data files (PST files). Each uses a combination of a file store for data storage and Microsoft's SQL Server for metadata, and all can work in both Exchange and Domino environments--Zantaz added Domino support this year. Each product interfaced with our Exchange mail store using Microsoft's MAPI, which means they all require an Outlook installation on the archiving server. They all include a Web interface for searching the archive. This class of software faces daunting technical hurdles. Electronic documents, including e-mail, can exist in every nook and cranny of your business. PDAs, smart phones, local Microsoft Outlook files on notebooks, removable memory cards and backup tapes all hold copies of e-mail. Technology alone isn't be enough--you also need strict policies to control where e-mail administrators and employees store e-mail--but your retention system must be flexible enough to address these concerns.

There's a lot of legitimate e-mail out there, and the volume is increasing dramatically. In 2004, the average corporate e-mail user processed 10 MB of data per day, according to the Radicati Group. By 2008, this figure is expected to rise to 15.8 MB. The good news is that all the e-mail archiving systems we tested off-load storage from your mail servers, reducing the size and number of servers you'll need.

Some e-mail archiving products, including Zantaz's Enterprise Archive Solution (EAS), make it possible for each user to decide which e-mail gets archived and which doesn't. There are big problems with this strategy. First, it assumes that all users are informed about company retention requirements, are kept up-to-date as requirements change and faithfully apply the same criteria to every message received. Second, it ignores the fact that the very e-mail likely to be sought during court-ordered discovery is what a particular employee probably chose not to archive--for the same reason it's now the target of discovery! As we say in our market analysis (page 36), you must suck it up and archive all e-mail--even stray Viagra ads that evade your spam filter.

So you'll require storage, and lots of it. The amount depends on your organization's typical e-mail usage pattern, both inbound and outbound. That's one more study to add to your to-do list before implementing an archiving system. The products we tested reduce storage needs by eliminating duplicate messages (single-instance store) and compressing messages. Because the bare minimum storage requirement is three years, with some industries stretching that to seven, your storage system must be scalable and flexible to keep up with growth in the amount of e-mail sent and received.

Two other important considerations: For search and retrieval to work, the storage space must be online; and for automatic deleting after the e-mail ages out, the storage space must be rewritable.

Let the Testing Begin

The degree of difficulty in setting up, managing and using the three products we tested varied widely. On one end of the spectrum, installation, configuration and administration of Zantaz's EAS are not for the faint of heart; you'll need to shell out for professional services. On the other end, it took us all of one minute to install MailMeter Forensic on our archiving server, and just 45 minutes to connect and test all the plumbing between our Exchange system and MailMeter using the two control-panel configuration wizards. Just about any organization will be able to get MailMeter up and running without a consultant.

In the middle of the pack is EMC's EmailXtender. Although we had a support technician in the lab during setup, most organizations will be able to get this product up to speed in a reasonable time frame without professional services.

When you're dealing with a large number of constituents, reporting is key. Zantaz EAS includes a number of canned reports that we could run from the administrator application. EMC's EmailXtender also has a limited number of canned reports. In both cases, we could create additional reports using Crystal Reports. But of the three products we tested, Waterford's MailMeter Forensic was tops in reporting. Its Web-based reports and search module, the Report Center, made it easy to pull together information about enterprisewide mail usage patterns and conduct detailed content analysis of mail conversations between two parties. Compliance support accounted for 25 percent of our score, and we gave Zantaz EAS a 5 (out of 5) here. It has the most flexible method for defining policies and rules, and it is the only product that lets you automatically delete e-mail that needn't be retained.

The 55 percent of our reader poll respondents who cited cost as the No. 1 reason for not implementing an archiving system should take a look at Waterford's MailMeter Forensic, especially if they're managing 1,000 or more desktops. As-tested pricing for MailMeter at that level was just $24 per desktop, compared with $40 for EMC's offering and $55.25 for Zantaz. However, heavily regulated industries or those companies that just want peace of mind will find Zantaz EAS's rich feature set worth the premium price. Indeed, when all was said and done, the Zantaz product was our Editor's Choice. Although it requires significant training, it automated the procedures we identified as critical, including retention and deletion, and has a flexible, powerful policy engine.

Waterford's MailMeter Forensic and EMC's EmailXtender made it easy to archive e-mail as it was delivered or received. But when it came to removing e-mail by date, both forced us to use manual methods, hurting their compliance scores.

Enterprise Archive Solution started life as Educom's Exchange Archive Solution in March 2000; Zantaz acquired the technology last year when it bought Educom. The EAS suite comprises EAS for Exchange, EAS for Notes, EAS for Files, EAS Search, EAS Storage Manager and EAS Supervisor. Its Exchange, Notes and Files components are the base archiving products for Exchange, Notes/Domino and file shares. EAS Search enables full-content searching, while EAS Storage Manager provides policy-based archive and retention rules for automating archival storage. EAS Supervisor enables monitoring of policy and regulation compliance through real-time review and post-process monitoring.

Zantaz sent us EAS for Exchange, EAS Search and EAS Storage Manager. The combined price came to $84 per user for a 200-user configuration, more than twice as much as rivals, but the comparatively high price is offset by a rich feature set that met all our criteria, including active archiving and automatic retention/deletion. With the help of an on-site Zantaz engineer, we created the rules and tasks needed to archive and age out messages automatically. The cost of the engineer is not included in the product price for any of our entries, but Zantaz told us all customer installations are done by the vendor or one of its resellers, unless the customer is Zantaz-certified to do installation. Pricing for this assistance will vary widely; check with your local reseller. Without the Zantaz engineer, we would have spent several days wandering in the desert, so plan on budgeting time and money for training and consulting services.

It was especially frustrating to find that this product, unlike the others we tested, offers no simple way to "turn on" archiving tasks. To get EAS to archive continuously, we had to create and schedule an archive task to run every day around the clock. And to run a near-real-time indexing task continuously, we needed to create the task, tie it to an archiving task and edit a configuration file. Standard procedures like these could easily be presented as check-box options.

EAS is extremely flexible, and most IT pros will accept complexity as a trade-off. But in EAS' case, some of the complexity could have been avoided. For example, the EAS Server can be run as an application or a service in the Windows server environment. We wanted to run it as a service, but the installation routine doesn't install EAS as a service even though we believe that would be the preference for most system administrators. Zantaz documents a manual process that we ran from a DOS prompt to install EAS Server as a service. Although not a big deal, this is characteristic of the gotchas we encountered while working our way around this product.

On the other hand, Zantaz adds significant value to EAS by including a slick utility for creating, editing and testing policies and rules. It uses a BASIC-like formula language that provides an exhaustive set of message variables; output variables and return values; operators; and arithmetic, string and date/time functions. There's power here, but you must know how to harness it--context-sensitive online help would have shortened the time it took us to code our archive and retention policies.

After trial and error, we did set up comprehensive, detailed rules to enable our test policies. Example formulas are available as templates, and these were useful. We also appreciated that we could test the formulas we entered prior to putting them in production. If, after taking some time to train, you can't define your policies and rules using EAS' formula editor, we suspect the problem doesn't lie with the software. Wearing our IT manager hat, we used the EAS Administrator application to configure the system; define, execute and monitor tasks; create and manage search indexes; and create formula templates. Users, compliance managers and HR personnel will primarily access the EAS archive through EAS' fully functional, integrated OWA (Outlook Web Access) or by using EAS Outlook extensions installed as add-ons to Outlook. In either case, EAS utilizes users' Exchange logins and provides access to a flexible and powerful search interface with the familiar AltaVista search engine to return results quickly.

Zantaz Enterprise Archive Solution 4.1, as tested, $16,875 for 200 uses, $55,250 for 1,000 users. Yearly maintenance, 20 percent of purchase price. Zantaz, (800) 636-0095, (925) 598-3000. www.zantaz.com

Waterford ships three versions of MailMeter. MailMeter MBA and MailMeter Insight both provide mailbox reports, attachment categorization and e-mail statistics for management, but neither provides body-text archiving. We tested the third flavor, MailMeter Forensic, Waterford's e-mail archiving offering for Exchange and Domino e-mail systems.

Setup was a breeze, and we were soon splashing around happily in the MailMeter Report Center, which requires Microsoft's Internet Explorer 6 and provides a context-sensitive search page that made it easy for us to search for messages matching our test criteria, including e-mail addresses, domain names, subject, body text and attachment file names (wild cards are supported). Each search report provided hyperlinks, including a formatted body link and individual attachment links, that let us drill down into specific content as needed.

We also liked the Report Center's large selection of canned reports, which provided a detailed analysis of mail usage by internal and external senders and receivers, domains, address pairs, mailing lists and attachment characteristics. E-mail volume graphs were broken down by hour, day, week or month, and we could save our ad hoc reports in Report Center's Forensic Workbook folder and run them anytime. As with search reports, all of MailMeter's canned reports provided drill-down links; we found exactly what we were looking for just by pointing and clicking. We could schedule canned reports to run automatically and to send results to any e-mail recipient--handy for keeping house counsel in the loop while maintaining a safe distance. We gained access to and managed archive contents through Waterford's MailMeter stored authentication database, in which users are defined and filters are implemented. The filters are based on criteria we defined through MailMeter's Exchange Configuration Control panel application and data contained in Active Directory for each user. It was a simple task to create both administrators who had access to the entire content of the archive and users with access to filtered data.

For our tests, we assigned AD users to our editorial department or our copy desk, and gave Network Computing's executive editor access to editorial department data. This filter blocked the executive editor from accessing copy-desk e-mail using the Report Center. Unfortunately, because the Report Center's authentication material is stored apart from the Exchange system, we had to assign a Report Center password to this account and ask the executive editor to log in and change the password during his first visit. Even though the process of creating users and filtering department data is simple, it would be better if MailMeter Report Center supported user access by authenticating with Exchange.

When MailMeter Forensic archived a message, it calculated a digital signature on each component of the message: attachments, message body and an overall signature. The signature was stored along with other metadata when the message was archived and was recalculated and compared with the original signature every time we retrieved our test message from the archive. Matching signatures confirm that messages haven't been altered--an important consideration in a forensic analysis.

MailMeter doesn't support automatic, policy-based deletion of e-mail that has aged past its retention period. You can bring your database people in to craft SQL statements to delete records from MailMeter's SQL database or use the database-maintenance facility in the MailMeter Configuration control panel applet. Each method will remove the archive's metadata so MailMeter searches won't return results from the deleted records, but neither approach gets rid of the archived body text or attachments, which are orphaned in the MailMeter's file store after the metadata is deleted. Body text and attachment data are stored in compressed flat files on the MailMeter server's file system. Because the file-creation date is likely to be different from the actual message date, we could find no good method to remove the body text and attachment data. Waterford is working on providing an automated procedure for aging messages out of the archive, but for now, we're stuck with a labor-intensive and error-prone manual process.

MailMeter Forensic, as tested, $8,000 for 200 uses, $24,000 for 1,000 users. Yearly maintenance, 18 percent of purchase price. Waterford Technologies, (949) 428-9300. www.waterfordtechnologies.com EmailXtender, which EMC acquired when it bought Legato in 2003, took only a few hours to install and set up--we were managing our e-mail stores before we knew it. Of course, it helped that we had a support technician in the lab during setup, but after going through the process and reviewing the installation and administration manual, most organizations should be up and running in short order and without a significant investment in training.

During testing, the only serious problems we discovered were an unwieldy archived e-mail deletion procedure and subpar retention granularity that might force you to keep e-mail around longer than regulations or your policies dictate. E-mail retention tasks are divided among three EmailXtender applications: Administrator, Audit Reports and EmailXtract. We used the Administrator application to set up storage and to configure encryption and compression settings for the archive. This is also where we created inclusion and exclusion rules to determine which messages to archive and to set archive retention periods. We set the retention period for all archived mail to 36 months.

The EmailXtender Administrator's Months view includes a "Months to Retain" column that let us quickly identify volumes that should be removed based on defined retention periods. Because EmailXtender doesn't remove aged messages from the archive automatically, we followed a two-step manual process. First, we right-clicked on the volume and selected the option "Dispose Monthly Data." You might think this would be the end of it, but EmailXtender merely moves the e-mails you'd like to be done with to the Lost and Found storage area, where they can be recovered if necessary. To actually remove the aged e-mail, we had to leave the EmailXtender Administrator interface and use Windows File Explorer to delete the folder where the aged data was stored.

This procedure is problematic for a number of reasons. First, the smallest unit of storage that EmailXtender recognizes contains a month's worth of data. E-mail sent or received on the first day of the month logically ages out before mail sent or received on the last day. But because EmailXtender's storage paradigm isn't granular enough to recognize these differences, you'll retain some e-mail longer than applicable laws, regulations or company policies dictate. In addition, e-mail removal from the archive store can't be automated. Someone, on a monthly basis, will need to apply a multistep process to delete data manually. Although we could set up a recurring task in EmailXtender to remove mail from the Exchange mail store based on age, we wish EMC had given us the option to apply a similar automated task to archived mail removal.

The EmailXtract application lets administrators, supervisors and users perform a variety of tasks, including archiving, short-cutting messages (removing message bodies), deleting messages from the mail store, searching for mail using a number of inclusion and exclusion rules, and analyzing the mail store. Tasks can be one-time or scheduled to repeat. Short-cutting messages left a message stub in our Exchange store that included the message header information while removing the message body. We created a job to stub messages older than 90 days on a mailbox that used 128 MB of storage space in the Exchange message store. After the short-cutting task finished, the mailbox size was reduced by 121 MB--a significant space savings. The user could still see the message subject, but the body had been replaced by a message stating, "Removed by EmailXtender."

Retrieving messages that have been short-cut is a two-step process. First, we opened the EmailXtender search plug-in from within Outlook and searched for the message subject. Then, in the results windows, we right-clicked on the message and selected an option to restore the message body to the mailbox. We'd like this procedure simplified by making it a right-click operation from the message display in Outlook. We could also search the archive via an Outlook plug in and by using an included Web application. These mechanisms will let your users search for their own archived mail, saving admin costs.

The EmailXtender Audit Reports application gave us access to five canned reports: Deleted Messages, Deleted Volumes, Users Accessing a Message, Messages Accessed by a User and User Accessed by Other Users. We could use filters in each to help find what we were looking for, but truly customized reports require Crystal Reports.

EMC EmailXtender Release 4.7, as tested, $8,000 for 200 uses, $40,000 for 1,000 users. Yearly maintenance, 18 percent of purchase price. EMC Corp., (877) 534-2867, (650) 210-7032. www.emc.com

Ron Anderson is Network Computing's lab director. Before joining the staff, he managed IT in various capacities at Syracuse University and the Veterans Administration. Write to him at [email protected]. We created a test environment for each suite by setting up Exchange Server 2003 running on Windows Server 2003; hardware comprised an Intel-based server with dual 3-GHz processors, 3 GB of RAM and two 36-GB SATA drives. We set up each product in its own Active Directory domain and installed separate copies of Microsoft's SQL Server 2000 SP3. We populated each Exchange server with mailboxes for eight of our most e-mail-prolific technology editors and seeded the Exchange store with a historical e-mail store containing more than 38,600 messages dating back to July 1997. The historical e-mail came from our production e-mail environment and included a good mix of sent and received mail from each of our volunteer editors.

In addition to the historical data we used to seed each Exchange server, we configured our production mail server to mirror live inbound mail into each test Exchange server to keep a day-to-day flow of unique messages into the system. During the three months our test setup was active, 53,577 messages were added to the message store.

Because we weren't doing performance testing, we loaded each vendor's archiving framework on a virtual server using Microsoft's Virtual Server 2005 and Windows Server 2003. Each virtual server machine was a member of each product's respective domain.

After installing the archiving products, we created a job for each to archive the historical e-mail that existed on the respective Exchange Servers, as well as a separate task to actively archive all inbound and outbound e-mail on a continual basis. We configured automatic aging to remove messages from the mail store and archive those that were past their retention periods, based on our 36-month retention policy. Because two of the three products we tested don't support automatic aging, we determined what steps were necessary to identify messages that exceeded their retention date and remove them from the mail store and the archive manually.

Finally, we evaluated the products' search facilities, both from Outlook and Outlook Web Access, by searching for specific messages by subject, body text, date and attachment information. All Network Computing product reviews are conducted by current or former IT professionals in our own Real-World Labs®, according to our own test criteria. Vendor involvement is limited to assistance in configuration and troubleshooting. Network Computing schedules reviews based solely on our editorial judgment of reader needs, and we conduct tests and publish results without vendor influence.

R E V I E W

E-mail Archivers

 Sorry,
your browseris not Java
enabled 

Welcome to NETWORK COMPUTING's Interactive Report Card, v2. To launch it, click on the Interactive Report Card ® icon above. The program components take a few moments to load.

Once launched, enter your own product feature weights and click the Recalc button. The Interactive Report Card ® will re-sort (and re-grade!) the products based on the new category weights you entered.

Click here for more information about our Interactive Report Card ®.

0