Because we chose to use Windows Advanced Server instead of Windows Data Center, we were limited to a two-node cluster. Those enterprises requiring a larger cluster (up to four servers) would be better off with Windows 2000 Data Center. For a detailed list of hardware and software used for our cluster, see "Parts List for Chicago Lab Implementation." The rest of our environment was similarly configured for redundancy, with redundant Cisco Systems Catalyst switches and Microsoft Active Directory (AD) domain controllers.

Parts List for Chicago Lab Implementation Click here to enlarge

The system we received from Compaq came perfectly configured as a high-availability Exchange 2000 system, but we decided to take the plunge and get the full end-user experience for ourselves. We reformatted the hard drives, tore open the cellophane and let the software fly. The endeavor was time-consuming and sometimes frustrating, but the result was worthwhile. Because our SAN/cluster arrived preconfigured in a Compaq half-height rack, the first order of business was understanding what we had in front of us.

Compaq's HA/F200 is a set of software packages, including Secure Path, that provides redundancy for specified hardware components (see "Building a Redundant Network"). Those hardware components comprise the MSA 1000 SAN with a redundant controller, a set of MSA fabric switches, two Compaq ProLiant DL380s, redundant 10/100 Ethernet heartbeat adapters and four redundant Compaq FCA (Fibre Channel Adapter) 2101 HBAs (host bus adapters)--two for each DL380.

The MSA 1000 offers several features that make it a highly redundant yet inexpensive hardware solution for a small-to-midsize company. Some of these capabilities include DtS (Direct Attached to SAN)--which allows for a quick but safe migration from legacy RAID arrays--and RAID ADG (Advanced Data Guard), which is a RAID 5 extension that increases the fault tolerance of a storage array by creating a second and completely independent parity scheme.

Building a Redundant Network Click here to enlarge

The MSA 1000 uses standard SCSI drives, as opposed to the Fibre Channel drives used in many of the other SANs on the market. The DtS architecture lets Compaq clients cash in on previous SCSI array investments and move their current RAID 1 and 5 DAS (Direct Attached Storage) units into the new MSA 1000. The real benefit, of course, is being able to keep one's data on the DtS array while migrating it over to the MSA, thus providing for a quick and painless migration to the SAN. This migration magic is made possible through Compaq's RIS (redundant information sector).

Upon initial creation of a storage array, Compaq RAID controllers write information relative to that array's configuration to a nonusable sector, the RIS, on the hard-drive array. By letting the MSA 1000's controller read this sector information, drive arrays from legacy ProLiant systems can be moved over and placed into the MSA 1000. The RIS is read by the MSA 1000, and the array remains intact.

RAID ADG (RAID 6)

Where DtS provides cost savings through the reuse of hardware investments, RAID ADG uses a second RAID parity scheme to provide protection by reducing the risk of array meltdown (see "RAID ADG Drive"). Although a typical RAID 5 drive array maintains only a single parity stripe across its array, RAID ADG maintains two, depicted in the diagram as Pri and Sec. By using two parity schemes, an array can survive two drive failures and remain intact. Of course, we had to check this out.

We put our Exchange system under a load, pulled a drive, let the array go into repair mode and pulled another. The array balked a bit, but the mail cluster never skipped a beat. To add more redundancy, you can assign one hot-spare drive to one array or to a set of arrays. This hot spare will replace a failed drive and become part of the RAID 6 array until the failed drive is replaced. This permits three failures in any given array without any loss of data (provided the hot spare has had time to rebuild itself).

RAID ADG Drive Click here to enlarge

This SAN has a couple of downsides. It supports only one cluster per MSA 1000 and arrays can be assigned only one hot-spare drive. Take, for example, a SAN configured with five RAID arrays and three hot spares: Hot-spare Drive 1 can be assigned to Array 1; Drive 2 can be assigned to Arrays 2 through 4; Drive 3 can be assigned to Array 5. Arrays 1 and 5 have their own hot spare, but Arrays 2 through 4 share a single hot spare. An ideal configuration would let one pool of three hot spares be assigned to all five arrays.

Also, the MSA 1000 does not provide for hardware-based site redundancy. If redundant site arrays are necessary, you have two options: The MSA 1000 can be mirrored by use of special software, such as Veritas Software Storage Replicator or Veritas Volume Replicator (currently in beta), or you can turn to a higher-end SAN solution, such as the MSA 8000, which has hardware-based site replication.

Exchange 2000 Services Click here to enlarge

Service interruption (albeit slight) can occur when one server in a cluster fails over to another, though, of course, you would want to prevent a failover whenever possible. There are many reasons why a server in a cluster might fail over to its counterpart: bad RAM, dead processor, crashed hard drive, dead power supply or even a dead motherboard. Aside from the last one, we're pretty much covered. We have redundant RAM, processors, hard drives and power supplies. When any of them dies, the other typically keeps things moving. It's not that simple, however, when you have failures such as an HBA dying, fiber connecting the server to the switch breaking or even a dead switch. Windows can't simply begin rerouting storage requests to the new device upon failure. Secure Path does provide the logic necessary to let storage requests reroute to a secondary HBA once the primary path takes a dive.

Windows Clustering

In conjunction with Windows 2000 Advanced Server, Exchange 2000 Enterprise offers the clustering components to implement an HA mail solution. Windows 2000 Cluster service is a vast improvement over its predecessor, MSCS (Microsoft Cluster Server), which ran on Windows NT 4.0. One such advancement is its ability to move active applications from one node to another immediately. Exchange 2000 also touts vast improvements over its forerunner, Exchange 5.5. For example, it can operate in active/active mode--using both systems at the same time. However, not all the components of Exchange 2000 can run in active/active; some Exchange services don't support clustering at all. For example, the MTA can be executed only in active/passive mode, and the Key Management Service isn't supported by Exchange clusters at all (see "Exchange 2000 Services").

From the Ground Up

Because Exchange 2000 requires AD, we began our adventure with the AD environment. We configured one primary domain controller with DNS, WINS and DHCP, and one secondary AD domain controller for redundancy.

To install Windows 2000 Advanced Server on both DL380s, we started with Compaq's ubiquitous Smart Start CD. With Windows installed, we were able to configure the MSA 1000 using Compaq's graphical array configuration utility. As with Compaq's other array utilities, this can be installed into Windows or run from the bootable CD. Unlike older bootable Compaq CDs, which ran a run-time version of Windows 98 or some other derivative, this one loads into a smooth Trinux-like environment--Compaq sure didn't get any help from Microsoft on that one! Although the Windows-based utility was useful for viewing our configurations, we had more luck with the bootable CD for reformatting our arrays.

Once our RAID sets were configured, it was time for Secure Path. Secure Path took some getting used to, basically to learn what it was useful for and figure out how the adapters failed over under various circumstances. A bit of reading and trial and error was all it took to be highly redundant in no time.

At this point we had to determine whether to deploy our system as active/ active or active/passive. You can't create your disk sets until you make this decision, and you must perform a capacity analysis beforehand. If you have or will have fewer than 1,900 concurrent users and less than 40 percent system load (supported limits as of Exchange SP2), you can use an active/active configuration. Otherwise, an active/passive environment is your only option. In fact, the engineer we spoke with said that Microsoft never recommends active/active clusters for Exchange 2000. The support for active/passive is the preferred solution, and with good reason.

Virtual memory fragmentation, which occurs under high load, prevents e-mail databases from mounting properly. This is because store.exe, which allocates virtual memory for use by Exchange databases, can't allocate enough contiguous memory. This fragmentation does affect a mounted database, but it becomes problematic during a failover, as the secondary node attempts to mount a failed node's EVS (Exchange Virtual System) databases. In an active/ passive environment, memory fragmentation is limited to the active node.

We decided to go with a active/passive environment because of load issues, which is ironic. Consider the following scenario: You have two mail servers, each handling about 2,300 concurrently connected users and averaging a 45 percent workload. For fear of losing either of these systems in a failure, you've decided that they should be clustered, allowing for zero outage in the event of a single system failure. Since neither system should operate at 90 percent load over an extended period of time, you consider an active/active environment, where both systems operate at 45 percent load, as normal, and reach 90 percent only in the event of a single system failure.

This sounds logical and is the main reason why many organizations consider an active/active environment. However, Microsoft clustering technology isn't quite there yet, because active/ active clusters are limited to having only those 1,900 users concurrently connected to any single server. In our previous scenario, we would be unable to serve all our users, and we'd be forced to acquire new equipment and operate in an active/passive mode, in which only standard server sizing issues remain a contention point. But what if we simply wanted to disperse e-mail load across two systems to boost performance while achieving redundancy?

Well, unfortunately, Microsoft does not support dynamic load-balancing for Windows 2000 Cluster services, so Exchange 2000 doesn't support it either. Because the Windows 2000 Cluster service is based on a share-nothing model, shared storage disks can't be accessed by multiple nodes at the same time and dynamic load-balancing isn't possible. Instead, load must be statically dispersed. This static dispersion is not facilitated by any automated process. It is the Exchange administrator's responsibility to assign load evenly across both systems. And most admins would agree that it simply isn't worth the effort.

The maximum number of storage groups permitted in a cluster is four (storage groups can contain up to five usable mail databases). If you are going to play the static load-balancing game, make sure your combined number of storage groups for both doesn't exceed this value.

Disk Allocation

When planning for disk allocation, keep in mind that the cluster itself requires a separate disk known as the quorum. The quorum resource disk is an important part of the cluster model because it holds configuration data for the cluster and is in charge of maintaining indexing checkpoints for both the cluster and the resource database. The quorum also plays an integral role in determining which server is active and which server is standing by. The node that gains control of the quorum first is allowed to form the cluster. Once the other node determines that the quorum is owned by another resource, it will just join the cluster and stand by.

While each cluster maintains its own group of resources, such as an IP address, a NetBIOS name and the quorum resource, each EVS must also maintain its own set of resources, such as an IP address and EVS name.

Each EVS will need at least one disk as well. Wherever possible, you should use two separate disks, one for the data store and one for the transaction logs. Simply placing your storage groups and transaction logs on separate logical disks of the same physical drive array won't achieve the same degree of redundancy that placing them on separate physical drive arrays will.

Cluster Implementation

Once all the RAID sets are in place and the partitions are created, the cluster service can be installed one node at a time. It's easy to lose track of the IP addresses that you'll assign to the various components, so have your plan on hand before starting your cluster configurations.

For example, a two-node active/active cluster will have at least five public addresses (one physical address per node, one cluster address, one address for each EVS), as well as at least two private addresses. The private addresses are used to maintain the cluster heartbeat. Two private network controllers should be installed in each cluster node, just in case one of the primary heartbeat NICs fails.

Once the cluster service is installed and configured, you're ready to install Exchange. Since Exchange is a cluster-aware application, it will detect the presence of the cluster and install accordingly. Since Exchange service packs can't be installed onto an active node, you'll need to fail the system over manually. Once the services have transferred to the failover server, you simply install the service pack. Upon completion, do the same for the other system. You would follow this set of procedures any time regular maintenance is required on a clustered system.

Migration Tools

We investigated two common migration paths, migrating from Exchange 5.5 to our Exchange 2000 cluster and migrating from Novell GroupWise 5.5 to our Exchange 2000 cluster. We looked at three third-party tools: CompuSven E-mail Shuttle and OpenOne Corp. Direct-To-1 for the migration from GroupWise, and Quest Software FastLane to migrate from Exchange 5.5. We also looked at Microsoft's native migration tools.

The third-party tools required a lot of testing, and we spent almost the same amount of time talking to tech support. If possible, get a demo copy before signing on the dotted line. We recommend testing a complete migration of the most complicated mailbox you can find in your environment. Our biggest complaint was the lack of documentation and, in some cases, bad documentation. Once you overcome the learning curve, however, some of these tools can add tremendous value to your migration process. For example, in an Exchange 5.5 migration, where user migration is also needed, FastLane lets you move NT SIDs (security identifiers) to the newly created Active Directory account.

To our surprise, Microsoft's native tools were the easiest to use. This wasn't the case when Exchange 2000 first hit the streets, but the last two service packs have improved the mail migration utility, Mailmig, tremendously. Migration is a simple two step process: migrating to .pst files, followed by migrating contents to the new mailbox. Mailmig also supports the ability to move Exchange 5.5 mailboxes between organizations.

Our solution provided a reliable mail system that could withstand quite a bit of failure. So is it worth the cost to deploy a cluster in your environment? Here are a couple questions that can help you make that decision: