Seeking SAN Ports in a Storm

Hurricane season is fast approaching in Florida, and for many IT folks that means making sure a disaster recovery plan is in place.

Brett McGill, IT VP of Clearwater, Fla.-based MarineMax (NYSE HZO) - a recreational boat retailer that generated $762 million in revenue last year -- spent the winter making DR more efficient after his company survived a few scares last year.

McGill took two steps to improve disaster recovery. First, he switched from Computer Associates International Inc. (CA) (NYSE: CA) BrightStor ArcServe backup software to IBM Corp. (NYSE: IBM) Tivoli Storage Manager (TSM) to reduce his backup window. Then he installed IBM SAN Volume Controller (SVC) to replicate data and virtualize storage on his EMC Corp. (NYSE: EMC) SANs.

With 71 retail locations spread nationally, MarineMax can’t just shut down its SAN every time a hurricane hits the west coast of Florida. So twice last year, McGill and his staff had to relocate data on tape to an offsite recovery center 21 miles inland in Tampa. Another time they had to hook up generators at their main data center after the building’s power went down.

MarineMax didn’t lose data, but McGill was determined to find a better way. “During last year’s hurricanes, we would take tape backups and carry some hardware and pieces of the SAN to our remote cyber center,” McGill says. “There, we’d bring up critical applications and bring up [retail stores in the] rest of the country."While none of the hurricanes crippled MarineMax, McGill says they took a toll on his eight-person staff. “If any little thing goes wrong, you have your staff out there trying to perform these functions when they should be home trying to evacuate their families,” he says. “We knew we had to automate this.”

Replication was McGill's answer. MarineMax now runs two EMC Clariions, one at its main data center and the other at its Tampa “cyber center.” This allows the company to replicate the coastal Clearwater site at inland Tampa. The replicated sites are connected to IBM BladeCenters. The SANs hold 3 Tbytes of storage used by 1,200 employees spread across 71 locations.

McGill chose IBM's SVC for virtualization and replication instead of waiting for EMC’s Invista virtualization product (see EMC Unveils Invista). “We have IBM blade servers. We just felt more comfortable with their solutions,” he says.

MarineMax still uses tapes for good measure, but McGill says backups take an hour or two with TSM as opposed to all night in the past. That makes it easy to prepare for the worst when the winds start blowing. The Tampa center stays open regardless of weather, and generally doesn’t get hit as hard as the main data center on the coast. But MarineMax doesn’t own the Clearwater building, so the IT staff has to prepare ahead of time in case the landlord shuts down after a hurricane warning.

“Last year we worked 35 hours straight, shifting people in and out, preparing for hurricanes,” he says. “That was cutting it close. You lose an hour or two in a plan like that because if something little goes wrong, that’s huge. You run out of time.”McGill hasn’t had to use his new DR system yet, although he did get a scare last week before Arlene turned out to be no more than a tropical depression.

“Having one come up the coast this early reminds us we have to make sure we have a good disaster recovery plan and business continuity for our stores,” he says.

— Dave Raffo, Senior Editor, Byte and Switch