Casino Rolls the Dice With a VTL

Foxwoods Casino and Resort was experiencing torturously slow restoral times and aggravating mechanical failures with its tape backup system. So, the resort determined that a change was necessary. Although replacing its tape system represented a significant six-figure investment, the resort decided to go ahead and deploy a virtual tape library (VTL).

Foxwoods, which has 11,000 employees, operates six casinos that house more than 7,000 slot machines and 400 tables for 17 different types of table games. More than 40,000 guests visit the resort each day, staying at the Grand Pequot Tower, the Great Cedar Hotel, the Two Trees Inn, or the MGM Grand at Foxwoods. The resort is owned by the Mashantucket Pequot Tribal Nation, so the IT staff is responsible for serving another 1,200 employees that provide municipal services, such as fire department and E911 communications, to the community.

A lot of hardware is needed to support the various operations. Foxwoods runs about 400 servers -- most run Windows, but there is also a mix of IBM AIX, Sun Solaris, and Linux servers. The various devices generate about 100 TB of information each day.

In early 2005, the resort was having problems with its tape backup system, an IBM 3494 tape library featuring 3590E drives. "Restore times could take as long as 24 hours, which was unacceptable," says Michael Grillo, principal engineer at Foxwoods. That amount of time was needed when a when a major problem such as a database management system crash took place.

Compounding the issue was the tape backup system's unreliability. "We were having four or five failures a month with the drives," Grillo says. "In some cases, a sensor would misread the tape and mark all tapes as invalid, so our technicians had to go back in, figure out what was wrong and then fix such problems."Adding more tape systems was one way to fix the problems, but it would have meant a significant investment. Foxwoods would have needed more hardware as well as a software upgrade. That approach could have quickly run into the six figure range, he says.

The resort decided to see what other alternatives were available. A VTL option was enticing because it had the potential to dramatically cut restore times, so, Foxwoods looked at products from EMC, Sepaton, and StorageTek, which is now part of Sun Microsystems>.

During a pilot, the Sepaton systems ran the best, about 20 percent faster than the other options, Grillo says. By the fall, the casino had installed Sepaton's S2100-ES2 VTL with about 60 TB of disk capacity. The installation went smoothly, and Foxwoods had the new system up and running within a couple of hours.

However, the purchase was significant, reaching the $250,000 mark. "The price was higher than buying a new physical library, but the added speed outweighed the extra cost," notes Grillo.

Since making the change, the resort has seen various performance improvements with its disk-based backup systems. The time needed to complete a system backup was reduced by one third, and application restore time was cut in half. The casino also reduced the number of physical tapes it was managing from 1,500 to only a few hundred, which has freed up technicians to work on other items.Another plus is the company improved its data redundancy. Tapes are now placed in an off-site data vault, which is located nine miles away and serves as a disaster recovery location.

Recently, Foxwoods has been looking at another technology that VTL vendors as well as other storage suppliers have been pushing -- data de-duplication, which allows companies to eliminate redundant information so they can store more data in the same amount of space. Foxwoods would like to take advantage of the technology, but has put such plans on hold for the moment. "Like other companies we now are in a zero growth period and simply not making any new major investments," says Grillo.

In sum, the casino's decision to move to VTL proved to be a good choice and has improved its backup procedures. Additional enhancements loom on the horizon but will not be realized until the economy begins to pick up, proving that not even casinos are immune to nationwide economic downturns.