University Makes Data Center Move

To increase its amount of stored data, the University of Texas (UT) at Arlington first had to move data centers.

UT Arlington researchers are part of the CERN-led Atlas particle physics experiment that will require a huge increase of storage capacity when it enters its next phase in 2007. While preparing for that, the university has also spent the past year on a multimillion dollar ERP project for a new student information system.

Those projects are paying off. Atlas brings contributing HPC labs millions of dollars in annual grants. And UTA's new Oracle PeopleSoft ERP system will go live Monday. But the Atlas and ERP implementation placed a heavy burden on the college's IT staff. Jon Harris, director of enterprise operations and systems, says the university's biggest problem wasn't storage capacity. It had run out of physical space for its data center.

In late 2004, UT Arlington maintained two on-campus data centers -- one for academic computing and one for business services. Neither facility had room to spare. The Atlas program will require a significant expansion in storage capacity from its current total of 70 Tbytes beginning next year. Harris says his EMC CX700 SAN has 170 processor nodes connected and will expand to 512 nodes. But there was one problem: "Literally, we could not put another piece of gear in those data centers," he says. "Without a new facility, we would not have had space for these projects."

That led to an ambitious data center consolidation. Because the 2005 move needed to be completed before student registration in late March, the university had six months to find a site and get about 650 servers, an IBM mainframe, a SAN, high performance computer (HPC) clusters, StorageTek tape libraries, and 16,000 tapes into it."Historically, moving a data center usually takes a year," Harris says. "I've seen planning alone take 18 months in some cases. So we were on an aggressive timeline. We had to make quick decisions -- do we build a data center on campus, or do we take an existing open area on campus and retrofit it? Neither option was attractive to our executive team. It was cost-prohibitive, and in a rapidly growing campus it's virtually impossible to get floor space."

Harris found a tornado-proof 51,600-square foot building in Arlington about seven miles from campus. The data center took up about one-third of the space, and the rest came in handy for the in-house consultants required for the ERP project.

While real estate might be all a matter of "location, location, location," Harris says moving the data centers was a matter of "planning, planning, planning." He maintains the key to the successful move was having every step and detail plotted out beforehand. To help with the physical move, Harris hired StorageTek's services division, now part of Sun's Data Management Group (DMG). (See Sun Sets on StorageTek.)

"It was like a military maneuver," he says. "We did several walk-throughs and table top exercises. Every single step was planned. Every piece of equipment that went out of the data center was tagged. We knew exactly where it was the whole time. When it came in, if it was tagged 'X,' we knew exactly where 'X' went."

The operation included 16 people from the UTA staff, split into a down team and up team, plus 10 from DMG. "The down team brought everything down," Harris says. "The up team came over here waiting for equipment to arrive. Then the down team came over and supported the up team. We had this thing scripted over and over again. We had scenarios on what happens -- what would happen if a server came over dead on arrival, what we would do."Outside of a few loose cables and a broken wheel on a server rack, everything found its new home and worked as planned. The teams moved half the applications over consecutive weekends, taking about 12 hours each weekend.

The only piece of equipment that they didn't move was a StorageTek Powderhorn Tape Silo. Sun/StorageTek provided a new library as part of the deal instead of breaking down the old one and transporting the pieces. Harris' team and DMG did move over 5,000 tapes for the silo, as well as 11,000 tapes for other libraries.

The university is happy enough with its new data center that it will serve as one of three regional data centers -- along with Austin and Houston for the University of Texas system. The idea is to store copies of critical data for all the state universities in the regional centers.

"Once we get that up and running -- and it might take several years -- it might not be necessary to have separate disaster recovery site," Harris says.

— Dave Raffo, Senior Editor, Byte and SwitchOrganizations mentioned in this article: