Data Storage Explodes in Nova Scotia

Storage requirements for the government of Nova Scotia, Canada, were growing at a rate rarely seen in any industry, including government. To make matters worse, the government's IT department had the same constraints as many government agencies -- its budget was fixed and there was little opportunity to grow the size of staff. Yet the demand for data storage was growing at 100 percent per year.

This explosive growth was fueled by a number of factors, including the government's decision to make more information available online for its citizens. "We had an increase in the amount of systems across the board," said Kevin Tillman, Nova Scotia's manager of corporate IT operations. "A large part of the increase is the large [geographic information system] that's come on line. We use that for all of the government departments. [And] everything has grown from email to electronic documents."

There was more. "We digitized all of our land records for the province," said Jeff Brunt, Nova Scotias technical architect for storage.

As a result, Tillman said, "We were in a panic to do something quickly. One hundred percent growth is substantial. We don't have an unlimited budget and the staff doesn't grow quickly."

The IT department had installed an IBM DS4400 storage-area network seven years ago as part of a plan to transition from its mainframe to a server-based operation, Tillman said. The department added storage as necessary to keep up with demand, but it became clear that they couldn't simply throw disks at the problem."We went to the IBM product called SVC [SAN Volume Controller] in March of 2006," Tillman said. He noted that he chose the SVC to retain compatibility with his existing IBM SAN. He also noted that while the SVC is from IBM, the servers he uses come from a variety of sources, including IBM, Sun and HP.

"SVC is our disk storage virtualization system," said Chris Saul, marketing manager for IBM's SAN Volume Controller. "SVC is based around the idea of a storage engine. They are Intel servers that run the SVC code [and] are always deployed in pairs for availability. If one fails, the other continues." Saul said that architectural approach produces a highly reliable system. "Our availability exceeds five 9s availability. It provides enterprise class availability." Saul noted that the redundancy and high availability are necessary. "If something goes wrong with a virtualization system that's a bad thing."

Ultimately, moving to virtualized storage meant Nova Scotia's IT team had to find a way to make the transition, while still keeping the storage and the information it contained accessible. "When we implemented SVC we bought a small number of disks. But we were able to reuse the existing disks to have seamless migration," Brunt said.

A seamless migration and integration was critical. "One of the other factors is, as these [services] become more available to the general public our ability to plead for downtime becomes strained," Tillman said. "SVC gives us our ability to do work in real time."

The process of moving to virtualized storage began in March of 2006, but still isn't done. "We have around 90 terabytes of storage behind the SVC," Brunt said. "We are about 90 percent migrated. We have some servers that are low priority that we haven't moved yet."During the migration, the IT team had to add additional virtualized storage controllers. "We're still on our DS4400," Brunt said, "but we've received notice of end of support. I can move all our data off of it, without a single outage. We have purchased this past year four DS3400s and one DS4800," which are high-performance IBM storage systems.

Brunt noted that outages have simply gone away for storage behind the SVC controllers. It also means he has much more efficient storage as a result of virtualization. "The big advantage is the ability to do thin provisioning," he said. "If someone comes to me and asks for 2 terabytes, I can give them 500 gigs and it looks like 2 terabytes and it grows as needed."

Tillman said that he expects storage growth to continue as the province moves more information online. In addition, he noted that his job is complicated by a number of factors, including the requirement for privacy under Canadian law. This in turn has caused him to worry about the United States.

"I need to need to know exactly how much storage we're going to need in the coming years," Tillman said. "The government could change its policy in terms of retention of data. We have various policies for retention of data. Our Department of Health has different policies compared to the Department of Justice."

And he worries about Sarbanes-Oxley, not because the U.S. law affects Canadian data centers, but because he expects the Canadian government to enact something with similar requirements. "If we get into a more SOX approach, we'll have to change our estimates. We don't have that, but it's coming," he said.Tillman also worries about the U.S. Patriot Act. "The Patriot Act means we cannot use outsourcing that you can use in the U.S. We have to make sure that data is retained in Nova Scotia," Tillman said. "We could not have EDS being an outsourcer and have our data residing in Plano, Texas." If he were to use a U.S.-based outsourcing company, Tillman said all of his data would be subject to inspection under the provisions of the Patriot Act. That's a non-starter. "We have strict rules regarding privacy."

The end result of these restrictions is that the government of Nova Scotia keeps all of its data stored within the province. Tillman said that the IBM SVC product allows him to maintain off-site backup storage that's easy to implement and is highly cost effective because he said that IBM doesn't require new licenses just to upgrade the off-site storage.

While Nova Scotia has its storage challenges under control, that doesn’t mean that they’ve disappeared. Tillman must still deal with departments that decide to buy their own storage in spite of having access to a huge pool of virtualized storage, and he must deal with consultants who install their own storage systems without even checking to see if the IT department has virtualized storage available. "Sometimes a project won't manage for the good of the whole organization," he said.

However, Tillman regards these issues as simply a matter of education. "We're making tremendous strides," he said, and those strides are critical. "We are constantly trying to ensure we have the capacity to serve the government of Nova Scotia and the people of Nova Scotia."