Market Analysis: Is Outsourcing Right For You?

Q: What do the PC revolution of the '90s and the disco platform shoe craze of the '70s have in common?

A: Both seemed like good ideas at the time, achieved awesome heights, but ultimately proved bad for operational stability. PCs and the client-server model let businesses innovate by neatly sidestepping ponderous mainframe development cycles, but IT operations distributed across hundreds of machines, all running independently, degrade faster than a Bee Gees album in the sun. As the number of PCs and servers to be provisioned, patched and maintained grew, so did upkeep costs.

How far can you stretch one data center administrator? In our experience, across 30 to 50 servers, depending on the OS. BladeLogic, Opsware and other autoprovisioning software sellers say that number can go as high as 100 servers per admin ... if you're willing to pony up hundreds of thousands of dollars and commit to care and feeding of their provisioning suites. A tough proposition these days: When we asked in our most recent annual reader poll about the greatest IT challenges in the coming year, controlling costs was second only to ensuring security. And for many business people, controlling costs is code for outsourcing.

Of course, we know it's not that simple. Best case, outsourcing data center operations can alleviate administrative workloads while cutting costs, without negatively impacting the quality of IT service delivery. It makes sense--large managed hosting providers like EDS, Savvis and SunGard have implemented autoprovisioning systems, or built their own, offering the benefits without the learning curve, capital outlay or process update.For "Data Center Dilemma", we issued an RFI to investigate data center outsourcing, including provisioning, networking, maintenance and systems support, for our fictional widget maker, NWC Inc. Here we examine select drivers, risks and benefits.

Cheap Trick

Although respondents to our reader poll cited cost reductions as the No. 1 outsourcing driver, increasing capacity on important projects, a desire to improve IT services in general and shortened time to market for core business imperatives also are considerations. The reality is, savings can be elusive, and not just because of corporate resistance to layoffs and the desire to use talent already on staff for other important projects. Another downside is the management complexity resulting from multiple outsourcing relationships, which are becoming common as a way to mitigate the risk of depending on a single contractor. Complexity is heightened by regulations such as SOX and HIPAA--outsourcers can aid in compliance, but ultimate responsibility remains with you.

Communication difficulties can be exacerbated by distance, culture and time zones. Output quality is tied to both employee skill levels and how they align to your infrastructure, and operational best practices adhered to by the outsourcer. Lingering trouble tickets may be traced to the administrators' troubleshooting skills, but root cause can just as easily point to poor change control.

Minimizing the impact of these issues is an ongoing task--outsourcing is not a set-it-and-forget-it proposition. IT managers should team with business executives to map the outsourcing purchase and set metrics for an ongoing relationship. Although it's natural for IT groups to feel threatened by the concept of outsourcing, if you're not actively evaluating what can be outsourced, rest assured someone is. And that person might not fully understand what it takes to turn the IT crank, what it costs and why things are done in a specific way. You are in the best position to evaluate and subsequently monitor any data center outsourcing arrangement.

SLAs Are Your Friends

Likewise, IT is also in the best position to negotiate service levels and remediation. In our RFI, we found that providers will offer outlines of suggested availability, incident-response, project-maintenance and problem-escalation metrics. It's up to you to determine if proposed levels are fair to both parties. It's also worth asking reference accounts--even competitors in different locales or market segments--for samples of their negotiated SLAs and remediation metrics.

You want the outsourcing relationship to succeed, so choose realistic metrics, then monitor them closely. A mean time to repair of 15 minutes is a ridiculous goal, for instance, but 15 minutes to respond to an outage isn't. Expecting a new system to be provisioned in a day is unreasonable, but living with a standard 15-day workflow and specifying a limited number of priority cases per year, with overages charged, is realistic.

Regularly reviewing the relationship helps nip problems in the bud. Review incident and trouble reports as well as project reports weekly, and attend monthly service-level-remediation audits. Also monthly if possible, but at least quarterly, set up a CxO-level meeting to review status of current operations and realign new directions as needed. If you wait for an annual review, incident specifics will no longer be fresh in your mind. Worse, waiting leads to infrequent course corrections. Misguided realignments drag on for too long.

Trust, But Verify

Most outsourcers are immersed in the alphabet soup of IT best practices and compliance. But claiming to have best practices is not enough anymore; formal auditing and certifications based on ISO procedures are necessary. The ISO framework issues compliance through its 9001:2000 guidelines. Drilling down, look for adherence to the ITIL (IT Infrastructure Library) framework, which is an outgrowth of years of practical IT operations. COBIT adherence is another good sign, and the BS 15000 standard is an objective demonstration of ITIL practices. (Click here for more on ITIL.)Security concerns are heightened when outsourcing. Insist on a security certification, such as SAS 70, an audit done by a CPA firm to verify operational and security practices and procedures within an organization.

Two types of SAS 70 documents identify security compliance: Type I does not certify an outsourcer's security precautions; it's more an inventory of what the outsourcer says it has done. A Type II document is a certification that the outsourcer has not only planned security provisions, but that those provisions are in place and working. Ask for documentation regularly.

In practice, security is applied at the application, data, system, network and physical layers. For data center outsourcing, security starts in earnest at the data level.

As long as there have been outsourcers, there have been concerns about the integrity and security of data collected and stored off-site. From physical storage to operational best practices, ensure your data is protected from unauthorized access. This may be through partitioning a SAN or doing offline data retention in rooms with monitored and limited access.Outsourcers segregate systems and networks through dedicated resources. All the vendors responding to our RFI rely on dedicated physical processors. Utility computing will virtualize this segregation by system, but you'll be sharing CPU cycles.

Network security also depends on a dedicated network infrastructure. Separate firewalls, load balancers, VLANS and network circuits are common. WAN circuits may be MPLS, ATM or IPsec VLANs. As at the system level, it's a divide-and-secure approach, so network rubbing of customer packets is avoided.

Of course, for outward-facing applications for which the outsourcer provides Internet connectivity, both system and network vulnerabilities must be closed. System patches and upgrades to OSs and hardware are the outsourcer's responsibility. However, you should review and authorize patches prior to application.

Ensure that your outsourcer monitors for DoS attacks using application-aware switching that can shut down the targeted port automatically. Because the bandwidth entering and within the data center is generally shared, discrete switching controls mean individual customer services fail while the outsourcer blocks offending source IP addresses.

Physical security of the outsourcer's campus and data center should include video surveillance, proximity badges and restricted access using biometrics or other secure methods. Ask about visiting vendors. It's common to require sign-in and escorted access to the data center, but find out if the escort is dedicated throughout the stay.A Little or a Lot

The least intrusive outsourcing option is collocation, where your equipment sucks the service provider's power, cooling and bandwidth while being controlled by internal IT. Usually you'll get some sort of on-site monitoring and help rebooting, but not much more than that.

For NWC Inc., we sought to outsource operations of our servers and applications. As we found out, that may involve shipping our physical servers to the outsourcing site, but more often the outsourcer provides hardware, storage and OSs--patched, hardened and ready to go. This means no longer owning, but rather renting, system hardware and OSs along with environmental and bandwidth resources.

A variation along these lines are services like DNS, voice, video and Internet being offered by telcos as well as more conventional service providers. In fact, in our RFI all the service providers had their own PoPs, and two had rather extensive networks as well as data centers, blurring the line between telco and service provider. Our poll respondents say they are most likely to outsource IP services like DNS and Internet connectivity.

SaaS (software as a service) is the current buzzword for outsourcing of applications. Remember ASPs? This is the same idea--renting software rather than owning it. Actually, the notion of licensed software is a bit of a burr in the side of data center outsourcers, as the model for client-server systems still tends to be one server, one license. This can leave it up to you, or in some cases your outsourcer, to negotiate and manage your licenses. Compared with mainframe apps, which were usage-based, SaaS has some growing to do before it's ready for widespread adoption; still, it's an outsourcing option worth watching (for more on SaaS, see "Tactical Services: a Winning Strategy" ).The Future Is Now

For a look at a cutting-edge midsize business outsourcer, check out 3tera. This virtual Web hosting environment is self-serve and completely removed from the underlying hardware. Say you need to run a load-balanced, redundant Web server, application server and database. Sidle up to 3tera's Web management console and drag the systems together. That's it--the systems are provisioned. Need the same setup for a big promo or some app dev? Drag, click, use and blow away. The underlying hardware is operated and maintained without direct reliance on the application operating environment. It's designed for low-touch support starting at $400 per month. It isn't going to run your home-grown apps or internal supply-chain complexities, nor is it ideal for outsourcing relationships that need a lot of support. But for the small to midsize Web site, 3tera is worth a gander.

Outsourcing: A Smart Career Move?

We all fear being outsourced, so it's a natural reaction for IT pros to scoff when the subject comes up. We can do it better, faster, cheaper in-house ... right? Maybe, but at what cost if your staff has no time to develop innovative ways that technology can advance business initiatives and boost the bottom line? Simply turning the crank on standard day-to-day operations is the IT equivalent of a hamster on a wheel. The main reason to outsource is to increase efficiency. What we fear as IT professionals is that efficiency is just a euphemism for our jobs being on the line. But this fear closes us off to the potential improvements that outsourcing could deliver.

In our experience it's rare that an entire IT shop is outsourced. Usually, and in the case of NWC Inc., getting some of the day-to-day operations off our back frees up staff to work on important strategic initiatives that are planned but not executed on. Of course, most outsourcing results in a reduction in operations staff. The cost savings is part of the attraction. On the other hand, during our discussions with these outsourcing vendors some wanted to know if they were going to pick up any of NWC Inc.'s employees. This makes sense from their point of view--not only do they transfer some operational knowledge, they fill vacancies without having to run a search. And for those that do make a transition, the service provider's focus on IT as a business can be a valuable career move.Of course this isn't always true, and sometimes outsourcing does put people out of work. But burying you head in the sand isn't the answer. Don't dismiss it--drive it.