Then There Were Four...

We invited all five vendors to participate in our test of Linux support services. MissionCritical Linux, however, didn't reply to multiple phone and e-mail requests. Perhaps the staff was too busy helping real customers?

So we settled into our Green Bay, Wis., Real-World Labs® and tested the services from Caldera, Hewlett-Packard, IBM and Linuxcare. In addition, we asked Linux Support Services to submit its enterprise offering but never heard back, so we looked at the company's free service, which is available only on the Web (see "You Really Do Get What You Pay For").

We didn't just examine the shiny exteriors--we hopped in and took each service for a test-drive. Under the guise of mythical company Surreal Systems and using pseudonyms, we called support lines to evaluate both technical competence and responsiveness. We are certain the companies did not tip off their staffs, though one clever fellow claims he quickly figured out what we were up to. Find out who penetrated our cover and see a list of questions we asked in "The $10,000 (Linux Support) Question."

We were delighted with the responsiveness and technical expertise of the support staffs. We based our grading of the technical expertise of each participant on both the empirical data gathered through our calls to the support line and the responses to our RFI, which requested data on certifications, average number of years' experience for support-staff personnel and other criteria (read the responses to our RFI).

As with any service, it's the value-adds that push one provider over the top. All our technical questions were answered with correct, though often completely disparate, solutions. What impressed us most were the extra lengths to which some of the vendors went when solving our problems. Linuxcare staffers, for example, spent time sharing their knowledge with us: They not only offered solutions, they took the time to explain why the fixes worked and expand our base knowledge of Linux. HP took extra security precautions when we called to ask how to retrieve our root password, verifying our identities and ensuring that we had physical access to the console.

We were, however, a bit disconcerted about the root-password scenario: Not one vendor speculated that our need to recover our root password might have been because of a compromised system. For shame! We suppose it's possible that losing the root password is a common occurrence, but it's also possible that an external or internal wrongdoer changed the password. Yet no one asked us to examine our system to determine if this might be the case.

Kudos go to all the support personnel who fielded our calls. Not one of them ever spoke to us in a less-than-professional and -respectful manner. With the exception of a few conversations with IBM staffers, we never felt as though we were being patronized. A few times we had to bite our tongues, such as when a call took longer to resolve than we felt it should or when a somewhat unorthodox method was used to solve a problem, but overall we were pleased.

And the Winner Is ...

After reading responses to our RFI and comparing our experiences with each of the support offerings, we found ourselves in a quandary: Linuxcare and Caldera were tied for the top spot by the numbers. We decided to give the win to Caldera's Technical Expertise Account Management (TEAM) Services. Not only were the staffers accessible and responsive to our needs, they also went the extra mile that's required of a support organization to garner loyalty and instill the peace of mind necessary for a successful long-term relationship. But we wouldn't have a problem recommending Linuxcare either. Both services provide unique features that make them valuable.

Aside from assigning a technical account manager who acts as a single point of contact--a big plus--Caldera also encourages yearly on-site visits and weekly status reports via e-mail regarding all support activity. And unlike IBM, which we had to hit over the head for information on how to access our account, Caldera and HP both sent out "welcome" e-mails with information pertinent to our account and detailed instructions on how to access technical support. Again, Caldera went the extra mile and scheduled a phone briefing with our technical account manager to get acquainted with us and our systems. Caldera also routinely called us after an incident to ensure it had solved our problem.

And Caldera offers Web access via its Online Manager system. We were able to submit incident reports, check the status of ongoing incidents as well as search Caldera's knowledge base for solutions to other problems. Linuxcare, which uses e-mail primarily for after-hours support, also accepts incident reports via e-mail. All our calls were answered promptly, and in only one case were we unable to reach Caldera over the phone. We reported our incident via the Online Manager and had a response within half an hour.

In general, responsiveness was not a problem with any of the participants. HP, for example, offers an outstanding three-hour response time via e-mail for online submissions on a per-incident basis, something no other provider matched. And Linuxcare's technical expertise and handling of customers are above reproach.

No vendor failed to solve the problems we threw at it. Support personnel at Caldera, HP and Linuxcare quickly admitted when they did not have a solution to our problem and promptly found another support technician who could help--many then remained on the line to get some on-the-job training.

Hardware support is not an issue for any of the participants except IBM, which restricts its services to customers with IBM or Intel-based hardware. Software support varies from vendor to vendor, though the major Linux distributions are supported, and most of the open-source applications you'd use--including SAMBA, DNS, sendmail, NFS and Apache--are supported unilaterally. IBM, of course, supports any IBM application that has been ported to Linux, and Caldera also offers support for Volution, its Internet-delivered software-management service. Linuxcare blew us away with its range of supported distributions, including Slackware.

Linux Support Service Features Click here to enlarge

Hours of availability depend on your needs. All the participants offer the choice of business hours (Monday through Friday, 8 a.m. to 5 p.m., Pacific time for Caldera and users' local time for HP, except for Linuxcare, which considers 6 a.m. to 6 p.m., Pacific time, business hours) or 24x7 service. All the services we tested were based on a 24x7 contract. After-hours support from IBM and HP is no different from support during regular business hours. Linuxcare offered us e-mail support after hours, with the promise of a callback. Caldera, however, not only offered us Web and e-mail access and phone support after hours, but also provided the cell phone and pager numbers for a technical manager as well as his backup. Caldera's TEAM Services is well named--support staffers feel almost like members of your own team and is the primary reason we chose Caldera over Linuxcare.

Pricing varies wildly, from HP's simple annual per-server fee to IBM's convoluted tiered pricing. We read the pricing models for each company repeatedly and all we got was a splitting headache. Finally, we again consulted the vendors and requested specific pricing for annual contracts based on the number of Linux-based servers supported--one, five, 10, 50 and 100. This still did not let us compare pricing apples to apples: Linuxcare's model is based on number of support hours and contacts.

Ultimately, we based our cost comparison on minimum contract quotes and per incident (or per hour, for Linuxcare) pricing. Many of IBM's figures are outrageous, and we won't harp on the additional cost for supporting clustered Linux servers. HP garnered our Best Value award because it has the best-priced services, but Caldera and Linuxcare are close behind.

Per-incident pricing ranges from $125 from HP to $230 from IBM and $299 and up from Caldera. Purchasing "blocks" of incidents from IBM or Caldera would bring the per-incident price closer to the $125 offered by HP. Linuxcare does not offer a per-incident service but does extend a $250-per-hour rate for incidental support, offered over the Web.

Linuxcare provides Tier 4 support for HP's Linux support services and provides Tier 1 to Tier 4 support for several PC manufacturers, such as Dell Computer Corp. To avoid a conflict, we ensured that our questions would not require more than Tier 1 or Tier 2 support.