Network Computingwww.networkcomputing.com

Five CSPs responded under the large category; we scored all of them. In the mid-tier, we based our selection on the size of the CSP's help-desk support staff, plus a matrix of reliability indicators including network and server redundancy, backup facilities and the offering of any service-level agreements. The five CSPs in the small category were selected according to the size of their help-desk support staff.

We began the scoring process by assessing, on a 1-to-10 scale, the relative value of each survey question with regard to small, midsized and large merchants, paying special attention to particular services, support and hosting performance. We did the same for questions with great import for mid-tier and large merchants. Finally, we selected another set of questions that applied only to small merchants' particular needs.

We then organized all the questions into five categories: the business focus of the CSP; its ability to deliver high-performance, high-reliability network- and server-based services; its support for e-commerce-specific services; its commerce payment processes; and its overall support in terms of staffing and availability. Each question was assigned a value corresponding to its 1-to-10 rating. In some instances, such as help-desk hours, we assigned different point scores up to 10 for specific levels of service. We then weighted categories to reflect merchants' priorities.

For medium-sized and large CSPs, our business "focus" criterion (not weighted) was based on the number of sites hosted, average and largest number of products for sale per site, ability to offer business-to-business services, and percentage of servers dedicated to a specific site or application. For performance/reliability (scores were multiplied by three, indicating utmost import), we examined the ratio of peak-to-aggregate bandwidth, service-level agreements, server and network redundancy, backup facilities and dynamic load balancing.

For commerce-specific services (scores were multiplied by three), we examined ability to integrate with ERP systems, site promotion, personalization features, frequent-buyer programs, e-mail notification of shipment to buyer, digital- and physical-goods fulfillment, ability to offer varied privilege levels that can be used by a business to manage its site, credit-card fraud-checking mechanisms, online query and analysis, site-design capabilities and search-engine support.

For payment systems (scores were multiplied by 1.5), we looked at e-mail alert and notification engine support, membership tracking, purchase order and invoice support, subscription services, online tax calculation, multicurrency support, bill presentment, online credit-card processing and certificate services for authentication. For support (again scores were multiplied by 1.5), we examined the size of the CSP support staff as well as the hours and days during which customer support is offered. A few CSPs declined to respond to some of the scored questions and thus earned no points for those questions.

The scores for CSPs catering to small merchants are based on that group's own unique weights, survey questions and point system. A more detailed look at this system, or even the system used with mid-tier and high-end CSPs, can be found at www.networkcomputing.com/923/923f1csp.html. Also, if you'd like to evaluate how other CSPs in our survey would have scored, you can plug their online responses into our online score charts.

While some security queries were folded into our survey, we made no attempt to rate CSPs on security services. Many CSPs follow policies of security through obscurity, making polling difficult. Even if this were not so, it would still be very difficult, if not impossible, to evaluate security using a polling methodology. Similarly, the many hidden costs and customized design features of commerce offerings make it impossible to adequately evaluate overall pricing.