We installed each product on a Compaq Computer Corp. 1850R server with 256 MB of RAM and a single 60-MHz Pentium III processor. We graded each product for its rapid-deployment technologies, enterprise scalability and high-availability technologies; the quantity and quality of included components (also called portlets or gadgets); and the developer's toolkit for the addition of custom content.

To judge scalability, we considered administrators' ability to evaluate load via the management interface and the product's ability to spread work across multiple processors and systems and to add processors and systems as needed. We evaluated high availability based on the product's support for clustering and automatic failover.

We placed emphasis on the quantity and quality of the included portlets that give users access to various content types and integrated applications. These components could provide access to existing business systems, thin-client technology, a virtual start menu, community components, document libraries and management, corporate search taxonomies, customization components, collaboration components, project management, business intelligence, application consolidation, and workflow. We also considered the vendor's ability to attract third-party portlet developers.