Managing Your Web Services

Along these lines, protocol support is needed if you work with a variety of consumers. You can let distributors and partners that may not be ready to deploy Web services submit data over HTTP/S, FTP or SMTP, and maintain a single interface within your data center. Transformation plus protocol support gives you a mechanism for extending the accessibility of your services without investing additional development time to implement support for such protocols.

Into the Lab

With these must-haves in mind, we invited AmberPoint, Actional Corp., Computer Associates, Digital Evolution, Flamenco Networks, Infravio, Itellix and Oblix (which recently acquired Confluent Software) to send their Web services management products to battle it out for the right to manage Web services at NWC Inc., our 24/7 production environment and applications testing lab in Green Bay, Wis. (For more on NWC Inc., see inc.networkcomputing.com.)

AmberPoint bowed out, saying a forthcoming release of its product would make a review of its current offering irrelevant. Flamenco Networks submitted its product, but after much trial and tribulation we were unable to get it working. Flamenco's dual-proxy-based architecture is, well, unique, to say the least. We could use the management console to define proxies. We could deploy the proxies. We could not keep the proxies running. After ruling out hardware as a cause for the crashes and working with Flamenco for a week to resolve the problem, we pulled the product from the review.Itellix submitted its Wisiba Management platform, but we pulled it as well, though for a different reason. Wisiba worked as advertised, but it's better for a service-provider environment than an enterprise because of the large amount of information--including billing and administrative contacts, billing rates and other such options--required to manage services.

We deployed the products from Actional, Digital Evolution, Oblix, Computer Associates and Infravio and evaluated each on their conventional management features and Web services-specific options.

Although the products offer both proxy-based management and agent-based models, we tested all products in a proxy deployment. There are pros and cons to both approaches, but given the performance and total cost implications of an agent approach, we're sitting firmly in the proxy camp. (Why? See "To Proxy or Not To Proxy?") Unfortunately, most vendors implement proxy deployments by deploying an agent within a local JBoss or Jetty application server, rather than running as a pure proxy application, such as that offered by Digital Evolution's Service Manager.

This review is not about security, but when it comes to Web services, you can't have too many security features. We were especially pleased by Service Manager's support of Web Services Security 1.0 (WSS). Its product's ability to map credentials and easily strip or insert WSS headers is a plus.CA's security support is accomplished through its partners, and by partnering with DataPower, CA's total package provides broad support for WSS and integration with an identity-management infrastructure. In truth, though, CA's WSDM fell squarely into the operational management space, offering only conventional management capabilities. Although this side of WSM is necessary, a service-oriented architecture (SOA) requires more than just operational management.

After putting all five products through deployment, configuration and performance paces in our lab, we awarded Actional's SOAPstation our Editor's Choice award. Actional provides all the necessary functionality, and its peak-performance numbers were excellent. SOAPstation gave us all the operational monitoring we could possibly want and then piled on SOAP-specific features, such as routing, support for Web Services Security and a flexible implementation architecture, all while outperforming the competition. Aside from some navigational issues that made the configuration cumbersome--a common affliction from which all products in this pubescent market suffer--SOAPstation proved worthy of our Editor's Choice award.

We deployed SOAPstation on the included JBoss application server and went with a default MySQL database as its repository. SOAPstation supports multiple application servers, including BEA's and IBM's, and can work in a .Net infrastructure. SOAPstation can be deployed in proxy mode or as a native agent on the systems being managed.

To configure a Web service for management using SOAPstation, we had to set up the application, an access point and the services to be managed. The application requirement reflected well on Actional: It let us monitor the application server. In fact, SOAPstation was the only product we tested that provides this capability. SOAPstation let us aggregate services into a group, clustering multiple services into a single, managed set, and apply transformations, alerts and other management features by group rather by service.

SOAPstation has some unique SOAP-traffic switching and routing capabilities and, like Digital Evolution's Service Manager, gave us a reusable object-based scheme for building security and management policies around services. We were pleased with the XPath editor, which helps define message fields. Message fields let administrators specify the path to an element within a SOAP message but let that path differ based on the message being processed within a service group. This is an excellent solution for managing multiple services that may contain a common field, such as purchase order or customer number, when the location of that element within the XML may differ. For example, the customer number for a business partner might be located within a "partner" element, while the customer number for a consumer would be in a different element.However, the XPath editor is available only for defining message fields, and we had to determine the XPath manually when we configured rules to route messages based on the amount of an incoming purchase order. Actional says the editor will be more widely available in future releases. In contrast, we were more pleased with Infravios XPath Evaluator's functionality, though it's an eyesore. Oblix's Coresv provides absolutely nothing to assist in defining the many XPath expressions required to configure routing.

After our performance tests, we discovered two things: SOAPstation is quite fast, and all the products experienced a performance degradation as routing, logging and SLA were configured, as expected. Our biggest concern with performance is that these products, which reside on or sit in front of application servers, must perform as well or better in terms of messages per second and total throughput as the products they'll be managing. Based on our tests of Web services platforms (see "Serving Up SOAP,"), we feel SOAPstation definitely can keep ahead of any services it might be managing (for hard numbers, see the performance charts below, in "How We Tested Web Services Management Software").

Actional SOAPstation and Looking Glass. Actional Corp., (800) 808-2271. www.actional.com

Service Manager ran neck and neck with SOAPstation in its security features, routing and logging capabilities. Service Manager, like SOAPstation and most products that manipulate SOAP and XML, uses a pipeline to manipulate data as it passes through the product. Service Manager fell behind SOAPstation, however, when we factored in pricing. In addition, Service Manager lacks scheduled reports and requires an external repository--Oracle or SQL Server--so implementing it is tricker than setting up SOAPstation, Infravio's Ensemble or Oblix's Coresv, all of which use a turnkey approach.

Service Manager takes advantage of WSDL to add services (to determine available operations) and yet it ignored the endpoint. While SOAPstation and Ensemble were able to pull the SOAP endpoint out of the WSDL, neither Service Manager nor CA's offering took advantage of this capability. Rather, both required us to specify the endpoint. In some respects this is advantageous--we could change or obfuscate the endpoint according to our own needs, something not possible with Oblix's Coresv.

Service Manager's pipeline configuration makes it a breeze to manipulate XML and SOAP messages. By adding a processing step to the pipeline, you can easily add logging, routing, transformation and security to any incoming or outgoing message. Unlike SOAPstation and Ensemble, however, Service Manager requires a usage component to the pipeline for each service you want operational statistics for. Without the component, you'll get no data on the number of requests for services or operations. Digital Evolution told us it would be adding this as a default to all services under management in the future.Digital Evolution offers pipeline components above and beyond what we expected. We were pleasantly surprised by the caching and compression options, a security bridge to .Net, and the ability to insert values into the XML document as it passes through the system. Although these features are available in most products via transformation (XSLT), Service Manager offers an easier way to accomplish these tasks.

We also liked the configurable ad hoc dashboard for viewing operational statistics. However, though Service Manager's statistical data for ad hoc reporting was much more granular than Ensemble's or even SOAPstation's, Service Manager does not provide a mechanism for scheduled reports. Digital Evolution says that because all data is stored in Oracle or SQL Server and nearly all enterprises have a corporate reporting system, such as Crystal Reports, such functionality is redundant. We can almost buy that ... but until report creators understand the data they're reporting on, it would be nice to have at least a high-level scheduled report for corporate consumption.

Digital Evolution Service Manager. Digital Evolution, (866) 99DIGEV, (310) 260-2717. www.digev.comWeb Services Distributed Management, appropriately named after the would-be OASIS standard, is spot on in terms of operational monitoring and integration with network-infrastructure frameworks, such as CA's Unicenter and Hewlett-Packard's HP OpenView. WSDM fits into just about any architecture but requires more implementation work because its architecture is based on remote agents embedded in application servers, such as IBM WebSphere and BEA WebLogic or, as we tested, deployed on an XML-proxy-based device, like DataPower's XS40.

CA's WSDM is purely for operational monitoring. All additional management capabilities come from its partnerships or the application server its agent is deployed within. By partnering with DataPower, WSDM gets a performance boost and can offer a total, flexible system complete with routing, SLA, load balancing and security features. On its own, CA's value proposition does not add up to its price. Its operational-monitoring capabilities are nice, but not $25,000-per-CPU nice. Coupling with DataPower, however, puts it on par with other products in the review in terms of feature set and price. Although the CA-DataPower combo, which starts at $115,000, appears to be the most expensive of the lot, that's only because rivals quote "starts at" pricing. When the "starts at" price is $75,000, it's going to hit $100,000 by the time you're finished.

WSDM's configuration is straightforward, requiring more work on the DataPower side than on the CA side. WSDM, like the other products we tested, deploys its management console within an open-source application server architecture. JBoss and Jetty with Apache are as common as Hollywood divorces in this particular market, and CA's product is no exception. Once the XS40 was configured to communicate with WSDM, all traffic flowing through the XS40 was mirrored to CA's product and analyzed according to configured monitoring rules.WSDM gave us some granular monitoring capabilities at the SOAP operation level, including total usage and latency statistics and interval-based data collection. Statistics can be monitored in almost real-time--the less traffic flowing through WSDM, the more "real time" the data. This is true of any product that sits inline and captures traffic before analyzing it.

As with the other products we tested, you can configure notifications and alerts to be sent based on thresholds, as well as directed to a systems-management console via SNMP.

Unicenter Web Services Distributed Management. Computer Associates International, (800) 225-5224, (631) 342-5224. www.ca.com/webservices

Confluent Software's WSM offering, recently acquired by Oblix and rebranded as Oblix Coresv 3.5, was very easy to configure, outdone only by Infravio Ensemble's exceedingly intuitive administration console. However, Coresv is missing many of SOAPstation's bells and whistles. It provides the basics for managing SOAP services, but not much else. The user interface's heavy reliance on XSLT and XPath knowledge dulled our excitement over its ease of use, and limitations on routing and SLA management pulled Coresv out of the running for top honors.

We were confused several times by Coresv, making for some blue air in the lab. For example, it offers a test engine that can import WSDL before you go ahead and configure the system, but it's limited in its capabilities and could not parse the WSDL served from our Spirent Reflector--even though the same WSDL imported into the management server was parsed.

Coresv, like many SOAP and XML security devices, obfuscates the SOAP endpoint to prevent accidental discovery. We were pleased with this feature but disappointed that we could not change the endpoint ourselves, because the obfuscation is not so difficult to understand--the URI (Uniform Resource Identifier) always began with the SID and was followed by a sequential number. Specific URLs for accessing services were used by Oblix, Infravio and Actional. DataPower uses a port-based model, and DigitalEvolution uses one or more management points (like an endpoint).Coresv also separates policy management from gateway management, which meant we had to move between two administrative consoles to manage the product and configure options like routing. Digital Evolution's Service Manager separates its management and configuration by function (alert manager, service manager); the tasks are integrated into the same console.

Oblix Coresv 3.5. Oblix, (800) 438-0626, (408) 861-6800. www.oblix.comWeb Services Management Suite We required that products for this review provide SLA management, and Infravio's Ensemble implementation was what we were talking about. Its abilities to easily configure SLA terms based on performance and to enforce specified SLAs is what we expect out of an enterprise management product.

Overall, Ensemble offered us the most intuitive interface, the most flexible configuration options and gave us everything we wanted. That's the good news.

Unfortunately, Ensemble's price and performance make it a far more expensive solution than the competition and drove it to the bottom of the list. If the product had performed better, we would have been able to say it kicked some major management ass. As it sits, however, performance is killing Ensemble's chances for becoming a force in the WSM arena.

Features like a WS-I (Web Services Interoperability) compliance report after importing a WSDL, an administrative tool to send a message (with or without authentication) from a Web page, policy versioning and load-balancing options make Ensemble a powerful management tool. The contract-based deployment of policies and easy configuration are so sweet. But then we generated load against the product and were sorely disappointed. Discussions with Infravio indicate that our performance numbers are close to what the company is seeing in deployments, and we're sad to say, it just isn't enough to justify the $50,000 price tag--even with all those management goodies thrown in.Infravio Ensemble 4.1 Web Services Management Suite. Infravio, (877) 246-3728, (408) 861-3000. www.infravio.com

LORI MACVITTIE is a NETWORK COMPUTING senior technology editor working in our Green Bay, Wis., labs. She has been a software developer, a network administrator and a member of the technical architecture team for a global transportation and logistics organization. Write to her at [email protected].Here are some of the results from a recently released Gartner survey of IT professionals conducted at its Application Integration and Web Services Summit:

» When asked which of the following best describes their organization's plans for Web services, 24 percent said they were developing for production, and 36 percent said they were experimenting. Nineteen percent said they have plans to develop within the next six months. Only three percent said they have no plans to use Web services within the next 24 months (why they attended the summit is anyone's guess).

» When asked which Web services-related standard or architectural style they preferred, 88 percent chose XML. SOAP ran a close second, with 79 percent of respondents indicating that they were using or planning to use it.

» When asked to state the three biggest challenges facing their application development/integration staff over the next three years, Web services skills availability was cited by 26 percent of respondents, followed closely by training existing staff on new technologies (23 percent). Technology standards came in third, with 15 percent.Think about it: A Web service can be as simple as buying a movie ticket online or as complex as the U.S. Army's procurement system. Anything that diverse and dynamic is bound to present significant IT management challenges, especially in terms of maintaining acceptable levels of security, performance and reliability. Predictably, vendors have seen this need and introduced specialized Web services management products to ease the pain.

We installed Web services management offerings from Actional, Computer Associates, Digital Evolution, Flamenco Networks, Infravio, Itellix and Oblix in our NWC Inc. business applications labs to find which provide the best logging and audit trails, SLA features, reporting functionality, request routing, and alerts and notifications while maintaining acceptable performance levels--bottlenecks need not apply. Flamenco's product went down in flames despite our and the vendor's best efforts, and we found Itellix's Wisiba better suited to service providers than the enterprise. In the end, Actional's SOAPstation earned our Editor's Choice award, thanks to its comprehensive functionality and snappy peak-performance numbers.

Management products in every market make a basic decision early on: To be a proxy or not to be a proxy and, for example, deploy native agents instead. The Web services management space is no stranger to this debate; companies are offering agent, hybrid and proxy versions of software designed to manage enterprise Web services.

Native agents are deployed within an application server, such as BEA's WebLogic, IBM's WebSphere or Apache Tomcat, and are implemented as servlet filters. This inserts them into the chain of processors that are triggered each time a request is received by the application server. Other products, including Computer Associates' offering, use a native agent architecture but require only that it be deployed on the same physical machine as the application server.

Proxy architectures act as intermediaries between the client and the application server. This lets the product sit inline and manipulate traffic as it flows through, providing AAA (authentication, authorization and auditing), message transformation, message routing and other capabilities in addition to the operational monitoring that is expected of a management product.There are pros and cons to each implementation. Proxy deployments can have a one-to-many relationship, with one proxy-based management system for multiple, even disparate, application servers. Agent-based systems, on the other hand, require one agent per server and generally require at least a single management point from which to aggregate statistics and manage the agents. If you want to manage an application server that isn't supported by the vendor, you're out of luck.

Proxy deployments cannot be nearly as educated about the nuances of every application server, nor can they necessarily (without agents or another method) manipulate the application server. Root-cause analysis can almost never be accomplished using the data from a proxy-based management system because its view of the data includes only that which can be captured on the wire. Agent-based systems, on the other hand, provide a much more granular view because they generally reside within the application server and therefore can provide a much deeper look into performance problems and other error conditions than proxy-based products can.

Agent-based systems are limited in terms of capacity by the application server. The agents can perform no better than the application server, and if the application server is overwhelmed, it's possible that the agent will be unable to perform its tasks.

Most of the products in our review can be configured to act as proxies or at least work in conjunction with a proxy-based product, in much the same manner that Computer Associates' WSDM interacts with DataPower. Actional provides additional management capabilities through its companion LookingGlass product, which makes use of application-server-specific agents to assist it in root-cause analysis and deeper monitoring of performance and SLA compliance.

We evaluated Web services management products on their ability to manage our B2B Web services implementation in our NWC Inc. business applications lab in Green Bay, Wis.For testing, a Spirent Communications Reflector simulated five servers, all Web services backends providing services for our partners. We configured a Spirent Communications Avalanche to simulate partners making requests, and further split the users up to send requests for different services with differing values to test the routing and SLA enforcement aspects of each product.

Each of the client profiles was further subdivided to simulate submitting different values for customer code and total amounts of the purchase orders, including a subset with invalid data.

We then installed and configured each product on a Dell 2650 dual Xeon 2.2-GHz, with 1 GB of RAM, a Gigabit Ethernet NIC and a 16-GB SCSI drive running Windows 2000 Server SP3.

Each product had to interact with external Web services plucked from the list at xmethods.net to demonstrate server-side interoperability with .Net and J2EE Web Services implementation (both the DOC/LIT and RPC/ENC models). Products were then configured to support our internal services as listed above, and put to the test in increasingly complex scenarios:» Basic proxy mode: No manipulation of the message.

» Routing: Products were required to route all requests with a customer code of 1 to a separate server, to simulate "gold" level service.

» Logging: Products were configured to log all requests and reply, in full, for auditing purposes.

» Routing and logging: Products were configured to do Steps 2 and 3 to see the effect, if any, on overall performance.» Authentication: Products were configured to require authentication for at least one service. HTTP Basic as well as WSSE UsernameToken were configured.

Products were performance tested at each of the four steps, with the number of messages per second, latency introduced and total throughput noted (results charted below). During the configuration of each of these steps, we evaluated the configuration mechanisms and capabilities of each product in terms of SLA management and enforcement, alerts and notifications, transformation of messages, security features and exception management.

R E V I E W

Radius Servers

Sorry,
your browser
is not Java
enabled

Welcome to

NETWORK COMPUTING's Interactive Report Card, v2. To launch it, click on the Interactive Report Card ® icon

above. The program components take a few moments to load.

Once launched, enter your own product feature weights and click the Recalc button. The Interactive Report Card ® will re-sort (and re-grade!) the products based on the new category weights you entered.

Click here for more information about our Interactive Report Card ®.

0