It's no wonder so many enterprises are looking to build out their SOAs--the agility a service-oriented architecture promises, within both IT and the business, is enticing and financially rewarding. The reuse of legacy assets, if not necessarily code, is a benefit long promised by EAI (enterprise application integration) but rarely realized. Granted, agility is one of those marketing words we try to avoid. But in this case it's warranted to describe the ability to modify business processes without code, as well as to describe the capability to modify the inner workings of a business service without affecting the clients and services that rely on the modified service. In a truly agile system, for example, a customer-lookup service can be changed to use an Oracle database from a COBOL copybook, and the client should be none the wiser.
Problem is, once you start digging, you'll find SOAs are a lot like your skeletal system: There are many moving parts, some of them essential and others--like the coccyx!--you may be able to do without. And like your skeletal system with its bones and ligaments, SOAs aren't just a single product. It's an architectural concept that must be implemented with many types of products, each designed to fulfill a purpose within the overall architecture.
To build a solid SOA infrastructure, you need an enterprise service bus. If the SOA is your skeletal system, the ESB is your backbone. It's the application infrastructure that joins disparate services and ensures availability of those services. It enables business services to be turned into business processes by BPM (business process management) suites. It's connected to everything else, and it protects the information
flowing from the SOA brain (the registry/repository) to everything else. It's the middle tier of the SOA infrastructure stack, the glue that binds process orchestration to services. An ESB isn't super glue, though--services aren't tightly bound together by the ESB because that would thwart the SOA's promise of agility. An ESB should link services lightly to a business service that can change rapidly with the needs of the business and the ever-evolving technological landscape of IT.
Stack It Up, I'll Take It
The SOA infrastructure stack comprises three tiers:
» The service-enablement layer. Applications are fitted, or retrofitted, with Web service interfaces. Conventional EAI offerings will fit the bill here, but we prefer products from companies like IBM, Iona and Software AG that specialize in exposing legacy technologies as Web services for use in more modern architectures. Playing on this layer requires both technical competency and coding chops.
» The service-orchestration layer. The ESB is an essential piece of this layer, providing availability and routing of services exposed at the service-enablement layer and orchestrating individual services into business services for use at the next layer. The registry/repository is also an essential player in this layer and will be required if SOA nirvana is ever to be reached. This layer requires technical competency but should not involve coding skills.
» The process-orchestration layer. BPM and BAM (business activity monitoring) take their place in the SOA stack at this abstract layer, providing support for business processes built on the business services integrated in the service orchestration layer. This is the layer where business analysts can interact with services and where, one day, they'll be able to compose SOBAs (service-oriented business applications) by using a drag-and-drop mechanism to orchestrate business services in a way that best suits their needs. This layer should support nontechnical business-oriented users.
Three layers, three sets of products. Some vendors, including Oracle and Sonic Software, claim to combine all three layers into a single SOA suite. But make no mistake: Even if all the requisite technology is packed into a single box, there are still three disparate layers in the stack, and your SOA should reflect that to maintain a loosely coupled architecture and realize the benefits of a well-constructed SOA.
The ESB should orchestrate services into business services--small orchestrations that bind lower-layer integrations into a business purpose. CustomerInfoRetrieval, for example, is a business service, but under the covers it may comprise several lower-layer services that communicate with multiple systems to retrieve the data required to present higher layer process-management systems or applications with all the right data.
Although you could shoehorn some ESB products into taking on the role of orchestrating business processes, most ESB suites are missing components--such as process management, workflow and BAM--necessary to manage and develop true BPM systems. An ESB should focus on providing business services to higher-layer applications and on ensuring the availability of those services.
To evaluate current ESB offerings we invited a lengthy list of vendors to make the trek to our to Green Bay, Wis., business applications lab and help NWC Inc. begin its journey toward SOA nirvana. BEA Systems, Cape Clear Software, Fiorano Software, IBM, Oracle, Sonic Software, Software AG and TIBCO agreed. We grew tired and sore from all the bumps and turns during our ride on each of the eight ESB implementations we tested. All eight entries drove us from Point A to Point B, and all hit a few potholes on the way. Overall the products from BEA, Oracle and TIBCO gave us the smoothest ride, while those from Cape Clear, IBM and Sonic Software actually made us break out the Mountain Dew and code for a while. Silly us, we didn't stipulate "no coding" in our invitation.
WebMethods and Sun responded positively but must have taken a left turn at Albuquerque; neither sent us a product. PolarLake fell victim to internal miscommunications and had to decline. Microsoft never replied at all, not surprising given its current marketing hype of having something "better than an ESB" and our requirement to support SOAP over JMS (Java Message Service); support of anything Java is necessarily absent from Microsoft products.
Iona joined us in the lab, but after we tested version 3.0.3 of its Artix product we decided it didn't meet our requirements. The company was in the middle of a major release cycle, with version 4 due only a few weeks after our testing window closed. Version 4 would have met our requirements, and we hope to get a chance to review it down the road.
We evaluated each product primarily on its core ESB features, examining routing capabilities, transformations and mapping, protocol support and orchestration mechanisms, as well as conventional management facilities. We also peeked at how well each ESB performs conventional integration functions for common enterprise tasks, such as interfacing with enterprise applications (like those from PeopleSoft, SAP and Siebel) and database interactions. Although such integrations would be achieved over Web services in the SOA purist's world, some situations and performance considerations require that typical integrations such as these be performed at the service-orchestration layer.
With that in mind, we evaluated each product by attempting to orchestrate a service to handle a purchase order. The service had to interact with NWC Inc.'s Oracle 9i database, be able to publish to a JMS queue, act as a Web services consumer and send a simple e-mail notification message to the customer. The ESB was required to transform data when publishing to the queue as well as when creating the appropriate e-mail notification. We used content-based routing to determine whether a message needed to be published to the JMS queue used by manufacturing processes. For more detail on our NWC Inc. scenario, see "How We Tested ESB Suites".
Architectural differences--most specifically those involving routing and management capabilities-- among the products we tested affected much of our evaluation. There is no standard architecture, no standard transport mechanism on the bus, no standards to address how Web services are implemented ... all of these decisions are left to the vendor, and no two vendors appear to be of like mind (see "ESB Transport Methods" below).
ESB Transport Methods |
Although Sonic Software's SOA Suite 6.1 and BEA's AquaLogic Service Bus 2.1 are both, technically, J2EE applications, neither is deployable on enterprise service platforms other than the ones with which they ship. Sonic's suite requires only a JDK and ships by default with IBM's JDK 1.4. BEA's ESB automatically installs into a lightweight WebLogic Server (WLS) 9.1 container, and the only time we needed to manage the WLS instance was to add a connection to get to our remote OpenJMS queue.
Oracle's SOA Suite and Cape Clear's ESB 6.5 requirements were similarly light on the enterprise service platform administration front, but both can take advantage of existing servers, supporting deployment on several J2EE containers including BEA WebLogic, IBM WebSphere and OracleAS.
What's interesting about products that have grown into ESBs from the conventional EAI area, such as those from TIBCO and Fiorano, is the requirement for an external service to provide Web services support. Although the products from BEA and Oracle also use external containers for Web services, both have integrated the process so well that it is transparent to the administrator and developer. In contrast, Fiorano's SOA 2006 Platform 3.7 and TIBCO's BusinessWorks 5.3 make it painfully obvious that the Web service container is an adjunct to the core ESB. Fiorano's container is Tomcat/Axis, and services with a Web service endpoint must be specifically deployed as such, separate from the orchestrated service. We also had to download and install Sun's JDK 1.5 to get Tomcat up and running. If it's a requirement, the JDK should be distributed with the product. TIBCO's solution to Web services is similar to Fiorano's, though we found its deployment process a bit more fluid; it didn't require as much in the way of installing external products.
Software AG's Enterprise Service Integrator employs JBoss, though it can be deployed in other enterprise service platforms, and its Web services support is in line with TIBCO's implementation, with one exception: In Enterprise Service Integrator we had to go to the Web administration console and define a "portal" (an entry point based on protocol, like SOAP or JMS) that is tied to a specific sequence (orchestration) configuration. As with Sonic's product, the line between development and administration isn't quite clear, and there's no definitive demarcation of responsibility for configuration between administrators and developers.
IBM's WebSphere Message Broker 6.0 was unique in that we had only to define a URI within the orchestration for its message broker to start listening for and handling Web services requests. IBM does not distinguish a SOAP listener from an HTTP listener, choosing instead to determine whether the request coming in on an HTTP port is SOAP or plain-old XML as part of the configuration of the orchestration. This was an interesting approach that let us define an orchestration to process a single SOAP message using both MQ and HTTP with little effort.
After IBM, configuring Sonic's SOA Suite 6.1 was like stepping into an alternate universe. Everything is based on JMS, with endpoints and exit points required for every service, including Web services (see screen at right). Endpoints are configured within the fat client-management console, and then set as properties on steps within an orchestration in the design-time environment. Although it wasn't too difficult to set up a process to be accessible over Web services, it was a bit daunting to force Web services into a JMS endpoint-oriented configuration paradigm.
Protocol support is a primary consideration when choosing an ESB. We found much differentiation among products in terms of inbound protocol support. Although every product supported HTTP and JMS, the similarities ended there. We could not, for example, easily implement inbound FTP or SMTP requests with every product. This lack of FTP support may be problematic, especially in terms of partner interactions, because FTP is still the main transport for a variety of business transactions.
Even more disconcerting was the lack of support for some fairly common protocols within an orchestration. Although JMS and HTTP were consistently supported, SMTP was frequently missing from the list of communication mechanisms (IBM, Cape Clear) as was support for simple queries to an RDBMS (Cape Clear, BEA, IBM). In all three of the latter cases, we had to write code to access our Oracle 9i database and expose the code as services before we could include them in our orchestration. IBM's solution to support RDBMSs includes its proprietary compute nodes that require coding in Java or ESQL (Extended SQL), which is very BASIC-like in nature. The only other product that required us to write code was Sonic's, which needs JavaScript to do any type of content-based routing within an orchestration. We preferred the XPath-based methods of the other products we tested, even those that required us to know more XPath than we'd like.
Sonic has a wide variety of protocol support, provided you can translate protocol requirements into a JMS world. All protocols must be turned into services--not unusual in the service orchestration sphere--but all protocol support also must be JMS-like entry and exit points, which was confusing at times and difficult to configure.
This varying support of protocols from within a service orchestration is at the heart of a controversy over just what constitutes an ESB. One camp asserts that ESB is the natural evolution of EAI and therefore must be able to integrate more than just Web services, which means supporting more than just SOAP over JMS and HTTP. Others argue that an ESB is a new creature, and while it serves a purpose similar to integration, it is at a higher layer in the SOA stack than pure integration technologies.
We're going to meet somewhere in the middle. Although we agree integration of legacy technology (CICS, COBOL, CORBA) should be handled at the service-enablement layer, some integration is so commoditized that it should be handled at the service-orchestration layer. Database access, for example, shouldn't require coding--aside from the SQL necessary to craft a query--and therefore should not require the overhead of being service-enabled in addition to the already performance-penalizing JDBC connection. Nor should pervasive enterprise applications that are generally integrated with application-specific database connectors or specialized API calls, such as PeopleSoft, SAP or Siebel, be placed at the service-enablement layer. Such connectivity is, or is rapidly becoming, commoditized and should be easily accommodated by enterprise applications whose purpose it is to communicate and orchestrate their services. Our grading of adapter support within the "integration" category reflects this view. Although it wasn't a high percentage of the products' grades--which reflects the small number of adapters we expect to see supported at this layer--it is part of our criteria because we believe you expect the same.
Core to the function of ESB are transformation and mapping. Data arriving in one format needs to be transformed before being sent to other systems, necessitating the mapping of fields within one document to another. The mechanisms to achieve this varied wildly, from Fiorano's most elegant visual mapping tool to IBM's multistep process.
|
|
We preferred Fiorano's and Oracle's visual tools for mapping data--being able to drag and drop fields and XSLT functions to construct both simple and complex transformations is a big plus (see Fiorano screen at left). BEA's drag-and-drop interface was good, but it entailed an understanding of XQuery/XPath that should not be required at the service-orchestration layer. TIBCO's BusinessWorks, like most products, relies on XPath and, like BEA AquaLogic, it constructed the XPath for us. But unlike AquaLogic, BusinessWorks doesn't necessitate an understanding of XPath to implement your scenario. Much better.
IBM's setup required us to import the appropriate schemas, create a message map between the two--using a visual mechanism--and then generate an XSLT from the map that we could use in our orchestration. Although the process was seamless, we were a bit put off. IBM has two formats for data, the protocol and the wire. Wire format is how data is formatted as it travels through the bus. Protocol format is how it's parsed when it comes in and "rendered" when it goes out. We found IBM's flexibility in this regard impressive, and its ability to distinguish between wire format and protocol format is useful, but we'd like to see something less complex for common scenarios, such as mapping XML to XML.
Software AG has further to go. With Service Integrator, we had to construct our XSLT from scratch, though it did provide an XPath expression builder that was accessible from most places XPath was required, but not all. More than once a copy-and-paste approach was necessary to provide the expected XPath expression, something we'd like to see rectified in future releases. Even so, Service Integrator is light-years ahead of Sonic's Stylus Studio, which required us to write XPath from scratch or attempt a copy-and-paste routine from within a hierarchical-tree-structure view of the XML document we were working on. Mapping the "PROD_NO" value from the inbound XML document to the database parameter, for example, required constructing an XPath query and then adding all appropriate name spaces to the step in the orchestration.
Orchestration and service management are at the heart of ESB management. Not only do developers need to orchestrate services, they also need to debug and test, and administrators then need to manage those services. Orchestration was not an issue for any of the products, though we were frustrated by IBM's complex modeling requirements. But debugging and testing the services we orchestrated was a mixed bag.
We were thrilled with the capabilities included in many of the products for debugging service orchestrations. BEA's testing tool offers a wealth of options as well as a detailed trace of each step in the orchestration and the data, broken down by headers and payload, right from the browser. Oracle's test tool is also browser-based, and it provides even more details than BEA's, including execution times on a per-orchestration-step basis. TIBCO BusinessWorks' debugging facilities are excellent, and we like its conditional breakpoint functionality, something we didn't see in other products.
Software AG includes a debugger within its XML Mediator Studio that let us set breakpoints and even modify values within documents as they moved through the orchestration. Software AG also provides a testing panel to execute Web services--including a nifty checkbox for WS-I compliance testing--but it still isn't as impressive as the test tool offered by BEA. Fiorano's test capabilities let us send and receive messages, but we couldn't view the steps in between. Hey, that's better than nothing, which is what we got with most products.
Although Cape Clear Orchestration manager also offers a view of execution times on a per-step basis, it doesn't provide a mechanism for viewing the actual message being passed to each step, something BEA and Oracle provide. Note that these testing tools and debuggers actually stopped the messages at each breakpoint we set, in real time, allowing for real-time analysis and modification (see BEA's AquaLogic screen, right).
TIBCO BusinessWorks provides details on a number of executions and execution times, but it did so on a process-by-process basis, which meant that we had to drill in and out of processes to find out the number of times our JDBC query had executed, and what the average execution time had been. Even though the orchestration containing the JDBC query is called from within another process, there are no direct drill-down capabilities in the user interface to get us from an outer orchestration to an inner orchestration. We like the attention to detail provided, but would like to see contextual drill-down capabilities within the process details in the future.
Other products tested simply don't provide a mechanism for viewing service-orchestration details, and only Oracle SOA Suite and TIBCO BusinessWorks gave us the option to terminate long-running or errant orchestrations. Cape Clear offers a minor configuration option during deployment that let us specify how the server should react to failed or exceptionally long-running orchestrations--deletion or attempted recovery--but that was it. We'd like to see more control over service orchestrations at the server level in all products.
We started this ride on the enterprise service bus with the intention of testing the performance of our scenario using NWC Inc.'s existing infrastructure. By the end of the trip we discovered that the ESB market has generally agreed on the protocols and standards that an ESB must support ... with two exceptions: JDBC and SMTP are hit and miss. Many suites support both, but some require the development of compute nodes (IBM) and Java Web services (BEA and Cape Clear). After much hemming and hawing we were forced to remove performance testing as a grading criterion because we were unable to perform a true apples-to-apples comparison.
Those products on which we could performance-test convinced us that, regardless of how the product performs out-of-the-box, there's plenty of tuning to be done. Take a look at JVM (Java Virtual Machine) memory tweaks, the number of connections allowed for HTTP connections and the code implementing a service; all are factors in the overall performance of any given scenario. In addition, the ability to perform in-memory calls between orchestrated services is imperative, especially when the orchestration involves SOAP over HTTP. Removing the overhead of a TCP session when services reside on the same machine is a huge performance benefit, and you should inquire about this capability to ensure it's available.
Another impediment to performance is the execution of complex style-sheet transformations or the use of encryption/decryption as per the WS-I Basic Security Profile. Being able to perform transformations quickly and efficiently will have a direct impact on the overall performance of any ESB. The choice of XML parsers, too, will have an impact, as well as imposing possible limitations on the size of XML documents that can ride the bus. For that purpose we asked each vendor whether the XML parser and XSLT engine provided with the product were pluggable--that is, could they be replaced or pointed at external hardware, such as a DataPower or Forum Systems device. As IBM has recently purchased DataPower, we hope to see support for pluggable XSLT engines in the near future in IBM's offering (see "XML/XSLT Support" right).
At the end of the ride, BEA System's AquaLogic Service Bus got our Editor's Choice stamp of approval, with Oracle coming in a close second, hindered by a large gap in pricing. Although AquaLogic Service Bus lacks the integration of Oracle's SOA Suite, BEA makes up for that with its awesome Web-based orchestration environment and elegant monitoring and reporting features.
TIBCO's BusinessWorks isn't far behind the leaders, held back primarily because of management and its less-elegant transformation and mapping capabilities. BusinessWorks did come in several heads higher than Fiorano's SOA 2006 Platform and Cape Clear's ESB, which both managed to take spots in our Top 5.
We set pricing grades based on our NWC Inc. scenario: Two processors, running Windows 2003 Server; adapters for Oracle 9i database, SQL Server database, JMS, SOAP consumer, SOAP producer and e-mail; five administrators, 10 developers, 100 users. The total cost to deploy ranged from BEA's low quote of $40,000, to IBM and Fiorano posting the highest prices, $170,000 and $170,995, respectively. In the midrange were TIBCO at $75,000, Sonic at $76,250, Oracle and Software AG both at $100,000, and Cape Clear at $105,000.
Lori MacVittie is a Network Computing senior technology editor working in our Green Bay, Wis., labs. She has been a software developer, a network administrator, and a member of the technical architecture team for a global transportation and logistics organization. Write to her at [email protected].
What's Up with BPEL
Analysts like to shout a lot about BPEL and claim an ESB should support the Business Process Execution Language spec, period. But there's some contention in the industry about BPEL, for several good reasons. First, BPEL 1.1 is merely a specification, not a standard. The second, more compelling, reason to ignore BPEL is that version 2.0 is vastly different from 1.1--and it's version 2.0 that's expected to be adopted as the standard.
Interestingly enough, BPEL 1.1 is often used as the orchestration mechanism for both services and business processes, even though BPEL lacks the notion of human workflow and focuses solely on machine-to-machine interaction. This makes it particularly useful for orchestrating services, as we noted with Oracle and Cape Clear, but not nearly as useful at the business process orchestration layer.
In addition, testing showed that BPEL as a specification implementation is not yet portable. Even though Fiorano, Oracle and Cape Clear all support BPEL 1.1, none was able to successfully import and use another's BPEL without modification. Oracle came closest to being successful--we cut and pasted the source of Fiorano's BPEL into its design-time environment and with a few changes, got it working.
Other attempts were not successful, and we came to the conclusion that, while BPEL 1.1 is just fine for service orchestration, it's not the be-all and end-all of service orchestration. Interoperability among BPEL engines will have to wait until a standard is accepted, and then perhaps we'll agree that BPEL is a definitive requirement to be considered an ESB. How We Tested ESB Suites
We jumped on each ESB by installing it on a dual Xeon 2.6-GHz Dell 2650 with 1 GB of RAM, running Windows 2003. Where out-of-the-box integration with NWC Inc.'s Oracle 9i database was not provided, we used the appropriate JAR (Java Archive) files from Oracle. Where coding was required, we used Java, even if support was offered for multiple languages. All SOAP messages were required to be DOC/LIT encoded, as per WS-I Basic Profile interoperability guidelines.
We attempted to orchestrate a business service to perform order fulfillment for NWC Inc.'s flourishing widget enterprise. The scenario was crafted to let us fairly evaluate core ESB features, such as transformation, protocol support, orchestration and content-routing capabilities. We used OpenJMS as an exercise in configuration and management of a JMS queue because it is not generally supported out of the box. We, therefore, could evaluate the mechanisms provided by each ESB for configuring foreign JMS servers, which, like the JDBC requirement, let us better understand the underlying architecture of each product.
While implementing our scenario we became familiar with each product's design-time interface and deployment mechanisms. Where BPEL 1.1 support was offered we exported and imported among vendors to evaluate compatibility (see "What's Up With BPEL?" for our results). We also examined the integration features offered by each vendor to meet common enterprise needs, such as database and application (SAP, PeopleSoft, Siebel) integration. We found a great deal of differentiation in this area, lending credence to the perception that the definition of an ESB is far from agreed upon.
Our implementation required each ESB to be both a consumer and producer of Web services, and we used the development of services and inclusion of external services as a mechanism to evaluate integration and use of a registry/ repository. Although most vendors whose products we tested take dual advantage of external UDDI-based registries and embedding of a registry within the bus (usually Systinet, but sometimes the vendor's own implementation), we discovered that the design and run-time role of the registry as it pertains to the bus is still somewhat in flux.
NWC Inc. implementation scenario logic:
1. Accept a SOAP-over-HTTP purchase order from the client
2. Call out to an Oracle 9i database via JDBC to determine the quantity in stock of the widget being purchased
3. Based on the quantity of the widget, perform a content-based route
a. If the widget is in stock
i. Call out using SOAP over HTTP to a shipping service
ii. Send an e-mail to the customer using SMTP
b. If the widget is not in stock
i. Put an XML message on a JMS queue that notifies manufacturing
ii. Send an e-mail to the customer using SMTP
All Network Computing product reviews are conducted by current or former IT professionals in our own Real-World Labs®, according to our own test criteria. Vendor involvement is limited to assistance in configuration and troubleshooting. Network Computing schedules reviews based solely on our editorial judgment of reader needs, and we conduct tests and publish results without vendor influence.
R E V I E W
ESB Suites
Verizon is demoing crowd analytics technology that uses 5G along with AI and mobile edge computing to give stadium operators insights into the flow of fans throughout a venue.
As we move closer to fully realizing the potential of Wi-Fi 7, it becomes evident that the RF, signaling, and throughput tests are not merely procedural checkpoints but the pillars of excellence in wireless communication.
With low earth orbit (LEO) satellites fast becoming the norm and 3GPP unifying telecom standards around the globe, the satellite IoT and machine-to-machine (M2M) sector ought to be looking at a bright horizon. But is this the case?
