Network Computingwww.networkcomputing.com

As for reporting, by storing the data in a SQL-based repository of the user's choice, custom reports based on specific requirements can be created outside Protocol Modeler. The built-in reports are sufficient for most testing and even include (where appropriate) rollup graphs and narrative explanations.

Crash-Test Dummies

Unfortunately, we faced some insurmountable glitches with version 3.06 (shipped to us for testing) that were especially painful given Protocol Modeler's hefty $25,000 price. The QA cycle that let this version out the door leaves us less than impressed. According to Cenzic, the changes made in 3.06 were all performance related, but the developers seem to have outsmarted themselves: Protocol Modeler frequently ran out of file handles and crashed spectacularly. Any intensive test fell prey to these faults. The company issued patches and the engineering team scrambled, but Cenzic could not fix the problems during our testing window.

The Web-crawling wizard experience detailed earlier left us with Microsoft Visual C++ exception windows covering the screen and a crashed version of Protocol Modeler. We encountered similar disappointments on other long-running tests. For example, we tried running a SQL Disclosure attack on a single user-supplied input on a Web page being posted back to a Web server. We watched the logs on the HTTP server (the user will often find him or herself monitoring closer to the target application) and got a glimpse of the types of attempted queries. We observed 8,600 attack queries before Protocol Modeler finally crashed (Cenzic said the fault injector was probably close to finishing, given that number). Unfortunately the product doesn't do any checkpointing, so we couldn't find out if any vulnerabilities were discovered in those queries.

A more critical feature gap is the lack of any indication of the approximate and relative run-times of the fault injectors, some of which can run for hours or days, depending on the size of the test. Even nicer would be a: "This test requires about 7 minutes per iteration, times 60 loops = 7 hours" message.

The Protocol Modeler experience is, in many ways, like staring at a blank 6-foot canvas with a full palette in your hands. Using the tool successfully takes creativity. We don't claim to be the Picasso of the Protocol Modeler world--frankly, much of our work was painting by numbers with the wizards, though we did begin to devise some interesting tests as we became more comfortable with the Protocol Modeler environment. Make no mistake: The product is difficult to use. Allow at least a full week to ramp up on the tool and assemble a preliminary test network. This assumes you have advanced knowledge of both IP networking protocols (at every level) and advanced knowledge of security vulnerability theory and practice. If not, allow much more time. According to Cenzic, a few days of on-site training come with the purchase price. We recommend taking full advantage of this.

At press time Cenzic had launched a new product, Hailstorm Web. The core engine technology is the same as Hailstorm Protocol Modeler but also includes extensive workflow management for structured use within an organization. Security expert and QA analyst roles let application security testing tasks be distributed logically. Predictably, the focus is on HTTP-based applications.

Hailstorm Protocol Modeler 3.06, $25,000. Cenzic, (408) 626-9004. www.cenzic.com

Patrick Mueller is a senior security analyst for Chicago-based security consultancy Neohapsis. Write to him at pmueller@neohapsis.com.

Post a comment or question on this story.