How to Handle Request Failures

This column is in response to a question from an NDCF reader. He wanted to know: Is there any way to handle 'in flight' HTTP request failures more transparently?" One example would be when a response to an HTTP request is sent fully from a Web server and the server crashes. The problem with this is that the request can fail at any point – both before and after it reaches the destination.

This is a very interesting question with a lot of nuances and shades of gray! The answer is neither a simple "yes" nor a "no."

The challenge in this kind of situation is that the surrounding components don't know how much of the request was digested and acted upon prior to the failure. As a result, it is possible that half a request was processed and some action started before the failure occurred. If a load balancer fronting the Web server were to send that request to another server after the failure, it is possible that an action gets duplicated. This could mean, for example, that a credit card gets billed twice.

In most Web applications, there are a few things that require a user's request to stay put:

From a security standpoint, if an attacker figured out the magic sauce to make a Web application stall or break, the last thing you'd want is the load balancer to detect the server going down and retransmit the attacker's payload to every other Web server!It is worth noting that load balancers configured to health-check servers will automatically reroute new requests to the remaining servers. This can still cause a bit of grief for application users that were set to persist with the failed server, as they will need to log in again to re-establish their state.

The bottom line is that sometimes it is better to see an error message or broken page so that the fact that something went horribly wrong becomes obvious. When actions half worked, but continued, and then later broke down completely, they are a lot tougher to track down and fix because it isn't clear where the problem started.

That said, there are some instances where load balancers can and do provide limited recovery from some kinds of application infrastructure failures:

It is architecturally possible to provide more advanced functionality than that addressed above. As such needs become more evident in the future, the load balancers with flexible request/response processing architectures (as described earlier) may evolve to provide more advanced recovery from server-side failures – while mitigating adverse side effects from any application layer attacks.

— Prabakar Sundarrajan, Chief Technical Officer and Executive Vice President, NetScaler Inc.0