To that end, I participated in the Interactive Testing Challenge at RSA last week (ok, I admit, it wasn't just for that reason -- I did it for fun, too). Carefully not called a hacking contest (by the organizers anyway), it was a three day event meant to exercise web application exploitation skills.
First of all, hats off to Security Innovation for a great contest. It can be really hard to find the right difficulty level for a live-fire scenario like that, and the sample online bank built for the event was perfect.
The most important factor in the contest besides basic web exploitation skills (cross site scripting (XSS), SQL injection, cross site request forgeries (CSRF), etc.) was speed. The top two contestants from each of the first two days competed at the end of the day in a best of three challenge for a spot in the finals on the third. The first day ended with myself and a technical staff member from the Church of Jesus Christ of Latter-Day Saints--not exactly who you'd expect to end up competing at the end of the first day of the biggest security conference in the planet.
The semi-finals each day were nerve-wracking. Announcers with microphones described the attacks and potential defenses as the audience stood around watching the two contestants on overhead displays, helping to increase the tension. Having both participated and watched, it certainly was much easier to spot the right answer when you weren't under the gun.