Security Innovations has been making the rounds at software conferences, hosting a challenge for security testers. At recent conferences including STAREAST 2004 and RSA--the leading testing conference and the largest security conference in the world, respectively--delegates lined up to take a shot at finding the security flaws in a sample Web site built expressly for the challenge.

In the staging area at STAREAST 2004, Security Innovations employee Maureen Robinson explained, "We've built a small site of about six or seven pages with known vulnerabilities to exploit. A questionnaire accompanies the challenge, and participants have to find the flaws… all of which are things you might uncover on a real site."

Security expert Hugh Thompson jumped in, "People are really getting into this. We've been open six or seven hours over two days [at STAREAST], and we've had to kick them out at the end of the day so we can close down shop."

Participant Richard Durham of Citrix in the UK agreed with Hugh's assessment of the enthusiastic response to the challenge. "The first few times I went by the challenge area, people were lined up to get at the three terminals. I finally got my shot late in the second day," he said. And what did Durham think once he tackled the challenge? "It was frustrating but fun. I didn't do as well as I thought I would. But testing is all about breaking things, and this was a chance to show what we've got."

Participants were expected to find login vulnerabilities, uncover confidential names and passwords, expose the name of the database table that stores account information, and cause browsers to fail when the user visited a certain page on the site. All of the answers were recorded on the questionnaires and some of the tasks required verification by Security Innovation staff. The grand-prize, a Microsoft Xbox, was awarded to the person at each conference who found the most security flaws.

Thompson has been impressed with people's efforts. "On the whole, people are doing well at finding at least some of the problems. The people here at STAREAST have actually done better on average than the guys at the RSA conference, which is all about security. That's cool."

What did participants have to say about the experience? Mitch Goldman of The Bankers Bank said "I couldn't [break the site], but knowing it could be done made me curious about what I didn't know." Others were surprised at the number of common vulnerabilities and how easily they can be exploited once you start understanding how to uncover them.

Tina O'Donnell from Chicago summed it up this way, "I was surprised to see how easy it can be to get account names. If you can do this in the real world, uh oh."

Though many tried, success varied greatly. Some were stumped right away. Others labored at it for long stretches and repeat visits. But at each conference at least one person cracked all the flaws. So who have the lucky Xbox winners been so far? At RSA the prize went to Jeremiah Grossman, CEO, WhiteHat Security. At STAREAST, two participants found all the known vulnerabilities, and the prize went to Nadeem Haq for detailing some additional flaws in the system.

The next stop for the security testing challenge is at the Better Software Conference & EXPO in San Jose, CA September 27-30, where it will go by the name "Hackerland."