The Case for Cooperation between White-Box and Black-Box Test Tools

[article]

have the advantage at finding unusual edge cases in the code. An online shopping site might process orders made on leap days (February 29) differently from orders made every other day of the year and a black-box tool might never test this condition.

On the other hand, source analyzers cannot analyze modules for which they don't have the source code. For Web applications, these modules could include controls or libraries purchased from third-party vendors, database-stored procedures, external Web services, or modules native to the underlying Web server or application framework. Also consider that newer Web technologies, such as Ajax, rely heavily on the client-side of JavaScript. JavaScript actually has the ability to programmatically modify itself at runtime. This makes reliable analysis of a complex JavaScript application virtually impossible when using only white-box tools, but a black-box analysis tool would handle this situation easily.

Given that neither type of tool provides assurance of complete code coverage on its own, your first instinct might be to use both a white- and black-box analysis tool and then combine the results. This is a great idea-when it comes to testing, as two heads are almost always better than one but, unfortunately, it still doesn't solve our problem. Remember the relative weaknesses of each tool: black-box tools can have trouble finding unlinked pages and unusual execution paths in the site, while white-box tools can have trouble with pages that use third-party controls or rely heavily on JavaScript. What if an application contained a page with both of these troublesome attributes, like an Ajax-enabled, unlinked administrator portal? Using two tools won't help in this situation, since the white-box tool will find the portal but won’t be able to test it; the black-box tool, on the other hand, won't be able to find the portal at all. This is not as uncommon a scenario as it might seem, especially given the exponential growth of interest in Ajax.

About the author

Bryan Sullivan's picture Bryan Sullivan

Bryan Sullivan is a security program manager on the Security Development Lifecycle (SDL) team at Microsoft. He is a frequent speaker at industry events, including Black Hat, BlueHat, and RSA Conference. Bryan is also a published author on Web application security topics. His first book, Ajax Security was published by Addison-Wesley in 2007.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Oct 12
Oct 15
Nov 09
Nov 09