Whether its focus is on detecting functional defects, performance bottlenecks, scalability issues, or security vulnerabilities, virtually every automated Web-application-testing tool on the market today follows one of two separate testing methodologies. The tool either analyzes the source code of the target application (white-box analysis), or it runs the application and interacts with it through its UI (black-box analysis). The relative merits of each of these approaches are hotly debated within the testing community, but the truth is that neither approach is adequate on its own. Black-box analysis is like chocolate, and white-box analysis is like peanut butter: Each is good on its own, but by combining the two we can create a much more delicious outcome and more reliable test results.
The main problem with using each type of tool individually is that one alone doesn't provide adequate code coverage. In this context, code coverage refers to the ability of the testing tool to identify the entire testing surface and thoroughly examine every path of execution of the application. Unless you can be sure that your entire Web application has been tested, you cannot rely on the results of the analysis. This assurance that all of the application has been examined is especially important for security analysis tools. A single functional defect in a page visited by only one user in a thousand may have little impact on the overall perceived quality of the application, but that same security vulnerability on that page may open the entire site to a devastating attack.
So, between white-box and black-box analysis, which offers better code coverage? Source analyzers definitely have the edge at finding every page in the application-they already have a list of pages from the source code. They don't need to resort to fragile techniques like spidering the site or prompting the user to provide a list of pages, as required by black-box analysis tools. Pages like administrator portals that are not linked to any other page in the site may be completely missed by black-box tools but would be found by white-box tools. White-box tools also have an advantage at finding unusual edge cases in the code. An online shopping site might process orders made on leap days (February 29) differently from orders made every other day of the year, and a black-box tool might never test this condition.