Testing in the Bazaar

Testing-specific mailing lists are maintained that help facilitate the discussion, as well as one-on-one email between team members. A second tool greatly used is Internet Relay Chat (IRC). Most projects maintain an IRC server for team communication. This allows for immediate, real-time communication-the closest thing to face-to-face contact a project can have.

Online access to the latest software source and builds. Testers need access to the latest project software-often both the source code and binary executables. To facilitate this, most projects provide build servers connected to the Internet from which anyone can grab the latest and greatest software. Build processes and source code maintenance are automated as much as possible. Build servers are often configured to provide software in the form easiest for testers and end users to get installed and running. This helps more volunteers get started testing project software quickly.

The Bugzilla defect tracking system. Open source projects need bug reporting and bug tracking systems. For smaller projects, bug reporting and tracking is typically managed via email and is the lead developer's responsibility. However, sizable projects must use a sophisticated system. The Bugzilla defect tracking system is used by many large open source projects (e.g., Mozilla, GNOME, and Nautilus). Bugzilla is a database for defect tracking coupled with an HTML front-end. This interface works well over the Internet. Testers can file bug reports quickly and effectively, and perform sophisticated queries. Furthermore, Bugzilla can be customized and tailored to an individual project's needs, since Bugzilla itself is open source software.

What Are Some Advantages and Disadvantages to Open Source Testing Methods?
Open source testing communities possess both advantages and disadvantages when compared to closed source, commercial testing activities. Two advantages are

Community involvement with few limits. When testing communities are built successfully, the open source project benefits. An able group of testers can expose bugs efficiently, write effective bug reports, and help deliver a better, polished project as development marches forward. Much of this is true for commercial development also, but open source testing communities are only limited by the number of volunteers they can attract and the team leader's effectiveness.

Early and constant exposure. The open source development model exposes project software to many users early in the development cycle. The software is available via the Internet to any and all who request it. With this limitless exposure, bugs are diagnosed and exposed quickly, design limitations are observed and corrected, and the project evolves quickly, making rapid gains well before its initial "release." Constant exposure of the software to testers facilitates bug detection and resolution.

However, these advantages can be a double-edged sword. Two disadvantages of open source testing are

Community development efforts require time and energy. While a testing community's strength lies in its volunteers and its leadership, community development efforts can take away from much-needed testing activities. For example, testing team leads have to constantly work to develop project awareness, recruit new volunteers, and steer team efforts. For a team spread across the globe, these activities can bog down a testing group. This is a disadvantage when compared to a commercial developer's core testing team with budgeted time and resources.

Goal determination can fluctuate. When a testing team's strength and effectiveness are in a constant flux, dependent upon the ebb and flow of volunteers, precise goal planning is difficult, if not impossible. The volunteer community environment makes it difficult to plan goals and schedules for reaching those goals, with no clear measure of productivity or team effectiveness.

Summary
For open source projects, test team building is an exercise in community development. Without the traditional resources that