- They gave me invaluable advice and provided insights about Web testing. Pick everyone's brain including vendors!
- For defect tracking at TRIP.com, we chose a Web-based tool, TeamTrack (now called TTrack), from a startup company called TeamShare. It, also was far less expensive than its competitors, but it was easy to implement and customize. At tensegrent, which is a brand-new startup on a small budget, we use the free Bugzilla and it is fine for our needs.
- configuration management, we again turned to a smaller, innovative company which produces an inexpensive, easy to implement and learn yet robust tool, Perforce. At tensegrent, we use freeware, CVS–it lacks some features, but the development team is small and can work around its drawbacks.
- Search the Web for resources. I would have quit TRIP.com after a week if I had not found an excellent Web site that points to information about testing Web applications and lists of tools. These sites, in turn, led me to more tools and information Here are some examples:
- Hire good help: It proved impossible at first to find experienced test engineers in our area, so we at TRIP.com hired bright but inexperienced people with the right qualities that make good test engineers: enthusiasm, dedication to the end user, and determination. A caveat: inexperienced testers who have no programming experience have a harder time learning a scripting language for an automated test tool. However, by using a combination of outside classes, hiring consultants and patient, one-on-one training, our testers learned UNIX, SQL, test scripting, HTML, configuration management, and other technical skills. You're going to spend money either way - paying high salaries for experienced test engineers, or training novice testers. Once you've turned them into pros, remember to keep them challenged, happy and well-compensated so other companies don't poach them!
- Get input about quality from all departments in the company. I formed a quality board with members from sales, marketing, customer support and travel to gather fresh ideas about error prevention and prioritization of regression testing. Whenever major problems occured, we held a quality review panel where representatives from development and information systems heard short presentations from the people who experienced and fixed the problem. The panel studied the issues and recommended steps to prevent such problems from recurring. This was a big effort and to be truthful, it was difficult to get follow-through. But give it a try. We also held post-mortems after all major launches to see what lessons could be learned. By employing these methods, we learned some valuable lessons!
- Insist on a test environment that is exact replica of, but is entirely independent from, production. You can't emulate a production load without the equivalent of production hardware and software. Since the production architecture is likely to change in response to increased traffic and other considerations, this is a moving target. The test environment will need to be updated in synch with the production environment. The architecture is key too. If the production servers are clustered, your testing had better be done in a clustered environment. If part of an application runs on a stand-alone machine, it must do so in your test environment. Establishing and keeping up development, test and production environments was a huge challenge. Even when the entire company is sold on the idea of a proper test environment, there are business and technical reasons (read: excuses) that get in the way of reproducing the production environment in for testing. Don't be complacent, and never give up. Make sure you have the best test environment you can get for each application going into production,
Stranger in a Strange Land
Bringing Quality Assurance to a Web Startup