Is Quality Negotiable?

• to write test results out in XML format and have your team write a tool that reads the XML and produces an HTML page with graphic representation of tests passed, failed and not run.
• Verify the minimum success criteria. As they say in the Air Force, if the minimum wasn’t good enough, it wouldn’t be the minimum.
• Apply XP practices to test automation. Do the simplest thing that works, continually refactor, pair test, verify critical functionality with a bare-bones “smoke” test.

Conclusion: Delivering Quality
In the abstract, I asked some questions that I’ve discussed in this paper. Here’s a summary of what I have concluded. Disclaimer: despite all most a year doing XP, I have almost as many questions as I have answers. Practice XP and come up with your own conclusions!

Is quality negotiable? If negotiation means a dialog between the tester and the customer to make sure that the customer has clearly defined his quality criteria and that the acceptance tests are written to reflect these, then quality is negotiable. Because you, as the tester, and the customer talk about all aspects of quality, the customer can be specific about what he wanted and perhaps even defined stories that address criteria such as stability and performance under load. The developers can accurately estimate stories, and the customer can get the quality he’s paying for.

How can we negotiate quality? By asking lots of questions of both customers and developers and making sure that nobody makes assumptions. By making sure the customer understands how XP works and what his role is in the planning game and knows what to expect each iteration. By putting a price on quality in the form of story estimates and letting the customer decide what is most important for his business.

What are internal and external quality, and are either or both negotiable? Internal quality is quality as measured by the programmers. XP works best when each member of the team is allowed to do his best work. Internal quality may actually save money. External quality is quality as measured by the customer. The customer has to pay whatever it costs, so the customer should set the standard. The XP team helps the customer do this by telling them how much the criteria for quality will cost, in the form of story estimates.

What is the XP Tester’s Quality Assurance Role? Help the customer set quality criteria and write tests that verify them. Provide a reality check for the developers. Mentor the developers in testing and quality assurance practices. Developing and testing share a lot of skills, but are distinctly different professions.

How far should testers go in helping the customer define acceptance criteria? As far as possible in the given timeframe. Ask all the questions you can think of.

The XP books state that acceptance tests don’t have to pass 100%. The closer you come to clearly and completely defining the stories and the criteria for proving the stories work, the closer to 100% success you will have.

Negotiating quality makes the end of each iteration much more constructive and comfortable. The customer is satisfied that the stories were completed. He knew what to expect and his criteria for quality were met. The developers are satisfied that they did their best work and produced functioning code that is up to the customer’s standards. The tester is satisfied that the customer got what he paid for, without the developers having to give away the store for free.

The acceptance test results may prompt the customer to change his mind about what his quality standards