e-Talk Radio: Rothman, Johanna - Test Management 101


kinds of features that you want to put into it, there is the number of defects and the kinds of defects that you are willing to ship with. You have a bunch of constraints. Do you ever have enough people? Probably not. Can you create the right work environment? Probably. So, there are a bunch of things that help you figure out what are all the requirements and constraints on your project. And the good enough quality people, of which I consider myself part, say let's talk about what makes sense. How fast can we ship it and still have something that people want to buy. When is this thing of value to a sufficient number of people? And the way that I actually find out if something is good enough, is that I use something called Release Criteria or Ship Criteria depending on how you think about it.

For me, Release Criteria is the time when the test manager and the project manager and the entire project team, hopefully the developers and the testers and whoever are on the team, the writers, anybody else, get together and say what is critically important to this project. What do we absolutely have to do before we ship? And then those become the criteria that you decide. Now, I have worked with some startups who said that we have to ship something on July 1st. It does not have to work very well. It has to have at least this feature in it, but if we don't ship it July 1st, we don't have a company July 2nd. Okay. So, July 1st is the one and only release criterion. But there are other companies where you say I don't want to alienate the current customer set. I want to make sure that I have good enough performance, which I am going to measure in this particular way for the next set of customers, and I have a couple other things that I absolutely have to do. Anything else, in a sense, is icing on the cake. And I would love to have it by the ship date, but if we don't have it by the ship date, I can live with that. There are some companies that say, we don't even care about the ship date, though there aren't too many of these right now. We will not ship this thing until we have this amount of coverage from our test, you know, we are actually going to measure basic test coverage. We will have to know that it has so few defects that it is safe for us to ship this project. So, release criteria helps you define, at least for me, helps me define what do I mean by good enough.

The other thing that I like about release criteria, is that it helps everybody on the project and everyone in the company understand what are our goals? What are we aiming for? So you don't, as a test manager especially, you never have to be the person who says this product is not ready to release. The project manager becomes the person who says, "We have not met our release criteria." Now, you can still choose to release, even if you have not met your release criteria, but at least you are doing it honestly and openly. And you are not saying, "Well, we sort of made that, didn't we?" No. No one's hands get tied behind their backs. It is not one of those, yeah, yeah, group think things. We are saying, "Yes, we met the criteria or

About the author

Johanna Rothman's picture Johanna Rothman

Johanna Rothman, known as the “Pragmatic Manager,” helps organizational leaders see problems and risks in their product development. She helps them recognize potential “gotchas,” seize opportunities, and remove impediments. Johanna was the Agile 2009 conference chair. She is the technical editor for Agile Connection and the author of these books:

  • Manage Your Job Search
  • Hiring Geeks That Fit
  • Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects
  • The 2008 Jolt Productivity award-winning Manage It! Your Guide to Modern, Pragmatic Project Management
  • Behind Closed Doors: Secrets of Great Management
  • Hiring the Best Knowledge Workers, Techies & Nerds: The Secrets and Science of Hiring Technical People

Johanna is working on a book about agile program management. She writes columns for Stickyminds.com and projectmanagementcom and blogs on her website, jrothman.com, as well on createadaptablelife.com.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!