e-Talk Radio: Rothman, Johanna - Test Management 101


kinds of features that you want to put into it, there is the number of defects and the kinds of defects that you are willing to ship with. You have a bunch of constraints. Do you ever have enough people? Probably not. Can you create the right work environment? Probably. So, there are a bunch of things that help you figure out what are all the requirements and constraints on your project. And the good enough quality people, of which I consider myself part, say let's talk about what makes sense. How fast can we ship it and still have something that people want to buy. When is this thing of value to a sufficient number of people? And the way that I actually find out if something is good enough, is that I use something called Release Criteria or Ship Criteria depending on how you think about it.

For me, Release Criteria is the time when the test manager and the project manager and the entire project team, hopefully the developers and the testers and whoever are on the team, the writers, anybody else, get together and say what is critically important to this project. What do we absolutely have to do before we ship? And then those become the criteria that you decide. Now, I have worked with some startups who said that we have to ship something on July 1st. It does not have to work very well. It has to have at least this feature in it, but if we don't ship it July 1st, we don't have a company July 2nd. Okay. So, July 1st is the one and only release criterion. But there are other companies where you say I don't want to alienate the current customer set. I want to make sure that I have good enough performance, which I am going to measure in this particular way for the next set of customers, and I have a couple other things that I absolutely have to do. Anything else, in a sense, is icing on the cake. And I would love to have it by the ship date, but if we don't have it by the ship date, I can live with that. There are some companies that say, we don't even care about the ship date, though there aren't too many of these right now. We will not ship this thing until we have this amount of coverage from our test, you know, we are actually going to measure basic test coverage. We will have to know that it has so few defects that it is safe for us to ship this project. So, release criteria helps you define, at least for me, helps me define what do I mean by good enough.

The other thing that I like about release criteria, is that it helps everybody on the project and everyone in the company understand what are our goals? What are we aiming for? So you don't, as a test manager especially, you never have to be the person who says this product is not ready to release. The project manager becomes the person who says, "We have not met our release criteria." Now, you can still choose to release, even if you have not met your release criteria, but at least you are doing it honestly and openly. And you are not saying, "Well, we sort of made that, didn't we?" No. No one's hands get tied behind their backs. It is not one of those, yeah, yeah, group think things. We are saying, "Yes, we met the criteria or

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.