likely you are going to find out how everything is supposed to work, and so your testing is not going to be as good as it could have been. But even starting in the middle understanding who I am designing this product for? Who are the developers designing the product for? And what kinds of things would they do? What kinds of crazy things would they do? What kind of completely off the wall things would they do? Those are things that help you figure out how to test the software. Even if you have not been involved from the beginning and you don't have any clue what it is supposed to do.
Carol: Right. Now, we have had some questions. Craig Byers who is the principal at American Management Systems Inc., sent me some questions for you that I will throw out and we can sprinkle them throughout our broadcast. I would like to give the toll-free number so if we have some people who are listening who have the guts who would like to step out of their shells and actually pose a question to Johanna. I have never known you to ever bite anybody. I have never bitten anybody. I think the callers would be safe calling in, don't you, Johanna?
Johanna: Absolutely. You know I'm in my office, so even if I spit it is only going to be here, not a problem.
Carol: And I am at a completely different place, so anybody who calls in, they are going to be safe. The toll-free number is 866-277-5369. Again, that's 866-277-5369. I would like to ask you a little bit about some measures that you might recommend for assessing the quality of the testing process and the test pieces or script. And as soon as we come back from these short messages, I will give you a chance to answer that.
Johanna: Okay.
Carol: We will be back with more of Johanna Rothman and Quality Plus! E-Talk...Welcome back to the show. I am Carol Dekkers, and my guest this week is Johanna Rothman. We are talking about Test Management 101. Before we went into break, I just posed her a question that was sent in by one of our listeners which is what measures would you recommend for assessing the quality of the testing process and test cases for scripts?
Johanna: Well, I measure a few different things. One is that for the test cases--how good are the test cases? You need to also understand that at some point the stuff that you measure about that is not going to be relevant. Let me try and explain what it is I measure and then why I think measurements start to fade in importance. I measure the defect find and close rates by week, especially over a long project. For a very short project, I have been known to measure them by day, but those are six-week projects. If any of you are doing E-projects, then you know how short these things are and you do want to measure stuff by day to get a better picture. But, I generally measure by week. I measure the defect find and close rates normalized by testing effort. So, if I have people, you know, five people in my testing group and they are for some reason not running tests or not developing tests for a given day or a few days, then I have to try to normalize that information; otherwise, I do not know how well I am finding defect.
Test scripts and test cases aren't just good
About the author
Johanna Rothman, known as the “Pragmatic Manager,” helps organizational leaders see problems and risks in their product development. She helps them recognize potential “gotchas,” seize opportunities, and remove impediments. Johanna was the Agile 2009 conference chair. She is the technical editor for Agile Connection and the author of these books:
- Manage Your Job Search
- Hiring Geeks That Fit
- Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects
- The 2008 Jolt Productivity award-winning Manage It! Your Guide to Modern, Pragmatic Project Management
- Behind Closed Doors: Secrets of Great Management
- Hiring the Best Knowledge Workers, Techies & Nerds: The Secrets and Science of Hiring Technical People
Johanna is working on a book about agile program management. She writes columns for Stickyminds.com and projectmanagementcom and blogs on her website, jrothman.com, as well on createadaptablelife.com.
