Testing Your Worth

[article]
Summary:

There's no doubt that the current job market is tight and a little shaky for test professionals. In a climate where entire test groups are being laid off or trimmed to the bone, Johanna Rothman notices a trend in test management priorities that you might want to consider. Follow the story of how one test manager determined tester ROI and how testers might approach increasing their value.

In this uncertain economy, testers are losing their jobs, and entire test groups are being laid off. I don't think testers as a whole are out of the job market, but I think there's a new trend in the testing world toward testers who can provide maximum value, either in reading and writing code, writing automated tests, being a subject-matter expert, or being an industry expert. 

Why do I think this is a trend? Many managers I know are working to justify their test group's (and their own) existence. Test managers need to explain the benefits of their current testing strategy, whether it's to senior software managers or to finance people. Managers may or may not know much about software, but they know when the services they're paying for (testing) don't seem to be providing enough bang for the corporate buck.

Amanda, a test manager, wanted to replace a manual black box tester, Ginger, who'd moved to another part of the country. Amanda's VP, Jim, asked what kind of a person Amanda was looking for. When Amanda said she wanted to replace Ginger with another manual black box tester, Jim asked what the cost-benefit analysis was for that kind of a tester versus other kinds of testers. Amanda was initially at a loss for the analysis. She then decided she could look at what people did, and compare their activities to the numbers and kinds of problems the testers found. (Amanda didn't need to compare salaries; salaries were close enough that they were not a consideration.)

Amanda came up with this table for the most recent release of the software, organized by who found the highest total percentage of defects:

Name

type of tester

number of high severity defects found

% of high-severity defects found by this tester

total defects found

% found by this tester

Bertha

Test developer

3

3%

175

22%

David

Test developer

21

24%

153

20%

Harold

Manual black box tester

9

10%

131

17%

Ginger

Manual black box tester

5

6%

114

15%

Edward

Manual black box tester

9

10%

92

12%

Cameron

Manual black box tester

7

8%

75

10%

Franny

Exploratory tester

33

38%

41

5%

Totals found

 

87

100%

781

100%

Table 1: Total Defects Found by Tester
There's not enough information here to make an informed decision about the value of each tester, but if you look at raw defect counts, the test developers look like they've found more overall defects.

Unfortunately, overall defect-find rate per tester is a particularly bad metric. We can all inflate defect counts with not-very-interesting defects. And what about Franny, our exploratory tester who only found 5 percent of the overall defects, but found a whopping 38 percent of the high-severity defects? We not only want to keep Franny, we might consider getting more Frannys in the group. So let's look at the table again, this time sorted by severity of defects found.

About the author

Johanna Rothman's picture Johanna Rothman

Johanna Rothman, known as the “Pragmatic Manager,” helps organizational leaders see problems and risks in their product development. She helps them recognize potential “gotchas,” seize opportunities, and remove impediments. Johanna was the Agile 2009 conference chair. She is the technical editor for Agile Connection and the author of these books:

  • Manage Your Job Search
  • Hiring Geeks That Fit
  • Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects
  • The 2008 Jolt Productivity award-winning Manage It! Your Guide to Modern, Pragmatic Project Management
  • Behind Closed Doors: Secrets of Great Management
  • Hiring the Best Knowledge Workers, Techies & Nerds: The Secrets and Science of Hiring Technical People

Johanna is working on a book about agile program management. She writes columns for Stickyminds.com and projectmanagementcom and blogs on her website, jrothman.com, as well on createadaptablelife.com.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Nov 09
Nov 09
Apr 13
May 03