Useful Metrics and the Problem with Performance Testing Programs: An Interview with Scott Barber

[interview]
Summary:

Scott Barber is the chief performance evangelist for SmartBear and an author of several books, including Performance Testing Guidance for Web Applications. In this interview, Scott chats about useful test metrics, communication, and the problem with performance testing programs.

 

Scott Barber is the chief performance evangelist for SmartBear and an author of several books, including Performance Testing Guidance for Web Applications. In this interview, Scott chats about useful test metrics, communication, and the problem with performance testing programs.

JV: I’m here with Scott Barber. You’re a familiar face around these parts. I’m glad to have you talking with us today. Can you explain your background to our listeners and readers?

SB: Sure. I’ve primarily been a performance tester for the last fifteen years. Although I say performance tester, I’ve been doing a lot of training, consulting, and speaking. I’ve managed to get a couple of books out there. That’s always a lot of fun. Actually, it’s just a lot of work. I started out, believe it or not, with a degree in civil engineering.

JV: I didn’t know that.

SB: Yeah, I paid for that with an Army ROTC scholarship, so then I was an Army officer for a while. I got recruited out to update one of the military systems that, needless to say, was a little behind the times.

JV: Yeah, I can imagine that.

SB: I bounced around a little bit before I ended up in the developing and testing space. It took a little bit of a winding path to get here, but I’ve enjoyed it since I got here.

JV: And you’ve got some upcoming sessions in STARCANADA, so I thought it would be good to talk about that. Why don’t we get into a topic about metrics? Explain to me how metrics can destroy your soul.

SB: Metrics are a really interesting topic psychologically, which is not what we normally think of when we’re talking about metrics, right?

JV: Right.

SB: You can think about it for yourself. Any job you’ve ever had or when you’re back in school, what’s the first thing you try to do when you’re taking a class? You try to figure out what the teacher grades on, what they want to get your A. As a student, that’s your goal in life, to do the least work possible and get an A.

Your goal is supposed to be to learn everything I’m supposed to learn. But that’s not what you do. You game the metric. It’s human nature. Some people take it to an extreme, don’t get me wrong, but it’s just human nature.

Now let’s take this into tester land. Let’s say your boss or the executive is interested in tracking bug counts.

JV: A quantifiable measurement, a number.

SB: That’s right. It’s a number. So, as a tester, you kind of end up finding out a sweet spot; if every week I report between twenty-five and thirty bugs, then the boss thinks I’m actually doing my job. But, I’m not reporting so many bugs that he thinks I’m gaming the system. So, every week I’m going to try to find twenty-five-to-thirty bugs. Which means this week, “Hey, you know what? I’ve got a bunch of great bugs on Monday. What am I doing the rest of the week? Not a whole lot.”

Most people are more ethical than that. I’m taking it to an extreme.

Pages

User Comments

2 comments

About the author

Jonathan Vanian's picture Jonathan Vanian

Jonathan Vanian is an online editor who edits, writes, interviews, and helps turn the many cranks at StickyMinds, TechWell, AgileConnection, and CMCrossroads. He has worked for newspapers, websites, and a magazine, and is not as scared of the demise of the written word as others may appear to be. Software and high technology never cease to amaze him.

Upcoming Events

Mar 27
Apr 13
May 03
Jun 01