Classic Software Testing Is Broken: An Interview with Regg Struyk


CP: Wow.

RS: That’s the sheer amount. That’s continually, exponentially doubling every year. That’s not going to go away. The current methods and the current tools that we have for testing, they certainly do not. They don’t actually have both the capability, and also our mindset will not be able to handle those sheer volumes and the integrity of that data. We’re going to have to behave differently and think differently how we test large amounts of data.

CP: That streams into my next question here. Is there any particular methodology or trend that you think has a lot of potential going forward to meet those challenges for testing?

RS: Yeah, I think it really comes down to hybrids. I wouldn’t necessarily say that there’s a specific hot trend. What I would say is that it’s a combination of things. I like the agile methodology, but let’s be honest here, that agile was written and developed by developers. It isn’t testing-focused. In fact, unfortunately, testing was an afterthought, which, a lot of times in this world, in our world, is that that’s an unfortunate struggle that we have to deal with on a daily basis.

Having said that, though, the reality is that inserting testing into the agile process—being able to be a key player to becoming one of the key stakeholders in those stand-up meetings, and having testing involved from the very beginning—really, I see that as the focus. Also, that whole concept I mentioned earlier about test early, test fast—I cannot say enough about organizations that succeed and are very competitive are the ones that view testing as a key component of the entire development process. I think that’s a key trend.

The other is the mindset is going to be a big difference. I’m seeing a lot of companies where the focus now is not on . . . Let me step back for a second. The traditional way to look at testing, from an organization perspective, was to test the behavior of a product, per se. You have a software application. It’s like, does it do what the requirement intended for it to do? I would say that that is the wrong way to look at it. I would say that the view needs to be, what is the customer expectation?

Companies that become customer-focused . . . I’ll use a name-drop. The Nikes of the world, the Apples of the world, they look at products and they say, “What are the customer expectations? How does the customer perceive this?” This is how we have to start testing. There are some companies that have learned this lesson through losing a lot of either both advertising and money.

For example, Xbox, a few years ago, with—I can’t remember what version of Madden football it was, but they released early. Product was really full of defects. Customer expectation was extremely high. That was the pressure to release it. What happened is that it was a flop at the time. There’s countless examples, especially in the gaming community, of products that if they’re not released and the customer expectations are high, they’re going to get torpedoed. I’d say that’s the biggest thing, is the customer expectations.

CP: OK. You talked a little bit about holding on to the traditional ways. How much success can a company really expect if they have a firm commitment to those traditional ways of testing?

RS: Yeah. That’s a great question. Some of these companies are actually heavily regulated. The medical device industry is one of them, now finance, et cetera. There’s a lot of heavily regulated environments. Not necessarily that that’s a bad thing, because obviously, if you’re using medical devices, you want the risk as a patient, a consumer, to be very low. However, that does foster the environment of traditional methodologies. Really, what that does is it slows down the pace to be very competitive and also release quality product.

You’ll see a lot of the game-changers in a lot of the—especially when you’re coming to apps and you’re looking at cloud-based products—a lot of the game-changers will actually be companies that are lean and that are fostering new ways to test and develop software. When they do that, they are disruptive forces within those industries.

Certainly, I’d say that you have to balance that, and that the traditional methodology doesn’t make sense to follow if you’re competing against companies that are using other types that are hybrids, that are more nimble, that are using more of—being able to adapt to the very quick changes that are happening in the marketplace. Yeah, definitely, those legacy techs really do, and we see it all the time. HP is a good example, but not just HP. There’s all kinds of different companies that they have old-legacy ways of doing things, legacy code, and lots of smaller, nimble disrupters that actually come along and change the marketplace.

User Comments

1 comment

About the author

Upcoming Events

Sep 12
Oct 01
Oct 15
Nov 05