Test throughout the Entire Development Process: An Interview with Andreas Grabner


Josiah: What do you think are some of the benefits of catching these smaller issues earlier on in development rather than just trying to take care of them all at the very end?

Andreas: I think there are multiple benefits. First of all, for developers, if they find, "Hey, this coaching that we just did had a significant impact,” for instance, they're now executing twice as many sequence of statements to get the same results. If they learn about this by changing the code, because they, for instance, used the frameworks in a different way… if they see that this small change has a strange impact, like, executing too many previews to the database or allocating more memory than necessary, then developers will maybe learn while they develop new features, on how to better use the frameworks and tools that they're using so they can start… they educate themselves with that.

On the other side, if you catch problems earlier on, you don't have to work to find this low-hanging fruit of the basic problem patterns the first time when you do a load test. Basically, the load-testing guys can then actually really focus on the tough problems and once again, breaking the application with only five users instead of 500 users that they should simulate on.

I think it's a benefit for developers. They will learn how to develop their software better and testers can really focus their expertise on real tool, large scale load testing because they don't have to deal with breaking the applications, right? With very small load.

Josiah: Can you give some real-world examples that you've experienced of major problems that were caused by just small changes that could have been taken care of early, but ended up snowballing and becoming something bigger?

Andreas: Yeah. I'm going to give you two examples. One is actually from our own software that we actually launched. We have internal… internally we use our backend services. We basically keep track of customers and their licenses. We recently introduced a free trial version of our product that you can download. Obviously we also have the reports. I, for instance, I can go in and see who downloaded the free trial after I have my talk at the conference.

We implement the report and our developers would implement the report, I'm sure at the best intention to make a nice report. Basically, what they did from one build to the next, they introduced, they used a new version of Hibernate, one of the more very popular OR networks out there. They spent no time on figuring out how to best configure Hibernate for their own purposes.

Now, in order to get my report, instead of executing a single-sequence statement that goes to the database and gives me the free trials that signed up based on my last talk, actually, Hibernate went off and executed 2,000 sequence statements with the same report. Obviously, that's a small change for the developers because they just flipped a new version of Hibernate. For them, from a financial perspective, everything still stayed the same. In the end, if you need to execute 2,000 times more database statements to get to the same results, it's a small change with a very big impact.

The other example, actually I wanted to talk about one example that was very prominent last year in the US. The healthcare.gov example. When that website went down, we did some analysis and the very small thing that they forgot were basically some best practices on that performance optimization. The first version that they released had fifty different JavaScript files on the page. Not many files, not merged together, and these are just some small things. But developers added more JavaScript features on the page because they needed it to download a chip and different plugins.

What they forgot to do is to follow some best practices and merge these things together. Again, on their desktop everything worked fine. Later on in production when millions of US folks tried to get to that site instead of downloading one JavaScript file, they have to download fifty JavaScript files and that was basically not possible. Because the pipes to the web servers were just clogged and not able to handle them alone.

Josiah: It's crazy to think something of that much importance of that big of a scale could have been fixed by just a few small things early on in development. I would like to transition a little bit into mobile, speaking of quality. Do you think the quality bar from mobile apps is currently high enough? Are apps launching on phones and tablets with enough testing to back them up or do you think developers are kind of leaning on updates to be a kickstand for their apps?

Andreas: That's a good question. I think we are in a transition phase right now. Obviously, there's a lot of mobile developers out there that develop web mobile applications. We also know that the majority, I’m not one to give a percentage, but I would assume 90 percent of all the apps that are out there have very bad ratings. Nobody's using them because they're either bad quality or… in most cases are bad quality.

They definitely get bad ratings and therefore, basically, all the investment that they made in developing these apps is basically wasted.

About the author

Upcoming Events

Nov 05
Nov 14
Dec 05
Jun 03