Making Testers Miserable: An Interview with Andrew Wulf


Andrew Wulf runs TheCodist blog, is the lead iOS programmer for Travelocity, and owns Idle Diversions—an iOS game company. In this interview with Noel Wurst, Wulf discusses his role as a coder to "make testers miserable," the need for clean code, and practicing agile before it was a term.

Noel: You and I are meeting because of your blog,, specifically, an entry you did back in March. "My job as a programmer is to make testers miserable." I saw the title, and I have to admit, your first sentence did not disappoint. "When I deliver something to a tester or a QA person, I want them to suffer terribly, to fear coming to work, to wish they'd never become a tester, to be drained by the end of the day." I love it. So, my short question to that long intro is simply—why?

Andrew: Quality comes from what the programmer does; if I don't deliver, no amount of testing can make it better. A tester's job is to make sure nothing gets missed, but they can't fix the problems— they only point them out. A baseball manager can encourage players, design strategy, and do all the right baseball things. But in the end, if the players don't hit, field, or pitch, then the game is usually lost. Likewise, programmers have to produce good code, ideally before a tester ever sees the end result. The idea that you churn out code fast and loose and expect the testers to find all the bugs never results in a good end product.

Before I ever worked as a programmer, I briefly had a job as a chemist where I worked for a product testing company. Whenever I was done with a test, I wrote up a report for the customer. The bosses complained I worked too hard on writing up a quality report, they expected me to slap it together and "the girls in the front office will shape it up." Besides the obvious sexist comment, it made no sense to me at all.

My job was to do the tests and deliver results to the customer. I think that experience impacted my work later as a programmer, in that I want what I deliver to be just about ready for primetime—even if it's only a test. That way the tester can focus on finding all the little things: the edge cases, and the nitpicky things that will drive customers nuts. If they push the first button, and the app blows up, then I'm not doing my job very well.

So if the tester has to work hard to find things (assuming they are highly competent) then I as the programmer am on the right track. I also think a tester needs to challenge the programmer to defend how things work sometimes; having more eyes look at the app is always a good thing and sometimes having to defend how you implemented something might make you think a little more about it.

Tester-programmer relationships have to communicate both ways; it's not about I code—you test, it's a team effort. By the time a customer sees the end result they should have no idea how the app was created because it works perfectly for them.

Noel: I quoted your blog in a recent interview I did with a tester who gives a session on alternative methods for awarding bonuses to testers. He recommends incentivizing collaboration between testers and developers, and other behaviors that lead toward eliminating what he sees as pessimism in the workplace that comes from "bounty-style incentives." I know you're a developer, and not a tester, but from your experiences with this "other species," do you see a reward program like that taking off, or being met with some skepticism?

Andrew: I don't really like creating artificial programs to try to make the tester-programmer relationship work better. Maybe I am idealistic, but after such a long career, I know the best programmers and the best testers don't need them. If neither is very good at trying to make them better with bounties or other means—it isn't likely to change much. Keeping both sides informed about what the customers are saying is more likely to be an incentive. No one wants to deliver a crappy app or whatever you are building. Just throwing money at people won't make it better; finding the right combination of people is going to pay more dividends. Of course that's not always easy, but I've worked with a lot of really good testers so they have to be out there.

Of course, managing development can screw up even the best combo of programmers and testers. I've seen where management decided to forgo QA because the customer thought it took too long and they went along with that! There isn't any better way to destroy a good team than not allow them to do their best. If the programmer-tester relationship is focused on getting the best possible result to the customer then random incentives aren't that useful.

Noel: I really like your thoughts on the need for "clean code," even if it's just a personal standard that you give for your own work, but it seems like there's almost a movement for clean, happy code. A developer I spoke to talked about the ability to "listen" to your code to see if it's "crying." He mentioned, "Code is talking to us" and that in many cases, design patterns in that code are "clearly telling us what it wants to do." With your love of clean code, do you also find it having that ability to tell us more than just "I'm error free?"

Andrew: Ever since I first learned to code I've basically been doing the same steps, and despite radical changes in technology in the ensuing decades, I get the same results today. I try out my ideas first, then I code in little steps and test each step until I completely understand what each one is doing—more importantly, that I have a plan for anything that can happen. It's not TFD or modern unit testing exactly, but has elements of both. It works for me and I still get the same level of quality in about the same timeframe. Maybe it's just a lot of practice!

To me you have to find a personal style and rhythm that works for you and gets the desired results. I have never bought into the idea there are magic bullets that can make anyone a great programmer. Programming is a combination of imagination, creativity, experience, tenacity and discipline. I still have no idea why some people master programming and others fail miserably. It isn't intelligence or some easily definable or discernible set of traits. If it were easy, everyone would only hire perfect programmers.

Of course the quality of code is more than just a lack of bugs. My favorite definition of quality in an app is "it does everything it is supposed to do and nothing it isn't supposed to do". Quality of an app is something best measured by the customer. But quality of code should also include maintainability and understandability if that's a word. Testability of course but there are many levels to how you test an app. Performance and other optimal runtime behavior is also a quality. A final measure might be how long it took to get to that desired combination of qualities but that's often the one thing that might take longer than you like.

Of course, if you can write code, keeping all these desired items in mind and still deliver them to the testers already in place, then you're doing great! That's the ultimate goal, but of course it isn't all that easy to do. But, if you aren't hitting any of these qualities, then maybe you need to find a different approach.

Noel: Another blog post of yours I really enjoyed was, "How I Did Agile Long Before it was a Thing." I actually hear this from time to time from developers who have been coding for decades, and it always makes me wonder, if people were "doing agile" that long ago, it made sense then like it makes sense now, why did waterfall-style development ever take off, thereby keeping agile "under wraps" per se, for so long?

Andrew: My first job was at a defense contractor and my first project there was clearly what we know as waterfall today. Of course, that name didn't exist yet, the term first appeared in the mid 80's as an incorrect description of a diagram in an article written in 1970. But that was actually the last time I ever did anything in such a stepwise fashion. Over those three years, everything was far looser and less organized. The projects I worked on were small and not simply writing some big application. So I learned how to organize on the fly and even do multiple projects at the same time.

In my first startup in the mid-80s, and in the following nine years where I lead three mainstream application teams, the other programmers and I never spent much time thinking about how software should be developed. I don't think we even realized that people studied this as a topic! There was no Internet and there were few sources of information to tell you how to do anything. You had to figure it out for yourself.

Agile is basically about communication and taking development in small steps. I think that's basically what we were doing even at the start. By the last application, Deltagraph (charting), where we spent five years delivering five major releases, we had figured out a comfortable way to make development happen. I basically operated like a spider and kept communication flowing in all directions between the publisher and my team, as well as coding.

We switched gears on features almost every day. Basically, we ran an interactive feedback loop. There were no static structures at all like you have today, no sprints or boards or anything. It was all people keeping information flowing by talking. We didn't even have email (we were in Texas; the publisher was in California) we had only phones and FedEx.

One day they called and the GM said they would buy thousands of copies if we added a special chart type (Radar) so we figured out what that was, built a basic implementation and sent it via FedEx in a couple of days so they could demo it. That wasn't unusual at all. What made such dynamic development possible was that we did full application testing every day, between our tester and the publisher's. By the time we got to a "beta" state, most everything was working; we rarely had something that wasn't almost ready to ship. Maybe the team was special and made it work, I don't know. Deltagraph was similar in size to Excel but our team was a tenth of the size.

Today, organizations fear putting too much into the hands of people and would rather have fairly rigid structure and process to guide development. Scrum and kanban really don't feel very agile to me at all, given my early experience—I guess lean is closer. I think given a good team and technical leadership, you can build processes that makes development work well without following some canned recipe. Of course, all my teams were fewer than ten, clearly the larger the team the less likely a self organizing process can work. I don't think of today's agile processes (Scrum, kanban, Extreme Programming, or even lean) are the end stage of software development.

I feel lucky to have experienced a decade of pure agility long before it got a name but I know we weren't the only ones. As time goes by I hope people don't get stuck thinking of agile as a limited set of rigid structures. The Agile Manifesto is such a marvelously simple philosophy, but people love turning everything in a recipe or silver bullet. Agile is supposed to be agile, not a yellow brick road; you have to follow and keep within the lines.

AndrewTexas-based programmer Andrew Wulf runs TheCodist blog, is the lead iOS programmer for Travelocity, and is the owner of Idle Diversions, an iOS game company.

About the author

Upcoming Events

Sep 22
Oct 13
Apr 27