Owen Gotimer

What is the biggest obstacle you have to overcome when you're trying to get people to understand the efficiency, the effectiveness, and the real need for automation across most software applications?

Chris Loder

I've been blessed in that a lot of the places I have are drowning in regression. It's taking too long to get releases out the door because guys are spending weeks if not months regressing a product. So people are turning to automation to make that shorter. And that's what I try to focus on these when I come into a company. I focus on what hurts the most? It's always regression. Okay, what in your regression is hurting the most? At InGenius, it was our telephony. A lot of telephone kind of tests, so we started automating those first, just to make life easier for the testers to free them up to do what they do best, which is test. So Michael [Bolton] and I will actually get along on a few things.

Owen Gotimer

I think it's funny, too, because I think a lot of people hear automation, and they get scared that that means not manual, which the automation portion of it is true, but to your point, what you're doing with automation, is doing the mundane, repetitive tasks that no one wants to do anyway, so that the testers can do the testing that they want to do all along.

Chris Loder

Exactly. It frees them up to do what's called exploratory testing. I know some great testers, they're like sharks, and they smell blood in the water. They're gone. They just kind of they hone in right on "there's the bug," they can put their finger on it. It's a skill set I don't have. I really, truly don't have that. But they're good at it. So we have to let them do that, we've got to give them the time to do that. If they don't have the time, it doesn't get done, and bugs go out into the product. So automation comes in and just takes some of the load off. I always tell people, my job isn't to automate; my job is to make everybody else's life easier. Whatever form that takes from building tools, writing scripts, to actually automating the product, right?

Owen Gotimer

It's so awesome, too, because everyone has different skill sets. Just because you don't know how to write automated scripts, or just because you don't know how to develop or do more traditional testing, doesn't mean that you're not a valuable piece of the team. You don't have to be able to do all of these things in order to be an effective member of the team. Of course, I think it would be nice if you know how to at least dabble in some of these areas so that you can help or maybe learn to grow your knowledge base, but ultimately, we're still gonna have specialists even as we move to agile. We talk about generalized specialists. People are going to have some more T-shaped skills. But ultimately, we're gonna have people that are still really good at the automation, still really good at the exploratory testing. So when you come into these companies, and you're talking to them about automation, and obviously, you're talking about what is the number one problem you're having, let's kind of narrow it down so we can start streamlining this process and work in a way where we can deliver this bit of value. Why is it important for you when you come into those conversations, to start working in that way, where you're not trying to say, "okay, let's try to fix the whole puzzle."

Chris Loder

Because you can't fix the whole puzzle. That's like taking an 1000 piece puzzle, throwing it on the table and saying, it's done. It's not. You got to take your time and the smaller piece you can do, the better you chance you have of success, right? We always say do a pilot: pick that one little thing that you can do that little box. So by doing those small, bite-sized pieces, and just finding that one little bit, you then also get a little bit of success, right? Success breeds more success. So you want to get one little thing working, and then just start doing the next and the next and the next. But I always sit down all the shareholders stakeholders, whoever's got a say in what needs to be done and what hurts the most. All right, what about that hurts the most? Now, can I do anything for you? Then we go from there.

Owen Gotimer

It's interesting, too, because you're talking about this kind of process as if it's second nature, like you don't even have to think about it. But we still have a lot of people – I think 30% I think I heard at a conference – who don't work in an agile way. Which you haven't even used the word agile, but what you're describing is kind of exactly what they're doing. Let's deliver small pieces of value more quickly, get feedback based on the value we deliver, and then go back and continue to try to add value in that way.

Chris Loder

I'm a firm believer in agile is just to word. The fundamental principle of agile is do what works for you. Find a small piece, that's what works for us. But you're right, agile is all the iteration, little bit, little bit more, a little bit more, a little bit more until the whole thing's done.

Owen Gotimer

I think we're moving in that direction, where agile is really just becoming a word. It's just the way that works for you. I think a lot of people are just seeing it as the most efficient way of building software is to iterate and to figure out the flow and the process that works for you. There's not a silver bullet. You can't bring in consultants into a client site that's going to give you all the answers. They're not going to hand out a template. They'll work with you, "these are the things that we see inside of your organization that's different from organization, A to B to C to D.' Obviously there are different answers to a lot of different problems, and a lot of different skills that people need to learn. As these problems are brought to the front of the room, when people are trying to learn these new skills, especially someone maybe who has a background in doing exploratory testing, and maybe they're interested in getting into that automation area. What are some steps they can take to try to become more skilled in test automation practices?

Chris Loder

Well, it helps if you can code. There are a lot of tools out there that you don't need to. You'll see vendors codeless automation and that sort of stuff. But it still helps. Because you still want to know what's happening and what's working and there's a million tools out there a million vendors will tell you that theirs is the best doesn't matter. Again, it's what works for you. Find the tool that works for you. And if you can code that makes life easier. Not saying you have to become an expert coder. I wouldn't say I am. Just that I've been doing it forever. Then find that little something that makes your life easier, and then do it. You may have to do it on the side, you may have to do it after hours. I wrote a lot of code when I first started in automation, because I started as a tester. When I first started doing automation, I would do it from the hours of 4pm to 11pm at night, because that's what I wanted to do, and that's the time I had to do it. I just kind of took little pieces, little things that I needed done that I didn't want to do the next day. That's what the other thing was: we had a lot of tests that we had to do almost every day, and I'm like, "why are we doing this every day when I could write a script to do it." So little things like that. Just find something that makes your life easier. And I'm not saying it has to be even automate your testing, but maybe write something that brings down your latest build for you. So you don't have got to go find it. Stuff like that, little tools and utilities, all that's part of that whole automation process. So just find something that you need and then figure out what you need to do to get it.

Owen Gotimer

I know when people talk about automation to you talking about tools here. Tools are a piece of the automation puzzle. But they are just one piece. The methodology you're using to work and the culture you created, it's not a one-size-fits-all approach. So I think a lot of times, we might hear, "oh, you have to go out and use this tool to get the job done." Well, you don't know what the business needs are. Businesses who are getting into the space might hear like, "oh, everyone uses this tool, so we have to use this tool." When in reality, we probably should be looking at it from the other direction, right? We should be looking and saying, what is our problem? What do we need to do to solve that problem? Maybe there's a tool that helps us get there.

Chris Loder

Figure out what it is you need to fix. Not even saying something's broken, what it is you want to do. Then, once you've identified that, then you go find a tool that will do it. It might be the one everybody else is using, but it might not be. And there's a lot of open source out there now. So you have to take that into consideration. Do I have that? Or should I go buy something? And I always tell everybody open source is great, but it's still not free, because you got to pay someone like me to use it for you. To write it, to code it. Someone still has to do all that magic for you. So whilst you're not paying licensing fees, there's still a cost involved, and a lot of people don't realize that.

Owen Gotimer

I think there's obviously a big push right now for the open source community. People think that there's an added value. I think that's some of the cool things about open source is that a lot of them are community driven. There's often a lot of support, especially the more popular open source tools. There's a lot of support from even enterprise level companies pouring their support into it helping improve the open source tool. So there is obviously value, but to your point, these tools aren't necessarily usable out of the box. A lot of them are not. A lot of them people might not properly understand what the purpose of the tool is, and they think for example, "if I take Selenium out of the box, it should be able to do X, Y, Z to do all my automation," when in reality, it just does a small portion of web UI testing.

Chris Loder

That's right. Do you know how many people I have to tell none of this only works in your browser? The minute that your file open, or your save as dialog appears, you have to come up with something else. Selenium only works in the browser and a lot of people don't realize that.

Owen Gotimer

I think one of the popular tools – we kind of beat around the bush to get there, but Selenium is one of these popular tools.

Chris Loder

It's a great tool.

Owen Gotimer

But there are specific use cases to use Selenium. There are other things that you're not going to be able to use Selenium in your test automation. And I think it's kind of to your point, doing your homework, improving your knowledge, and learning these things. But also, again, coming back to the point that we need to figure out what are we trying to improve? What process are we trying to improve? What's your problem? Will these tools help us get there?

Chris Loder

What hurts the most and which tool will help it hurt less?

Owen Gotimer

Obviously, really important. I know we talked about this a little bit at the top, you talked about being lazy. And you kind of pride yourself a little bit on being lazy. Do you want to talk a little bit about why you find it effective? Some people kind of see that word as a negative, but you kind of use it as a way to empower yourself and say, I'm lazy because I'm finding new, creative solutions to do boring, mundane tasks.

Chris Loder

Absolutely. And you'll get find out all about that on Thursday during my keynote, but the CliffsNotes version of it is why should we have to do these things over and over again? Especially anything that's routine or mundane. Let's find a way to automate it. Let's script it. I had to rename all my test cases one time just to remove underscores becauase we're running out of file space. I wrote a script to do it, because I know that I would accidentally press the delete key twice on one, and then everything would fall apart, stuff like that. Just automate just about anything, but just do it, so you don't have to do it. Our automation that InGenius is fully unattended. It starts at six o'clock every night, done by morning, picks up the latest build installs, it builds our automation, creates the test plan for us, does all that, runs the automation, puts the results up, and sends an email for you waiting for you in the morning. Because I don't want to do that stuff. I used to have to put up my sites. That's painful. I mean, that would take time. So we programmatically do that. Building the test plan. We know which tests we want to run. It's regression usually. You want to run every all those. So we programmatically do that running the automation that's already done. And then the reports they come out in the morning. We have a whole report suite that's built off our result runs. You get an email or a Slack message, so I don't want to have to generate reports or sift through data. It's all done for me. Because I'm lazy.

Owen Gotimer

You get that report in the morning. You're not done?

Chris Loder

No, that's just where it starts. You look through it, you analyze the failures. Are they a script problem? Are they an actual bug? And if they're a bug, what do you do about it? You got to reproduce it. Usually, if I find a bug, I'll walk over to the manual folks, because they're better at this stuff than I am. I'll show them. Here's my screenshots. Here's what failed. Here's what happens on my system, does it happen on yours? Sometimes I get lucky, they take pity on me, and they'll log the bug for me. I find that if you log the bug, it's yours forever. So that's usually how it goes is you have to sit there and analyze it and that falls to my team. The minute I automate something for you, it's no longer your problem. So if I take tests off your plate and they're automated, the responsibility is on me and my team to make sure that they get run. And if they can't get run automatically, then it's my responsbility to make sure they get done manually. So if something happens and the automation can't run, it's on us to make sure those tests are done, because I've taken them off your plate. So you can do other stuff.

Owen Gotimer

You're freeing up that space for the exploratory testers to go in there and do their hands-on work. But quality—we're talking about specifically automation and manual testing—but something you're talking about is the responsibility of your team to get this done. I think a theme we've heard over the last few years is quality as a team or corporate wide thing. We have to have developers and coders who are writing quality code. Really we even start before then, right? We have BAs who have to write quality requirements. We have coders who have to write quality code. We have automation testers that have to write quality automation scripts,. We have to have manual testers doing quality manual testing. And then the cycle has come all the way back through the process. Because we really do need quality because the thing is it makes your job more difficult if the software itself isn't written well, where you can understand and your automation can understand what's happening. That probably makes your job a lot more difficult. So from a quality perspective, how do we get teams to understand that quality really does fall on the whole team and not just the "testers" in the room?

Chris Loder

Absolutely. I've been blessed where I'm at now, the developers get that. They go above and beyond. They include the automation team on their pull requests, on their code reviews, whenever they do anything in the UI, so we can make sure that they put locators in for us, so we can actually find those elements, so the automation will continue to run. It's a company mindset that we have to have. And it really starts with all of us, and I've been in meetings where we've got the developer, the manual tester, and the automation person, and we sit there and decide who's doing what. What's covered under unit tests? All right, the developers will take care of that. What's going to be automated and then whatever's left the manual folks have to deal with. We have those meetings to decide who's doing what out of a feature, which is really cool. It's good to see that everybody's buying in because the last thing you want is a bug to come in from the field on something you worked on. No greater shame, is there? In the software world? I don't think so.

Owen Gotimer

The cool thing you mentioned there is that the developers like going out of their way to say, "Hey, we're making this change that you I will make sure we add these locators" and having that open communication with you. It's almost from your perspective, as someone who is empowered to produce quality and to check for quality and test for quality, empowering that the people who are actually writing the code also care so much about it and are willing to work with you rather than against you or throw it over the wall to you to help you get that job done.

Chris Loder

Absolutely. It was it was cool to hear a couple of weeks ago, I hear over the cubicle wall, senior developer to a junior developer, "you can't do that you'll break all of automation." It's like, wow, we made it. We're in the mindset now. We're part of that machine. They're thinking of us, which is fantastic. And that's where you want to be. You want your developers thinking about testing. They have to build testable code, then it helps us to build automatable code. If it's testable, you're halfway there, right? And then you can add those extra little things that you need for the automation folks. Makes life a lot easier.

Owen Gotimer

So one of the things you mentioned was locators. What other kinds of things help from an automation perspective? What other things could developers do to help teams that are working on automation testing?

Chris Loder

To share how much coverage they've already done with your unit tests. No sense in us writing the same tests again, to do something that you've already got covered, especially if you're talking API tests. A lot of them will write unit tests to cover that. We would only then have to focus on the external, the public facing API, that sort of stuff. So having that conversation is a big one, because we're saving time on both sides, because they know we're going to cover it off over here, so they won't bother. Locators are a big thing for us, especially because a lot of what we do with is web UI related. I'm truly blessed. Our APIs are not actually exposed for anybody else to hit. So our guys built us a web page that just has all the APIs on it, so that we can hit it and test the APIs using the UI. They went above and beyond for us. I cannot stress enough, sit down with your developers. They're not all scary people, right? There's some good ones out there. Sit down with them, spend some time figuring out what it is do. There are things you can do to help them. They've got their own problems, their own headaches. See what see what you can do to help them take some load off their plate?

Owen Gotimer

When you're able to do something like that and help them out, they're more willing to say, "Hey, you know, Chris is always pulling through for me. He's always able to help me get a problem off of my plate. He's asking me to put locators in my code. It's really not that difficult to add locators to my code. I'm going to go out of my way, not even out of my way, but I'm going to make sure that I have my locators here, because I know that's going to help Chris."

Chris Loder

It's a give and take. It helps a lot to have good people that you can have that back and forth with.

Owen Gotimer

The communication, obviously, is such a big part of it. The shift in mindset, such a big part of it. That really starts with that initial conversation between the whole team. Really it's across the whole team. You mentioned starting with the pilot team at some level, obviously getting this up to a company wide initiative where everyone is thinking that way and everyone is saying, "you know what we want to be quality driven, want to deliver value frequently, nd we want to be more lazy in accomplishing those goals."

Chris Loder

We want to get as much as we can with the least amount of effort. And let's do that together. The only way you're going to do that is together, you really do. You have to work together.