Automation in Sports Video Game Testing: An Interview with Fazeel Gareeboo


Fazeel: Well, I would say what's appealing is really automation, to me. If I'm doing something that's repetitive, I usually start with if somebody else can do it for me, and then I extend it to, well, maybe a machine can do it for me. That's even better, because a machine doesn't need breaks, typically doesn't make mistakes. I mean, there are other challenges with machines, of course. But that's where I start off.

If I look at something that I'm doing on a regular basis that takes a significant amount of time but that's repetitive and does not really require a lot of thinking, a lot of creativity, I'm looking for a way to automate it. And checking whether a game launches or just runs to the start screen really does not need to require a human. Checking whether a game can go through a set of menus without crashing really doesn't require a human. Other types of testing do, but this particular type—I would call it my stability testing—that doesn't require a human.

And the other side of it is, the more of it we can do the more frequently, the easier we can pinpoint where the defect was introduced in the game. So then we can go talk to the software developer who has introduced it. And because we're fast—so the software developer does something that checks it in and that breaks the game—if we can go back to them five days later, it takes them a long time to go figure out they did, what could be wrong. Whereas if we turn around within fifteen minutes, we say, "Hey, you checked something in that broke the game and here is where it broke," they still have it fresh in their minds and it's a lot faster for them to fix it.

Cameron: So, you're kind of saying that automated testing or automation can kind of be a prerequisite to manual testing.

Fazeel: Yes. I would say it frees the manual tester from a lot of the repetitive stuff so that a human being can then focus on really the added value that the human being brings to the testing process. A computer doesn't know that—well, currently, at least. It doesn't know that, "Hey, we've got the wrong face on this player," or "This particular character is not running right," or "Hey, the game is just not fun to play." The current state—at least of our systems—don't do that.

So there's other things that we need human beings to do. Because at the end of the day, we're selling it to human beings—we are not selling it to computers. So we need human beings to try and test it.

Cameron: You know, not everyone who will attend this session that you're giving will work on video game software or has worked on video software. So, what is something you would like everyone attending the presentation to learn? What is kind of the grand takeaway?

Fazeel: I would say there are a couple of things. But the main thing is identify repetitive parts of your testing. Automate them, but also include this automation as part of your development process. There is a movement in the software industry now called continuous development. So basically it's you develop, you test, and then you release, and then you keep repeating the cycle—as opposed to in the old days, we worked on specifications for a couple of months, then we work on the program for maybe a year, then we release. So we're moving to smaller bits of development being more agile. But I think a big component of that is having automated testing in that loop so that you're not having to rely on an army of people to test the software.

Cameron: Now, you have somewhat of an interesting link to your career. You developed a computer-aided design software and you wrote some device drivers for Windows and MicroStation. But you've also done some development for none other than Bill Gates's home. Can you speak a little on that?

Fazeel: Yeah, sure. That was an interesting thing. I had just started this job at a graphics card manufacturer in the UK. And my first task was to do this 2048x2048 Windows driver for a card that we made, and it was meant for Bill Gates's house. Apparently, he has these $10,000 monitors in his walls and he can change which artwork is displayed through some software. So, that was my first job. And there was an existing Windows device driver—I think it was Windows maybe 3.1 or 3.5. It was one of the early versions of the ones that kind of work, not the first ones. It was definitely a very interesting first job at that place. The cost of the graphics card itself was $10,000; the cost of the monitor was $10,000. So, I guess it's not surprising that he was Bill Gates.

Cameron: So, I take it you don't have any monitors in your walls?

Fazeel: Not as of yet. I'm working on it, though.

Cameron: OK. And you also have a passion for creating teams for the kinds of work that computers can't really do yet. So, what kind of work are you really referring to?

Fazeel: I think it's more the creative side. Computers are good at, you know, you can program it and then it will do a certain set of things. So you have to know beforehand what is it that you want it do, whereas human beings, you can ask them to create something as a team. And as a team they will create something, but you typically have no idea what it is. I would say some examples would be a performance, a piece of music, a piece of theater. I don't think we're at the stage where we can get computers to create those for us, because they just don't get it. I mean, they aren't meant to create those types of things. But typically most difficult problems these days require teams of people. And that's the other thing that I'm interested in, is how to get teams of people to work together and create great things.

Cameron: And, you know, you talked about computers being able to kind of decipher creativity. Do you think there will come a time when computers are able to do that and kind of take on the role of humans in testing? And when do you think that will be?

Fazeel: I don't think they will be able to completely take over that role. I think there are more and more things that computers will be able to do. But again, it's got to be more along the lines of automating something that's repetitive. There is a new area of artificial intelligence where people are using kind of neural networks as a way to, in a sense, mimic human behavior. That may be an area where people could, in a sense, train neural networks to be able to do the job of the tester. But we're not there yet. I don't know when we're going to be, if at all.

Cameron: And if you did know when it was going to be, you'd be a very wealthy man in that aspect.

Fazeel: Right! Yes.

About the author

Upcoming Events

Sep 12
Oct 01
Oct 15
Nov 05