Paco Hope talks STARWEST 2014, his goals to reduce the stigma around security testing, and how you don't have to be a wizard to test the safety of your software. Paco also covers how security testing is like cooking, the future of security, and how he became interested in security testing.
In this interview, Paco Hope talks STARWEST 2014, his goals to reduce the stigma around security testing, and how you don't have to be a wizard to test the safety of your software. Paco also covers how security testing is like cooking, the future of security, and how he became interested in security testing.
Cameron Philipp-Edmonds: Today we have Paco Hope of Cigital, and he is giving a presentation at STARWEST which is October 12 through October 17, and his presentation is titled "Softwarts: Security Testing for Muggles." Paco, to start us off can you tell us a little bit about yourself?
Paco Hope: Right, so I'm a software security consultant, so I work with a lot of financial institutions, retail companies, hospitality companies, folks who make software that do useful things for us in our lives. I work with them on making that software do the right thing, be secure, sometimes a white hat hacker if you will. The kind of person who helps developers get their job done, get it done right, and use technology the right way.
Cameron Philipp-Edmonds: Okay, now your presentation, it's titled "Softwarts: Security Testing for Muggles," and it seems like the presentation is humorously played off a very popular fictional wizardry book series. What led you to that association?
Paco Hope: That's right because frankly I work with a lot of folks who seem to have the opinion that security testing is this magical thing and you need to have a pointy hat and half-moon spectacles and a magic wand if you're going to do something productive in security. Actually it turns out it's pretty far from the truth. You can do a whole lot of really valuable security testing if you know anything about just good old software testing. That's the great thing about STARWEST is a lot of people who take software testing seriously, it is their job, it is their career, it is something that they're passionate about.
I really feel like we need to disabuse the industry of this notion that to do something meaningful in security you need to have special tools and special knowledge and all these techniques that only the wizards have. Actually you can do some really useful things and really powerful things as an everyday software tester who knows what he's doing.
Cameron Philipp-Edmonds: Great, now you consider you presentation a lesson in defense against the black hats which is kind of a defense against the dark arts to go along with that association. Can you cover the difference between a black hat and a white hat security tester?
Paco Hope: Yeah, it's a pretty easy distinction. The folks who are doing something good, trying to make software better, hoping that we all enjoy and benefit from technology; these are the white hats. The black hats are the ones who are trying to do bad things. It's taking advantage of mistakes in the software in it's design and implementation, and trying to do things that it shouldn't do. The reason we make this white hat black hat distinction is because there's actually a lot of techniques that are very similar between the two, but I'm looking to find the bug and the flaws, and the same thing with software testers.
We're looking to find the bugs and flaws to make the software better, not to in some way abuse the software or use it to our unfair advantage.
Cameron Philipp-Edmonds: Okay, so why do a lot of teams and projects stray away from security testing?
Paco Hope: Well, there's a couple of reasons. For one thing, there's a misunderstood notion that software testing is little more than checking, so we're just making sure that the software does the right thing and we deprioritize hunting for the wrong thing that might in fact hurt the business if it did the wrong thing. We're so focused on making sure the software does the right thing that we sometimes overlook the bad things that it could do to burn the house down.
The other side of that is people I think sometimes perhaps discount their own abilities. Well, I'm not one of those software security hacker-type people. I couldn't do what they do. I can't do something like security testing. I just know how to do regular testing. The two things are very, very similar. They're not that different.
Cameron Philipp-Edmonds: Okay, that leads into my next question here. You're a strong proponent of the ability to seamlessly blend security testing with existing testing practices with a little bit of extra work. Do most testing practices lend themselves to the addition and inclusion of security testing?
Paco Hope: Yeah, they do and I think that limitation comes from a limitation when we're thinking about software requirements. We start talking about what software does. We're usually talking about what we would want it to do in the good case along the happy path when people are trying to do the right thing, how will the software work? To be fair, of course, we have to get that right or the software isn't worth writing in the first place. Having said that, if you sit down and write your requirements only from the perspective of doing it right, then your testers will only test from the perspective of does it do the right thing.
That's the key concept. If you were to write security requirements, use cases, user stories, around what's the right thing to do when somebody is trying to do the wrong thing. Well now you've just written something that a tester has to go test. What do we do when the guy does the wrong thing? Suddenly security testing becomes just an extension of the rest of the testing that you're already doing.
Cameron Philipp-Edmonds: Okay, so it's becoming more exploratory.
Paco Hope: Exactly, it definitely trends towards the exploratory testing, and I feel like most organizations that are employing a significant number of software testers are actually almost squandering this opportunity to use highly qualified, highly trained people who know what the heck they're doing to actually go look for security issues. I work in a penetration testing capacity fairly often which means someone says there's our app, can you break it? Okay, I can try, but very often security testing is time boxed. It's like well, you've got ten days, or you've got $5,000, or whatever boundary around how much can I spend looking for security issues?
Contrast that with someone who does testing on this application for a living, they've got themselves a nice script. Every single capability that the software can do, they have a way to go exercise that functionality. They can go deeper, they can go more specific, and they understand how it's supposed to work. If they were to just toss in a few extra little test cases here and there deep down in some function that I as an outsider in a time boxed capacity, I won't be able to do that. It's sort of unused capacity in your own testing organization. If we just gave them a little bit of a more renip to do more things against their software, then the security guys would actually have a lot less to find.
Cameron Philipp-Edmonds: Okay, so let's say down the line organizations stop squandering what they have in front of them. Security testing becomes a part of day-to day testing regularly. What will that mean for the future of software?
Paco Hope: Well, I often make the comparison between security testing and security requirements, and performance testing and performance requirements because it's another one of these non-functional requirements it's really important to get right. If we were to start testing these things earlier while we're still building the software then A, the security results come on sooner and they're a lot cheaper to fix if we find things sooner. B, then the security testers spend their time doing the stuff that really does take a security tester's capabilities to find, rather than have security guys come in and pick low-hanging fruit and say ha ha, look what we did.
Cameron Philipp-Edmonds: Now before we go on, I kind of want to know what really got you interested in the realm of security testing?
Paco Hope: I've been doing security for fifteen years. I've been doing it for awhile, and probably seven or eight years ago I attended a talk by somebody who was a software testing specialist. Just someone who does use software testing. I looked at how they were organizing their work, and how they were really approaching it with this professional mindset and I thought 'Good grief! We security people don't do nearly that level of professionalism and that level of diligence. We just go in and start shooting at stuff and we manage to hit a few things, and we call it job done.'
I went to a STAR conference probably the first time, I want to say 2006 maybe, 2007. I was just really blown away by the level of professionalism and diligence that people were applying to what I as a non-tester had always thought just make sure the thing works. I thought gosh, look at the level of professionalism and how diligent they are. Man, if we applied even a fraction of that diligence to security testing, we would achieve so much more. Likewise, if these people who are so diligent and thorough and conscientious about what they were doing, they started doing half the security stuff that I know then I'd almost be out of a job. The testers would get it right then I would have almost nothing to find.
Cameron Philipp-Edmonds: Okay, there's been a whole lot of high profile security breaches happening recently. You talked about how security testing overall has become more professional and it's become more and more part of that day-to-day testing, so with that in mind, if the knowledge of the area, the confidence of the subject keeps going, is security testing going to keep up or is there trouble ahead?
Paco Hope: I think ... There's an interesting effect that I've been seeing lately which is the ever shortening time between finding something that's a possibility and saying 'you know, I think the software might do this' to a weaponized payload of yep, I can exploit that thing time and time again. That's what the black hats are doing. There was an event just very recently with an app called TweetDeck where they had cross-eyed scripting in TweetDeck. A tweet could come across your Twitter feed and as a result, bang. Your web browser just did something that you didn't even have to click to make it happen.
The fact that black hats can rapidly bundle and package their exploits also helps testers because it means we can rapidly bundle and package up regressions that will spot the same sorts of issues. If we tap into that packaging and rapidly reusing security payloads and security test cases, the testers will benefit from that same speed up.
Cameron Philipp-Edmonds: Okay, so to kind of put you on the spot here and I understand you're not Nostradamus or anything, do you expect there will be more security breaches like we've seen with Target and eBay and what not?
Paco Hope: Yeah, a great example is OpenSSL because SSL underpins everything we do and it's really, really important, and what we discovered was holy cow, there have been two or three major bugs laying around in that code base for two years.
Cameron Philipp-Edmonds: And one for like fifteen years?
Paco Hope: Yeah, there was the one that was laying around since 1998, and there's the one that had been laying around for a couple of years, and so now everybody and their brother is picking through OpenSSL with a fine-toothed comb. Now OpenSSL is flashy because it's so well known and it's used everywhere but there's a lot of software out there that we're reusing everywhere.
I think it's woke everybody up to holy cow, there's a lot of stuff we're leveraging and nobody's really picked through it carefully. We're going to see a lot more before it finally starts to taper off.
Cameron Philipp-Edmonds: Okay, now to move on here a little bit, you've co-authored a few books such as "The Web Security Testing Cook Book" which I think is a great name, and that offers recipes for security testing. It's pretty intriguing that you made that analogy. Is testing for security kind of like cooking?
Paco Hope: Yeah, in fact it's a lot like cooking. If you think about ... I'm no chef but you know that there's some rudiments like making a roux which is a very basic white sauce, and everybody knows the ingredients to a basic white sauce are just some butter, some flour, some milk and cook it in a certain way. So many security testing payloads are exactly that. Oh, I want to test fro cross-eyed scripting. Well, I've got these three or four possible payloads that I should try, and here's three or four typical ways that I'm going to put those payloads into a test case, and so it really is like having a cook book of some recipes. Oh look, I've got web interface that takes in some free text. I've got five good recipes to test that kind of an interface.
Oh, I have a database extract transform load, an ETL application we call them. Well, I've got some standard recipes for the sorts of malicious stuff I could stick into an ETL app and look for the results.
Cameron Philipp-Edmonds: So it's fair to say there really is a recipe for a lot of security testing?
Paco Hope: There really are. That's right, and then given that recipe, like anything else, like pairing a wine with a meal, you can say this is going to go well with that, and you know how to recognize a vulnerability. You know how to recognize a failure or a finding. It becomes very repeatable.
Cameron Philipp-Edmonds: Okay, and you have experience over a number of different platforms such as web applications, business-to-business transaction systems, and even online gaming. Is there a particular medium or platform for software that tends to be more vulnerable or more susceptible to security attacks?
Paco Hope: Yeah, so there's two ends of the spectrum. You have in the middle of the bell curve, you've got all these apps that were conceived of as web apps from the beginning. Many of those are not so bad because we thought about them being on the web to begin with. There are two things that are kind of at the ends of the spectrum. One is the apps that we never thought would be on the web in the first place. There's still actually an awful lot of financial, and transportation, and infrastructure sorts of systems that are 20-30 years old and they were not really built with the web in mind.
That's the new wine into old wine skins kind of problem. The other end of the spectrum are things like mobile. Mobile apps are web first. They're mobile first. There are whole groups of people who are building an app because they never existed in the first place, and it's going straight to a mobile platform. What I think a lot of people don't realize is just because you're mobile, you still have all the same security concerns that we've always had.
Then you have some new ones, too because you're on mobile. It's really like we understand the middle of the bell curve and then these two tails of legacy systems that never anticipated the web, and mobile where they think they're different but the truth is they're not. Those two are places that are really active in security.
Cameron Philipp-Edmonds: Okay, so going forward, in the future the most susceptible is going to be the past. Not only the past, it's stuff that people think is immune because it's mobile.
Paco Hope: That's right. I'll just say that if you think of a mobile device ... A tablet, a phone ... If you conceive of that as a Windows XP PC chock full of malware then you have the right mindset. That's the kind of security that you need to associate with a mobile device, and a lot of people don't. They think of mobile devices almost as appliances. How could you hack that? It's self-contained. You can't get under the hood, but in fact we can.
Cameron Philipp-Edmonds: Okay, now going back to your presentation, is there one thing you would like for attendees to take away from it?
Paco Hope: Yeah, the biggest thing I need people to do is to stop thinking of security as magic, and think of security as perhaps a special case of what they're already doing. Maybe you do things a little bit differently. There's a bunch of testing 101 stuff. Boundary value testing, equivalence class partitioning, and when I first got introduced to those concepts as a security person, I thought well, we do that, we just don't call it that. We didn't have a name for it. That's what people need to see when they're thinking about security. They need to see it and think of it as yeah, this is just what I'm already doing.
Cameron Philipp-Edmonds: Okay, and so going along with that title you have in place which is "Softwarts: Security Testing for Muggles," and how you're going to try and get rid of the veil of magic that's been shielding people's eyes, is it fair to say that you will not be wearing a wizarding cloak and wizarding hat during your presentation?
Paco Hope: Actually, I fully intend to come on stage wearing the full on pointy hat, magic wand, beard, and half-moon spectacles. That's part of the attraction. Yes, to show that in fact, underneath all of the wizarding costumery there's actually just a regular guy.
Cameron Philipp-Edmonds: That's fantastic. I really like that. Is there anything else you would like to say to the delegates of STARWEST before they attend the conference?
Paco Hope: I think bring an open mind and check the staff at the door.
Cameron Philipp-Edmonds: Okay, well that wraps up our interview, and this was the magical Paco Hope of Cigital. He'll be speaking giving a key note presentation at STARWEST 2014 which is titled Softwarts: Security Testing for Muggles, and we look forward to seeing you all October 12 through October 17. Thank you so much, Paco.
Paco Hope: Great, thank you.
A principal consultant for Cigital, Paco Hope has deep experience in securing software and systems. Paco’s experience covers web applications, online gaming, embedded devices, lotteries, and business-to-business transaction systems. He has worked with small startups and large enterprises in architecture risk analysis, secure code review, penetration testing, and other consulting. Acting president of the London Chapter of (ISC)², Paco serves on (ISC)²'s Application Security Advisory Board, authoring questions for the CISSP and CSSLP certifications. He coauthored the Web Security Testing Cookbook, Mastering FreeBSD and OpenBSD Security, and a chapter of Building Security In.
