Pete Chestna -- SAST, DAST, and IAST. Oh My!

0:00Chris RomeoGreetings, friends. On this episode of the Application Security Podcast, we are joined by Pete Chestna, and Pete talks to us about SAST, DAST, IAST, many other different types of application security tools, as well as how to implement them for those starting a new program and for those that are a little bit more established. We hope you enjoy. The Application Security Podcast. Here we are.

0:31Pete ChestnaHere we go.

0:47Chris RomeoHey folks, welcome to the Application Security Podcast. And on this episode, which is episode 5, we're joined by Pete Chestna. Pete's going to talk to us about SAST and DAST and application security programs and many different things. And so Pete, on this podcast, our first question we always ask all of our listeners is, what's your security origin story? Where did you come from? How did you get into application security?

1:18Pete ChestnaSo it's a funny story. I was at a company that was privately held and I, let's see, I'd been in the industry probably, I don't know, 15 years, and I was looking for— I had already been as part of a startup. So at this company, I was like, hey, there's no equity here. Why is there no equity? And then I finally looked at myself and said, you know what? I want to go build something. I want the blood, the effect of my labor to be something that's meaningful, not only to the company, but also to me. So I left the company and moved on and found another startup, which happened to be an application security startup. doing a SaaS-based product. And it wasn't about security that drew me there. It was finding that startup. So I started there and it was, I was a developer. And then you start to get in the face with all of the application security components of this. And I learned as I went. I've been there almost 12 years now. I love the space. I'm super passionate about it. So that, you know, that's how I got into security.

2:17Chris RomeoAwesome. So many, and many of our guests come to us via the developer route. So it certainly is a, I think it's a good path for people to really understand how things work from a code-level perspective because that just— it gives you a different perspective on application security when you get here, when you have that foundation.

2:39Pete ChestnaOh, for sure. And I speak publicly now quite a bit at DevOps conferences, at AppSec and InfoSec conferences. And the one thing that I can bring to the table is I can speak both developer and AppSec. So, when I'm in front of a DevOps audience, Hey, let me explain this security thing to you. It's not as hard as you think. And when I'm in front of the security audience, hey, let me explain how development actually works so that way you guys can do a better job at it.

3:01Chris RomeoYeah, that's very cool. So, the topic that we have for today is really to unpack this idea of SAST and DAST and how those fit into the application security programs. And just for our listeners' benefit, this actually— this episode, the genesis of this came to us from a listener question. A listener was asking, How do we deal with kind of the false positive sides of using these application security tools? And so that's— so just so the listeners know, when you ask a question, we may just go find somebody who can talk about it and bring them on here to answer your questions. So Pete, I want to start by laying the foundation of what are SAST and DAST, because some of our listeners may not even know. The person I was talking to earlier today has a brand new program that he's trying to stand up. And so he's trying to figure out all this stuff at the same time as rolling it out. So lay a foundation for us on what is SAST and what is DAST.

3:55Pete ChestnaOkay, so SAST is one of the fundamental technologies inside of application security today. It's static analysis security testing, which is what that acronym means. Think of it as an inside-out. Some companies do it via source code, some companies do it via binary, but in the end This is part of your CI portion of your CI/CD pipelines or the static analysis portion where you're running lint, you're running, you know, like your SonarQube type code complexity tools. You would also put this kind of SAST product in that space to be able to look at those check-ins to understand the security implications of the code that was checked in or be able to run it from your IDE. So one of the things that I always stress when I'm talking to people is whatever happens in that assurance case in the middle there where I'm doing my static analysis, that should be an open book test. That should be able to be run from my IDE prior to check-in so I know I'm going to pass. So as you get your teams higher up the maturity curve, they become aware and intolerant of people not doing the right things. So as a developer, I didn't run my unit test because I was in a hurry and I needed to get out because my wife's getting aggravated with me, so I'll just check this code in. And it breaks the world. So you have those tests afterwards to prove that you actually followed the right process. So that's SAST, it's an inside-out technology. It does not require your code to be running. Now, if we look at DAST, dynamic application security testing, this requires a running instance. This is automated pen testing, very simplistic automated pen testing, right? I can't, it won't do the things like infer business logic from the attack it just did, but it will do things like look at your security headers, look at input forms, make sure you're doing the right validation, look for things like—

5:41Chris RomeoCross-site scripting.

5:42Pete Chestnacross-site scripting, look for things like information leakage. So are you publishing the fact that you're using Tomcat version thus and such, or Spring, or what have you, so that way it gives information to attackers that they can then look and use against you? So dynamic is an outside-in technology. It becomes— the application becomes a black box, and I can look inside of— you can look from the outside of that application. Now, as we talk about these technologies, these aren't the only 2 in your application security portfolio, and it's important to understand that you want to look at it from multiple angles. SAST is a great tool. DAST is a great tool. Now we have IAST, or interactive application security testing. So if you think about this, this is a runtime agent that sits inside of the JVM or what have you that watches code execute. So if a dynamic attack is able to leverage something it can see it, record it, and it becomes yet a different way of looking at your application dynamically while it's running. On the really fringes of this, we have RASP, which is Application Self-Protection, and think of that as that IAST product but in blocking mode. So you take a look and say, hey, I see this SQL injection about to get triggered. Instead of doing the wrong thing, I'm going to log it and I'm going to return zero records. So it's a runtime protection portion of that dynamic analysis that kind of builds on everything. Then of course you've got things like software composition analysis to take a look at your open source tools. So there's a gamut of tools and you wanna make sure you're looking at it from multiple angles, including pen testing, because nothing is better at finding logic errors inside of your application than a pen tester, someone that can infer behavior from results.

7:25Chris RomeoSo when you look across these, I think you kind of worked across the list. based on history, meaning SAST was first, then DAST, then IAST, then— IAST and RASP, I guess, were pretty similar software composition about the same time. So would you say that at this— in this day and age, SAST is the most mature out of all these different types of technologies, or are they pretty similar?

7:49Pete ChestnaAbsolutely.

7:49Chris RomeoIt is? Okay.

7:49Pete ChestnaNo, it's followed quickly by DAST, but SAST is definitely the foundational technology for most application security programs.

7:57Chris RomeoOkay, and is, uh, now in looking across the industry, is this— are these tools primarily cloud-based or are these tools on-prem, or how are people using these things today?

8:10Pete ChestnaIt's really a mix. There are some companies, and if we look at government, that's one of them, where they have air-gapped installations that require on-prem tooling. But one of the things that companies are finding, and you'll find that most of the people that started as an on-prem tool are also creating a SaaS-based version of that tool, is because the total cost of ownership includes things like standing up a hardware farm, building and buying people to go and run it, and doing all of that stuff that, you know, if you have a SaaS-based product, I plug into the wall, I get all the capacity I want, no different than, you know, plugging into the electrical grid. So, Anyone that's doing on-prem is also trying to do SaaS for those reasons. But on-prem, there's a small portion of the user base out there that absolutely requires it. Some of them are highly regulated industries, but for the most part, it's ones that are old enough that they're just hesitant to have anything go off-prem.

9:08Chris RomeoOkay, so I know that, you know, SAST as a tool, when I come in and I look at somebody, you know, somebody's program, or somebody who's trying to start a program, I always tell them that's a great place to start from a tooling perspective. But I know that we get a lot of pushback from developers when they start to use these tools. And so, I'm curious as to what you've kind of experienced both as a developer and as somebody who works with these tools and deploying them. What are the struggles that developers are having with these tools?

9:45Pete ChestnaThere are 3. The first one is speed. So this needs to be something that doesn't slow me down, right? So if it takes me an hour to run, I'm not going to like that very much. If it's a couple of seconds or a minute or something like that where, you know, I go up and refresh my coffee or, you know, look over the cube wall and talk to my neighbor, that's something I can deal with. So that's the first one. The second one is around accuracy. And really, The industry is focused a great deal on this thing, where if the developer has to weed through a whole bunch of stuff to find, you know, 1 out of 10 that are true positives, they're going to abandon the tool. It's a waste of their time, and they're going to push away from the table. In most cases, security is a function for the security team. They're responsible for it, so it's not my problem. I get paid to ship code fast, so anything that's going to get in the way of that by saying, I'm doing a bunch of useless work. I spent 2 hours to find out this is a false positive. This is great. So it has to be accurate. And the last is it has to integrate into the tools that I use every day. It has to be part of my IDE. It has to integrate into my build tools. It has to integrate into my bug tracking system. One of the worst things that company security organizations do, here's the spreadsheet, here's the link to our systems for you to go find the work. It's like, nope. I run Agile. Here's my backlog. It's in priority order. This is where work goes. So if you don't get into that, then you're not work to them. So it's easy for them to ignore. In terms of false positives, you mentioned accuracy. What kind of typical false positives come out of a SAST process? There are 3 flavors. So the first one is an incredibly important one, and this is one that companies need to help developers understand as they go into their baseline scans. We don't trust input. One of the things that a SAST tool will do is take a look at something that comes from the outside and gets used on the inside and say, I don't trust that, I don't know where it's been. So you have to train those tools to say, hey, I know I read from this database table, how did it get there? Well, it's It's written at installation time, it's read-only, here are all the compensating controls around it to make sure that no one else can modify it. It's safe, I trust this. And so that's one flavor of what I would call false positive, is it's that training aspect of the tool, because whether it's reading from the user or a file or a database or somewhere else, we just don't trust inputs and you need to tell us that they're safe and prove that they're safe to your internal constituents. The second one is around compensating controls that you might have in place. So maybe you have a WAF, or maybe some design in your network or the application actually protects it. So what we're seeing is not something that's exploitable. And that's the second level of false positives in there, is going through and putting in those compensating controls and documenting those for your security team such that they can accept or deny those requests for Approval of that risk.

12:51Chris RomeoAnd just to add on, so on 2 compensating controls, WAF, this is web application firewall, right? So this is some type of technology that's being— it's an infrastructure type of device that's going to be looking to detect and block individual attacks that are coming into the application, right?

13:11Pete ChestnaCorrect. And the other one that might become more popular now is IAST. So maybe I'll say, hey, I've got IAST in front of this, so SQL injection just won't happen.

13:20Chris RomeoOkay.

13:22Pete ChestnaAnd then the last category, and depending on the vendor, you're going to see different levels of results, is around a real false positive, something that the software should not have found. And those numbers range anywhere from 5%. I've seen applications and products that go up to 50%. And more. So those are the places where you're going to struggle. You have either, if it's a SaaS-based tool, maybe it's something that you buy with the service to do false positive reduction. Otherwise, you've got a security team that maybe tries to do it. If they force it on the developers, then the developer's gonna throw up, walk away from the table and say, you're slowing us down, and the business is gonna be behind them on that. So finding that product that has a good level of false positive, and every product is gonna have some, But being able to actually train the product over time to get better at that process is critical.

14:18Chris RomeoSo, is false positive really the only— so, we talked about, I guess, 3 different types of struggles. Speed, it's not— don't slow them down. Accuracy, cut down on the junk. And then integrating into the daily tools. Anything else that— I mean, have you ever seen like a lack of knowledge as a struggle there where The developers don't even think that security is really that important?

14:44Pete ChestnaOh, so yes. So not just that the security isn't that important, but I was never trained on it. And I'll say that for myself. I'm more than 25 years a developer. When I graduated from college, there was nothing about application security. Even in today's universities, if they have a security course, it's probably network security. and it's not required. So the few places that have it don't require you to do it. Our entire workforce is untrained in the ways of application security, not only why it's important, but how to do it. If you look at the OWASP Top 10, it's the things on that list are in some cases could come out of your fingers secure if you just understood them.

15:29Yeah.

15:29Pete ChestnaSo bringing awareness to cross-site scripting, bringing awareness to SQL injections, like don't do it that way, do it this way instead, means that I didn't write it in the first place, so I don't have to find it, triage it, fix it, retest it, all of that cost associated with me just not knowing any better. So part of the struggle for a security team is to really build in that training and help developers understand how easy it is to prevent some of this stuff.

15:57Chris RomeoYeah. So when we think about the Kind of how you make a program coming out of this. So, I kind of want to ask the program question from 2 different perspectives.

16:11Okay.

16:11Chris RomeoThink of kind of 2 different people that might be dealing with this because I could think— and going back to a little bit about what I said earlier, if somebody's kind of building, starting up a new program, the last advice I'm going to give them is you should go find every one of these types of tools best of breed, and you should deploy them all, and you should, uh, you should see what happens. Which I already know what happens if you do that. You're not working there anymore. You're fired, you know?

16:37Pete ChestnaRight.

16:37Chris RomeoUm, so let's— so, so the first, I guess, from the programmatic perspective, I want to ask you, how do we, how do we create a truly effective AppSec program? But the first option is for somebody who's brand new to this, just like the guy who emailed me earlier today about he's setting up this new program. He's using the podcast here as a way to learn about how to— the types of things he can do to, to try and make successful program. So for him, as somebody who's brand new, what do you recommend that he does here in the beginning as he's kind of looking at SAST and trying to figure out what to do? What are some strategies for him?

17:12Pete ChestnaAll right, so let's just say we haven't picked a tool. SAST is probably a really good choice, but in this particular way of thinking, it doesn't matter what you have. You have data, right? So think about your application inventory. And what's been scanned and what hasn't. So what are you essentially in ostrich mode on, and I know nothing about it, versus I know something about it? And then understand how to move that balance over time towards me knowing something about everything. And then the next level is to start fighting the battle of integrating it so that way I have accurate measures. So one of the things in a good AppSec program is knowing whether you're getting better or worse.

17:53Yeah.

17:53Pete Chestnaas you apply training, as you apply tooling, as you measure more frequently, are we going up or going down? Are we getting better or are we getting worse? Are they fixing faster than they make them? Because if they're not, then that's my problem.

18:05Chris RomeoSo, when you're integrating for accurate measure, so is that before you're in a policy enforcing mode? And what I mean by that, is that before you're actually— are you doing that before the developers are really kind of being forced to make the fixes to the code, or is this happening all at the same time?

18:27Pete ChestnaGood question. So we'll take 2 different worlds. In the greenfield world, where I've got a blank sheet of paper, either I've just started a company or this is a brand new product, it's something that you can have that bar set very high up front and help them understand what they need to do and provide the tooling to measure it. That's not the case in 99+% of the companies.

18:49Chris RomeoYeah.

18:50Pete ChestnaSo for this individual that wrote to you and said, hey, how do I build an AppSec program? Understand you're gonna find a lot of stuff that you're not gonna like, and the fact that you found it is good, and you're gonna need to find some way to get after it. So putting the big gate up or wall and saying, you guys can't ship anymore until you fix this, it could be years. You won't have a company by then, and you certainly won't be in that job anymore. So understand that you're going to find things that you don't like, and you're gonna need to find a way to pay that down at some respectable level. That might require you to get additional resources to help, but understanding the problem is the first step. And it could be thousands or tens of thousands of vulnerabilities. I've seen them that high. It might be hundreds. You don't know, and you don't know what the severities of those. So I wouldn't go putting a policy in place and just saying, good luck. Understand what you have. You may need to have different levels for different mature— maturity levels of teams. How good are they at fixing these type of things? And maybe you only start with very highs, and then you ramp it up to highs, and you ramp it up to mediums, or maybe you're adding one CWE, Common Weakness Enumeration, at a time into their workload. But this has to be a very deliberate process if you're talking about a brownfield project. It can't be the case that, you know, it took you 10 years to get here, but overnight we're going to make it secure. That's just not going to be— that's not—

20:10Right.

20:10Chris RomeoYeah, and I like your— I like that idea of adding one CWE at a time. I've actually made that same recommendation to people because I think that's one of the places where people really struggle when they're integrating one of these tools from the beginning is that they come in and they say, you know what, we just spent all this money on this tool, we better get our money's worth right from the beginning. And they bring up that policy and they click every checkbox possible, and then they make that part of the—

20:34Yep.

20:34Chris RomeoAnd then you walk up to the developer with your 10,000-page report that you were mentioning security bringing earlier, and they look at you and they go, what am I going to do with that? And then they throw it in the trash can or recycling bin and they move on with their lives. And so yeah, that one CWE at a time, I like to get some, get some success, you know, let the developer actually see something that the tool does actually work. And then they start to go, well, this thing isn't a piece of junk. It actually found something that was a problem in my code. So, okay. So after—

21:02Right.

21:02Pete ChestnaI mean, they're just, they're just strapping on their, their hiking boots and you're putting them in front of Mount Everest. Saying good luck, right?

21:08Chris RomeoYeah.

21:09Pete ChestnaSo the critical aspect with that is, if you're going to do this one CWE at a time, and again, I would look at the initial set of baseline results before I made any decision on how I would treat that particular application or application team. But if I'm going to say, hey, this is the CWE we're going to deal with first, I start that with training, I start that with tooling, and then I measure the impact of that over time. And you'll see whether or not your training was effective because you should see a lower incidence rate of new vulnerabilities coming into the system, and you should see the number of total vulnerabilities going down over time. Yeah. So it's critical to put those kind of measures in place. Again, am I getting better or am I getting worse? And maybe every team's a snowflake to start with. You certainly can't scale that out to 3,000, 4,000, 8,000 applications, but as you bring this program online, you'll find buckets of those teams to be able to put the measures in place so you know that you're actually having a positive impact.

22:05Chris RomeoYeah, and I'm gonna— I'm actually gonna just reiterate that because I think that's— I think that's a— that's a brilliant idea that you've got here. And I think a lot of people fail in this regard. They don't set those metrics up early so that they can truly show a track record over historical time. They get a year or 2 years into a tool deployment before they start thinking, oh, we should really be tracking these metrics and being able to demonstrate what the value proposition is here. So, I think that's, you know, for our listeners just to consider is when you're building one of these things out, think metrics early, because if anything, it'll justify the additional budget you're going to need as you want to grow your program over a period of time.

22:42Pete ChestnaSo, so here's the best-case scenario. I've built out a bunch of tools, and let's say I'm spending $5 million a year on my AppSec tooling, and they come to you and say, hey, you don't have any vulnerabilities in the, in the applications. Why are we spending $5 million on the tools? You can go back to those metrics to show through good training, through good tooling, we're covering everything before they check in, we're reducing all that risk up front. You know, you are able to show the value because you're now switching the way you measure from reactive to proactive, which was the next phase, to say, now I'm trying to deal with things before they check in. So I've got those IDE integrations, I've got them checking in secure code, which means my assurance scans, I expect them to be green all the time for these particular teams. That doesn't mean I don't need the tools anymore. You still have to do that assurance case. It's trust but verify. So prove that they've done the right things and then you move on. It's like, God forbid that we have zero vulnerabilities on our application. You can show them where you came from and how you got to where you are, the fact that they add value.

23:46Chris RomeoYeah, definitely. Okay, so is there anything, any steps for our— for this new program after they're adding in one CWE at a time? What's there? Is there like a final step or is that kind of where we leave that case?

23:58Pete ChestnaSo when you get up in the maturity level to be proactive, so the team, when they get to a certain point, will start to say, hey, we do not allow any new vulnerabilities to come into our code. Now, maybe we're still paying down the old technical debt, but we've now set up a barrier to say, if you introduce a new vulnerability, we will break the build. That's the next level of maturity, is to disallow new stuff from coming in. Up until that point, they're gonna make mistakes. You know, some of this is going to be just sloppiness or getting the program underway. Some of it's gonna be a team member moved from a different team, or I've got a new employee. I start to see these new vulnerabilities crop up. That's where you need the data to go say, what just changed? At this date, I started to see things that I hadn't seen before that we've eradicated, like polio. I eradicated that years ago. How the hell is this coming in now? Oh, new teammate. How is your onboarding process helping developers understand what you expect of them as they come into your teams to make sure that, again, as you roll this program left, really making sure that you've covered all those bases to make those zeros be true for all time?

25:06Chris RomeoYeah. Okay. Well, let's kind of change gears and move into that other case. You know, we're moving away from kind of the brand new person into kind of a, I guess, a little bit more of a mature program that's going to be looking across all these different types of technologies. But first, I want to ask you, because I've heard you have an interesting answer to this, from your perspective, what is the purpose of an application security program?

25:30Pete ChestnaSo, I've danced around it quite a bit during our talk. It's about secure developers. If I can create secure developers in my company, they don't write vulnerabilities. Now, a secure program and secure outcomes are a byproduct of that, but not your goal. Because if you look at this, where are all the places you can get vulnerabilities in, whether I'm proactive or reactive or what have you? If I can get the developers not to create them in the first place, and then again, if we look at the OWASP Top 10, some of those are super easy to train on, then I have actually made them faster developers because it comes out of their fingers secure. It's not something that they have to go back and rewrite history or unravel a whole bunch of code to go rewrite because they made that mistake. If you can train them not to make that mistake, you've added value and velocity to the company and really brought something that no one else could. And that's different than quality, because from a quality aspect, logic error is going to happen all the time. But some of these things in security are black and white. You did it right or you didn't. You used secure random or you didn't. You cleansed your inputs or you didn't. So training on those things will only make them go faster.

26:40Okay.

26:43Chris RomeoSo then when we start to think about more of a holistic approach where we're looking across all the different types of tools that you explained to us, SAST, DAST, IAST, RASP, even maybe WAF and other things. So I guess how does this look different if we're A little bit more of a mature program that maybe has some of these pieces but not all of them, and they're looking to go to the next level. How do you see all these tools kind of fitting together into a truly mature program?

27:14Pete ChestnaSo you want that data convergence, right? You need to bring these metrics together so that you understand the value of the tools that you bring in and whether they are adding or subtracting, whether they are really worth the the price that you pay for them. And also, when you get up to this level of maturity, this is where I would start to bring in people. I would start to bring in red team and blue team into my requirements phase to say, here are threats that we see to the business today, or here is where your red team was able to compromise you yesterday. So those kind of things, that's the next level of maturity, is to really pull what's happening in your production environment forward and left into the very earliest stages of the application teams. Now, the team itself has to be mature and have already dealt with all of those simple things. So the nice thing about these tools is they can find vulnerabilities for pennies on the dollar for what it would cost you to have a pen tester or a team of pen testers go after it. So when you actually employ them, they have to work hard to find real things. And then you could take those things and bring them back and take the developers to the next level, have them think like attackers, have them understand when I'm writing code, what are the possible ways that this could be compromised? And I have now upshifted. I'm not just like some tool monkey that takes this tool input and knows how to write it right the first time. There are logic things that can happen as well, and I start to learn those over time.

28:43Chris RomeoSo data convergence. So this is, this is the idea of all the metrics kind of coming together into one. So I guess if we kind of continue the thought when you're talking about red team, blue team, does this— I mean, if I have a solid set of these tools together, do I need— do I even need pen testing anymore? Do I need bug bounties or any of these other types of things, or can I reduce or even eliminate the spend I have on outside pen testers and bug bounties and the like?

29:20Pete ChestnaSo I think you can definitely reduce. I don't think you ever get rid of it. Like I said at the, at the very beginning, pen testers infer behavior from what they see on the screen, how the application reacts to them, whether that be the amount of time that it takes to do one thing versus another, or things that they see reflected back on them in the screens or in the headers or, or something else that they see. Those are the things that only a human can do today. So if you're interested in getting to that level of scrutiny, then that's the place you need to have those pen testers. So they're absolutely valuable, but you can shrink down. What I don't want, I don't want to spend hundreds of dollars an hour on a pen tester for them to come back and show me all the SQL injection flaws that they found. That is way too expensive to find those. I can shift that into my organization with good tooling. What I want them to find is more of the logic-based errors. And by the way, when we start talking about you know, our key infrastructure, whether it be authentication, authorization, cryptography, those are the places where I'm going to want to have a human eye in that code to look at it, to scrutinize it. Did I cleanse the inputs properly? You know, my custom cleansers, are they working properly? Am I not doing something wrong in my crypto? Am I using all the right ciphers now? Those are the things that require human beings. The tools won't tell you those things, and for some of those, you're gonna wanna pen test, but just be smart about it and use it for only those critical pieces.

30:46Chris RomeoYeah, I've seen some organizations that have been— have tried to jump in way too early to bug bounty, and they end up getting responses back or results back that you could have found with any of the tools on the market. And you just have to kind of chuckle and be like, you know, okay, bug bounties are cool and they're one of the hot things out there, But bug bounty doesn't replace a solid foundation, strong foundation in your application security program where you're taking care of the easy stuff first.

31:16Pete ChestnaFor sure, for sure.

31:18Chris RomeoSo, I wanted to just quickly touch on or talk about the kind of open-source world. I'm always thinking about, you know, there are people out there that don't have large budgets and things, and they're always trying to scrape by and figure out how do we do something, especially in the beginning when I can't just write a check. So what does the open source world even look like for SAST and DAST? Is there even anything out there that's worth people taking a look at, or are they really just no options there?

31:44Pete ChestnaSo I'm not very familiar with the DAST portion of that, but for the SAST, there are— they are mostly tools that do quality checks, so your type of lint or code complexity tools that have added in some flavor of security. So there is something in there, but it's not robust. And the other ones that are available are for specific technology stacks. So you might be like your Breakman for Ruby on Rails, where it's very specific to that technology, but it spits out its own version of the information. So you try to integrate that with, try to find your best of breed in open source in every one of the languages that you write in, C++ and C# and Java and all these other things, and then try to correlate that stuff together. That's why you pay commercial companies to go and do the single pane of glass for all of your languages and support, that they can provide more universal and uniform results for you across your application portfolio.

32:45Chris RomeoYeah. All right, cool.

32:46Pete ChestnaAnd you get the— you also have this false positive tuning too aspect of it, right? So being able to understand that aspect of it, that might not be the primary benefit of building one of these open source tools. It's just, here's all the stuff I found. So you still have the same problems or similar problems. You have to deal with false positives, you have to evaluate the results and determine what you do next, and so on. So it's not going to solve all your issues by not writing the bigger checks or whatever. Well, and there's no leverage for you, right? So when you're purchasing a tool, you can go back and say, I need this language or I need this fixed. That's not something that unless you're willing to cough up resources to go and do yourself and give it back to the open source community that you're necessarily going to be able to get. So especially in your very large organizations that spend millions of dollars on this, they have no, no more leverage than anybody else in the open source community.

33:36Chris RomeoYeah, it almost seems like you're at a disadvantage if you're going to start trying to kind of piece together open source tools. It's almost like from my perspective now, I'm thinking it— there might be more risk in trying to go cheap, because in my experience, if you lose the developers early, it's very, very hard to win them back. Even if you say, I just spent $10 million on this new tool, they're still going to say, yeah, I remember the crappy tool you gave me and how it didn't work and how you generated a bunch of bugs that, you know, there were 10,000 bugs in my queue that I had to look at. So I'm almost thinking there, they might be better from a programmatic perspective to make a small investment in the beginning to ensure you have success in what you do.

34:18Pete ChestnaFor sure.

34:19Chris RomeoAll right. Well, hey, Pete, thanks for taking the time today to walk us through all these different types of technologies and provide some strategies for how folks can be successful. And we look forward to maybe a future conversation about metrics. Might be a good conversation. I just wrote that down as we were talking here because that's something that a lot of folks are struggling with. So, love to have you back on to talk about that. And thank you for your time today talking about all things SaaS, DaaS, IaaS, and RASP.

34:46Pete ChestnaThank you.

34:47Thanks for listening to the Application Security Podcast. If you enjoy the podcast, please do us a favor and visit the iTunes Store and give us a 5-star rating. Our intro music is 8-Bit Kung Fu by Born and TJ, and the outro is Southern Delight by Stefan Cartenberg. You can find us on Twitter @AppSecPodcast or on the web at www.appsecpodcast.org.