James Berthoty -- Is DAST Dead? And the future of API security

0:00Chris RomeoJames Berthoty has been in technology for over 10 years in engineering and security roles. An early advocate for DevSecOps, he has a passion for driving security teams as contributors to product and built Latio Tech to help connect people with the right products. He lives in Raleigh, North Carolina with his wife and 3 children and is pursuing a PhD in philosophy. James joins us to cover the age-old question, Is DAST dead? We also explore his AppSec Kool-Aid statements, which are things that he disagrees with, and we get into it a bit. And I'll let you in on a little secret. We don't always agree.

0:39The Application Security Podcast is brought to you by Security Journey. We provide application security training for not just your developers, but for all roles in your SDLC. Learn more at securityjourney.com. Hey folks, welcome to another episode of the Application Security Podcast.

1:00Chris RomeoThis is Chris Romeo. I am the CEO at DaVinci, a threat modeling company, as well as a general partner at Kerr Ventures. And Robert is not with me today. He's somewhere traveling around the globe. Um, but I'm super excited to have James Verzotti with me here today. And, uh, James, I feel like you've really been stirring things up a little bit in the world of application and product security. And I love that because we need more of it, but we'll get into a lot of that as we go. But let's, let's start out, James. What's your security origin story? How'd you, how'd you get into security?

1:34James BerthotyYeah. Um, really, I think that story is why I've had the, the stirring up that I've done as far as, um, my background is in IT operations up through like help desk into systems admin and systems engineering. And then with cloud moving into more like DevOps, type roles supporting growing cloud infrastructure. Um, so I was at like Spartan Race, which, you know, like a medium-sized company. Um, and then went from there to ReliaQuest where I was a cloud security engineer on our internal platform. Um, but being at a managed detection response provider, that really was the first time that I was able to, um, drink deeply from seeing how security's handled, uh, and From primarily an operations standpoint. And I was building out a lot of the early days of like, all right, so we buy a CSPM and we start sending those alerts to a SOC. What are those guys going to do with those alerts? Um, and a lot of the nightmarish experimentation of, uh, really application security, which now I can look back and understand that traditionally only giant enterprises had the money and time to really build out meaningful application security programs because they were the ones who had the risk and incentive to do so. But with the advent of cloud, all of a sudden people like me who are just sort of like DevOpsy type people are getting alerts from tools like Snyk and then slowly getting involved with like SaaS scanning, more in-depth DAST type scanning. And, uh, that process is what led to ultimately now, uh, I went to a startup called Very Good Security, worked on it in the SOC 2 automation space that gave me a love for startups and Uh, a love for the sales process and trying to build product, um, all around compliance automation, went back to ReliQuest, and now I'm at PagerDuty, uh, where we are FedRAMP in process. And so it's my first true taste of enterprise stringent, um, regulatory type environments. And so a lot of my opinions come from that. Like, I implement Snyk and I see all the alerts and there's like 2 of us and it's like, oh my gosh, what are we going to do? With all of this crap. Um, and so that's what fuels a lot of the, the desire I have to like actually try to see results in, especially in like mid-sized companies, uh, where, who are just now getting exposed to application security for the first time, as this has come down from like cloud and code becoming more standardized as just the way that everything is deployed and managed.

4:10Chris RomeoMm-hmm. So when I think about your trajectory here, so you've, you've come to the world of application security, not through the developer track, not through the system administrator track, but more from the operational side, the cloud, the cloud engineer, cloud security engineer track. And so do you, are you, have you experienced coding languages and things like that? Or like what, what kind of like as a cloud security engineer, how deep do you go? Do you code? Do you write stuff in Python? Do you just, do you just not have to do, is, does code not really intersect into that side of the operational hurdle?

4:51James BerthotyYeah, I think this is why I advocate so much for product security is like everyone is learning how to code together at this point. Um, everywhere I've been, I've noticed more and more operations people asking for like, hey, can we get permission to install VS Code on our system? We want to write a Python script to do this thing. And really it's, it's about how I don't, I think everyone has to code now. If you want to manage infrastructure in the cloud, if you start managing like Terraform deployments, you are one step away from like Python scripts to just start doing things. Um, and so that's why in my day to day currently, I've gotten better enough at coding that now I feel comfortable calling myself a developer. I've built everything with Latio Tech from the ground up using a bunch of different frameworks and technologies, but, but that journey to get there. Uh, I think really speaks to what's happening more broadly with cloud where sort of everyone is dabbling in scripting now. And once you start doing that, you're getting very close, especially because of the popularity of Python and JavaScript. Like it, you're getting very close to, uh, more formal software development. And I think that's part of the importance of the, of the training piece as well and how all of that fits in.

6:06Chris RomeoYeah, that's cool. That's, I, I, I very seldom do I get a chance to interview folks that have come to AppSec and ProdSec from your trajectory. And so that's why I'll probably have some questions throughout as we're going, just because I'm normally maybe talking to people that have come from the development route, people like me who came from the sysadmin route to cybersecurity, to landing in AppSec, you know, later in our careers. And so, um, but I know there's a, there's a large percentage of folks that have made the same trajectory, same journey you've been on. From, from, uh, coming from an operational, starting looking at things operationally, infrastructure, and then, you know, coming closer to, like you said, intersecting, starting to intersect with the SaaS tools and the other technologies that are, that are, we're using.

6:47James BerthotyYeah.

6:48Chris RomeoSo let's start with this idea of DAST. And so anyone who's listened to me for the last year plus has heard me many times rail against DAST and, and Um, draw the ire of DAST vendors everywhere because I've come out and said I don't see the value proposition in DAST any longer. And so I know you're somebody through, through Latio Tech, you're, you're doing a lot of analyzing the market, kind of analyzing different categories and things like that. And so, um, I would love to get your take on this. Like, do you in fact believe like I do that DAST is dead? Or do you think there is something more to it than that?

7:31James BerthotyYeah, I, I, uh, was an early adopter of StackHawk, like soon after their Sneak partnership. And that was sort of my, because of my background, that was sort of my first exposure to DAST. And I've only realized later, like what a blessing that was. And now I can sympathize better with people who say that DAST is dead. Um, 'cause I think when people say that, they're more thinking of what I consider like legacy DAST providers where they're just pointing like a web crawler. It's like going URL by URL and trying just like dumb fuzzing into the input boxes.

8:07Mm-hmm.

8:08James BerthotyUm, and now that I've used some of those tools, I totally agree that they're like totally useless. Uh, it's nothing but false positives. Everything's, uh, uh, if you have like one cookie with, if you didn't set secure only on it, you're going to get like 1,000 alerts. To, to add the secure only attribute to your cookie. Like, and it's something that, because browsers default to that anyway, so it's not even super meaningful. Um, and the time to investigate those alerts is really tricky because then you're trying to like create little mini POCs and see if it really fixed, worked or not. Um, so I think that that kind of DAST is dead. And I think that really it's been rebranded to API security. And I was really interested in, well, first of all, the first time That I, I was evaluating DASTs against, uh, an application that we were using that was GraphQL, uh, heavily based. And I was looking at the logs of the scan and like every log was just like, it was just trying to send like dumb payloads that weren't even in the right format. So of course it's going to get nothing but like 200 responses. Um, and so it was just like the most meaningless thing ever. But then when I pointed StackHawk at it and it actually interpreted the endpoint correctly, and I saw that it was actually fuzzing the, the true data endpoints in the correct format. That's why I don't think DAST is dead is that is extremely valuable when you can actually try injection, um, against the endpoints directly in a format that they understand. And I think that, uh, there is a big demand for API security. And I just wonder if people don't realize that like most API security vendors who aren't on like the runtime side are really just doing DAST. Um, but they're amplifying it by like looking at network logs and building, uh, API documentation from the logs and trying to build this picture of an application that I think provides a lot of, uh, value to it.

10:00Okay.

10:02Chris RomeoAnd so when you, when you're seeing these API scanners these days, are they, are they able to in fact Figure out what all the various endpoints are and really get a good profile because that's been one of the challenges to your, to the, and you even kind of alluded to it here a little bit with the legacy DAST products is they just, they just, they just, they just don't do a good job of really understanding the modern application. And so then they just default to junk, like, you know, your cookie example where it's a consent cookie. Like I didn't really need that to be secure, even if I, you know, it's just a consent cookie there. They're a layup basically for trying to say we comply with GDPR. But from, so, so are you seeing like these API scanners, are they really, can you just, is it point and click or do I have to do some work to get it to the point where it's going to really get some value?

10:54James BerthotyYeah, this is what I, what I usually tell people is like, they'll, they'll start going down the route of StackHawk, who I keep using, but there's others like Pint or Escape or Appto. Um, and they all, uh, require more setup than people want. Like part of why people like DAST initially was because you like put in your URL and then it's like, oh, it just works.

11:16Chris RomeoMm-hmm.

11:17James BerthotyUm, and then, then you look at, I, I, I remember starting at a company, to your point, this is me on the DAST is dead side. And I looked at our, uh, scan results for the last year and every scan was just getting nothing but 403s because the scan was blocked by our WAF, like an IP block. And I, and like, we were turning that in to auditors. Like, is no one like looking at this?

11:39Chris RomeoUh, they didn't know what a 403 was for a minute there. They're like, hey, what is this?

11:42James BerthotyWell, it's working. There's no findings. Go figure. Okay.

11:45Chris RomeoSo if you want, if you ever want to test—

11:47We're super secure.

11:47Chris RomeoWe're super secure. There's no, look at this. There's no findings.

11:49James BerthotyYeah. Um, and so that goes on the, the side of like, I, if you try to do DAST the easy way, it's going to be bad. And so you have, you do have to be willing to say, I want to take the extra time. to implement DAST correctly. And so for StackHawk, that's like deploying YAMLs in each repo that give it the information about like where to get the API documentation. But there's others like, uh, Levo and Operant, um, have an eBPF agent. That's a really interesting approach to where you could, they're looking at the logs that are coming in from the network and then recreating the API docs on the other side. It's just another way to get the granularity, but from the runtime as opposed to the documentation. Um, Pint has a really tight integration with, um, Postman collections to get some of the docs that way. And so all of that to say with these different examples is that they have ways to get this stuff, but it's definitely not trivial. Like it requires some developer implementation. And I think this is why DAST is always the, the ugly duckling in the like ASPM scanner bucket. Like everyone goes outsource for it because it's, you can't just like set up a webhook. and just start auto scanning DAST. It does require some amount of instrumentation. And I think that's what has made people really hesitant, um, from a vendor side to properly support it.

13:07Chris RomeoSo are you still using the term DAST or have you replaced it with API security? I know from more from a Laceo Tech perspective, when you're looking at categories.

13:15James BerthotyUm, I, I still call it both. I, because I never want to like, I don't want to hurt company sales just because like people don't want to look for DAST. They look for API security and it's like, oh, we're doing DAST. I wish everybody was able to see that it's just DAST, um, but I'm not going to try to change the market forces either. Um, I do, I think the issue we have on the API security side though, is that you have like predominantly runtimey kind of people that are like looking at the logs from the ingress portion and then creating WAF rules around it basically. Um, and that's a lot of what people mean by API security, but then you have this, like the DAST type of configuration doing API security. So I do think there's confusion as far as like needing 2 different buckets there. And then you've got a couple people, I think Impart might be the only one I'm aware of that's doing like both the runtime and the DAST at the same time.

14:06Chris RomeoSo when we were having a previous conversation, we started talking about CVEs and reachability analysis. And I remember I wrote this down because you made a statement about this. You said, and maybe I made the statement. I think it was you though. Um, I mean, in regards to the fact that CVEs and reachability analysis have led us away from the real problem that we have, this is something you said, I think. So what, what are your thoughts right now? I mean, there's been a lot of noise about CVEs and NVD and everything and the, the delays and all the things, but what are your thoughts on CVEs? And let's start with CVEs.

14:48Yeah.

14:48Chris RomeoWhat's your, what, what are your thoughts on this?

14:51James BerthotyUm, there are many people much smarter and more ingrained than me. I'm very sensitive to the fact that my opinion comes heavily from this like DevOpsy operational midsize business thing. So I, from that perspective, I hate CVEs. Like they're almost all false positives. They're scored entirely wrong because it's like the worst case scenario scoring when like no one uses applications in, in the way, like Usually people are running applications in weird ways to get a CVE so that they can publish it and then they can write a blog about like the amazing CVE they discovered. Um, I think it makes it really hard to know if you're a mid-sized company, like what should I actually fix when everything looks like it's a critical nuclear meltdown failure? Um, but I'm sensitive to the struggle of like, I'm really grateful for the NVD. Like part of going to VulnCon was Realizing like internationally, like people are just trying to get similar programs spun up and like they can't get funding. Like there's so many challenges to trying to make something like that. So I find them super annoying, but they are critical, uh, a critical backbone to have a standardized way to disclose this stuff or else you have nothing.

16:09Chris RomeoMm-hmm.

16:10James BerthotyUm, so I never want to come across as like a CVE hater. 'Cause I'm really grateful for like all of the work that comes into doing it and making them, it's important. Um, I think the key problem is with the vendors who have sort of lazily slapped CVEs into a security product and then said like, there you go, like go fix all this stuff. Um, so I think, I think we on the vendor side have to do more to, to enrich And make it, make the data usable for people.

16:41Chris RomeoOkay. That's helpful. What about reachability analysis? I know this is, it appears to be all the rage in software supply chain, software composition analysis, scanning tools, reachability analysis. And I'd even seen it discussed in RASP and IAST contexts as well. Is this really all the rage right now? This, this idea of reachability.

17:06James BerthotyYeah, I, I'm, uh, uh, reachability is an important feature to have, but we need to stop acting like it's like the thing that fixes all of the problems. Um, 'cause we're really, the problem that everyone has with their vulnerability programs is they've only ever seen the number go up and to the right. Like I have never seen vulnerabilities go down. And the whole idea of shift left, the, the, the reason people bought these products was because they thought it was going to be I implement this tool and stop the bleeding, and then I can slowly churn down my backlog. And so they had this idea that like the VOM graph would go up and then it would sort of level off and then slowly go down. But in reality, it just keeps going up. And reachability, uh, really just hides that problem by shoving another filter on there. And that's why it, to me, like you can, if it's internet reachability, if it's runtime reachability, if it's function level reachability, all, all of those types of reachability are just filters on the dashboard to try to pretend like it's going down, even though it's not.

18:08Chris RomeoOkay.

18:09James BerthotyUm, and I, I think people have done it, um, because our entire workflow, I think the fundamental problem here is our workflow needs to totally flip where, uh, when, when Patch Tuesday comes around with Windows servers, like nobody sits there and goes, am I really vulnerable to that CVE? Am I really vulnerable to that CVE? Am I really like, you just, you just patch the thing and move on. And I think vendors have fallen into this trap of listening to security people who are, who, when they get an alert, their initial impulse, and it's really hard not to do this, is to start trying to create like a mini POC thinking about if you're vulnerable as opposed to just trying to fix it right away. And that's created, yeah. And so that's created this workflow where vendors are saying like, oh, people need help figuring out if they're really vulnerable or not. When people actually need help, uh, patching things and knowing how hard it's going to be to patch, creating playbooks for, to help people patch, knowing with container vulnerabilities, if you just need to redeploy the image or if you need to change the version, like there is so much work that needs to be done to help people fix things. And I don't, that's not to say reachability doesn't matter. It's just that it's not the primary problem.

19:22Chris RomeoIs patching really still that hard? Because it was hard 26 years ago when I started in security and even years before that as a sysadmin, when I was kind of making my way, you know, beginning my career journey. I would have thought we would have solved this by now. It seems like my macOS can update whenever it wants now. And I just sometimes come in in the morning and I get a new, and I'm at the login prompt again. And I'm like, oh, it must've applied an update overnight while I was sleeping. And like, why is patching still so hard? And maybe that's a whole, that should be the name of my 7th book. Why is patching so hard? But like, why is it though? I mean, you're on the operational side. You're more, you're more in the weeds than I am on, on, on operational types of things. And maybe I live in a glass house. I don't know. But like, why, why is it so hard to patch?

20:12James BerthotyYeah, I, I think it's the same reason that technical debt happens, um, where It's never hard to patch the service that everyone's already working on. Everybody's already like, it knows what it's doing, invested in. It's hard to patch the thing that like somebody that the old principal architect made that like ran under his desk and then like he left and it's gone and whatever. Like that's the stuff that's really hard to patch. And then I think what people, because it's not just like Windows is going to issue a patch, like Ubuntu patching is not that bad because it is like pretty standardized. Um, it's when you start throwing in all the SCA stuff on top of it, and it's like you have unmaintained packages versus well-maintained ones. You have people who know how to semantic version properly and people who don't, like there's no enforcement upstream on this stuff, which is why I really love like what Tidelift's doing to try to actually help standardize some of this stuff upstream. Um, because otherwise, like that's the core of the difficulty is that You never know how hard a patch is going to be until you start going down the path of fixing it. And I remember like when Snyk released auto pull requests was my first exposure to like, oh, I'll just check the box and then like the devs can just merge all the PRs and it's fine. Um, but go figure, like every PR had a ton of breaking changes. Like maybe it worked, maybe it didn't. Um, the only vendor I'm aware of that has like interesting data here is, uh, Mend because of their Renovate product. like the open source free, uh, version upgrading product because they can actually tell you like the, how difficult it seems like it is for people because it's like failing a certain amount of times when it tries to auto update.

21:50Chris RomeoOkay.

21:50James BerthotyUm, and so that I think is the key issue is, is when you're talking about patching like a, a, a patched version of some open source colors JavaScript package, that's like 2 seconds. But migrating from Spring 4 to 5 is going to be like a major cross-company, uh, rewrite of all of your different dependencies. Um, and so that's why in this space, I'm actually the most excited about vendors who don't even really market as security. Um, so like Grid and Modern and, uh, SEAL backports patches, but Grid and Modern especially focus on like writing playbooks to make migrations easier. And I think that lies at the heart of like how to actually help people get stuff fixed instead of raising a million alerts that all tie back to like your Spring Framework version.

22:43Chris RomeoSo the million dollar question then, is AI going to solve the patching problem? Is there a future world you see where an AI and LLM driven solution can detect those potential future challenges that could come when trying to integrate a patch or can do that integration and then fix whatever problems come out of it because it can be trained in such a way to know how to fix things that are happening when there's a merge problem or something like, what's your, what's your thoughts on, on AI specifically in the patching space?

23:18James BerthotyYeah, I mean, that's exactly what, uh, Grit and Seal are doing. Um, as far as like Using LLMs to increase the speed at which they can deliver some of these playbooks for migrations for people. Um, I, uh, I, I definitely think that it can be done. Um, it's just a question of creating the, the augmented queries properly, where you give the, the AI the changelog since the current version, and then give it the current codebase, and then let it suggest the migration path.

23:51Chris RomeoYeah.

23:51James BerthotyUm, I think that's totally plausible. Um, and then something that Grit's doing that I really appreciate the detail on is, uh, also helping you write the tests for it because that's the key. Uh, when you try to do that, all the tests are gonna fail. So those also need to be part of the consideration of that migration.

24:11Chris RomeoYeah, that's a whole other, whole other avenue to this is And you would think LLMs would be fairly decent at writing tests based on, you have, and even if you had something, a custom solution where you could load up your tests into an LLM, you could train an LLM in a custom way with your own tests. So it got a feel for what good tests look like in your world. That seems like that could be something that would be, uh, but it'd be a pretty powerful feature. Yep.

24:43James BerthotyAnd that's the asterisk I have on this whole thing. Well, first of all, what I usually say, if I were given a bunch of V— if I was trying to VC raise to start a company, it would be exactly this. I think this is the biggest opportunity is applying LLMs to each of the ASPM scanning challenges and trying to holistically rethink how, how we approach them. Um, but the asterisk is that a patch has to be available in the first place. And this is why I, uh, this is why I love Tidelift so much is they're the, they're literally the only ones out of all the vendors I've talked to, uh, who actually are just trying to help create relationships with maintainers. Because if they don't make a patch, if it's just this like dead repo somewhere, like there is no migration path or, uh, update coming. Uh, and you still have other risks of like zero-day malware, like we saw with XZ. Uh, utils, like it, Tidelift to me is the only upstream solution to try to actually pay maintainers and create relationships with them so that we can actually fix things.

25:46Chris RomeoYeah, it's, it's one of those issues. Like when I think about the software supply chain, I'm like, it should be so simple, but it's not. It's so complex and so complicated. Like if we just did some things to your point about, you know, working with maintainers, getting to the point where the top 10% of open source projects all versioned the same in the same standard approach that they all follow the same set of rules and the same constraints. And like, it just, it seems like it should be an easy problem, but then you introduce the 100 million developers that are maybe not 100 million, but The 10 million developers working on all these different open source projects, some of them in their basements and closets and whatever, because they don't have funding. And that's, that's really what it all comes back to. I just had a great conversation with, uh, Mark Kerfe and Simon Bennett, uh, Simon as the project leader for Zap. And just coming back around to like one of the biggest challenges in open source is just people that use it don't pay for it. And even companies that build products on top of open source projects aren't willing to contribute. And that's what we got to change in our industry because that, and it doesn't make any sense to me because I'm a startup person. Like I've started companies. If I built a product on top of another piece that was open source, heck yeah, I'd be investing in it because my, my risk register at the top would say open source project maintainers decide to quit. And then I no longer have a foundational thing to build on top of, but people just don't think that way. They don't, they think about their bottom line and they think open source is free and I can just get it. I can just take it. I can consume it because the license agreement allows me to.

27:32James BerthotyWell, that's the thing that drives me the most crazy, um, is the, when I see people submit like CVE scans against open source repos as like issues, um, because they're treating the open source repo like a vendor. Uh, because you, you sort of have to from compliance purposes. Uh, and then you're just like sending like, oh, hey, I ran the Sysdig scanner against your image and here's the vulnerabilities. Please let me know and it'll be fixed. It's like, but that's crazy, uh, to, to think that you can do that.

28:03Chris RomeoYeah. I mean, fix it yourself and generate a PR, submit a PR against the project. Like that's what people don't do though, is they do the, what you just said, they, they contact the maintainers. I have a problem with this. You know what? This is open source. You have a problem, create a PR, and then it'll go through the approval process and whiz bang, you'll have your solution done. Like it'll, it'll be fixed according to what you need for your compliance scan that makes your auditors happy and makes your company money at the end of the day.

28:31Yep.

28:32Chris RomeoThat's the challenge that we have here. And that's, I, it's just becoming more and more apparent to me. The more I talk about software supply chain, the more I realized that's the core. It's about the money. But with most things, follow the, follow the money. It's how you, uh, you know, it's how you, uh, it's how you investigate international crime. It's also how you deal, solve, or figure out the open source challenges of the world. Okay. So you, uh, are a prolific LinkedIn poster and I, I enjoy, uh, following you and, and, uh, interacting with your posts. There was one, uh, the AppSec Kool-Aid statements I disagree with. And I gotta say, I was chuckling as I was reading it because it was, it was, it was, it was some hot takes on a number of different things in our industry, which is, and it's funny, you, I just looked at number 3 and I didn't even realize this. X tool is just a wrapper for open source project Y. I kind of walked into that one already.

29:26James BerthotyUm, that's funny. I hate when I hear it. Yeah. I, I, it's always the posts that I put the least amount of time into that get the most traction. 'Cause this one was just me, like I started. I, I just wanted to talk about DAST is dead with this. And I was like, oh, here's some other stuff that like I get super annoyed about too. Um, and then like the most—

29:44Chris RomeoIt resonates. It just tells you though that it resonates with people because other people are like, yeah, that is kind of annoying. So let's deal with the first one you had on the list, because this is one that I've also taken a stand on in the last year or so in various conference talks and in writing written form. Uh, the first one from your list was API security is just Wow.

30:05Yeah.

30:06James BerthotyAnd that's, uh, that gets at the heart of those runtime API security vendors. Um, and I, I definitely assumed, uh, that this was the case until I started Laceo, met with a bunch of API security folks. Um, you know, why would I ever get, uh, something like, uh, I haven't done full evaluations of like no-name and stuff, so I don't want to say what they do, uh, specifically, but. My sense of them was always like, why would I get something that looks at my outbound API ingress points when I already have like AWS WAF up in front of it? And, uh, I generally agree with that, but I think it misses a lot of the value details that come from the more, uh, holistic solutions. And it misses some of the innovation that's happening in the space of API security generally. Um, so I still, uh, I agree with some, some API security vendors are just doing WAF rules essentially. Uh, but I think there's a level of depth and, and, uh, interesting features that mean that we should continue to tease out, uh, API security as like an end-to-end solution, whether it's stuff like we talked about with the, especially when you start throwing eBPF agents in there. Um, and building these network maps of how the application's talking, looking at like doing data tagging, redaction in, in line. Um, there's a lot of just interesting options that come with like end-to-end API security. And conversely, I think the, it's a mistake to think that the legacy WAF vendors do like just as good a job on this stuff as the API security ones do. Um, because they have a lot of limitations that like when you're writing these big gnarly GraphQL queries to a backend. You're, there's a decent chance you go over the AWS. I think they just bumped it to 16 kilobyte, uh, limit, um, on the WAF side, but like, you're going to, there's a decent chance you go over that limit and there's a lot more, uh, to the payloads themselves than what traditional WAF rules are looking for. So there's a sense here that I agree with it. I just don't like the writing off of the category of like, oh, we don't need API security because I have a WAF. That's, that's, that's really what I was getting at. Yeah.

32:22Chris RomeoAnd I'm, I mean, let's, let's just talk about WAF and, and it's specifically about that technology type as well. So I had a, uh, previous company had a security review process we were going through for a large company and they said, well, you don't have a WAF. That's a problem. You have to have one. I said, no, I don't. And they were like, well, why not? I said, well, because I run Our application in a containerized environment. It is bare bones. It's a stripped down container. There's nothing in it other than what we need to run the application itself, you know, and the, and the runtime for it. I said, uh, I've got a RASP agent attached to it, which does my blocking for me and my visibility into it. I said, you tell me, explain to me what a WAF is going to do for me. And then you'll, maybe you can try to convince me that, that I need to install one. Well, well, I mean, we have to do that. This is part of our requirements. I'm like, I mean, I don't care. That's not part of my requirements. And so it was that moment where I realized like people use something like a WAF as a crutch and it's not a, it's really not adding a whole lot of value to the world. Because if you think about the cat and mouse scenario of WAF rules, right? What happens? Yeah. You create a rule to block something, attackers go create some new bypass, uh, message that's going to, you know, pass, allow people to pass through. And then the vendor comes back with a new, you know, you're always in this cat and mouse game. And I'm like, it's just not, it's, I don't think that, I don't, I mean, does it hurt? Do, does it hurt to put a WAF at the front end of an environment and just leave it there with base rules that are just blocking the noise? Great. Okay. We're, if we're, if all we're doing is using it as a noise prevention tool, then fine. I mean, I'm not going to be mad at you because you have a WAF installed, but if you tell me I have to use it as a primary security control, I just don't think there's a value. I don't think the value proposition is there. I don't think it's worth investing the time in maintaining it today, but you're on the operational side. So I'm curious what your opinion is.

34:26James BerthotyThat's exactly how I view it. I view it as both, uh, uh, stop auto scripts from doing anything. Um, and then the only thing I would add is I do think there's benefit in like having you, sometimes you need a quick operational response to something. Like if you have a set of IPs that shows up on an IOC list from a vendor or something, or you have, um, uh, like Log4j, like just a way to monitor or block like JNDI in the payload.

34:55Chris RomeoMm-hmm.

34:56James BerthotyUm, it's just a helpful low-hanging fruit way to respond to things. Um, but I definitely would not view it as like, we're secure, we have a WAF, so we don't need anything else.

35:07Chris RomeoI think that's still how it's perceived.

35:10James BerthotyAnd, uh, um, I, yeah, I do think there's a lot. I, I wish people would reevaluate their tool stacks in light of modern web architectures. Um, I remember I was really shocked by a post that was on the cybersecurity subreddit that was like, if you had an unlimited budget, what tools would you buy? And it was all CrowdStrike, Palo Alto, and F5, um, around like very focused on like WAF, firewall, EDR, like server endpoint detection. Um, there was no consideration in any of the replies for like even CSPM or CNAPP or, uh, any SDLC scanning, any of that stuff. Um, and that just really, I think a lot of people still think of the, the security stack in the way that you're describing of like, I have a WAF and an EDR. And so like that, I'm good. Uh, when that just doesn't really relate to modern architectures at all.

36:11Chris RomeoYeah, I think that's, uh, that's definitely true. Well, you got a bunch of other good things on this list, but, uh, I do want to talk about Laceo Tech. Um, and so I will just tell people to, uh, find this post. We'll put a, we'll put a link to it in the show notes so you can find this post on LinkedIn if you want to interact with it and see some of the other goodness. And if I remember correctly, there were some good comments that came attached to that too, as other people, uh, tried to challenge your, uh, your conclusions. So, but, you know, the statements like this, they do get people thinking, and that's really what I've been trying to do as well, is like with my DAST is dead campaign and don't, and WAFs are useless. It's, I'm just trying to challenge people to think about the stuff that's in, like, to your point, I wish people would just start, or would just go back and reevaluate the things that they have and really do some critical thinking about it and go, what is the real benefit of this? Is it really worth the fact that I'm spending a quarter of my budget on these tools? Are they really providing me that much value? Or are there newer things that I should be investigating that can, you know, I can take some some of that spend and reallocate it to things that are going to give me better results upstream and downstream.

37:21James BerthotyYeah, and that's, especially in FedRAMP land now, uh, it really makes you realize how hard it is to change processes, um, as organizations scale. And I think that becomes the key challenge is like, you have to be, if you're a security engineer at some major manufacturing company, like you have to be willing to really stick your neck out and do a lot of work. to say, hey, this DAST scanner we're using sucks. Like, let's go with an API security solution. Um, like that's going to be a big project with a lot of work that you're signing up for. It's a lot easier to just say like, oh, here's the new, the DAST results. Oh, it's a bunch of 403s. Like we passed.

38:01Chris RomeoOkay. Next. Like, celebration time. Yep. We passed. All right. So Latio, Latio Tech, what, what is this thing, this entity? What are you trying to achieve with it?

38:14James BerthotyYeah, I, the heart of it, um, is the, the list and sharing all of these vendors to help people do realistic evaluations. Like I just never wanted, there's so many times in my career where I've selected a product without knowing that there was like a better fit that was out there just because I was unaware that they existed. So my primary goal is to make it so that that's not the primary blocker. And then around that, I've created educational material through the YouTube and the newsletter to try to help people both understand what vendors are doing and the innovation that's out there. Because I think in cybersecurity, for as much as we, like, as a community seem to hate vendors, they're the ones who drive, like, 90% of the innovation. So I think it's important to, like, stay up to date on what they're doing. And really trying to create a community around innovation within product security as a whole. So that's like the Discord and the YouTube, the newsletter to stay up to date on this stuff. Um, down the road, uh, the next thing I'm working on is like a subscription version of the app that allows you to see more granularly, like what features every application that's on there has. Uh, so that you can do like real product comparisons in there. Because right now every product just has a couple sentence blurb, um, from, you know, my hot take of the day or whatever on it. Uh, and I'd really like to, to have that become more valuable to it. Um, so that's the next thing that I'm working on.

39:45Chris RomeoYeah, I think that'd be very valuable to have a clearinghouse, an independent clearinghouse of just information. Um, almost like a G2 for cloud security and AppSec.

39:58James BerthotyThe, the, I've been so disappointed by like the existing, like G2 and Gartner sites out there have like, like if you look at any of their SCA buckets, like you've got products on there that don't even do SCA. You've got like all the major providers of course are listed on there, but there's like so much innovation happening specifically in SCA. Um, and it just doesn't get reflected on those sites because they're pay-to-play. They don't reach out to small vendors. They're not aware of like the small nuances and what people are trying to accomplish. Um, so it's really just trying to be focused on, uh, on that piece.

40:33Chris RomeoI mean, when I, when I consider what you're doing here, you're bringing a practitioner perspective to these individual products where the G2s, the Gartners, These, the folks that are, that are compiling this information, they don't really understand that they couldn't deploy one of these. The people that are, that are, the people that are allowed to write about them couldn't deploy them if you put them in a room and gave them a week and a, a printed manual. They couldn't.

40:59James BerthotyThis is Chris Romeo saying that just for the, just for the publication.

41:03Chris RomeoOf course. Of course. I'm always, uh, I, I've reached a point in my career where, uh, I can just, uh, You know, save on things.

41:12James BerthotyIt gets at the CNAPP thing that I've been writing about a lot most recently is like, it becomes a list of feature checkboxes. Um, and you're comparing platforms against each other, uh, rather than trying to take a step back and saying like, what do I actually want this thing to do? Why would I buy this?

41:29Chris RomeoYeah. Yeah, definitely. So, all right, let's do a couple of lightning round questions and then we'll, uh, we'll wrap up our conversation. So. Uh, the first one we like to ask in the lightning round is what's your most controversial opinion on application security? And then why do you hold this view?

41:45James BerthotyUm, yeah, I think it's the, uh, my most controversial opinion is that we still don't know how much funding we want to give security. We still don't, we still don't know the organizational value of application security, especially when it comes to mid-market. Um, people are still figuring out, like when you saw that small economic crunch happen, all of a sudden it was like, whoa, like what, what's the right amount of application security we need? Because we've just been sort of like doing it. Um, and so that's my hottest take is that it's probably still pretty undefined, like the actual ROI on AppSec for mid-market stuff.

42:28Chris RomeoSecond question is, if you could have display a single message on a billboard, at RSA or Black Hat, you can choose whatever you could, whatever, or both of them. What would that single message say to our communities?

42:42James BerthotyYeah, I think it would, it would be that, uh, fixes greater than findings. Um, as far as like, it's so much harder, it's so much more beneficial to focus on actually helping people fix things than like, I discovered some new, uh, piece with my scanner.

42:58Chris RomeoYeah, that's great. That would be a very simple message. It would be powerful though. Fix. Fixes greater than findings. What about, uh, book recommendations? And they don't have to be in the world of cybersecurity. Any book recommendations you would, you would provide for our audience?

43:12James BerthotySure. The one that I thought of was, uh, What Is Art by Tolstoy. Um, he has the best philosophical definition of art and he calls it the communication of emotion. Um, and I, I'm, I'm getting a PhD in philosophy very slowly. And so that's the, the background. Uh, but it's a good— Yeah.

43:36Chris RomeoCool. That's, that's one that I will definitely look up. So any kind of key takeaway, call to action, something you want the audience to do as a result of our conversation here today, James?

43:46James BerthotyYeah, I, I really, it's just about, um, taking time to learn product security, no matter what role that you're in. Um, and usually that starts with either making a pull request or learning or setting up like kubectl and getting connected to Kubernetes. Those are the 2 like on-ramps to really start getting your hands dirty and, and building relationships with developers. Uh, we can't approach this problem as like a detached, uh, I'm just going to look at scan results and like tell devs what to do all day. Like we have to be able to empathize with the workflows that they have and what's going on. And the only way to do that is by getting hands-on with the, the stuff that they're doing. So.

44:29Chris RomeoWell, James, thanks for, uh, being a part of the podcast. Keep up the good work with LatioTech. I know I've been a consumer of that, uh, database as well, just in looking things up and seeing how things are organized. So definitely I'm a, I'm a big fan of what you're doing there. A big fan of, uh, all the stuff you're up to. So, uh, keep at it. Keep going. And, uh, we'll look forward to chatting with you in the future about, uh, next year's set of hot takes.

44:54James BerthotyCool. Thank you very much for having me.