Your Developers Are Letting AI Answer Your Security Findings

0:00We're now roughly between 35% to 60% of the PR comments that we're responding are— don't have a human writing it, right? I think it's going to become 80%, 90%, maybe even 95% or 99% is just agent to agent. One agent's coding, another agent's finding security flaws, that agent's triaging things and responding back to the original security agent.

0:21Chris RomeoAhmad is the founder and CEO of Corgia, an AI-powered application security platform that helps developers find and fix insecure code. A 3-time startup founder, he's focused on helping teams ship secure software without slowing development. Welcome to another episode of the Application Security Podcast. This is Chris Romeo. I'm flying solo today, and I'm super excited to be joined by Ahmad Saadeddin. And so, just a quick note up front so you know, Ahmad and Corgius sponsor the show, but listen, that doesn't mean they get special treatment here. They get the same grilling that every other guest gets to experience here. And this isn't going to be a product pitch because we just won't allow it. And we're going to talk about some of the research that Ahmad's been doing here, and he's seen some pretty interesting things about developers using AI and whatnot. And so we'll dive into that deeper, and I'll grill him on a lot of what he's seen, and we'll, we'll discuss and debate it, and we'll have some fun with it. But Ahmad, welcome to the show. We always love to start with people's security origin story. Really helps us and our listeners to set the tone, the context of where you're coming from, from a security perspective?

1:35Yeah, no, of course. And first of all, thanks for having us and thanks for allowing us to sponsor the podcast. We love what you guys are doing and always tune in, so we thought it would be appropriate. But, you know, let the grilling commence at any point in time. We're always happy to share as much as we can. And so my security origin story kind of started during my university days. you know, just hacking around, playing around with things. I remember that was the time I started getting into Linux and Ubuntu, and it was like, just to date myself a little bit, it was like Ubuntu 6.4 was when I kind of started on that journey. And so, you know, playing around with Nmap and Wireshark and, you know, some of those, you know, still very relevant tools to today, you know, trying to do man-in-the-middle attacks on my roommate. You know, having fun with it, playing around. And so was always just kind of like a security geek, always kind of hacking things around, building things, got into software engineering. And then I did grad school, and while doing grad school, even got into even more, started, you know, bypassing network controls during exams to have fun with it.

2:41Chris RomeoThis is all more than 7 years ago, right? Oh yeah, this is just to check the, uh, You know, yeah, this is like, we're not gonna get you in trouble.

2:492010, grad school was 2010. So this is like 15 years ago or so. So aging myself a little bit here, or 16 years ago now. So yeah, just having fun with that. And then got into different fields, but always kind of had this like software engineering curiosity about things and just kept building and hacking with things and playing around with it. And then did a second startup, we got acquired. And I had moved into a role where I used to head up payments for that company. Coupa, a big spend management company. And whenever you're moving money in the billions of dollars, security is like a really critical piece of that. Like, you know, it's probably one of the top 3 things you need to do, right? Because you don't want to be the product owner that, you know, sinks thousands, hundreds of thousands, if not millions of dollars of customer funds because of a security incident. So we work very closely with our CISO and the security team, and we had patented a lot of stuff on secure money movement. how to move money securely, like virtual cards, how to validate bank accounts, how to protect against fraud and payment fraud and remittance fraud. There was a lot of that kind of stuff. We witnessed a lot of, you know, what I could speak at least publicly, like, you know, you would see a supplier impersonation kind of problem. Like we saw, you know, public reports on that with that Lithuanian guy that was able to exfiltrate like hundreds of millions of dollars from Facebook and, you know, a bunch of tech companies by just submitting fake invoices, right? So we did a lot of stuff in that. I used to own a lot of those security pieces, incident response, a lot of those. And I was looking at a lot of the tools and they didn't change from my university days. And I was like, oh, the space really hasn't materially moved in the last decade. Maybe we should go start a startup around this, especially with the advent of AI. And we were at the time seeing ChatGPT 3.5 emerge. And so it was interesting to see, hey, this stuff is actually getting pretty sophisticated.

4:40Chris Romeo3.5.

4:42was hallucinatory as hell, right? And then 4 came out and then we started seeing the writing on the wall. And so we started saying, oh, code is becoming a primary use case for these LLMs. So therefore security must become a primary use case if code generation is becoming easier. So that was kind of my like long-winded security journey through the whole thing.

5:01Chris RomeoYeah, it's helpful to set context. And I'm guessing in that environment where you were responsible for so many different payment technologies and such a large amount of money, security budget in that scenario, I gotta imagine, is pretty easy to come by. You're not facing some of the challenges that maybe other companies are facing just because I would think that folks that are running that company know the risk well than a lot of other places would.

5:32You know, it's interesting you ask that. I don't think we ever talked about budget per se as dollar values. I think we looked at it as like, Hey, can we— I used to own a lot of the product, so can we just build things inherently secure from the very get-go? Can we like, you know, a lot of these like principles are free by design. Like if you plan things and orchestrate them correctly and actually design with security in mind from the get-go, then you can actually get away with not having to implement a lot of point tools. There's a ton of examples around this. Like one of the things that we did that was very unique and very cool was if you as a supplier submitted a new remittance detail to the customer, and we've seen that remittance detail in our community of customers before, we would tell that customer, hey, we've seen this before across other customers. And so we would feel more confident that that remittance detail was true. If it was the first time, we'd say, hey, we have never seen this remittance detail in the community before. You might want to call them or you might want to verify through another means. Right? Because we can't guarantee that that is a legitimate rate. You have wire fraud that exists. People intercept emails, people intercept, take over emails. They do all kinds of things. And so that was free to us, right? We would develop that within our product. That was pretty cool. And there's a ton of things that we used to do from just like fundamentals of security that were just free, right? And obviously there was tools, right? We did code scanning, pen testing existed, you know, a lot of red teaming stuff, tabletop exercises. So yeah, it was an important thing for us, but I don't think we ever thought about it as budget. I think we thought about it as engineering budget more so than a dollar budget.

7:17Chris RomeoThis episode's sponsored by Corgia. Design it, build it, ship it. Corgia secures it. Corgia is an AI-native app security platform covering your whole software lifecycle, from the first architecture diagram to in production. It brings design reviews, AI-powered code scanning, and autonomous pen testing into one platform, so your team spends less time chasing noise and more time fixing what matters. Visit corgea.com, C-O-R-G-E-A.com. Corgia, one security platform for the entire SDLC. And you raise a good point about when building something from scratch, you really do have the opportunity to do it right from the very beginning.

8:05Right.

8:06Chris RomeoAnd I think that's where we are with AI right now. And my fear is we're going to miss the boat. Like everything that we've learned and everything that we've known and talked about over the last 20 years since AppSec's been alive or around, we were constantly chasing the software development practices of the past. And I'll go as far as to say we never actually caught up. We never, I mean, people wanted to shift left, they wanted to shift right, they wanted to shift up, shift down. We did all the shifts, but we never actually caught up and made it in the end. And so I feel like we're at this point right now with AI and things that are being built with AI, where if we set ourselves up, we could be, we could have a solid foundation for the next 10, 10, 20, 30 years. But if we miss it again and we let it go, we could be doing the same thing. AppSec could be chasing AI for the next decade.

9:04100% agree. I mean, there's a big opportunity now with especially agent skills and processes and building software factories to embed security by design from the get-go, right? You're seeing a lot of security design review companies, including ours and many others that exist in the market that are starting to think about it. I think you can embed security from the moment an idea pops up or an agent starts to plan to embark on something. Hey, let me go check this other service and see like, what do I need to do from a security center? Or let me look at the skill and think about it. And you can guardrail a lot of these agents and a lot of the secure development process. But it's interesting because we have an opinion about this, that the problem with LLMs is they're kind of inhibiting human taste. And I think security is taste. It's a very tasteful exercise actually, because there's a direct friction between the user experience and security, right? There's an inverse correlation. The more security you need to make someone go through, the less of a pleasurable experience it is for the user. Look at the TSA, right? No one wants to go through the TSA, but we all have to go through it. And the more TSA makes it more scrutinous, the less of a pleasurable experience it is to get to your terminal, right? And so I think that still has to have human tastes, right? Spinning up a new service. Well, should we treat this as a zero trust service or not? That requires a level of intellect to know, like, what is our risk standard and policies and what do we really want to do here? And I fear that LLMs still don't have that taste unless they're instructed to think about it that way. I don't know why you think about it, but that's my fears around the space in some way.

10:49Chris RomeoYeah. And there's nothing enforceable at this point is also the challenge because you can inform the model, but you can't mandate the model to do something. It feels, and I think some of the challenge there is it feels like you can, but living in a non-deterministic world, You might get it right 9 out of 10 times, but if the one time that it didn't fire the way you expected it to, you could end up with some type of critical vuln that gets pushed into production because you're relying on the AI too much and you're letting things just rip through an automated process. And so, that's why I keep coming back to, there's gotta be some human in the loop somewhere. This whole idea of dark software factories and Agents building software, testing it, reviewing it, pushing it into production. Maybe for the cutting-edge, bleeding-edge parts of the world, of parts of technology companies, maybe that's possible. But like, think about the average bank. Like, no bank is gonna sign on to a software factory, an autonomous software factory. They're gonna be like, we have regulations that prevent us from doing anything like this. Yeah. Yeah. How is this— Large financial penalties as a result of not, of not having the right checks and balances that exist. And so, you know, my bank's website is never the latest and greatest, and I'm fine with that. Like, if I came to my bank's website and realized they were using all technologies from the last year, I'd probably switch banks because it'd be like, these folks are moving too fast, they're creating too fast and going.

12:29I mean, you don't want Admiral ChatGPT, right, with the nuke codes, right? That's why for the longest time the US arsenal was on floppy disks. Right? It was reliable, it was tested, it was rigorous, it was proven, right? They moved to CD recently, I've heard. But I think there's like, for sure, these dark software factories are scary for a lot of industries. I mean, I was talking to a robotics, a surgical robotics company a couple of years ago, and their fear, which one of their competitors had an incident of a hacker actually being able to hack the robot mid-surgery. And that's scary. You can start to do ransom at an un— like with someone's like life, right? Like you don't want that company to be a software factory type of company. You want some really smart developers and AI. Like AI can be wonderful when paired with developers at spotting a lot of stuff, but you kind of want them in tandem with each other working towards like very secure software. I mean, I don't know if you've seen NASA publishes really great coding standards and they talk about their coding standards and their opinions, right? I don't think NASA or SpaceX or any of the space or Blue Origin is vibe coding rocket telemetry software, right? You just don't want that crashing into a city or flying around. So I agree with you 100%. There's still industries where you will have a lot of this kind of stuff where it's too high stakes not to have a human in it.

13:53Chris RomeoYeah. Well, let's get into the research that you've been working on because I want to unpack a number of different pieces of it, but why don't you go ahead and just lay it out for us? Put it on the table.

14:03So, we're a pretty data-driven company overall. Like, when we build our solutions, we try and go into the data as much as we can in terms of understanding how things pan out. And we have a lot of data to talk about today, but maybe we'll start with a theme of, like, developers and software factories and shipping fast, right? Because I think that is in a realm, like, when you and I were talking earlier, not really spoken about from a security lens with, like, factual data. And we're seeing this interesting bifurcation between the opinion of security engineers on our customer bases versus their developers. Like, when you look at the surface level of some of the developer feedback, they're like, oh, we see a lot of false positives. In our benchmarks, we don't. Like, when we go and benchmark and build and actually, like, develop our software, we're seeing, like, very low false positive rates. So we started digging in. Okay, where are they seeing it? Are we missing certain context? Is there certain, like, things? And of course, there's always context that we don't have, like, cross-repo type of things or outside of the code base. Maybe it could be some control on the network layer. It's excusable things from like scanning code. But when we started digging into it, we started seeing a lot of the responses to our bot PR comments look agentic.

15:18Chris RomeoHmm.

15:18Like we started digging into like, hey, what are developers actually commenting on and why are they commenting on them? And what is the context that we're missing here? So, we said, let's pull the data. And so, some of the data that we've pulled shows that there's a rise in developers using their agents to triage security findings in pipelines. And so, we wanted to measure it. Now, this is a kind of a difficult thing to measure because how do we know something was written by a human or not? Now, when you contribute code into GitHub, sometimes you'll see like a Claude or a cursor be appended to the user.

15:51Chris RomeoYeah.

15:51That does not happen on PR comment responses. It doesn't say this was an assisted thing. It just shows that the user responded to that comment, right?

16:00Chris RomeoYeah, I've seen that as well. I've seen that where I get some type of, like, my user account is associated with the comments that went in via the agent. So I've definitely seen that.

16:11Correct. So this is a graph that measures how many times do we see— this is the number of comments we get on a pull request. response that have an em dash, right? No human uses an em dash, right? All the agents use an em dash.

16:29Chris RomeoSome of the— somebody does somewhere because they invented it and they started using it. I don't personally, I don't even know when I would use an em dash correctly. So.

16:37Right. I don't know on the keyboard how to invoke an em dash. It's not like, but humans aren't like, unless you're writing like, you know, actual stuff, like long-form text. you're not using em dashes, right? And developers are definitely short-form writers. And so, we're seeing almost 36%, a third of responses to our agent comments are written with an agent. Like, just, it has em dashes, it's very clear, we've measured it, and you can see it's like rising very rapidly. Like, just in June, it was 11%. In May, it was 2.5%. Probably the spike was a customer or 2 of ours that we onboarded that were just going wild about it. But we're seeing this reliably go up since May, it was 2.5%. Now it amounts to a third.

17:27Chris RomeoSo let's stop here for a second and kind of explore, like, is this a bad thing or a good thing? Like, I still think it's, I think it's okay. With some caveats, like if a developer is looping an agent to another agent and just creating a token burn scenario where they're trying to make the reporting agent go away, and maybe they're burning 10 times the tokens that they would've if they would've walked with an agent through a fix, for it. But I mean, what are your thoughts? Like, is this okay that this is where developers are going from a philosophical perspective?

18:15Yeah, I can argue both sides of the equation. So the negative side of it is, are developers weaponizing the agents to just mark things as false positive and aggressively overmark things as false positive? So one of the things that we noticed in our data was the second highest CWE that's marked as a false positive is hardcoded credentials. And we know we do a very good job at marking hardcoded credentials as false positive natively as a product. So we started digging into it and we realized like probably, and this is like anecdotal, not data that I've crunched, but anecdotally, 7 or 8 out of 10 times it is because they're contributing dev credentials into the repo. And every security team is like, that's a no-no, especially after the GitHub attack. Everyone's like, do not contribute any credentials, including dev credentials.

19:09Chris RomeoSo these aren't like test cases that have fake credentials or something. They're not true false positives.

19:14They're like, maybe they could be like development OpenAI keys. They could be development like infrastructure keys. Like these are actual keys used by developers to do their job.

19:28Chris RomeoOkay. And so they're trying to sneak 'em through by saying they're a false positive. And I think the motivation for the developer is it does make their life easier. Let's be honest, security often makes things harder, but it, there are some things that we have to do. Like we know we can't put credentials into a GitHub repo because some eventually at some point they'll be snarfed out by somebody in some scenario and reused, whether it's someone else inside the company that takes the key home and starts a startup using the development key that belongs to the enterprise. Like, there's so many different scenarios we could go into, but it's kind of the deceptive nature of that that's kind of the more troubling part.

20:13Correct. And so now, could people just prompt and like, an agent to say, as much as you can, mark things as a false positive, and then for all the other ones, fix them. And so we're seeing a lot of gun, like, trigger-friendly behavior here, right? So there's that world, right? Like, could you, could you not maliciously, but kind of like bypass a lot of the security stuff by using an agent to find any corner case to justify that this is not a true positive, right? So there's that aspect. I think then the other negative is it kind of is like, and you know this, Chris, from your background, like it's starting to remove developers from being good at security, like actually seeing what did we make a mistake of? How did we make a mistake? Why are we doing— like if you're now succeeding a lot of the security triage to an agent, are you as a developer becoming better? And I think this is a fear across all disciplines of software engineering? Are you architecting better? Are you doing better performance handling? And are you doing better bug management and exception handling? And so, so I think it's making developers weaker at security, right? And it's a, it's a very interesting paradigm. Now, the plus side of it, when an agent actually gives this feedback and gives very rich feedback, right? That's great. That can feed back into the engine. And we have a learning process that will take that feedback and go corroborate it and create custom rules. Wonderful. Like, they're much more verbose than the human developer. Human developer says false positive. Okay, why? Why is it a false positive? Is it because it's not reachable or not exploitable, or is there a network context that we're not aware of? Is there something we do not understand or know? And some of that has a learning opportunity either for the tools themselves or for the developer themselves, right? And I also find that the agents are helpful because developers are asking, hey, could you explain this to me more? I actually don't understand. So they're might be some learning opportunities for the curious, right? And so I don't know how to make sense of this. It's scary on some parts, it's good on some parts. The other part of this is we wanted to actually see like, okay, these all have em dashes, right, over time. And this is just a cohort of like a sampling. When we look at it, we started saying, well, what if they don't have em dashes? Right? Because now these model providers have caught on to kind of like, we need to start to suppress em dashes, right? And so we started saying, what if we included like long-form responses, like something that would take a lot of energy and effort to respond to, like over 100 characters or so long, right? Longer than a Twitter response.

22:59Chris RomeoWhich no developer is writing. Chapter 1 of my PR comment, the early days of this PR. Like, nobody's writing that level of— they're literally using shorthand as much as possible to communicate in the least number of characters. It's like in the early days of LLMs where people thought they could like abbreviate words to try to use less input tokens when things were super expensive, right?

23:24Correct.

23:25Chris RomeoSo yeah, so it's definitely a machine pattern. of, you know, greater than 100-word PR comments. I mean, if you've ever reviewed a PR in your life, you know this, that was written by—

23:38You just see this hyper-verbose response to a PR comment. You're like, you clearly did not take the time to write this on 10 comments. Like, I know, I know I was talking to you while this was happening, right? And so we see it go as high as roughly like 60-ish percent, right? And that's kind of even crazier. Like now the range of possibilities between the floor of the MDashes and what's kind of fuzzy for us to measure is anywhere between 35 to 60% of responses to security ticket— to security comments on pull requests is now generated by agents. I think it's probably somewhere between the 40 to 50 range.

24:18Chris RomeoYeah. And it's This is just making me come back to the determinism versus non-determinism, right? Like having an agent fix a security vulnerability is not necessarily going to consider the depth of the issue. And it may also leave pieces out because I've done a number of experiments myself in this type of an area.

24:50Yeah.

24:51Chris RomeoAnd like when you have an agent review code for you, you can never trust the first run.

24:58No.

24:58Chris RomeoBecause if you, if you, and I've done, I've done some experimentation with looping a development agent with a PR reviewing agent and just having them both monitor the output of the other one on the PR. And I burned through my max plan. on Claude over one night. I left them going, I went to bed, I come up the next morning and I've got 0%, um, of my plan left for the weekend. It's like Tuesday and it doesn't reset till Sunday. And I'm like, it literally went back and forth all night long between the 2 different agents coming up with new problems, fixing them, creating new breaks in the process. It eventually got close to an answer, but it never even finished it it ran outta capacity. But it just shows like the effect of non-determinism on security in the, on the PR is it just, it's, it's, we've gotta do some more to be able to really trust the value that comes out of it. You can't just say, review this code.

26:05Yeah.

26:06Chris RomeoI mean, take the results and go, I'm good, I'm done. It's only found one, it found one nit out of all the things I wrote. I wrote 1,000 lines of code. One nit was all that it found.

26:14It's super fascinating you're mentioning this. I have a nice way to assimilate, to think about it. If you give ChatGPT a photo of Miss America and ask it to criticize Miss America, it will find 10 things. If you ask it to find one thing, it will find one thing. If you ask it to find 100 things, it'll find 100 things. There's a reward function in these LLMs. They're designed with a specific reward function to perform with a certain like, you know, success rate back to the request, right? And actually, you could argue the reason of the OpenAI Hugging Face hack was because of that reward function.

26:55Chris RomeoYeah.

26:56Right. It was like, go do this. Okay, I'm just gonna hack Hugging Face, right? And so because of that, you actually like, it will just go infinitesimal on this. So when do you stop, right? How nitpicky do you want to get and when do you want to stop, right? We actually did this really cool experiment, the data of which I can't share yet because I just don't have it synthesized in my head. But we did a— we pulled in public repos, their code review PR comments for some of the leading code review products. Almost none of them agreed with each other.

27:38Chris RomeoHmm.

27:39Like, they almost— none of them agreed with each other when we ran them against each other. And they all kind of had different focus areas depending on how they were prompted and what they were most incentivized on. Some of them focused very heavily on like error handling, others kind of focused a little bit more on security, others focused— so like it Really, your mileage really varies on what truth of quality looks like in these, like, code review, quality review feedback loops. And it sounded like you just built an agentic, you know, tornado that just kept spiraling into itself and just never ending.

28:15Chris RomeoYeah. And costing me money as it was going. So, what else do we see? Like, this data makes sense about the length of comments and whatnot. Like what else, what else did the data kind of set up for us?

28:31Yeah, I think another thing that's interesting is we're seeing the number of commits or the number of reviews per PR also start to go up over time, right? And we're starting to see this trend. And why I'm showing this is it ties to, look, developers are also coding a lot more and contributing a lot more. They're shipping more. Now, this is like anonymized across a lot of industries and verticals, and it's a sample size, but the interesting kind of takeaway from here Is we've seen it as low as 4 commits or 3 commits per PR. Some of our customers that are really like building software factories, it's as much as 24, 25, 26 commits per PR.

29:09Chris RomeoSo what do you attribute that to? Like, what's, what's causing all of those commits to happen?

29:16The agents themselves commit as a safe, as a, as a, as a save point during their development process. So they use the commit more as a functionality. Oh, I built this, let me save it. I built this, let me save it. It starts to do a lot more commits than the typical human would. And typically, agents don't bifurcate PRs. Now there's stacked PRs, like GitHub introduced stacked PRs recently to like start to bifurcate like logical blocks of work against each other. Humans tend to be a little bit better at that. They're like, I don't want my PR to get too big because the review process is going to get too big. And now I have to wrestle with this very big PR. Agents don't have that problem. So like we, we at the company have succumbed to the same problem where we're like, look, your PR is too big. It's clearly like, you know, you went ham on the, on the agents. We're not reviewing this. Shrink components of it to like specific PR sizes so that humans can actually review them because we still have a human review in a lot of our stuff. Right. And so what this is showing, like, hey, we're seeing at the beginning of the year, we had 3 commits roughly. It went to almost 4 commits. Now we're at 4.5. And we're starting to trend to 5, and we're not even done with September, right? And so the line is clear. We're seeing an upwards trend, right? Some of our customers are very traditional shops. They still don't even have like coding agents, while others are like pushing the boundaries on that. So I think this problem will exacerbate the number of vulnerabilities we find, which exacerbates the developers' frustration with security, and they're going to start to ask for agents to respond and triage more and fix things more and just kind of move into this what we call headless AppSec mode.

30:58Chris RomeoHmm. So when you say headless AppSec mode, what do you mean by that? Like, just define that for me as a starting point.

31:06I think it's something we're trying to define as this market is evolving, but I think the— it's agent to agent now. Like we're now roughly between 35 to 60% of the PR comments that we're responding are don't have a human. Writing it, right? I think it's going to become 80, 90, maybe even 95 or 99% is just agent to agent. One agent's coding, another agent's finding security flaws, that agent's triaging things and responding back to the original security agent, that agent is fixing things and asking that agent to rescan, and it just becomes headless, whereas no human is going into a UI and reviewing these findings anymore, right? No human is triaging them anymore. No human is fixing them anymore, right? It's become kind of agents taking care of the messy parts of software development and kind of being like, I'm done. Here's a product you can use. Do you like it and have any feedback for me? And I'm sure you've done it. I mean, you were just saying like you got these agents to kind of go into this loop and just woke up one day and like they exhausted your Claude tokens, right? We're moving towards this like headless software development. Where even the IDE is dying. Like you look at Cursor, Cursor's experience hides the code, right? Replit's experience hides the code. We're starting to move into this like headless mode of software development. Some people are calling it software factories. Some people are calling it, you know, vibe coding. I don't know what it'll be in half a year. I mean, vibe coding as a term is, wow. 14 months old, 16 months old.

32:43Chris RomeoYeah. It seems like it's been around for so long and it's not even, it's a year and change old. I guess the tension that I think exists in what you just described of this headless AppSec future is more on the cost and the token side. 'Cause I've been seeing a lot of people publicly discussing how much they're spending on AI, like not their total numbers, but just the fact that their token burn is at such a higher rate than even it was. It's trending on a line like you've been describing for, you know, number of commits per PR. Their token burn is trending as well. There was a lot of hotness to AI at the beginning of this year from a budgetary perspective. People were taking budget from elsewhere and slot moving, you know, we're going all in. Everybody went all in on AI. It was the best possible moment of like, everybody said, we're all in, everybody's all in. Because if you don't say you're all in, people are going to think you're a dinosaur company and you're not going to be around in a year. But I think some of those decisions now are starting to have real dollar values associated with them.

33:58For sure.

33:58Chris RomeoPeople are spending a lot of money with Anthropic and with OpenAI and Grok and Cursor and You know, there's OpenRouter if you want to route to your own, like there's a lot of different ways you can spend money on AI now. But I think that will be a limiting factor in the short term on this idea of headless AppSec because it's going to cost maybe more than having a human being driving an agent, at least keeping some control on it in the short term. So, I mean, what do you think about the cost and the token burn perspective?

34:33Yeah, no, it's interesting. We built a calculator for this actually for code review. that we're publishing sometime maybe this week or next week. And it's actually an interactive calculator because a lot of people— the problem with LLMs is they've given people what we call LLM courage. I can build anything, right? And I want my way. Like, it's Burger King. I want it my way. I'm going to customize it up the wazoo to our business needs. And so, we're seeing a cohort of companies that are like, I can build my own AI SaaS. Like, I can build my own AI, you know, pen tester, I can build my own AI. And so you're probably seeing like every week there's a new open source repository of some new AI security tool, right? And so what's interesting is you're talking about code generation, but let's talk about security review also on the PR. Is that costing you $1 to $2, maybe $3 or $4 or $5? And now you're telling me the number of commits because you scan it per commit, right? You're scanning it per push. Now a PR could cost, you know, $5 to $25 per review. Some of our customers are doing 100,000 a month. That is not cheap. That is not cheap. Like compared to like getting it through vendors and whatnot is a fraction of the cost, right? You have to figure out a ton of stuff like. It's not as just simple, hey, LLM, go like scan this code and find me stuff. Like, there's semantic context, there's traversing the code base, there's actually being operationally efficient and fast. And how fast are these PRs? So we're seeing some data around this. Like, some of these commit pushes could be anywhere from half a dollar to a few dollars depending on the model that you're using. We built a calculator that actually estimates this, and it will actually ask you how many PRs, how many commits per PR, how many developers do you have, And it'll actually give you based on— and you can actually pick the model, like I can pick Opus or Mythos or Fable or like GPT-5.6 or 6, and actually give you an estimated cost of how much you want to pay. And so when I talk to a lot of, you know, prospects and customers and users, that is still very real. They're feeling the budget burn, especially because now we're starting to see more code being generated at an unprecedented pace. And I think what's going to happen, if you were to ask my prediction, what's the like vision over the last— over the next 6 months? I think when companies show their, their yearly earnings after January, right, and depending on when their yearly earnings— a lot of CFOs were able to justify, hey, we're gonna like put more token spend as long as we don't hire more people, and we may even let go of some people. Like, you're telling me through automation we can Do more and save more. I don't think that's the reality. I don't think that companies are getting the same ROI that they were hoping for a year ago when they embarked on this. So might we see budget cuts and more scrutiny? I think so. I think we will start to see some of that because people are not seeing that return. Like, who talks about Mythos anymore? from code scanning. 6 months ago.

37:47Chris RomeoNobody can get access to it. So—

37:50No, but the customers we've spoken to that have gotten access to it, they say, well, we just went from our traditional SaaS to now Mythos, and we've not solved our problem. We now have 2 vulnerability datasets, and we're not remediating fast enough, right?

38:03Chris RomeoI mean, and Mythos has got the same struggles, the same non-deterministic struggles. That like a SaaS scanner that runs in a deterministic manner just doesn't have that challenge, right? Because you run it and it, on the same piece of code, it gives you the same results time and time and time again. Even with a frontier edge cybersecurity stoked model, you still don't get determinism out of it. That, that hasn't been solved yet in any, any model providers. Right.

38:37Yeah, it hasn't been solved. We've minimized it a lot. It's not a solvable problem, but here's the trade-off. Do you want to trade off true positives and false positives for the determinism? I think like, look, in general, our stance on it has always been like, look, do you get 2 times more true positives for a loss of some determinism and a much better false positive rate rather than like you're just finding the basics of security? Because the basics are not interesting. Like a static scanner can find a lot of them.

39:07Chris RomeoYeah.

39:08It's the IDORs and the auth and the business logic. Those are the things that are escaping people, right? And humans are not deterministic. Like, you know, you go do a PR review. If I didn't have the coffee, if my coffee in the morning is going to lead to very different results versus me being very caffeinated and well-rested as a human reviewing that code.

39:27Chris RomeoThat's a good point that I haven't really thought about. Humans are not deterministic. No.

39:34In a code review.

39:35Chris RomeoHumans are very non-deterministic because you and I have had different development experience, different security experience. And so, all of those things that make up our biases, our kind of context when we're looking at a piece of code, you may have seen a business logic flaw in another system, and then we're both looking at the same PR. If I've never seen it, it's going to jump out of the across the page at you, you're going to be like, whoa, hold on. That part right there, I've seen that before in another product. And that is, that's a challenge. Whereas if I haven't seen it, I may not, it may not register for me. And so there is this kind of idea that humans are deterministic in the security approach to the world, but in fact, we're not deterministic.

40:24No. And then keep in mind, human slop is real. Our code wasn't any better. Like, hell, my code was much worse than a lot of the LLM stuff that's being outputted. Honestly, sometimes LLMs are way better code than I am.

40:36Chris RomeoThe stuff that I used to write, like people would, like senior developer would look at what I wrote and go, how does this work? How does it, 'cause it got, it gets the right result. It works. And he would look at it and go, this really shouldn't work the way you did this. It shouldn't be possible. But somehow I think I was parsing XML or doing something like that. Like it was a, and I should have done it with a library, but I was doing it by hand and it was, he's like, this shouldn't work. So yeah, I mean, I'm with you. My coding agents are far better software engineers than I am. And I think a lot of folks would say that right now.

41:15And I think we expect machines to be very Boolean. I put something in, I get the same result out. Right. And I think that's why we scrutinize them much harder than we scrutinize our peers. is LLMs are non-deterministic, right? We always say that, but are they better than the human level of determinism? Like, I'm telling you, like, there are days where I'm reviewing so many PRs or like looking at things where I'm rushing through it, right, to get it done. Whereas in some days I'm like, you know, this is a really critical thing. Something just happened in this context. I'm just going to pour my energy into it and like grind into this PR with this developer.

41:49Chris RomeoYeah, that's the part that we are missing in the world of AI today. Well, I think we got to land the plane here. So, Iman, what are some actionable things that our practitioners out here that are listening in and watching, what are some actionable things that they can take away from this research?

42:13I mean, I'll share this last stat, which is interesting, and I think this kind of segues into like that ask that you're, you're, you're, you just asked me. We measured that roughly 45% of developer comments are wrong when they mark something as a false positive. And some of our customers have done human review and seen it go up to 70%. And so the fundamental question is, have agents truly made our security experience better for the developer, right? I think we're seeing definitely way more true positives. I think false positives are definitely on the decline, but I don't think that developers have been better equipped with agents overall to like handle security volume because they're just producing way more attack surface area and the agents are making mistakes and they are making mistakes prompting those agents. I mean, the quality of the output of those agents depends on the quality of the developer driving those agents, right? And so is security training still a thing? Absolutely. Like trusting the agents to helping people triage things and just fix them automatically. Yes, they can help be a megaphone for that developer, but if the developer doesn't understand what the security finding even means in the first place, the difference between an IDOR versus, you know, improper auth, you're not going to get good results. Right. And so I think the problem is getting bigger, not smaller. I think the complexity of the problem is getting bigger, right?

43:38Chris RomeoYeah.

43:39More PRs are being generated, more commits per PR are being generated, more findings are being found per commit. More agents are triaging these things. So we're moving to a world where I feel like the developer is becoming extremely disconnected. And unfortunately, security teams are not equipped for that world, right? A lot of security teams don't know and grok that code. So I think you still need to do good security training. I think you need to, you know, educate developers on how to do it. I think you need to give a lot of these agents secure guidelines in their agent prompts or in their tools. I think you still need to have humans in the loop. I know as much as possible, like people are moving to software factories, including us, but it's really important to distinguish. Like we actually have what's cool, and maybe I'll share this. We actually have a risk score per PR. We have an agent that goes and puts a risk score on every single PR, and if it does not meet a certain risk in PR, you cannot merge. You have to drop below a certain risk.

44:39Chris RomeoHmm.

44:40It's frustrating, but you take a risk-based approach to these things because people are trying to move fast, but you want to move safely too.

44:47Chris RomeoHow do you get determinism out of a calculation like that though? That could— an agent A could give me a 7 out of 10 and agent B could easily give me a 9 out of 10 if I just run it once.

44:59Yeah. I mean, look, these things are pliable for sure. Like it's not a hard boundary. Like some things might flex like this. Some things might flex like that. But the way we see it is, look, between very responsible developers that are well-trained and educated and like are focused, which is what we hire for, and I know that's hard when you start to scale to 100, 200, 500, 1,000, 10,000 developers, that becomes very hard to enforce. So education plays a huge role. The second thing is we have a lot of stuff going on on every pull request. We're doing security scanning, code quality scanning, performance testing. We have test suites that are running, right? And then that risk score, it's another thing. It's not the final thing, but it is another—

45:41Chris RomeoSo, it has some deterministic measurements that are feeding into it that are just simple arithmetic that came from other tools. And maybe the model is doing some analysis, but it's, the model's not just doing a cold look at the PR and going, feels good.

45:59It actually looks at the diffs in the PR, all the code quality, code security, and human comments, and then looks at all these static runs that we do from our test suites and end-to-end tests and integration tests and all of that. It takes all of that and then provides a risk score. And it's actually, I would say, more biased to being more conservative than liberal, which you can influence the agent to do, but it's one of our—

46:23Chris RomeoWhich you want. Which you want. And that type of a— decision point, you want it to fail lower, to drop the score lower, because even though maybe that PR was better than, you know, than going the opposite where it says, ah, it's actually better than the data shows. You kind of want it to be more conservative in how it draws that conclusion.

46:44Correct. And so in NetNet, we see it as one of the many tools that we have in our pipelines. And you just have to, I don't wanna say just put as many tools in the pipeline as possible, But put different divergent tools that focus on different aspects of what makes good quality software, right?

47:00Chris RomeoYeah.

47:01We even did something really cool. We have a canary deployment. Like we took it from Netflix as an idea. Like Netflix actually has canary deployments where they'll route some of their traffic to see how it responds. We have a canary deployment. We'll reroute some of the pull requests traffic from our customers to make sure like when things like, do things break? We'll run it for 2, 3, 4 hours. There's a lot of things that we can do today. But yeah, I think that's not net some of it. I know that was pretty verbose answer to your very straightforward question, but I hope that—

47:28Chris RomeoNo, I think there was some action. I think you left us with some action, and that's what we always try to do here is give people some things that they can do. And I think the data you shared is a pointer to folks listening to go do some investigation on your own. You've got models that can do summarization of data. You can literally generate the data that Ahmad just shared for your own organization. in 30 minutes or less. I think you should be. I mean, if you've got GitHub access to the repos, it's just pulling data and summarizing it and looking for em dashes, looking for length of comments. And then, I don't know, think up some other rules and send them to Ahmad so he can incorporate them into future research that he does on this. But Ahmad, I want to thank you for being a part of the show. It was fun to just go and explore some different areas of how AI is impacting development right now. And I think it's, this is something that, uh, it's gonna be constantly changing over the next number of years, but these are some good reminders and some actionable things. So, thanks for sharing that insight with us.

48:31Thank you, Chris, for having me. It was a fun session.

48:34Chris RomeoThanks for listening to the Application Security Podcast. If you enjoyed this episode, subscribe, leave us a review, and share it with a friend or colleague who would enjoy the conversation. We'll see you next time.