Isaac Evans - AppSec in the Age of AI

0:03Chris RomeoIsaac Evans is the co-founder and CEO of Semgrep, an application security platform he started in 2017 alongside Drew Dennison and Luke O'Malley. He holds a degree in computer science from MIT. Quick note, we're opening up a few sponsorship spots on the Application Security Podcast. If you want to get in front of real AppSec practitioners, the folks actually making decisions, this is a great place to do it. Just reach out to me, Chris Romeo. I'd love to chat. You can find me on LinkedIn. Hey folks, welcome to another episode of the Application Security Podcast. This is Chris Romeo. I am one of the co-hosts of said podcast, flying solo today, and glad to be joined by Isaac Evans. And Isaac, our audience loves when we jump straight into people's security origin stories, so I'm just going to go there. If your security career was a comic book, what would we find in that episode 1?

1:11Isaac EvansOh, I love that. I guess it probably goes back to early interest in cryptography, back when crypto meant cryptography and not something else. And some of the first programs that I wrote were all about, you know, cryptanalysis of Caesar ciphers and things like that. I'm not exactly sure why I was so fascinated by it, but I really enjoyed it. And then I also really loved robotics and I was at MIT for undergrad. And then my junior year in school, there was a tsunami that hit Japan. The Fukushima nuclear reactor was destroyed. And one of the things that later I learned happened as a result of that tsunami was that there were a bunch of essentially firefighter tasks like closing valves, moving pipes that they needed performed inside the plant. And they looked on the market for what robots could be used to go perform these tasks because we're really concerned about what the radiation levels are like in there. And they couldn't find any robots that met their requirements. And so they ended up sending out a request for volunteers and retired plant workers went in there to do these tasks and potentially expose themselves to dangerous levels of radiation. So that story was actually the inspiration for DARPA's Robotic Challenge, DRC-3. DRC-1 was like drive a vehicle through the desert. DRC-2 was like vehicle in a simulated town. And DRC-3 was build a firefighting robot that could perform these kind of tasks. And so I was lucky enough to be one of just 3 undergrads that were on MIT's team, and it was a lot of fun. Our robot didn't actually do that well, but one of my other big takeaways from my time on that team was that the PhDs who were the majority of the team working on the robot, it was almost like they had negative security knowledge.

3:08Robert HurlbutThey knew enough about security to bypass security controls so that they could get their code onto the robot faster so they could get their thesis done. And graduate. And this robot was built by Boston Dynamics.

3:24Isaac EvansIt looks like Terminator. Boston Dynamics, these days, their robots look a lot more friendly, but the idea of this sophisticated robot with this deeply insecure code was really compelling in terms of some of the reasons why I started the company in the first place.

3:42Robert HurlbutSo, that's a little bit of the background.

3:43Chris RomeoYeah. Yeah. And then, uh, so you, you're one of the founders of SemGrup, so you kind of, so you've been there since the beginning. Have you been in the CEO role since the start?

3:54Isaac EvansYes, I drew the, uh, the short straw.

3:58Chris RomeoAnd it's always how it goes with a bunch of co-founders, but that's right.

4:05Isaac EvansOkay.

4:05Chris RomeoWell, let's, uh, let's jump into what's catching a lot of attention right now, this whole AI shift that we're seeing and it's impacting AppSec just like everything else around the world. Love to get your take on how do you see AI fundamentally reshaping AppSec over the next 3 to 5 years?

4:26Robert HurlbutYeah, I mean, it may not take 5 years, it may just take 6 more months.

4:34Isaac EvansLike just in the past 6 months, I feel like it really has, starting with Opus 4.6, started to dramatically influence how AppSec professionals do their job and look at security problems. I think the biggest thing that I see is that we used to treat CI as the kind of control point, right? Like, oh, developers are going to ship their code, they're going to do whatever over there, but then once it gets to CI, we're going to have code review, we're going to have these security checks. And that was an improvement over, well, right before it goes to production or background scans, some of the original tools in this space. Scanners might take days or weeks, right? And those were too slow for modern development practices. But now I think that CI is starting to lose its place as the central security control point. And as a replacement, there are 2 things that are developing. On the one hand, you have like deep background jobs, right? So you're like large, large models, say like a Mythos-class model, And we were actually fortunate that we had early access to Mythos. We helped make some improvements to it. We used it to improve some of our stuff. And now that we have customers who have also been using it in conjunction with Semgrep, we have a lot of data about how it performs. But the gist of it is, you're going to want to spend days of compute, if not weeks, just hunting for interesting vulnerabilities. And one of the other things I think is true is, Attackers don't have access to Mythos today. Maybe they will in the future, but the overlap between what, say, like a DeepSeek is going to find and a Mythos can find is not complete, right?

6:15Robert HurlbutSo you may actually be better off saying, hey, like, what are the threat actors using today in terms of models?

6:22Isaac EvansAnd obviously there's some random variables here, but maybe I want to try to get as much coverage of what the attacker will see if they look at this application using this model in that deep background job mode. But, you know, you and I both know you don't achieve security by removing bugs, right? Like, it's fundamentally like a whack-a-mole game. So the real opportunity is even further left of CI. And maybe 9 months ago, I would've said, well, like, if a code— if code is written by a human or written by an agent, we don't really care. It's just code. We'll scan it the same way. And what I've realized now that we've launched a product that's really focused as a plugin for the coding agents, so Cursor, Claude, Codex. This is actually our fastest growing product ever. It's explosive in terms of popularity. The advantage of being in the loop of the coding agent is that you can sit there and force the coding agent to try again and again and again until it gets it right. And Semgrep is one of the only tools that's fast enough to sit in the loop there. And it's not going to take 2 minutes to scan a fragment of code. Like, no, we can scan it in less than a second.

7:32Robert Hurlbutand then be like, nope, this does not have the security properties we want.

7:35Isaac EvansRegenerate it, regenerate it. And there's, you know, the models can generate functionally equivalent code 1,000 times, but only some of those may have the security properties that you care about. And so I think that that is the thing that's really surprising me in terms of, wow, you know, like the CI story seemed pretty good. And now if companies are not even doing human code review, a lot of these like checks in CI that we used to be treating as the kind of like primary layer may be disrupted as well.

8:02Chris RomeoSo what are you seeing as far as findings or results when you're causing the AI, the models to regenerate? Like how, I mean, how good are they? Is it taking literally thousands of times to get something that reaches? And then what's the threshold? Like what is good enough? when you're having this thing regenerate?

8:27Robert HurlbutIs it 90%?

8:28Chris RomeoIs it, I mean, is there any type of metric to that these days?

8:30Robert HurlbutYeah, well, so, you know, a lot of people, like, there's some vendors in the space who are like, you know, the models are probabilistic, you know, like, we can't trust them to have security properties.

8:43Isaac EvansYou know, humans are also probabilistic security reviewers, so I don't think that's a, like, fundamental barrier. But what we're seeing teams do is, like, on those far-right background job workloads. So, hey, you were using Mythos, you found some issues, can you root cause those and write some really simple Semgrep rules that would've prevented these things in the first place? And now take those, insert them back into the plugin to the code generator and say, we just never want to see you generate code that looks like this.

9:11Chris RomeoHmm.

9:12Isaac EvansSo, that doesn't even have to be vulnerabilities, right? That can just be weaknesses. And there's good evidence that not just from a security perspective, but from a general bug density perspective, the models do better with more constraints, right? So models are generating better code if they're being type-checked, for instance. And so what we're really talking about is, they're not having to generate it 1,000 times. It's generally once, maybe twice, that the model has to change what it did to pass. Essentially, you're like, very quick security tests inside the code generation loop. Okay.

9:51Chris RomeoSo, it's almost like test-driven development from a security test perspective to drive. So, you're creating effectively a super secure coding agent because you're adding more rules and you're almost putting a wrapper around the model, which some people have been talking about doing via prompts. Like, can you, can you Can you force secure coding just by telling the model enough context about don't do this, do this, this is bad, but it sounds like you're doing it at scale.

10:20Robert HurlbutYeah, I think that's right. And it's not just, you know, code vulnerabilities. It's also like, check this package for CVEs, you know, because like the model doesn't know about a CVE that was published yesterday.

10:30Isaac EvansIt has a training cutoff date, right? Or like, hey, like here's this secret pattern that like we never want to see you commit a secret that looks like this because that's like what a token looks like in our organization. Model may not know that, maybe it does, but it's just actually faster and cheaper to check it with the deterministic tool over on that side. We're still doing a lot of probabilistic stuff in conjunction with the models for customers as well. So like, we're not ideological, but in the coding agent case, the determinism ends up being pretty valuable. And then the thing that I think is the biggest opportunity that I've ever seen in AppSec is the idea of like steering the model even more generally. So for instance, we have, you know, like guidance rolled out for Claude Code in our organization. about dependencies, right? Hey, like, here's our opinion on dependencies. We don't want you to introduce a dependency if it was published yesterday and it has 3 stars on GitHub.

11:22Robert HurlbutNope. Like, we'd rather not have that as part of our supply chain risk. And previously, right, good luck getting developers to follow your, you know, high-minded dependency guidance.

11:34Isaac EvansThey're just going to do what they want and it's going to show up in CI and you kind of have deal with it. But with the models, the model will be like, well, I guess you're right.

11:41Robert HurlbutOkay. Like, whatever you say, boss, I will write the code myself, right?

11:45Isaac EvansI won't introduce this dependency, et cetera. And so, we now have this ability that previously only by putting developers in front of whiteboards, right? And giving them security training, which I've been through, it is not the most exciting thing in the world and has limited knowledge retention.

12:01Robert HurlbutThat was our best way to influence the mind of the developer.

12:04Isaac EvansNow we have this incredible capability to influence the mind of the model when it comes to the security properties that we want. And so I feel like the job of the security team is really becoming, hey, like teach this model what the business case for security trade-offs in our organization looks like.

12:24Chris RomeoSo it sounds like there's a lot of customization then in that customer by customer in the way you're approaching this. It's not universal. It's not like a single set of universal rules, contexts, knowledge that applies to everybody, which is what I would say how a SaaS, for example, worked in the past was everybody had the same set of rules. You could certainly do some customization to it, but you were doing it by hand.

12:49Isaac EvansMm-hmm.

12:49Chris RomeoSo it sounds like now I'm buil— you're building a tool that is customizing to the context of my organization. And it is refining against my organization, but it may be completely different from what Acme Corp is doing in the way that they're building software based on how I'm building software.

13:08Robert HurlbutThat's right.

13:09Isaac EvansAnd I mean that, you know, like Semgrep has always been an outlier in terms of the customizability of the open source and all that. And, you know, just to give you a sense of how we use it internally, we dogfood when we get comments like code review comments in CI, from an LLM, we tell the LLM, oh, also generate a Semgrep rule that would prevent this from, you know, like that would detect this like class of issue.

13:37Robert HurlbutAnd so, you can kind of look at it in CI and be like, hmm, like actually, I really don't like that.

13:41Isaac EvansI never want to see that again. I'm going to suggest that this rule be promoted to the code generation loop across the entire org. So now, like we're all kind of working together to train the consciousness of, you know, like that, the code generation agent to understand like who we are as a company and what we need in terms of security properties, right? And in fact, we're quite conservative. Other companies might be much less conservative, right? Or have different concerns. So I really think there's something kind of exciting about this opportunity to, we've just never had those kind of capabilities before. Yeah, absolutely.

14:18Chris RomeoSo, what does the role of a security engineer look like? Like, it sounds like this is a lot different from what classically, I'm talking about 5 years ago, I'm calling it classically now, right? But 5 years ago, security engineer meant something very different from what you just described there. So, like, what do you see as the role for the security engineer as these models get better, the tools get better? And, and, uh, a security engineer's not sifting through tool results like a lot of AppSec engineers had to do 10 years ago. They were spending a lot of their time scanning things and going through results and then generating tickets for the ones that were important. Like, what are they going to do now?

14:59Robert HurlbutYeah, I, you know, I, I feel like we have to almost start by answering the question, well, what are software engineers now?

15:08Isaac EvansAnd, uh, There's an Anderson Cooper interview where he's interviewing this guy who's like a music producer for a bunch of famous artists. And Anderson asks this guy, so you have— but he has no technical ability. And so Cooper's like, you have no technical ability whatsoever.

15:26Robert HurlbutAnd he's like, no, no, no, no, no. And Cooper's like, so what are you paid for? And he's like, for the confidence that I have in my taste. Right. And, you know, for software engineers, I'm like, that's actually not so far off the mark. This is the meme for us internally in terms of what's going on.

15:48Isaac EvansBut, you know, it is good taste and good judgment and the ability to kind of predict if we build this software this way or this system this way, is that going to like serve us well with where we're trying to go? in the future, right? I think so. The engineer now has been elevated to essentially a manager of junior engineers, as well as some like PM-style, you know, thinking, more product thinking leverage. And I think that, you know, my favorite persona is the security engineer, right? Someone who is really thinking about things in terms of an engineering mindset. How do we get leverage over these problems? And so, I think the security engineer becomes the same kind of thing, right? Like, hey, like, I'm going to manage a fleet of virtual security agents. They're going to do most of the toil work, right?

16:40Robert HurlbutLike, no more looking at findings for me, except unless there's some like weird outlier case where my agents can't figure it out.

16:50Isaac EvansAnd now, I'm thinking about how do I get out of this world of bug whack-a-mole? Right? And into a world where I'm actually like being able to certify the right level of security guarantees such that the business is, you know, like operating at an acceptable risk level, right? So that's kind of like the known knowns and then discovering, right?

17:12Robert HurlbutOf like, well, what's out there that like new things that people are talking about that could affect us.

17:17Isaac EvansAnd I think that's actually a lot more fun than You know, a lot of the drudgery of application security jobs. So I'm optimistic that there will actually be more need for that in the future because I think that LLMs are just like, they don't find the simplest solution to a problem. They find like the fastest solution to a problem and it might be much more complicated than what a developer would've come up with. And so complexity, as we know, is the enemy of security. So there'll be plenty of security need. But hopefully the work that we do on it will feel a lot more leveraged.

17:53Chris RomeoYeah, it feels like they're going to be, it's going to be higher up the stack based on what you just described, as far as security engineers are not going to be down in the nitty-gritty anymore. They're going to have an opportunity to layer up and oversee more along the way there. So, what would you say to somebody who's been, let's say, let's just pick a random number. They've been in AppSec for 10 years. They're kind of mid-career. right now? Like what, what would, what do you think they should be doing to prepare themselves to be on that, that to be able to, to, to go up the stack and not be trapped in the layer of, I guess, going against the AIs, trying to look at results and find things faster than the AI. It doesn't sound like a great job. So what would you tell those folks that are listening here that are trying to figure out how they're going to survive the next 10 to 15 years?

18:39Robert HurlbutWell, I mean, a lot of security knowledge already has a pretty short half-life.

18:46Isaac EvansSo I think that the advice to optimize for agility when it comes to security knowledge is really important, right? Like you want to be on top of like, well, what is the latest trend here? Like what's going on? Like who are the security teams that have figured out how to get some good leverage in this new era and what are they doing? So I actually think that the importance of kind of socializing with the security community, going to conferences, understanding how the teams that are really effectively leveraging virtual agents is just greater and greater. So that's, you know, part of why I'm delighted to be on a podcast, right? Like talking with other security people about what are you seeing out there?

19:32Robert HurlbutAnd that's probably my first piece of advice.

19:34Isaac EvansAnd then my second piece of advice is, be hands-on with the technology. Like there's never been a better time to be an IC actually, and the leverage that you can get as a solo IC is insane. Yeah.

19:47Chris RomeoSo let's switch gears a little bit and talk about the foundation model providers because it seems like OpenAI, Anthropic, they've been teasing this idea that they're going to have tools that will do AppSec things that will come from the model itself. And so, do you think, I mean, are you seeing a world where there is plenty of room for independent security vendors to keep being best of breed, or do you think there is some danger with what these foundation model providers are bringing?

20:23Isaac EvansWell, I think there certainly is danger in the sense that You know, the disruptive wave of technology, even if the model companies are not trying to directly compete with AppSec companies, you know, kind of like what we just talked about earlier, I think a lot of the like AppSec market is just obsolete, right? And all these companies that had product market fit in the old world do not have it in the new world. I mean, that applies to Semgrep as well, which is, you know, why I mentioned like this new product that we're working on, Guardian, which is very focused on being the agent for the models. And like a lot of the stuff, and in fact, probably entire categories like SaaS and SCA just won't look anywhere like what they used to. So at the same time, when we saw the cloud transition, hyperscalers showed up, they're like, great. Like a lot of people asked, hey, isn't AWS just going to solve cloud security?

21:27Robert HurlbutAnd turns out there was a lot of, there was a lot to be solved in cloud security.

21:34Isaac EvansAnd so, I think that the nature of the problems is going to change dramatically, but there will still be plenty of space for independent security vendors. It may be different from where those independent players are playing today, if that makes sense.

21:49Chris RomeoYeah. And the cloud example's a good one because I mean, AWS and Azure, they've been around for what, 20 years at this point? Is it that? It's hard to even remember when they hit the scene, but it's probably been about 20 years, and they were really in their infancy back in the, right after, you know, 2005 in that era. But it has taken them 2 decades to get to the point where we say now, okay, no matter what cloud provider you're working with, they have a solid set of foundational security pieces that can be plugged together. Whether that's key vaults and management, managing keys in the cloud environment, whether it's firewalls, whether it's authentication providers, but it's taken 20 years to get there, which if we, if I would've answered that question in 2010, I would've said, oh, in 2 years from now, they'll have solved all these things, but it took a lot longer. And so there could be some of that same, AI's been moving a lot faster than even cloud, but they're still, it's still gonna take, it would take them some time to—

22:50Isaac EvansRight.

22:50Chris RomeoKind of make their way around and find their way.

22:53Isaac EvansRight now, even though I say that, Chris, like there is a possibility that we need to consider that the advancements on the models, like if they continue on an exponential trend, may really upend our understanding of like industry and technology in a way that we can't appreciate.

23:15Chris RomeoRight.

23:16Isaac EvansLike if we are on the path to something like AGI, however you define that. So I'm not fully certain on, you know, if like AGI is coming, well, what does that mean for like any company outside of the foundation labs? Obviously you see some of that in like stock prices of public security companies, but I don't know if you have a take on that. It's a little bit more—

23:40Chris RomeoI mean, there's a lot, there's so many unknowns in that. I've kind of stopped making predictions. I did that a year or two ago and like, I'm one, all the predictions are being blown out quickly. Like things are happening so fast now that, but I think there's still going to be a place for people that really understand a problem in great depth. And I, and if we do get to AGI, then maybe all bets will be off. But for now, what we're seeing is kind of the trends. The foundation models may be able to do things like hunt bugs at a much greater speed.

24:17Isaac EvansMm-hmm.

24:18Chris RomeoBut does the foundation model then know how to influence an organization to get all of the products updated? And there's a lot of other moving pieces before you can get to a point where you say, now I have a, I mean, what'll be scary is the fully automated company. Like I do a start, like I literally type one prompt into a model. And then 3 months or a month later, I've got revenue coming in and it's created all the bank accounts for me and everything. And all I'm doing is playing on my farm, digging a hole while money, and all I did was type a single prompt. Like that could be the future, but that seems like it should be so far away. But like I said, I'm afraid to make predictions.

24:57Isaac EvansRight, right. Yeah, I agree with you. I agree with you. And, you know, there are companies like, you know, startups like Factory, right, that really are working on the, like the dark factory Hey, like, you know, it's just agents building. There's no humans in the loop, right? So totally plausible that we enter a world that is much, much more automated in terms of software construction. And, you know, AppSec is again going to look very different in that world than any of the existing capabilities.

25:31Chris RomeoWe've still got the trust problem though. Like you just mentioned, factory and, and, you know, agents building everything along the way. I've been in security since around 1997, so I don't trust anything in general. And most people in security don't, right?

25:45Isaac EvansAnd the models, as someone who used to work on a reverse engineering tool, right? Like the models are the most inscrutable binaries we've ever seen, right? Like our ability to reverse engineer and predict behavior of the models is almost nonexistent. Even the most obfuscated binary is more tractable to predict the behavior of than a model.

26:06Chris RomeoYeah.

26:07Isaac EvansAnd Anthropic published some great research a few months ago where they looked at, if you were trying to do an attack on the model where a specific environmental key could trigger some other unintended behavior, how many samples would you need in the training set to have that effect?

26:26Robert HurlbutAnd the answer they found is super disturbing. It's not highly dependent on the number of parameters in the model.

26:33Isaac EvansSo, you have a million parameters, you have a billion parameters, it's about 250 samples inside the training set.

26:40Robert HurlbutSo, this makes something like Ken Thompson's Trusting Trust paper, like with a compiler, child's play, right?

26:48Isaac EvansCompared to how are you going to reason about the security of a model that you don't have a full understanding of the training corpus because it's too large to be feasibly understood.

27:00Chris RomeoYeah, well, there'll always be a place for the human in the loop in my book, just from a trust perspective. I can't trust— somebody has to understand. And one of the challenges I see that's on the horizon here is we're going to reach the point where you're not going to have anybody who can explain the system in its entirety. Whereas in the days of old, you know, we had system architects and it seems like such an archaic position for somebody to think, oh, you had a person who just thought about how the system all worked together. But in the age of AI, you could reach the point where nobody understands. Oh, the AI understands it. Okay, now, well then how do you trust it? Well, we just trust the foundation model providers.

27:41Isaac EvansRight.

27:41Chris RomeoSo, well then who's, trust but verify has always been the thing that we think about. So how much do you trust the code coming out of agents right now? Since I went to trust, I'm just going to keep pulling on that thread.

27:55Isaac EvansYeah, that's a really good question. I will say that like, I trust code generated by agents. I feel like it's actually more trustworthy because it's by its nature, you know, can be mechanically explained. It's deterministic, right? You can reason about that structure. That to me is actually more trustworthy than I ask an agent to take an action and the agent is my code, right? So I like the use case of agents generating code. I think it's a great use case, especially when you can get an oracle of like, hey, is work or not, like yes or no. That's a— agents do phenomenal when they're put in that kind of loop.

28:36Robert HurlbutLike you saw this with Mythos producing exploits for Firefox, right?

28:40Isaac EvansLike the exploit is a very verifiable condition, right? Oh, we have control of the instruction pointer, right?

28:45Robert HurlbutModel, don't stop until you have control of the instruction pointer.

28:48Isaac EvansBoom, right? Excellent, excellent use case for models. So I'm pretty optimistic about models as code generation. And I mean, obviously crazy success with Cursor, Claude Code, people are really enjoying that. I think they're going to be better than humans at writing code. And I think in most cases, they already are there. So now maybe you have a follow-on question there about, well, what if there's some malicious behavior or—

29:20Chris RomeoRight, that's where I always go.

29:22Isaac EvansIf they still generate plenty of security vulnerabilities, like we've found, like now I guess the majority of vulnerabilities we've found in the past couple of months have been generated by Claude Code, which is why we've been working on that plugin for Claude Code. But yeah, go ahead.

29:38Chris RomeoNo, I mean, that's what it comes down to though, is, you know, once again, how do you ultimately trust it? If there has to be a verify, for me to trust something, I have to have some proof. And in your Firefox exploit example, the proof, the verification was, okay, it created an exploit, the exploit worked, we were able to get access to the browser through it. But in a scenario where we're using a coding agent to build a new feature, how do you, how do you get to the, how do you say, hey, I trust it? Like, what are, what are the things that go into it? Do you, is there a, some type of a security review that we still need to do that's outside? Do we pit one model against another from competing companies? And I know some people are doing this now. They're saying they're having Claude code generate the, The code, and then they're having Codex or whatever kind of look at the output and do the code review on it.

30:36Robert HurlbutWell, 100%.

30:37Isaac EvansI guess like it actually comes back to maybe that like that shift left, shift right dichotomy that I was talking about earlier where I'm like, okay, you know, shift right, add more code review, more agents, but it's fundamentally kind of like a token slot machine, right? It's like, oh, you might find a really good vulnerability, I don't know, just keep throwing tokens in, right? See what comes out. You're not going to achieve trustworthiness or security by just like throwing tokens in a code review, right? You might find some great stuff, but, um, so that's why I'm much more optimistic on the generation side. And what we've talked about so far is, hey, like if we provide plugins to the models that essentially are like additional security tests, I think you can start to get some really good properties that are like, okay, like, you know, we can trust that The model did not generate code that had any SQL injection possibilities. Okay, great. That's a good security property. Even more fundamentally, I'm really encouraged by the idea of rewriting software in more secure languages by default. And I think that there have been, well, we'll see where all this ends up, right? But if you are able to write in some languages that have stronger security guarantees, so the canonical example would be Rust as opposed to C, you should be able to just eliminate good classes of defects.

32:05Robert HurlbutNow, you'll still write code that has security vulnerabilities, and there's a great example of this.

32:10Isaac EvansThere's a coreutils port to Rust that just had like 15 CVEs published in it, even though it's written in Rust. But over time, I think that We will probably invent some new programming languages that have even better security properties that are like more optimized for the new LLM era. And I think that could be very exciting in terms of how we can actually trust these systems and programs. Yeah.

32:39Chris RomeoAnd the models will build it for us. They'll build the new language. So then we gotta figure out how we look at the language to see—

32:45Robert HurlbutYeah, that's gonna be trippy.

32:47Chris RomeoYeah. It's almost like we're in a loop here. We just keep going around like, where are we gonna find the trust? Maybe it's in the center. Okay. So yeah, just one of the other things I was thinking about when you were talking about SQL injection, it seems like it's relatively easy for a model to understand what SQL injection is and then look for it. Seems like there's going to be a rise of the business logic flaw again, because that's not as cut and dry easy. I mean, we think about SQL injection, We should have eliminated it 10 years ago because it's not hard to find. It's a very specific pattern. You don't customize a lot of SQL injection vulnerabilities. It's the same set of 3 or 4 things you just didn't do, which would've made it go away. But with a business logic flaw, it opens to a much broader ocean of potential because even like a very, it could be a very small business logic flaw that's hard to see and understand. And maybe the models will get to the point where they can sniff that out, but that seems like an area of research that could provide some interesting results.

33:57Isaac EvansYeah, I'm fascinated by that.

34:02Chris RomeoYeah, I mean, business logic flaw is just something that's, you know, I've spent a lot of time focused on threat modeling and my last company that I sold was a threat modeling tool company. And so business logic flaws, it's also caused me to see that when interacting with AI, when we talking about going up the stack again, there are things you can do from a design perspective to once again, scope the AI, give it more context, not at an individual feature level, but potentially at a system level to let it build you something that's more secure. Yeah.

34:39Isaac EvansYeah, I think that the LLMs are just really good at business logic flaws as well, right? From an elimination or identification perspective.

34:50Robert HurlbutNow, can they produce code that is free from business logic flaws? I don't know.

34:56Isaac EvansThey can find them pretty well, but those are 2 separate categories of problems, right? Identifying some vulnerabilities versus generating code that is free from vulnerabilities.

35:06Chris RomeoYeah.

35:08Isaac EvansSo the other capability that I've been most impressed by is LLM's ability to reason across module or repository boundaries really successfully about those business logic flaws, right? And so we've been building a bunch of tools to kind of like cache context across the organization, like manage context so that, hey, like you have 100 different repositories, you need to reason about interactions between them. How can you get that context to the LLM in the right way? And I mean, there's been a bunch of people also talking about like, hey, how do you build some harnesses around the LLMs that steer them correctly? But we've seen, even with customers that have had access to the very large models like Mythos, the difference between a team that kind of just throws the repository in is like, find vulnerabilities, versus a team that is using a tool to steer the context of like, hey, go look at this, now go look at this, and with this context. It's like a 10x difference, not an exaggeration in terms of the number of like CDEs at the like tail end of that process. So I find that like quite fascinating actually. And there's a debate there of like, you know, hey, like, well, is it the model? Is it the harness? Like it's a combination of both for sure.

36:22Chris RomeoSo I gotta explore one other area here because this is actually a question I've been I've been thinking about. Vibe coding, it's still a thing. It's not as popular of a term as it was a year or two ago, but it's still, it's ingrained in organizations now. You have, I've been talking to some other folks in AppSec and you have enterprises that are adding 200 folks from marketing to GitHub, right? And just turning them loose at this point.

36:51Robert HurlbutWe're seeing this with our customers too. They're like, wait, where'd these new developers come from?

36:56Chris RomeoYeah. And so, I guess the question is, where does static analysis and AppSec tooling fit in, in this new world when you have a person who's writing code that doesn't have a software engineering background? And I'm not saying there's anything wrong with that. I think it's fascinating that this is the direction we're going, but you can't— if a model tells them you have a SQL injection, that doesn't mean anything to them. And so how do we build something that wraps them and protects the organization and protects the code that they're writing so that they don't repeat the sins of the past 25 years that we've started to get a tiny handle on?

37:35Isaac EvansRight, right. Well, I think the model companies are certainly thinking about this, right? And like something like SQL injection, probably eventually it's just like the models won't generate code that does that. Today they're actually worse than the average developer. And I mean, that's why we're building plugins for them, right? So that we can add some security tests and guarantees, but we're kind of in uncharted waters, right? For a lot of these new developers. And I can only imagine that this must have been what it felt like if you were an assembly programmer and people started coming along and using these higher-level languages like C. They don't understand what's going on with the registers and Like just no understanding of these underlying capabilities, but the productivity gains are worth it, even though the security and performance and the demand for software in the world is just so high that I think we're just going to see a massive surplus of software production. Well, not necessarily surplus, but just like more and more people are going to be writing software. And again, going back to the security engineering thing, What's probably going to become more important is not like being the world's best software engineer, but understanding the security, the performance, like testing guarantees of these systems, right? One of the things I've noticed for myself with LLMs is they do great when they have a really good test suite, right? It's actually much easier to reason about what they're doing, but engineering that test suite, even if you're using an LLM to do it, requires some expertise that your vibe coder probably doesn't have, right? So I think there's, it's almost like all of the ancillary functions to like writing the code end up becoming like way more important and way more valuable to have like professionals in, even though like the average coder may be, you know, vibe coding, never looks at the code sort of experience.

39:37Chris RomeoYeah. Yeah.

39:40Robert HurlbutAnd potentially, like, what your take is on that same question, you know?

39:43Chris RomeoYeah, I think— Both of us have the curse of knowledge.

39:46Robert HurlbutWe can't exactly forget how to program.

39:48Chris RomeoYeah, it's true. No, but I think I see a world where agents will be used to create guardrails for those environments. And yes, I keep talking about trust. We may be trusting those guardrails based on the model's capabilities, but it's going to be better than just letting people vibe code things and release them into production in one step without anything else. So, I see a world, at least in the short term, where you almost have 2 classes of code that's being produced.

40:25Isaac EvansMm-hmm.

40:26Chris RomeoThere is the traditional engineering route where you have people that understand the underlying pieces of an app, for example, that's going to be pushed out to production. You have, then you have things that were vibe coded. I could see 2 different profiles for how we use agents to ensure that that vibe-coded stuff is meeting some level of goodness. And like your SCA example, pretty easy for the model to have an SCA agent that just sits there and says, I'm looking at this vibe-coded thing. I'm making sure there's nothing that they brought into this thing because maybe the model said, or maybe they got prompted in the midst of vibe coding. Like I have, you know, Do you mind if I just put in this other package that I want? And maybe they're just gonna automatically hit the yes to all button. Just do it, do your thing. I need an app built, right? So I think there's almost, there's 2 profiles. I see a future where there's 2 kind of profiles of those that are building software. I think they're gonna converge over time. Once again, I'm not predicting when, but I think there's gonna be a convergence of this because people that are It's going to take us a while to build a more engineering core in all functions, but it just seems like that's where everything's funneling towards. It's hard to imagine a world where you're in a marketing program 2 years from now, studying for a marketing degree in university, and you're not getting a deeper technical foundation because unless the higher ed just doesn't read where we're going in the future. So that's kind of my thoughts. I mean, what do you see a world like that or do you think I'm just out there?

42:05Isaac EvansYeah, no, I think that actually already seen some of that play out in the open source community. I don't know if you know Mitchell Hashimoto, one of the HashiCorp co-founders, right? Started this project called Vouch, which is all about kind of, hey, like, here's my quality bar for engineering and code, and I only want to work with people in the open source community who have a similar, you know, bar and are vouched for by somebody in my network, right? Oh.

42:33Robert HurlbutAnd I've also seen this where people are posting their vibe-coded projects and then the comments they're getting are like, hey, this is kind of garbage. I prefer to, if I was going to use this, I would just vibe-code my own garbage, but then it's my AI slop rather than your AI slop. So I don't know.

42:56Isaac EvansAnd I've poked around with some of these products actually in software-defined radio where it's very clear that the person who used it really had no idea what they were doing. Uh, half of the features don't work.

43:07Robert HurlbutStill fun because half of them do and there's more than there would've been otherwise.

43:11Chris RomeoYeah. And it's, it's somebody's kind of passion project, right? You know, maybe they have a core expertise in something else, but they're— and that's another, another avenue for vibe coding, right? Is, is there are, there is a whole group of startup founders out there that have an idea, a problem, But just didn't know how to, didn't know what to do with it. So it's going to be where there's already companies that are starting.

43:33Isaac EvansWhen I was at MIT, there was this recurring trope that literally happened where, you know, someone from Harvard would email an MIT mailing list and they would be like, you know, I have an amazing idea.

43:45Robert HurlbutI'm going to build the next Facebook. Just need a technical co-founder. But now I think a lot of people are like, oh, Claude is my technical co-founder.

43:54Chris RomeoI mean, I think there is a case study that's being written right now. It may not be being written actually, but the story's being— they're in the midst of that story where people are able to do that. And I think that's the future.

44:08Isaac EvansThat's where we're going.

44:08Chris RomeoYeah.

44:09Isaac EvansNo, it's exciting, right?

44:10Chris RomeoIt is exciting. It's, I mean, may you live in interesting times, right? Is a curse because that's where we sit right now. But I mean, this is Things have changed more than I've ever seen in my 30-plus-year career of working in tech. Like, it is more different now than it ever has been. And can we stay on this exponential rocket ship, or does it event— do we eventually hit a point where things start to standardize and play out? I think time will tell.

44:39Isaac EvansYeah, totally.

44:41Chris RomeoSo what are some key takeaways then, or calls to action? that you can leave with our audience here today?

44:48Isaac EvansYou know, I think the biggest thing is look at the models as an opportunity, right? For us in security. Yes, they're going to destroy some product categories. They're also going to create a bunch of new security issues. But the most exciting thing is, hey, you're going to be able to influence the models in a way that you could never influence developers. And so if you can steer the models and think about how to get leverage of them, both as a kind of like fleet of virtual security agents that you're going to manage, but also as this like factory that's going to build the software inside your organization without much human involvement. And like you can be involved in like setting up that factory to be really aligned with what your business needs from a security risk reduction standpoint. Those are huge opportunities, and we've just never had that before, right? Like, you know, your previous alternative was like training all the developers in your organization to think this way, and that was just never going to work as well as it's going to with a model-like system. So, I think that's my biggest, you know, there's some great reasons to be optimistic here. We, as security people, are not generally optimistic and hopeful. But I've got to acknowledge that those opportunities are there, even though there's also going to be a tremendous amount of mess that's created from LLMs finding zero days all over the place. So much patching is going to happen this year. So many more vibecoded apps are going to be rolled out by people with very poor security posture and properties, and security people are going to be left to clean up the mess. That will all happen. Hopefully we can get through that to a world where some of the really exciting things like programming languages that just have even fewer possibilities for these security problems, systematizing the security knowledge that we have. One of the things that I thought was really cool was one of the flagship bugs from Mythos in FreeBSD or OpenBSD, it turns out was an exact copy of a patch that was on Kerberos 7 years ago.

46:56Robert HurlbutSo it was like, looks super impressive, but actually ended up being in the training set. And to me, that's actually still a huge win because a lot of security people are fixing the same kinds of issues over and over again, or it's already been fixed at some other company.

47:11Isaac EvansHow do we just get leverage over those problems rather than one after the other? So I'm excited about that.

47:18Chris RomeoNo, I think that's a message of opportunity. is a really great key takeaway. Like you said, those of us that have been in security for a while, and even new security people, we tend to be more pessimistic by nature.

47:32Isaac EvansYeah.

47:33Chris RomeoBut there is opportunity, and I think those people that embrace this opportunity are going to be the ones that rise to the top. If you try to hold on to the things of the past, just like in anything, it's not cybersecurity specific, it's just general. Like the the people that were like, no, these horse and buggies are going to be here forever. This Model T is not going anywhere. This automobile, right? There were, I'm sure there were a lot of people that had that type of a reaction to it and they got left behind in that process. And so I think as security folks, we have a chance right now to, we have an opportunity, like you said, to get out front and lead from the front and help tie these things all together. And so that's—

48:12Robert HurlbutI love what you're saying there.

48:14Isaac EvansAnd I think like, The teams that I'm really enjoying talking with on our customer base, right? They're not the ones who are trying to like hold AI back.

48:21Robert HurlbutThey're the ones who are trying to be like, oh, this is an opportunity for us.

48:25Isaac EvansLike, we want to help you build that factory so that we can get in there with like, these are the things that need to be set from the get-go when it comes to security. And that, those, those people are really thriving. So be one of those people, you know, not the person who's watching the Model T stuck in the mud and— Get a horse. Definitely.

48:44Chris RomeoWell, we'll have to, we'll have to talk again in a year and kind of revisit this because this, this is a lot of things are going to change in the next year, but it'll be interesting to go back and see how some of these things played out. So Isaac, thanks for taking the time to share your wisdom and knowledge and whatnot with our audience. This has been a very powerful episode for me. I've learned a lot in this process.

49:04Isaac EvansYeah, pleasure, Chris.

49:06Robert HurlbutReally glad to do it.