The Future of Open-Source Threat Modeling

0:00Vikramaditya NarayanWhen you suggest the system to an AI and says, threat model the system, you throw in a data flow diagram, a sequence diagram, you actually get a very plausible sounding threat model. One could try it right now with ChatGPT and it's going to give you a list of threats. But when I try this with humans and propose this system, they don't even get as far as a threat model. There's so much of discomfort in their faces as they try to digest the idea because the idea feels wrong.

0:26Chris RomeoThis episode's sponsored by Corgea. Design it, build it, ship it. Corgea secures it. Corgea is an AI-native app security platform covering your whole software lifecycle, from the first architecture diagram to code in production. It brings design reviews, AI-powered code scanning, and autonomous pen testing into one platform, so your team spends less time chasing noise and more time fixing what matters. Visit corgea.com. C-O-R-G-E-A.com. Corgea, one security platform for the entire SDLC. Vikram is an application security leader focused on how AI is changing threat modeling, trust, and decision-making. And today we're getting into where AI helps, where it falls short, and why human judgment still matters most. Welcome to the Application Security Podcast. This is Chris Romeo. I am one of the hosts and as always joined by my good friend, Robert Hurlbut, who, if you're watching on YouTube, has a black background, which he says he learned from our good friend, Brooke Schonfeld. And I'm in the Brooke Schonfeld fan club. So I wasn't probably the original member, but I joined pretty early. So if Brooke's Brooke has a Black background. Robert wanted to be like Brooke. There could be a whole advertising campaign. Be like Brooke. If I could be like Brooke.

1:57Robert HurlbutThere you go. Right. Robert Hurlbut, threat modeling architect. And yeah, you know, just trying different things out. But hey, good to be here.

2:05Chris RomeoAll right. Well, we're joined by Vikram. And Vikram, we like to just jump right into people's security origin story, giving them virtually no time to warm up. is our goal.

2:17Vikramaditya NarayanSure. So first of all, thank you so much for having me here, Chris and Robert. And I've learned a lot from your talks as well as Robert's thoughts on threat modeling. And my own story is that I don't come from a security background like most of your other guests. I have more of a developer, AI engineer kind of a background. And years ago, when social media was a thing and self-publishing was becoming a thing, I launched a book marketing site and it grew pretty well at that time. We scaled it up to about 15,000 authors. And then we came up with this bright idea of, hey, let's give away prizes for people to play games. And in those games, they're going to get exposed to the books of our participating authors. So it's like advertise your book inside the game, games like where you match the covers of the book and play hangman and win actual real prizes. And on paper, the math made perfect sense, right? So because you're collecting all this money from these authors, you're spending a little bit of money on prizes, and then you got people playing games. And what could go wrong? Turns out almost everything. The site was overrun by people who weren't actually into reading books, wouldn't be buyers of these books. They weren't even in the target geographies. We had countries from non-English speaking countries constantly playing these games and trying to win these prizes. And so then, you know, you start off saying, okay, where's our traffic coming from? Great, we're growing, but actually the traffic's coming from some other country. And so now we need to go back and explain to authors, and then you start shutting off IP addresses at 12 in the night, right? Make sure that your deadline for the prize game is not overrun and the iPad does not get won by somebody else from a non-target country. And so trying to make up rules along the flight. And that was really my first exposure to— I didn't have a word for it then, but that was threat modeling. I wish we had done threat modeling at that point and asked one of the basic questions of, hey, what are we building and what can go wrong? And we never asked that question. We just were so happy. We just thought through the happy path, but not through the abuse cases. Anyhow, but the site did do well because we had a lot of other engines out there to help authors. But then I was feeling this sense of dissatisfaction and I felt like I needed to learn more. And I had an ex-employee who said, you know what, I'd love to take over the site and run it. And so he jumped in and I was able to exit that business. So I had some time and money and then I said, I want to learn everything about machine learning and AI at the time. So I started getting into it quite deeply, all the way from from the ground up, from multivariable calculus and linear algebra to building deep learning models and different kinds of machine learning models. I became a contributor to an open-source machine learning library in the Julia programming language called SciML and Surrogates. And I was quite happy in my world doing these contributions, being a consultant, all kinds of little things. But then ChatGPT happened. And at that moment, It felt like something really revolutionary happened. And so I again got back into the game, created a course on large language models, started going out there and training corporates on this new emerging technology called large language models. During the course of that work, I got pulled into another startup which was building RAG-based chatbots for surgeons in New York City, for a hospital in New York City. So we went out and built these chatbots, RAG and all of those things. But we weren't again able to go past the POC stage because the surgeons loved it, but the hospital's administration and the governance committees, they had serious concerns.

6:09Chris RomeoSo the fact that— I have this vision of a surgeon in the operating room and they're like, hold on a second.

6:15Vikramaditya NarayanYeah.

6:17Chris RomeoYeah, okay, let's go this way. Like, now I'm slightly afraid.

6:21Vikramaditya NarayanYeah, actually it was more for patient assistance. So imagine someone undergoing a knee replacement surgery and they need to ask questions like, how long will it take to heal? Can I take this particular medicine or not? What's the best way to recover? When do I really actually ask a doctor and escalate the question or not? Those kind of things. Like the first line of answering questions was what these chatbots were for. But the hospital, the surgeons loved it because that would take away save them actual time. But the hospitals were concerned with that 1% of hallucination that might happen. And so we actually turned around and said, we're going to build out this AI governance tool for hospitals, which did well. But I had to step away from that business for personal reasons. My father passed away, and then I had to take over the family responsibilities and handle a lot of family issues. And so I had this long break. Away from doing anything again. And at that point, I had already been exposed to the risks of agentic systems, and the emergent risks of multi-agent systems began to interest me because as you have agents interacting with each other, all kinds of new kinds of problems start developing. So think of a traffic jam, right? It's just a bunch of cars, but all of a sudden you've got a massive traffic jam that explodes, and sometimes you don't have a traffic jam happening. And so I applied these similar concepts to the risks of multi-agent systems, wrote out a paper, and Robert was also involved in that in terms of mentoring me. And so I got an opportunity to go to ThreatModCon in DC. And over there, we presented the paper and also got exposed to a lot of the tooling out there in the threat modeling space. And coming from a developer's background, and it seemed like I could build this and actually create a tool that would be open source. And if we could bring down the barrier to entry for threat modeling by open sourcing a tool that was on par with the major commercial vendors out there, then you'd have a lot of organizations doing threat modeling. And so it became a personal passion, and I got joined on my journey by another person who's the head of threat modeling at a major European bank. And so he had all of this domain knowledge. And then I spoke to maybe about at least 100 AppSec leads and integrated their learnings into the tooling. And it's still a work in progress, but right now it's got adopted by OWASP and we're just continuing to build out. It's, we're about to release 0.3 in a couple of days. And so that's where we are. And so, yeah, that's, that's a long answer to my security origin journey.

9:01Chris RomeoYeah, no, it's good. It's good to—

9:03Vikramaditya NarayanIt's like, it's like a fire hose right now. I'm just trying to learn as much as I can from all the folks out there. And I must say that the community is incredibly supportive. So be it on Slack, we've done LinkedIn, there's a lot of support and you have a question, you get multiple perspectives. So yeah.

9:19Chris RomeoAnd we were talking, Vikram, before we started about Bangalore, the fact that you live in Bangalore. And so I got to tell you this story real quick. So when I was at Cisco, I helped to run some conference series and we had a Bangalore edition of the conference series. And it was in the time when IoT was really big. Cisco was big into IoT for everything, you know, connecting devices for everything. And the chief security officer at the time is a guy named John Stewart, and he's doing a keynote for us remotely from San Jose. And he uses this big example at the beginning of his talk, and he's like, you know, like, imagine if all the traffic lights in Bangalore went out. And I had been to Bangalore a couple of times. I'm like, and so like everybody in the room is just kind of looking around at each other like, wait, what? Because in my time in Bangalore, I think I saw 1 or 2 traffic lights in the entire city. And so he uses this example, like, what if all the traffic lights went out? And people are like, yes, so what? There's like 2 of them. It wouldn't do anything. So that was my fun.

10:21Vikramaditya NarayanIt is chaotic, but somehow things move and things happen. And it's such an interesting way when you look at it, it's Like self-organizing systems. Like when I see a traffic jam in Bangalore, it's all snarled up and yet someone will just get down from his vehicle and start voluntarily just waving the traffic around and moving them around and things just resolve themselves in like 5, 10 minutes. And yeah, it's pretty interesting to see the way things— that humans can actually interact with each other even if there is a breakdown of this organizational infrastructure.

10:57Chris RomeoThere's an AI illustration in there somewhere. I think the guy getting out of the car is the agent.

11:01Robert HurlbutYeah.

11:02Chris RomeoAnd there's— I can't put it all together right now, but—

11:06Robert HurlbutBut the orchestrator or something like that. I love it. Hey, Vikram, you wrote an article, a post on LinkedIn quite recently, and this was based on a talk that you gave in Vienna at OWASP, Vienna 2026, and it was related to AI, where AI is and the Threat Modeling Manifesto. It was really, really interesting talk and certainly an interesting article. But one of the things you mentioned was you quoted a practitioner who said an LLM threat model isn't worth more than a compliance checkbox. Does that match what you're seeing in the field, or do you think that's a bit harsh?

11:46Vikramaditya NarayanSo when you have a time pressure and you have a tool that allows you to overcome that time pressure, and you have a law that says you need to do something, and you again have a tool that says— that allows you to comply with that law, I think most humans would take that tool. So you have so many of these emerging laws and standards that insist on some kind of risk assessment. And then you've got AI, that's AI-written code, which is moving really fast. There really is no time to sit down and do a threat model. So the natural reaction is to actually use AI. So yes, and there is a personal story here. Now, I had another business which I since exited, but this was a couple of, maybe a year and a half ago or a little more than that. But we had to get a contract from a bank and the bank required us to do threat modeling and it's a bunch of developers in a room and We did a threat model, but now that I know everything that I know about threat modeling, I would say that it was not the best of threat models out there. And the moment you have a pressure that says, hey, this contract you get if you get this certification or compliant, send in this document, and then you've got this date and you're not getting paid for it either. I think that human behavior tends to gravitate towards the shortcut. It's just human nature. Yes. So I think that the person who made that code, he's also an extremely experienced threat modeler, security person. And I think he was spot on target that when you just say, AI, just do the threat model for us. And it generates something that looks really plausible, well thought out, well mapped out. And you can't at first pass find anything wrong with it. You tend to accept it, submit it and move on.

13:35Chris RomeoYeah. And I think, I think, I think this is a lot. The same for a lot of things that AI is touching. Like, for example, I could go to Claude or ChatGPT right now and say, I need to perform heart surgery. Give me a plan to, for, to be successful with the, an open heart surgery that I need to do right now. And I bet you it'll give me the steps that I, that I could do if, but nobody's going to let me operate. Robert, you're not going to let me operate on you, right? Yeah. He's like, no way. No way. Not going to happen. Right. So it's, but it's the same, it's the same situation. It's just the stakes are higher with heart surgery. And so yes, somebody will say, oh, I'll just have it do the threat model because it's easier. And I guess it's not easier to have me do the heart surgery based on what I could learn from an AI bot. But it's really the same challenge, same underlying challenge that I see is that it's, it's when there's human life in the picture and, and part of the potential danger of, of harm or something. then people are like, oh no, I would never do that, but I'll do it for this where it makes something easier. And so it's really about the kind of the attitude towards the outcome. I don't know. I mean, are people, do people just not understand? Are they just not valuing the outcome, Vikram, of the threat modeling process? Is that the challenge?

14:52Vikramaditya NarayanSo I think it's also an issue of education and communication. When you have most, a lot of tooling out there promises threat modeling without using you know, without any humans in the loop. So, I mean, there is an open source one from AWS itself, which says, you know, you have a— just feed it the diagram or not even the— yeah, you feed it the diagram. It's going to decompose the diagram into the constituent components. And then for every component, it's going to keep running it through a loop until you're satisfied that enough number of threats have been generated. And these are thought leaders whom everybody else is following. And when you see an open source project doing that, it's compelling, or you start questioning yourself whether you did it right or wrong. And the second thing that happens is there's also this desire to avoid strain. I think humans just are wired to avoid anything that feels like heavy mental burden. We want to spare glucose for our brains. And the moment you're asked to, if there's a shortcut to avoid that kind of strain, you would take it. And another aspect is when there is— the consequences are not known right now. So what happens if you have bad code? You see the consequences almost immediately if there's a bug in your code. But if there is a bad threat model, the consequences don't emerge for a while until the tool is built and until the application is built and until it goes live and then something goes boom. So it's like the perfect mix to actually use AI and move on.

16:27Chris RomeoSo, so in your talk, you gave what you described as a thought experiment with this Wi-Fi sense, but I'm more interested in kind of the conclusion you drew about the human reaction versus the AI's output. I think it's going to help to inform us about why you kind of lean towards human in the loop being important. And I agree with you, by the way, just, just for the record. I think human in the loop is the way we have to do threat modeling. But what, what in that thought experiment helps us to see The difference between what the AI gives us and what the human gives us.

16:59Vikramaditya NarayanSure. So imagine that in your hotel room, there is a new kind of technology, which, by the way, it is a real thing. There is a new standard, 802.11b, or one of those new standards IEEE approved, which says that you can actually track human pose and do pose detection based on the way the Wi-Fi signals are bouncing off the body. So now imagine you have such a technology which tracks your body movements in your hotel room and tells you on your screen, on your TV screen, with offers, which gives you offers on your TV screen. And so you might say, okay, I see Chris looking tired today. Maybe I should offer him some kind of maybe drinks, or he looks stressed. And so when you suggest the system to an AI and says, threat model the system. And of course you throw in some technical details, throw in a data flow diagram in there, a sequence diagram. You actually get a very plausible sounding threat model. I know one could try it right now with ChatGPT and it's going to give you a list of threats, one of which will say privacy violation. But when I try this with humans and propose this system, they don't even get as far as a threat model. It's generally like— There's so much of discomfort in their faces as they try to digest the idea because the idea feels wrong. And humans live outside the threat model. So what I get back— humans live outside the context window. So what I get back from them is, why would you even build this? It's just such a crazy idea. But ChatGPT never says, why would you even build this? And so that reaction is one aspect. And if you really think about it, the way these models have been trained is on text data, right? So there's mountains of text that they fed into it, but all of the text data is secondhand information. The threat model, the LLM never experienced a situation where at 2:00 AM your book marketing site was getting overrun by bot traffic.

19:00Chris RomeoIt'll tell you it did. It'll be like, yeah, I experienced that. Sure.

19:04Vikramaditya NarayanYeah. But that feeling, that cold sweat of losing money or that feeling of which we so badly want to avoid. Humans want to avoid pain at all costs, and we tend to be far better threat modelers simply because that's built into us. It's innate. And we feel it in our gut, we feel it in our heart, it's all over. Whereas it's just reduced. There's this dimensionality reduction that happens with LLMs which says everything is just text. And I can pretend to be like a human, but it's, you can sense it. You can, there's a tell.

19:40Chris RomeoIt's almost like you're describing a binary reaction that human beings have to this particular example. Like it's either a 0 or a 1 for us as humans. And we may not even be able to explain why it's a 0. Like nobody should do this. You should never build this system. Do you have a name for what you're calling this? Have you, have you come up with a term to describe this human phenomena? You call it the human phenomena.

20:06Robert HurlbutI don't know. Yeah.

20:07Vikramaditya NarayanYeah. gut reaction. I mean, we tend to just respond with our gut. And like you said, the LLMs are going to say something that sounds like a gut reaction. But yeah, you know that it's not so, and you feel it within its writing and the way it responds to things. And yeah.

20:26Robert HurlbutIn terms of the tool that you put together, Precogly, when you were putting that together and thinking about your own product design, how did you think about that line between AI as an accelerator and AI becoming the new hero threat modeler, which is something we also talk about in the Threat Modeling Manifesto, and a bit about what you were talking about in your talk as well, that does that still matter? What's that line? And curious, how did you work through that in the tool?

20:57Vikramaditya NarayanSo one of the key ideas behind Procogly is this notion of library packs, and these are threat libraries. So you could use it, use AI, at design time. So even before you start threat modeling in an organization, you know what kind of technologies they use, what kind of components they use, and then you use the AI to generate these library packs of components, threats, countermeasures, taxonomies, and standards that you're trying to comply with. And you put it all together in a library pack and get humans to curate it. And in our opinion, that is a good use of AI because it's doing the grunt work, going out to MITRE ATLAS and CAPEC and all of the PCI DSS and trying to put it all together saying, okay, here are the components from AWS and these are the reference threats that are out there. And here's ASVS and I connect all of this. That's a good use of the AI's time or AI resources. On the other hand, and then when humans do the threat modeling and you say you drop a component and you show up all of those Related threats, countermeasures, that's okay because you're just still doing freeing up human cognition to work on design-level threats. So rather than answer the question, what kind of things can go wrong with this S3 bucket? You're delegating that, pre-delegating that to AI because that generated all of those threats. And then now you've got time to discuss what else could go wrong with the way we are designing our system. And so we are freeing up time over there. The anti-pattern that we want to avoid is saying, hey, here's our system, can you generate a list of threats? And so now you're exposed to situations where it might have hallucinated or it might have missed some crucial component or threat. And even in the way we're trying to answer the 4th question, so the 4 questions, what are we working on? What can go wrong? What can we do about it? And did we do a good job? The way we're trying to answer that question, we're trying to track if in the threat model, which of the components came from outside of the library packs. So if it came from outside the library pack, of course, if it's humans, we are okay with it. But if an AI generated a threat model or assisted in the threat modeling activity, and you have a number of components that were not there in your library pack, then it requires a second look by humans. So that's another way. But on the other hand, we are also pretty agnostic. We're creating this skills layer where like a CloudMD file, you have a bunch of skills that humans can author. And this is where we're heading towards. And so one might say, I want a skill for generating threats for components. Another might say, I'd like another skill for going out and collecting information from the various cloud service providers and then putting together a first cut at a DFD, at a data flow diagram. So we are agnostic on that and we are just creating the foundation for the skills layer. So it is still possible to go down an anti-pattern that's not recommended by the Threat Modeling Manifesto that lands up shortcutting the process and kills the journey of understanding for humans. It's still possible to do that, but the way we've set it up is at least the human must be able to see what the AI did in such a situation. and why it did so. That's the worst-case scenario. And in a few years, let's say AI is writing all the code, and then AI is doing all the threat modeling, then what? At least the humans— the risks are still owned by the humans. And so there must be a way for humans to be involved in that final risk assessment and say, OK, this is a risk that we mitigate, or it's something that we transfer out. So yeah.

24:35Chris RomeoSo Vikram, if I go into Precogly today, create a data flow diagram, Where does AI kick in? Like, what makes Precogly an AI, you know, a tool that uses AI? Like, what are the features that are going to use AI?

24:52Vikramaditya NarayanYeah. So right now, Precogly is the foundational layer. It is still very much a CRUD system. It's, you know, create, read, update, delete. And the layer on top, which is what we're building out, right now is the one that will allow for AI to exist at various points. So we call it AI affordances. So you might drop an AI affordance in the threat analysis workspace where you actually do the threat analysis. So you've got the components and you say, okay, what else can go wrong with this particular component that's outside of what the library packs have suggested? You might decide to drop an AI affordance or an AI feature in the DFD and say, okay, help me generate a DFD from our C4 diagram. So that's where the skills layer comes in. And so we are creating the foundation for something like this to happen. So based on the organization that wants to do a particular skill, they could do that. And of course, if this works out, there's gonna be a skills marketplace where someone might create a skill for generating threats in some way. Or maybe there might be a PASTA wizard that says, this is a PASTA wizard and we're gonna go through the 7 stages of the PASTA framework and help you generate a PASSTA-compliant threat model. Or it might be for a particular industry like healthcare. Maybe someone's trying to comply with FDA guidelines, and so it'll walk you through that and be a wizard for that. So the skills layer is what's going to facilitate all of that.

26:24Chris RomeoSo would you say that with Precogly, the human is still at the center, or do I have the ability to say, Precogly, do threat modeling for me? And then I just take the output.

26:35Vikramaditya NarayanOne could very well use it in the way that you described, where you say, hey, Precogly, do the threat model for me. But the hope is that it forms a pane of glass where a human as well as an agent can actually see what was made and can actually decide on it. So the DFD becomes even more important, like the sequence diagrams, because humans reason about things visually, whereas The LLMs rely on text, and so the artifacts become more important for this collaboration to happen. But yes, one could potentially misuse the tool and say, hey, Pricogly, do threat modeling for me. But we are trying to set it up such that even if it does it, there's some sense of accountability of the LLM. So you know what was the LLM's output, what came from the library packs. And the bet that we are making is that people will use LLMs to generate the library packs and be less reliant on the LLMs downstream if you have high-quality library packs.

27:37Robert HurlbutGotcha.

27:37Chris RomeoOkay. Well, I'm going to jump to one of the most fun questions in here, but it could also be the most controversial. And so you, towards the end of the article, you mentioned 2 camps in regards to amending the Threat Modeling Manifesto. Now let the record show on the interview here today, there are 2 authors of the Threat Modeling Manifesto amongst a whole bunch of other really awesome people, Brooke and the list, there's 15 of us total. But you're basically laying out an argument for should the Threat Modeling Manifesto be amended to include something about AI or should it be left as is? People over tools being the concept that that defines AI because we were so future-looking when we wrote it. AI wasn't even a thing really when we wrote this.

28:26Vikramaditya NarayanIt's amazing that you folks thought about that.

28:28Chris RomeoI give all the credit to Brooke with his future-looking way of doing it. But yeah, I mean, so what do you think? Where are you landing on this? Do you think that we should amend the Threat Modeling Manifesto to say something about AI or should we leave it as is?

28:44Vikramaditya NarayanYeah. There's 2 ways to look at it, right? So is AI just another tool? And if you look at it, yes, it is. So how is AI different from Microsoft Word? And why are AI toolmakers trying to anthropomorphize this technology and give it names like Claude or something else, Gemini? They're trying to make it feel like it's human to us. But when you strip it out at its very core, it's just autocomplete. And so the argument is, should we be treating it as something more than a tool? Should we? Because you also have things like co-written by Claude, right? So when you make a commit these days and Claude does it for you, it says co-authored by Claude. And so it's almost like they're trying to make us believe that this is humans. So the argument for the first part is should be a tool. And I personally think that it is very much a tool. and it should not be amended. But that's today. But what happens in 3 years' time? What happens when AI is actually getting even better? What if they figure out a breakthrough beyond the current transformer technology into something else where there's actual human-level discernment and judgment happening? At that point, does it need to get amended? Should the Threat Modeling Manifesto be that forward-looking? But that future is coming really fast. 3 years ago, we didn't even have something that, you know, right now it doesn't faze us when we have machines talk back to us, but 3 years ago, that would have been magic. So I think it's a moving target, but for now, I personally think that AI should be treated as just another tool, you know, with smart autocomplete and nothing changes really.

30:28Chris RomeoYeah. Robert, where do you land on this? Do you want to get the band back together?

30:32Robert HurlbutYeah, same. No, I think it's a tool. It is still, it's math, it's software, it's a tool. And so I agree. Now, future, I don't think so either, actually. I think it's going to continue to be that even as it gets better, because it's still based on not current, it's based on past. The data, datasets, it's always going to be not current, it's going to be past. And, you know, as humans, we're, we're making decisions based on, of course, our own history and thoughts and experiences, but also what's around us, what's our current context, what's happening now. And we could, we can, you know, make those decisions. So again, I don't think there's a comparison. I think it's still tools. People are definitely over tools. So I'm in agreement not to change it.

31:18Chris RomeoYeah. Until like to Vikram's earlier point about the gut reaction to something, until you can, you can program gut reaction. You can't, you can't really, because like that, like, like Vikram was sharing with the Wi-Fi Sense example, the AI is just going to start turning on a threat model going, this Wi-Fi Sense thing where we violate people's privacy at the utmost level is awesome. It doesn't make any judgment on it. And that's going to be a lot harder to code that level of judgment. It's one thing to regurgitate things based on what we saw in the past. But to be able to teach morality to an AI, and I know Anthropic, for example, has these AI philosophers, I think is what they call them. I might be, I might be missing the name. I should ask Claude to see what they're actually called, but they have people that they pay a lot of money to that are thinking about these problems. Like, how do you, how do you get an AI to have some basic level of morality or good and bad, you know, correct and bad, like whether it should do something based on making that judgment. And that's a whole other level of complexity beyond just regurgitating the next word based on statistical analysis or whatever, whatever's happening there. So yeah, I'm with you guys as well. Like, I think this is a, this is a tool for now until somebody proves it differently. I'm still going to value the human in the loop no matter what happens, because I don't trust— I've asked, we've been, we've been doing this series on AI and how it's impacting AppSec. And the concept of trust keeps coming up and there's just not a good answer. How do you trust it? No. Can you explain how it works? Not really. There's only a few people on planet Earth right now that can explain how AI works behind the scenes at the level of being able to predict perhaps what it's going to say. And even then it'll deterministically go some other direction and come back with something that even they didn't think it was going to do. Right. And so, When you can't understand something, how can you trust it is what I keep coming back to. And the same thing with threat. I mean, threat modeling output is so valuable and crucial to the development of a feature, a subsystem, an entire system. Being able to have that kind of level of insight as to what can go wrong and what we're going to do about it, something we got to trust. I got to be able to trust, like I got, I'd rather have a Robert-generated threat model all day long over something that you pump through, through AI. And who knows, maybe Robert uses AI to do some research to get to, to, but he's still at the center of that thing and he's still gonna come up with some things that the AI could never possibly think of because they're just, it's never been done before. That's the beautiful thing about threat modeling is you can come up with the wackiest thing you want. And when I teach threat modeling kind of workshops and stuff, I always, I always tell people, you can't say that's not possible. Those words are not allowed in this room. You cannot say that's not possible because with enough money, enough time, and enough human resources, anything is possible. Could it take a million years? Yes, we don't care. Maybe we're— that might play into whether we're going to mitigate something if it would take a million years for that to come to fruition, right? But the idea of that's not possible is just not a thing that's allowed in that. And AI just doesn't have the ability to to wrap its collective head around concepts like that.

34:42Vikramaditya NarayanCorrect. Yeah, totally agree.

34:43Robert HurlbutRight. So one last question here. If an AppSec lead is under real pressure to hit a compliance deadline and AI feels like the fastest path forward, what would you want them to keep in mind before leaning on AI too heavily?

34:59Vikramaditya NarayanSo the big idea is, is the AI-generated threat model like the ASVS, like a checklist of items based on the components that you're using, or is the threat model something deeper where you're asking other questions like design-level issues and getting them to work on the design-level issues first and then simultaneously pull out the ASVS-style checklist items? So it's like a 2-paddle kind of thing where the humans are actually doing what they're really good at. which is taking a step back and trying to look at the system as a whole and looking at interactions between multiple systems and seeing what kind of failure points are there. But delegating all of the rest of the grunt work to AI seems like a plausible solution to a team that's having a compliance deadline. Yeah, that's the way I look at it.

35:54Chris RomeoOkay. So Vikram, as we come to the end of our conversation here, What would be a key takeaway that you want to leave with our audience?

36:02Vikramaditya NarayanI think, you know, this is from my background as a developer right now trying to build a tool. You know, 2 or 3 years ago, I thought I was good at some things like Python and JavaScript and React. And now just a year or so of AI coding, I'm already atrophied in those areas. So if you ask me to write raw JavaScript right now, I'd probably struggle. And we need to fight the AI. Don't accept it, but learn to fight the AI. And it's super easy to just hit enter. And the way the tooling is getting set up these days is it says, would you like me to do this? It tries to predict what you did, what you might want to do next, and all you have to do is hit enter. And it's crucial for us, and I think it's almost a generational challenge that we have to actually begin to fight the AI, ask questions. Why should I not do it this way? or ask it to coach you and say, coach me through this and teach me. And it might take a little longer, but in the long run, if you're trying to optimize for global velocity as opposed to local velocity, I think it's going to matter. So fight the AI is something that I tell myself all the time. And yeah, that's—

37:13Chris RomeoThat's a t-shirt. That's worthy of a t-shirt. Fight the AI. And it's something you said there about just Pressing enter, pressing enter, pressing enter. I've noticed that in the current, some of the, in the current generations that are coming up right now, we've lost the ability, like critical thinking is no longer something that people have the ability to do. And it's just fascinating to me that we have, that we've reached this point and we've dropped this idea of being able to solve problems. And some of it I think is just atrophy from letting AI You know, the reason you're saying fight the AI, right? It's because people are reliant on it for everything. They'll say, you know, uh, Claude, what's 2 2? Come on, you know 2 2, but why are you asking AI what 2 2 is? You need confirmation?

38:01Vikramaditya NarayanMm-hmm.

38:02Chris RomeoBut like, this whole idea of being able to solve problems and critical thinking is something I'm telling the next generation, anybody that I talk to, like, all you gotta do is be able to solve a problem without an AI at this point. Like, that's going to be a sought-after skill set that just, it's going to go away.

38:22Vikramaditya NarayanYeah. And years ago, I tried to learn programming and struggled with it for a while until I ran into this book called Learn Python the Hard Way. And basically the whole book was set up to make you struggle, just struggle, struggle, struggle. Everything was about struggle. He would never give you the answers. You had to struggle until you found it, and then he'd give you the answer. And that really built skills. And I think so becoming comfortable with struggle is something that we need to keep sharp. I think that itself is a kind of a muscle that says, hey, I want to struggle and I'm going to enjoy that struggle. And it's getting really hard to do with AI. So struggling is becoming a struggle.

39:00Chris RomeoWell, Vikram, this has been great, this conversation. I've learned a lot. You've helped to catch me up on some of the things that are happening in the cutting edge of AI threat modeling right now. And, you know, we spent a lot more time talking about, not about the technology, but about some of the issues that are swirling around the technology, which I think is like a lot of people are thinking about these things. So Vikram, thank you for being on the show, for sharing your experience. And we look forward to tracking what you're up to and following along with Precogly. Your talk that you did at OWASP Global Vienna, by the time this episode airs, should be out available. I was going to suggest that some, that you just ask your AI to tell you where it is. But because Vikram challenged me to fight the AI, I guess you're just going to have to go to OWASP.org and just start clicking around until you find it. But the videos will be out there. So Vikram, thanks for being a part of the show. That's it for this episode of the Application Security Podcast. If you found this useful, share it with someone on your team. And if you're on YouTube, subscribe and drop a comment. On Apple or Spotify, a quick rating helps new listeners find us. We'll be back next week with another conversation. Until then, keep building secure stuff.