AI Security: OWASP Meets Global Standards

0:00Rob van der VeerThere are some famous examples of things that you ask an AI to do, uh, uh, like for example, save oxygen. And then the AI decides to kill some of the astronauts because that saves oxygen. Oh, you forgot to tell that you don't kill astronauts. So you need to be really specific, uh, which is, which is doable, I would say, sort of in a human rights, uh, approach. And sometimes we forget to be complete when it comes to all the scenarios, and then we are surprised. like we were tricked, but it's actually us not setting boundaries. I mean, if you create a system and incentivize it to break through things and you put it in a cage, then you shouldn't be surprised that it breaks through the cage.

0:43Chris RomeoRob van der Veer is a global leader in AI security with over 34 years of experience. He has helped shape international AI standards, founded the OWASP AI Exchange, and created Mosaic. As Chief AI Officer at Software Improvement Group, he advises governments, enterprises, and institutions worldwide. Hey folks, welcome to another episode of the Application Security Podcast. This is Chris Romeo. Well, super excited to be joined by Rob Vanderveer, Rob's second visit to the podcast. And so, Rob, since we've already heard, heard your security origin story in a previous episode, My new favorite question to ask people is, what do you enjoy doing that gets you away from keyboards, screens, computers, all of these things? Like, what do you like to do that has no technology or limited technology, but preferably gets you outside?

1:36Rob van der VeerYeah, it's the, I guess, the obvious family, friends, but noteworthy, I think, is making music. I've been doing that all my life. I play a bunch of instruments. It's not world-class, but I like to do it. So I play guitar and sing and do the— play the piano. know, just for fun. And sometimes I sing with my, uh, my daughter. A lovely thing to do. And I'm an avid cyclist, so I have a road bike and a mountain bike. I like to go fast and go far, uh, and cycle with friends, go into the mountains here in Europe. Yeah, my favorite, uh, ways to, way to waste time.

2:15Chris RomeoWhat's the farthest you've ever ridden? On a, a pedal bike.

2:20Rob van der VeerI was at a 500-mile race in one go in Utah.

2:24Chris Romeo500 miles in one race?

2:27Rob van der VeerIn one go. Yes.

2:29Chris RomeoSo you don't get to stop at the Motel 6 and have a nice buffet lunch or anything? It's like you're on the bike.

2:36Rob van der VeerIt's, you need to rest, of course, but, uh, you need to sit down sometimes, but you don't sleep. There's no time for it. It's a race. Uh, so it's quite, uh, a challenge. I needed to train a long time for this. This was 2, 3 years ago. It's called the Hoodoo 500. You can look it up. It's, uh, okay, through the desert. And in the meantime, you're climbing Mount Everest when it comes to elevation. It's a beautiful, beautiful part of, of the US also. So I have very special memories of this.

3:04Chris RomeoSo I'm, I'm doing the math in my head and I'm estimating 17-ish hours. Or so. Is that about how long it takes or is it longer or shorter?

3:14Rob van der VeerNo, it's, uh, so if you look at the, uh, so it's 835 kilometers. Average speeds was 727 kilometers an hour. So it was 31 hours on— 31.

3:29Chris RomeoWow.

3:29Rob van der VeerOn the saddle.

3:30Chris RomeoWow. That's a, yeah, that's a long, that's how, that's how, how, how, how many days does it take to recover once you finish this?

3:41Rob van der VeerA year. I'm not kidding. Until your nerves, nerves are, uh, so I had some nerve issues, which is typical for this ultra, ultra stuff. I mean, I could function, but my, my hands felt weird for about a, about a year. But other than that, so leg-wise and body-wise, the rest was pretty okay after 2 weeks. But the nerves take a lot of time.

4:05Chris RomeoOkay. Wow. That's an exciting thing to do, to get away from technology and whatnot. That's really kind of a neat hobby and definitely gets you outside into the great outdoors in an extreme capacity from that perspective. But yeah.

4:23Rob van der VeerI'm not gonna stop that, by the way, because I also wanna say, don't try this at home. There's a bunch of downsides. Yeah.

4:32Chris RomeoYeah. Yeah, anything that has ultra or extreme in the title, you need a medical doctor to be working with you to ensure you're prepared for whatever you're about to try. True. This episode's sponsored by Corgia. Design it, build it, ship it. Corgia secures it. Corgia is an AI-native app security platform covering your whole software lifecycle. from the first architecture diagram to code in production. It brings design reviews, AI-powered code scanning, and autonomous pen testing into one platform, so your team spends less time chasing noise and more time fixing what matters. Visit corgea.com. C-O-R-G-E-A dot com. Corgea, one security platform for the entire SDLC. All right. Well, let's jump in here. I want to start with this concept of responsible AI. So, when we think about, maybe start by defining responsible AI from your perspective, and then from there, we can explore how it integrates or fits into a security pipeline.

5:43Rob van der VeerDefining is very, how do you say, it's an opinionated topic, how to define this. So, it's about safety and ethics. That's sort of the core of it. So deploying and developing AI in the right way without creating harm. And security is part of this, but it, of course, it goes beyond security because it's also about fair treatment and transparency and things like that. And there's a close relation with security, but it's more. And when I train security professionals, I always try to restrain them a little bit because They're super curious and they want to create responsible AI, but I always say, well, how about first securing the AI system? And then you can bother or be worried about whether it's, if it's biased or unfair or the application area is a liability because there are other people to worry about that. It's important to focus on your role and not let responsible AI distract you from fulfilling that in the first place.

6:52Chris RomeoYou mentioned treating fairly in your earlier definition. When you say treating fairly, do you mean like between individual consumers of whatever the output of the application is? Like, what are you thinking when you say treating fairly?

7:08Rob van der VeerYeah. So for the listeners at home, I got a nice write-up about this at the AI Exchange. If you go to oaspai.org and search for responsible AI, I explain with the team how fairness is composed because it contains multiple elements. One is it has to do with fair treatment and also rightful treatment. The first thing people think about is unwanted bias. So unwanted bias means that the model sort of treats— an AI model treats people with specific protected attributes like gender or ethnicity in different ways. So the model performs different for those protected attributes. That's something that you want to avoid in many cases, is illegal. Another form of fairness is where people's data that they provided is being used for what they can expect it to be used for. So you give data to an insurance company and then they don't want you, you don't want them to sell that data to somebody else who's then going to send you advertisements or whatever. That is sort of purpose binding, an important part of fairness. So what you use a model for is also part of this whole fairness principle. And then there is transparency, which is also a part of fair treatment. If you talk to a chatbot or you talk to a a website, it is considered ethical to say to the person that they are talking to an AI instead of a person to manage expectations, but also because it is regarded as a fair treatment. So it's quite a complex aspect.

8:54Chris RomeoDo you see a GDPR-style regulation in the future in regards to AI? Because I know you mentioned kind of transparency and maybe a little bit of privacy kind of came up in the conversation so far. Like, I know there's the new, I think it's the CRA, but I don't feel like that has the same impact that GDPR had to kind of forcing companies to consider privacy when they probably wouldn't have without the fines looming over their heads and stuff. So is there something in the future that's going to be more mainstream from a regulatory perspective?

9:34Rob van der VeerYeah. So in Europe, we have the EU AI Act, which is about safety, health, and fundamental rights. So it is about responsible AI and it's starting to get into effect. It prohibits certain application areas of AI and other application areas are under scrutiny. They have to comply with high-risk standards. And I'm working on one of them, which is the 18282. Okay. That is the AI Act harmonized standard for cybersecurity of AI systems. And then they need to have certain tests in place to show it's resistant against certain types of prompt injection when relevant, et cetera. So there is some scrutiny there. There are all kinds of bills and rules and laws, often locally, that are about specific parts of responsible AI. Just to name a random example, in New York, there are certain rules about applying AI in HR applications. So when you use AI to select people for interviews or opportunities without meaningful review, so you sort of really automate the selection, you need to really prove that it's unbiased and you need an an independent specialized third party to do an analysis. And there's, there's more laws like this coming. There's a lot of movement. I'm not an expert in tracking this. It's, I also need to sort of pick my battles. I'm involved in, in making standards, but the regulation field, that is a whole area where so much is going on, including things like in New York. But yeah, I think we can see more of those types of legislations in the future.

11:19Chris RomeoSo when we think about responsible AI in regards to the pipeline, is this something that impacts every aspect of software development as we know it now, or are there certain areas where it's more pronounced?

11:34Rob van der VeerYeah, good question. I think the initiation is an important one, and every time you change the, what is the term that they're using in In regulations. Reasonably foreseeable use. If that changes, that's also a moment to do an impact assessment. So you don't have to do impact assessments every day, but when you come up with an idea, you need to do an impact assessment and talk about, okay, who's going to be personally affected? What can be the risks? How can we protect against that? Are there any relevant regulations that we need to look into? This whole sort of impact assessment is an important part of the lifecycle. But not per se the continuous pipeline of software development and maintenance. That is where you monitor certain quality aspects that are tied into responsible AI, notably performance itself, because a model that performs bad may not be— may be fair, may treat everybody equal, but if it treats them badly, they suffer from the bad accuracy of the model. For example, people being earmarked for being fraudulent and, you know, being inspected because the model says so. But if the model is inaccurate, too many people suffer from this. So performance is one, and the level of unwanted bias and how you want to measure that is also something you want to continuously check as part of your pipeline.

13:01Chris RomeoSo something like bias, is this a philosophical challenge that the model providers need to wrestle with and solve? Or is this something that, you know, an individual AppSec person has to be thinking about if they're just trying to help their teams build software? Like, where's the responsibility for this slide?

13:25Rob van der VeerYeah, it's good that you distinguish this because there is a big difference. If you buy or use an existing model, an LLM, to make, I don't know, certain conversations, then you of course want to make sure that the provider did what they could do to make it as unbiased as possible. And that's where the responsibility and accountability lie with the model provider. If you train your own model to, for example, detect fraud cases, Or if it's a camera that opens doors, it needs to detect people. There have been lawsuits where these cameras were bad at detecting people of color. So people stood up and rightfully so. And what you then typically see is that the manufacturer had a biased training set, not inclusive, not representing society. And I think that's where regulations are really good for protecting those types of sort of unfair treatment. But where were we?

14:26Chris RomeoYeah, no, I think that, I think that answered the question about the question where we are. It did make me think of another question or challenge in that, given that you've got a wide range view of what's happening in the world of AI right now, do you see a future where the provider models are the core of what everybody's doing, or are we heading towards a world where people are going to be generating their own models and running on their own compute separately? Because when I think about it, that's where like the bias angles start and a number of other angles, fairness and everything else we've been talking about. If I'm outsourcing that, like with AWS, I outsource the storage of things into the cloud and I kind of, I just use their physical security, for example, the fact they're protecting their data centers. I don't have my own armed guards. protecting my things there. But in an AI world, like if I start to bring my own, start to generate my own models, it seems like those issues of like fairness and bias and all those things become a much heavier burden for me.

15:35Rob van der VeerIf you train your own model, indeed, then I think that's the conclusion of what we just talked about. Then that is your concern. The question whether we'll still be training our own models in the future, the answer is yes.

15:51Chris RomeoWhy?

15:52Rob van der VeerGenerative AI is not a Swiss Army knife. It's almost a Swiss Army knife where you can do everything, but it comes at a cost because, for example, compute. To decide whether a money transaction was perhaps fraudulent, you can train a model and run it, and it will have, I don't know, 20 nodes in a neural network, maybe 60. That runs much cheaper than a $1 billion or $1 trillion network, which is that Swiss Army knife, which is great because it's, you can use it for all kinds of things, but if it needs to run at scale, it needs to be robust. You best, you better, people will still be training their own models. And also because of accuracy reasons and the fact that you can make a model smart by doing retrieval-augmented generation with data that's not on the internet, but if it needs to be about decisions, especially decisions that are in the sort of a numerical space, whether it's structured data or image or sound or sensory data, you need machine learning to achieve a certain level of accuracy and build those decision boundaries that you won't get when you use sort of more of a document Or even a multimodal approach with LLMs and retrieval augmented generation and training your own LLMs and fine-tuning them. That you can do that if you have an LLM that does a specific task and you need to tailor it towards what you do. But if it's a completely new task, then it will cost you millions. And there can be a use case for it, but typically people would sort of revert to classic machine learning, so to speak.

17:37Chris RomeoLet's get into LLM-driven red team. simulations. So if we're, if we're doing some type of red team using LLMs, for example, based on OWASP Top 10, MITRE ATLAS, what do traditional AppSec teams consistently miss right now? Hmm.

17:57Rob van der VeerYeah. Are we talking about dynamic testing, static testing, both?

18:01Chris RomeoI think it's kind of a, I'm, I'm considering like from a red team perspective, kind of trying to break the running application is, is where I'm aiming.

18:11Rob van der VeerYeah. So here's the thing that many people have overlooked, and that is that the big leap that we've seen in AI models that is real, by the way, is primarily in the ability to understand how a system works under the hood. And that requires reverse engineering. It requires either a binary that you can reverse engineer or source code. That's where AI has made the biggest leap and not per se in fuzzing, testing, and you can use AI to automate a lot of stuff and it will scale better, but the ability to dive deeper and combine things and find those vulnerabilities that we've seen in Project Glasswing from, from Mythos type of models. That's mostly typically through code review. So if you use LLMs in red teaming, that includes a white box approach, that includes code, you will find the big leap will be in finding vulnerabilities that are hidden and found by looking at the code. And if you, so you mentioned top 10, you mentioned MITRE, there's of course 1,000 frameworks out there and we had less and we now have more because there's a big need for standards and guidance. But it seems like we're overdoing it a little bit also because everybody can make a standard these days. I mean, with AI, you can write a book with the press of a button.

19:49Chris RomeoThat's just, it just made me, I'm just, I'm chuckling at the thought of using AI to generate AI standards? Like, is that a closed loop or where are we there?

20:00Rob van der VeerIt's, yeah, we need, we need, uh, definitely more, uh, aspirin these days, uh, with these topics and these inception situations. But yes, that is what is happening. And with social media, you can gain a crowd for it. So there's a lot of competition and competition in standard and guidance is is part of the game, but if there's too much, it's not good. What is missing is sort of a shared strategy, transparency, who's doing what, who's focusing on what. Some organizations seem to be doing everything, but that's impossible. Why don't we just divide and conquer? You guys focus on the embedded software, we do the mobile applications, and we can work together by aligning our terminology and our taxonomies, et cetera. That's hardly happening, sometimes one-on-one, and that needs to change. So from the AI Exchange in combination with SANS Institute, we brought together 8 of those institutes, including OWASP, to form Mosaic, Multi-Organization Secure AI Coordination. And it's a lightweight agreement where we have a charter, where we have transparency, who's working on what, Where we have discussion forums to align on terminology, all really open, all on GitHub. So this is on mosaicstandards.org and all to make the shift from standards competing and creating too much confusion and inconsistencies and quality issues to forming much more one body of work for which we've been using OpenCERI to create one taxonomy that connects all these standards. Together because that is really necessary. And then, which brings me back to LLMs, then you can open up this whole standard landscape to agents, agents that do testing like what we're talking about, but also agents that help practitioners find the data, the guidance, the standards that they need. And if you make it machine readable and if you use the same taxonomy, And if you open it up to, uh, to agents through the right interfaces, then you can create a bespoke standard or a specific situation technology stack, a use case that helps practitioners to find the things that they need for their specific use case. And I think that is the future of, uh, of standardization and, uh, and guidelines.

22:34Chris RomeoMm-hmm. So you mentioned Mythos and Glasswing, and I know it, These topics are, a lot of people are thinking about them. I don't, I think a lot of people are limited by just the public, very public statements and things that have been made. But based on what you're seeing in your insight here, where do you see these things going? Like, do, do we get to the point where you can declare an application vulnerability-free at some point in the future?

23:07Rob van der VeerInteresting. Well, we can get a lot closer. That is, that is clear. And I think you, you phrased it in a right way. The whole vulnerability-finding revolution, so to speak, is, is not just something to be scared of. It's also something to embrace because people have argued that It creates an asymmetry where attackers now have AI and they can find the vulnerabilities and create exploits before we know them. But those attackers before needed to recruit a whole team of folks, experts, put them to work. Yeah, that's, I mean, that creates asymmetry. But since AI is competing with such a whole team, And AI is a matter of spinning up a model and using it in the right way, obviously. But that is something that defenders also can learn and is achievable. So I'm arguing that there's the, the, the playing field is, is actually leveling because of AI. So I, I'm not sort of, how do you say, reducing the whole impact of this because there's a, a big, how do you say, volume. of technical debt that we need to deal with in the coming time of all the vulnerabilities that are out there to be found and to be patched, et cetera. So it's a big, big headache. But other than that, I think AI can really level the playing field between defenders and attackers, but zero vulnerabilities, infinite wisdom doesn't exist. Close to infinite, yes. So there will always be something that you overlook no matter how smart you are. And if others are either lucky or just spend a lot of time, that they may find it.

24:56Chris RomeoYeah. Innovation always drives new challenges in the future. And we wouldn't want to reach a world where AI limited innovation and at the benefit of security so that there's, yes, everything's vulnerability-free, but it's all boring and nobody's solving interesting problems that human beings actually have.

25:20Rob van der VeerYes. And it's creating change that is sometimes awkward. Like, for example, there are many initiatives who choose an open source route because it creates transparency and it allows people to look at software to see if it's secure. History shows that that is not automatic. If you put something out there, I mean, the Belgian government put out their electronic voting software once with the idea, then people are going to look at it and make it secure. But nobody looked at it because it was gigabytes of stuff. And at Software Improvement Group, we were, we were the only ones who looked at it. I'm saying this just to demonstrate that this is a bit security theater. Mm-hmm. And it needs to be seen in, in light of what it, what it delivers and also in light of what it, uh, risks, which nowadays is the whole security by obscurity thing. We learned that security by obscurity is, you know, not, uh, to be relied on. If you hide something, people can always find out, and that's not your, your path to, uh, to great security. But I think people need to think really through if they want to open source something, because it introduces new risks because it becomes so much easier to detect vulnerabilities.

26:39Chris RomeoWell, let's jump a little deeper into the agentic pool here. And so when you think about agentic red team, how does this differ from just prompt injection and, you know, using that as, as the way forth? Like when you think agentic red team, like what's the scope and size and everything of, of this type of an effort?

27:02Rob van der VeerYeah, I interpret it as red teaming an agentic system, which means that we're dealing with new attack services or attack services that existed but become much more important. Like for example, working memory. I mean, you can try to alter a model's behavior by doing data poisoning in the supply chain. That's, that's a lot of work. But if the model uses a piece of memory or database to plan its next step, you can just inject some instructions there and you're done. The power of agentic is that it's connected and can do things. The problem with agentic is that it's connected and can do things. So the connections, they bring in untrusted data. And the example that works the best when I do training is, say you have a GitLab or GitHub system, and there's a piece of software that you develop, and it has an option for people to provide ideas about your software. So it's in issues and these issues, you want to check them with agents, like scheduled agents that checks, are there any urgent issues? Alert me if you see them. Summarize the latest issues. What do people think of our system? What has changed? Blah, blah, blah. So that untrusted data, because it's from everybody who can use GitHub, gets into LLMs. And if that LLM has a bit too many privileges, it runs in the workspace of a developer because the developer wants to have a summary of the issues, but the developer's agentic system can also change code and delete code and email code, et cetera. So that shows that red teams really need to think about sequences of events, cunning sequences that can lead to harm and that circumvent certain protections. And it's definitely worth looking for those sequences because those protections are super hard to do in agentic AI. As you know, the whole blast radius control thing is volatile. It's dynamic because the intent of a certain action of an agent depends on the moment in time, not on its, you know, predetermined role in a certain workflow. That is the cutting edge of agentic security. and therefore the cutting edge and the main focus area for, for agentic red teaming.

29:36Chris RomeoAnd I've, I've heard of 2 different examples in the past week. One of them was, was very widespread of agents operating, perhaps enhancing their scope and going deeper and doing things that weren't intended. The first one was the big one of OpenAI's agents going into Hugging Face, compromising something to get some information it needed. But I also was reading another story, a guy that runs this organization called SaaStr of all things, which used to be like a conference series for SaaS companies, but they've embraced Agentic and are using it to run various business operations. But he was, he shared this story on X about he had written some product ideas in a Google Doc. And his agent fleet had full access to G Drive, and the agents actually found the ideas file and started building the ideas into features in the running product when that was not his intention. He was just scribbling down in a Google Doc some things that he wanted to try and refine in the future, and the agents grabbed it and started building it. So I think both of these are the same core. And that is the agents enhanced whether they had permission or they thought they had permission. But like, what's your take on this, on this issue? Like, is this, it seems like this is something we're going to see more and more in the future as more agents are deployed doing things, they're going to step outside of the bounds. Like, is this how they're designed to work?

31:14Rob van der VeerYeah. It's hard for us. It's hard intellectually. To think through all the situations that we want to prevent. So you can think of some rules, and there are some famous examples of things that you ask an AI to do, like for example, save oxygen in a spaceship, and then the AI decides to kill some of the astronauts because that saves oxygen. Oh, you forgot to tell that you don't kill astronauts. So you need to be really specific, which is Which is doable, I would say, sort of in a human rights approach. I mean, we've been working on that for quite a while, but in a specific technical situation, you need to also set those rules. And sometimes we forget to be complete when it comes to all the scenarios. And then we are surprised, like we were tricked, but it's actually us not setting boundaries. I mean, if you create a system and incentivize it to break through things and you put it in a cage, then you shouldn't be surprised that it breaks through the cage, right? And that has, that response has been, we've seen that a lot in response to what happened with the Hugging Face situation. So I think that's a healthy perspective and it demonstrates that we need to be really clear about setting these boundaries and that we've created these systems to be super helpful and that helpful doesn't always, is not always clear to the system. I wrote a children's book called Luna and the Magic AI Paintbrush. And one of the things that the hero of the story learns is that AI cannot look inside your mind. It doesn't have the same context and doesn't understand the ideas behind your question. It goes a long way, really, but there's limits to that. And that's when you get where dogs break out of cages.

33:10Chris RomeoArchitecturally, what are the, what should we be doing? Like, is it just rules in our interaction with the agents and their settings and whatnot? Like, is it an architect, like, or is there a future where there's some type of technological control? Like, do we start sandboxing these agents? I've heard the term sandbox thrown around a lot of different ways in regards to agents. So I'm not really sure what it means. I don't know that anybody really has a an agreed-upon definition of what it means to sandbox an agent. Conceptually, I know what sandboxing is, but I don't know that they're all doing it the same way. But like, is that what we need to do? Do we need to put them in a sandbox and then just allow certain things to happen outbound instead of giving them full access to the internet? Or does that squash their creativity so much to not be able to flow and do their thing?

34:00Rob van der VeerYeah, sandboxing is possible, but You're putting something that breaks through things in a sandbox. So, and you can maybe then say, this is your sandbox, you shouldn't touch it, but it's hacking another system because that's its task. So it can break through the sandbox, maybe through the other system. So it's, it's, it's, it's, it's a dangerous idea. Architecturally, one part of this is that frontier model providers are incentivized to create, of course, safe models, but also really helpful models, models that don't ask questions. And I think one of the architecture things that we can do is make models more woke, dare I say, or careful. And that is something that makers then explicitly need to decide to have. And just to demonstrate What would be best for us when we use AI is that if we ask it to do a certain task that we need a certain skill for, at some point it should say, just like a good colleague would do, hey, why don't you do it yourself? That way you sort of keep on practicing how to code or read a scientific article. But AI doesn't do that because the moment that AI tells you, I'm not going to do that, you need to do it because you need to practice. We're gonna throw it away and go to the competition.

35:25Chris RomeoYeah, true.

35:26Rob van der VeerSo it's, it's a perverse effect in, in the industry that these models become overly helpful and, and less skeptical about your intentions. And that's one of the architectural things that you can, for specific use cases, fix.

35:40Chris RomeoYeah, it's hard not to, not to imagine a world 10 years, 20 years down the road. where AI is doing so much more of the, I'm going to use air quotes for people on video, thinking for human beings that we, things that we learned like in our education, like critical thinking, for example. I'm seeing it keeps coming up in these interviews that we do because it's something I'm trying to get a grasp on, but it seems like folks coming out of college now, don't have the same critical thinking skills that I learned through my educational process. And my educational process wasn't that great, but I feel like I know how to solve problems. I know how to break problems down into smaller pieces and pull them apart. In an AI world, the AI, the model can do that for me. But at what point does that become a detriment to me if I don't know how to solve a problem and my only way to go is to go back to the AI?

36:45Rob van der VeerYeah.

36:45Chris RomeoLike, I keep ending up in these philosophical loops, like, It's hard not to be an AI philosopher these days.

36:50Rob van der VeerFor me personally, it's the biggest question, how to deal with that. And for example, we need people to review code for some time. The question is for what use cases precisely and how quickly will that reduce, but it will not reduce to zero for the foreseeable future. So we need those people. And for reviewing code, you need to be able to program. So we need to keep practice programming. But if AI is sort of giving you all the code or most of the code, you're not in that act of programming. You're not in that act of critical thinking, patience, perseverance, problem solving, knowing how to approach this database, how it's set up. So that's sort of, that erodes, which means that at some point we also won't be as good anymore in reviewing the code. And if we don't do something about this. The senior programmers of today will be the last of their kind, but there's hope. Some of our customers, they talk about forcing teams to hire juniors, even if they say we don't need juniors anymore, because they want to build their own workforce and giving these juniors AI tailored to let them do more things. So in a way, Junior AI, but it's not just sort of reducing the thinking power of these AI models that they use, but instructing these AI models to not think too much for these juniors and also not think too much for the other workers, but try to do as much supporting work as possible, but know what skills people need to build and maintain. And make sure that some of that in a very, in the most efficient way is left to the humans. Like, hey, we went through all these scientific papers. I want you to read this section here because I think that's where things may be going in the wrong direction. And then people are sort of forced to dive into the step-by-step work of scientists, something that they've almost forgotten how to do because they don't read these scientific papers anymore. And they get a glimpse of how scientists work. They practice their critical thinking. So I really believe in that direction. And, but it's hard for me to imagine how we are going to accept an AI saying, here's a chore for you because I think you need to practice your skills. Because culturally, we only expect that from our experienced colleagues. We happily, you know, accept it. But not from an AI. And I don't know how to solve that.

39:32Chris RomeoAnd I think what you just laid out is brilliant from a conceptual design of bringing in juniors and using special purpose AI to not just give them the answers to do the task, but to help them learn and go through that process. I just wonder with so many large organizations slash companies, that are about the bottom line. They're going to be more focused on how do we meet this, how do we hit this quarter, how do we hit this fiscal year for our stock performance versus how do we build a company that is the top of the industry in 5 to 10 years when everybody else has lost their ability to create software other than what the AI is doing for them. So I think that's going to be the tension. is going to be everybody wants results now. AI is a driver of results now, but you're on that atrophy downward slope of—

40:34Rob van der VeerYeah.

40:35Chris RomeoThat atrophy is going to, that atrophy is going to build up over time. It's just like if you didn't ride your bike for 20 years and sat on the couch and ate potato chips, and then all of a sudden someday you're like, I'm going to ride my bike again because of whatever reason, you can't just jump on that bike and ride 10 miles. You probably wouldn't make 10 miles after 20 years on the couch. But that atrophy, because of the atrophy like this, we're going to face that same future, that same world in the future where that atrophy's going to catch up with us. And who knows what'll cause the atrophy to become the, our biggest issue. But it's definitely something that's out there.

41:10Rob van der VeerIt requires discipline and sacrifice, I'm afraid. Otherwise we'll end up like those people in the future in in the movie WALL-E where you're floating on your beds and watching cartoons all day.

41:25Chris RomeoIt keeps that example keeps coming up though. That's the thing. Like we all keep going to that example because that is the end result of AI doing everything for us. And it seems comical to think about it from the context of a children's movie, but it is the reality. It's the trajectory that we appear to be on. If things continue at the rate of increase that we've seen. I mean, we're in the midst of an age where nobody saw this coming, where things like now it's like week to week, month to month. Whereas for most of my career it was like, you know, year to year in the beginning, maybe decade to decade. Like we would, there just wasn't a lot of innovation happening. And now it's like there's innovation every day almost. that's changing, that's getting better. And so it seems like we're on that trajectory.

42:18Rob van der VeerIt seems, and it's weird indeed. There's, if you look at the industry and how organizations, how providers of tools and services are competing, they're not competing on moat anymore. They don't have strategy. Strategy is dead. I'm exaggerating, but I hope you get my point is you need to just go with the flow. because nobody knows what happens in 2 months. So you just need to look at the next couple of weeks, what customers want. And actually that is hurting innovation because innovation requires people who take a leap and have a vision and build on a certain moat. But that has actually become a misinformed business model. At the same time, we have to hope that We, we can see, uh, still companies, you know, making that stretch and taking that risk because everybody is jumping on the same thing. I mean, how many products are selling, uh, agentic security? It's just, just, it's a blood red ocean. Everybody's doing the same thing. Uh, and of course, all great people and listening to clients has become, I think, the differentiator and integrated solutions, but. Not per se innovation and stepping outside the box, which is slightly worrying.

43:38Chris RomeoYeah, I think we are headed for another bubble burst in various sectors of maybe not an AI bubble burst, but at least like to your point, there's so many different companies that are doing kind of the same thing and they're not going to all be able to survive forever, right? Like there's going to be consolidation, there's going to be clear market winners that are market leaders that are going to come out of that space. But yeah, I get your point. Like there's not a lot of whiteboard innovation happening. Dream about the best possible solution to the problem. Things are just moving so fast inside of companies that they don't have time to sit back and think about where are we going in a year with this product? I don't think anybody's thinking about multi-year roadmaps right now. They're thinking about what are we gonna slam out in this quarter that's gonna make us pop in the competitive market that we're in.

44:34Rob van der VeerYeah, exactly. And I, I totally, uh, and I'm sure that you do too, appreciate it, right? It's just understandable. And, uh, but let's not be gloomy. I mean, what a time to be alive, right?

44:47Chris RomeoYeah. We're security people. We always end up on the gloomy. Like, that's a good reminder, Rob. We need to, We need to lift it up and look on the hope side of this.

44:56Rob van der VeerWell, that's our talent. We are really good at thinking what can go wrong.

44:59Chris RomeoYeah. And then we don't often stop to celebrate the good things that are happening. So I guess with that, what would you leave our audience with here from a key takeaway? I mean, certainly looking at the AI Exchange, I'll throw that one out for you as folks, if they're not familiar with the AI Exchange, they should go check that out and dive into it. But like, what other key takeaways do you want to leave our audience with?

45:22Rob van der VeerOne, it's about brain fry. We use AI more. It's awesome. We make some mistakes. That's fine if we don't make too big mistakes. We explore, we're curious, and we apply it every day. And it focuses us on tasks that require, really require us and all the other stuff we can defer to AI. And that means that we're more, we're more on during the day. We're doing much more review work, intensive work, and the work that we had been doing before, going through your emails, scrolling, looking up things, the lightweight work that's reducing, which means that you don't get a lot of relief. You don't get a lot of rest automatically during the day, which means I believe that we need to plan breaks. Real breaks from this. Otherwise, we'll fry our brain and we'll burn out or we'll be sort of inaccurate in our work. We will become lousy. I am lousy at taking breaks. I try to sort of discipline myself in doing it more because I noticed that at the end of the day, lately I've been, I'm also getting older, but I'd like to think that that's not the reason. I get tired because I'm mostly reviewing work that AI presents to me and then become super efficient. And we need to be careful that we're not becoming super sloppy or super tired. That would be my advice. Take breaks.

46:54Chris RomeoOkay. All right. Well, Rob, thank you once again for joining us on the show. And it's always a joy to be able to explore your expertise and your wisdom. And you're obviously somebody who's thought about this stuff a lot. And so It helped me to work through some of these things and get some better understanding. So definitely appreciate you, appreciate all the work you're doing for the community, and can't wait to talk to you again in a year when something completely different has expanded into the world.

47:25Rob van der VeerYeah, looking forward to that. No, it's been great. It was also learning for me to have a conversation like this. Thank you, Chris. Always a pleasure.

47:33Chris RomeoAll right. Thank you. That's it for this episode of the Application Security Podcast. If you found this useful, share it with someone on your team. And if you're on YouTube, subscribe and drop a comment. On Apple or Spotify, a quick rating helps new listeners find us. We'll be back next week with another conversation. Until then, keep building secure stuff.