---
title: "Josh Grossman--AI & SAST: Is it a match?"
url: https://appsecpodcast.com/josh-grossman-ai-sast-is-it-a-match/
date: 2026-06-02
duration_seconds: 2429
season: 13
episode: 5
guests: ["Josh Grossman"]
topics: ["Security Testing", "API Security", "AI and LLM Security"]
audio: https://www.buzzsprout.com/1730684/episodes/19275442-josh-grossman-ai-sast-is-it-a-match.mp3
video: https://www.youtube.com/watch?v=OCBosHWLiAQ
transcript: true
---

# Josh Grossman--AI & SAST: Is it a match?

*June 2, 2026 · 40 min · Season 13, episode 5*

with [Josh Grossman](https://appsecpodcast.com/guests/josh-grossman/)

on [Security Testing](https://appsecpodcast.com/topics/security-testing/), [API Security](https://appsecpodcast.com/topics/api-security/), [AI and LLM Security](https://appsecpodcast.com/topics/ai-security/)

[Audio](https://www.buzzsprout.com/1730684/episodes/19275442-josh-grossman-ai-sast-is-it-a-match.mp3) · [Video](https://www.youtube.com/watch?v=OCBosHWLiAQ)

## Show notes

Traditional SAST is deterministic but shallow; AI can reason about context but may answer differently every time. Can the two approaches make each other better? Bounce Security CTO Josh Grossman explains why he built AGHAST, an open-source framework that combines static discovery with LLM analysis to investigate authorization, business-logic, and organization-specific risks. He walks through reducing false positives, importing SARIF, controlling token costs, and deciding where AI-assisted checks belong in developer workflows and CI. Josh also shares how he used Claude Code to build most of the project while retaining the architecture, product judgment, and code-review responsibility himself. The episode closes with AGHAST's roadmap, supported languages, practical adoption advice, and a guided demonstration of the tool.

Connect with Josh Grossman:  
→ [Josh Grossman on LinkedIn](https://www.linkedin.com/in/joshcgrossman)  
→ [OWASP AGHAST](https://aghast.owasp.org/)

Mentioned in this episode:  
→ [OWASP AGHAST](https://aghast.owasp.org/)  
→ [Semgrep](https://semgrep.dev)  
→ [Cursor](https://cursor.com)  
→ [Claude Code](https://claude.com/product/claude-code)  
→ [SARIF](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html)  
→ [NDC Security](https://ndcsecurity.com)  
→ [Black Hat](https://blackhat.com)  
→ [DEF CON](https://defcon.org)  
→ [ISACA](https://www.isaca.org)  
→ [Manicode Security](https://manicode.com)

Chapters:  
00:00 Meet Josh Grossman  
01:11 Why Josh built AGHAST  
04:22 Will AI disrupt AppSec tooling?  
06:09 How AGHAST combines static analysis and AI  
08:48 Deterministic rules and pure AI checks  
10:23 Reducing SAST false positives  
11:49 Using SARIF from existing scanners  
12:46 Building AGHAST with Claude Code  
14:14 The product specification and human judgment  
17:48 How much code did the AI write?  
19:05 The architect and product-manager mindset  
21:08 Token economics becomes its own industry  
22:54 Authorization and business-logic checks  
25:55 Context makes custom rules valuable  
28:15 Where AGHAST belongs in the workflow  
30:11 Languages, frameworks, and COBOL  
32:10 The AGHAST roadmap  
34:20 Key takeaway and call to action  
36:56 Training and conference appearances  
37:29 AGHAST demonstration

## Transcript

*7,790 words · assemblyai*

**0:00 Chris Romeo:** Josh Grossman has worked as a consultant in IT and application security and risk for 15 years now, as well as a software developer. This has given him an in-depth understanding of how to manage the balance between business needs, developer needs, and security needs, which goes into a successful software security program. Josh is currently CTO for Bounce Security, where he helps clients improve and get better value from their application security processes and provides specialist application security advice. His consultancy work has led him to work, speak, and deliver training both locally and worldwide, including privately for ISACA and Manicode and publicly for OWASP's global AppSec conferences, NDC Security and Black Hat. Quick note, we're opening up a few sponsorship spots on the Application Security Podcast. If you want to get in front of real AppSec practitioners, the folks actually making decisions, this is a great place to do it. Just reach out to me, Chris Romeo. I'd love to chat. You can find me on LinkedIn.

**1:11 Robert Hurlbut:** Hey folks, this is Chris Romeo. Welcome to another episode of the Application Security Podcast. Super excited to be joined by Josh Grossman here, who has been on the podcast before a few times. And we're going to be diving into this new tool that he's created called Agast, which has kind of mingling together some AI and a bunch of other things. So I guess, Josh, the best place we can start here is A lot of security teams, they're dealing with a lot of tooling. You know, it doesn't always work the best way that they need it to. So for you with this tool, AGAST, like what was the moment where you decided to build a solution rather than just try to find a workaround with existing tools?

**2:02 Josh Grossman:** So I guess I sort of had a front row seat last year to just the absolute sort of takeover of AI in the development industry. I was sort of working with one particular client, and I basically, in the space of the first half of the year, I watched them go from sort of rough POCs, okay, we'll evaluate this AI provider, and like this AI IDE, and this is early 2025. They started off doing that, it will take a few weeks, and then suddenly like, yeah, we just bought Cursor licenses for the entire organization. And then I just sort of got mine set up to help work with the client. They're like, oh, no, everyone's using Claude Code now. And this was like within the space of a few weeks, one after the other. And I just saw the pace that it started moving and the pace that developers started moving as well. And that sort of culminated in a— I was lucky enough to be at Black Hat and DEF CON last year, and I was helping out with OWASP community. And I said, you know what, I'd really like to do a discussion about this sort of whole AI coding thing in general and trying to discuss with other people who would be interested to come to OWASP community, okay, how are we handling this? What are we doing about this?

**3:10 Robert Hurlbut:** Yeah.

**3:13 Josh Grossman:** And this talk ended up being, I think, a Sunday morning. Sunday morning at DEF CON's already a little bit quiet. People are already thinking about, you know, they're finishing off, they've had a few big late nights. And I suddenly see like 10, 20, 30, 40, 50 people come into this, into the Ask community to have this discussion, because clearly everyone is seeing the same thing. Everyone's seeing this sort of massive rise. So it became clear that we couldn't just ignore this. We had to be involved in this somehow. And I think part of it is about at the actual writing code stage, but certainly what I was seeing at the client, what I was most worried about is, okay, well, how can I take advantage of this to actually figure out what's going on in code? How can I take advantage of this to sort of discover what's going on in this new volume of code that's being generated? And I'd already seen that static tools weren't really enough. And sort of generic rules, off-the-shelf rules, they weren't enough to begin with. And now this extra sort of AI dimension gave a lot more flexibility and a lot more possibilities of, okay, what can we actually discover? How can we use this sort of extra intelligence to discover better findings and find sort of more accurate findings as well?

**4:22 Robert Hurlbut:** Before we dive deeper into AGAST and how it works, I'm curious, when you think about the kind of state of AppSec tooling, where do you, where do you think, are we heading for like a complete disruption from your perspective where we can think of classic tools, SAST, DAST, IAST, RASP, OhMy, you know, the kind of classic things that are there, are those just, in your view, are those going to disappear and be replaced by other things, or how do you see that future?

**4:53 Josh Grossman:** I, I think that for the past few years even, I think people have sort of gotten to a stage where those tools, they're using them to a certain extent, but I think they're gradually realizing that they're not covering a lot of the actual risks or a lot of the things that are really causing trouble. And I think that maybe part of that is because maybe we're writing slightly better code than we were a few years ago, and therefore the sort of the generic rules aren't finding as much. Part of that is the limitations of these tools, limitations of these static tools. So, I previously, I came on podcast a few years back to talk about my course around using these tools more effectively. And I think that having these tools in place and sort of well-configured is definitely valuable, but I think the genesis of that course was organizations spending all their time on these tools. And I think there now needs to be a much more careful examination of how long are we actually spending on these tools compared to how much benefit we're getting. Maybe we want to put a little bit of time up front. get something out of them, then move on to something more valuable without, you know, making sure that we're not having all our time taken up by them. I think that they have this place as a baseline, but they are no longer enough. And I think we're gonna see different things coming up and more sophisticated tools coming up as well.

**6:09 Robert Hurlbut:** So, when we look across the tool landscape right now, there's a lot of hype for AI in the code scanning space. You got Anthropic shipping their own tool, which I think we're all kind of still holding our breath on, 'cause we're— the idea of it seems like it could be industry-changing. I don't know that what it provides at this point is industry-changing.

**6:36 Chris Romeo:** Yeah.

**6:36 Robert Hurlbut:** But OpenAI's kicking around the same idea of getting into the security tooling space. But going back specifically to AGAST, like, what does AGAST do that those tools fundamentally can't? And kind of walk us through what AGAST is doing.

**6:52 Josh Grossman:** So the idea behind Agar.st, but the idea behind sort of the way I approached building the tool and designing the tool was a few things. First of all, I didn't want to throw out the whole sort of static rules. I wanted there to be some element of sort of deterministic analysis, some element of sort of traditional finding points in code using A traditional sort of static rule. Maybe that's a custom static rule, but I didn't want to throw that away entirely, but I wanted to blend that with the AI approach. So, the most powerful mode for a Ghast, it's not the only mode, but the most powerful mode is sort of this hybrid mode where you identify particular areas of the code where there might be an issue, and then you feed those into an AI provider and say, look at this part of the code and tell me, does this part of the code have this particular issue in? And that means that you're not asking the AI, okay, go find bugs, and it's sort of running all over the codebase, which has cost implications, but it also has effectiveness implications, as eventually the LLM will choke on a very large codebase. It means that it's looking at a particular area, and it's focusing on a particular problem to sort of come up with a targeted answer. Do I have this problem? Do I not have this problem? So, that was the overall idea. And there are various other tools that people have been working on. I saw one just today, I think it was called DeepSec. I think Vercel released it, which was a similar idea, but I think a lot of the tools that have sprung up around this have been very focused on, okay, we'll use LLMs, and then we'll use LLMs to drive the LLMs, and we'll use LLMs to orchestrate the other LLMs. We'll have agents and sub-agents. Whereas I didn't want to completely leave behind the static aspect as well that I think can still be valuable. I don't like using an LLM to do something that I could do statically and therefore get a definitive answer out of? Because when I do something with an LLM, I'm not 100% guaranteed to get the same answer each time or the full answer or complete answer.

**8:48 Robert Hurlbut:** Determinism, the one thing that's missing. That's a good t-shirt idea right there. Add that to my t-shirt idea list. So is AGAST then tied to the static rule bases?

**9:05 Josh Grossman:** No.

**9:05 Robert Hurlbut:** In any way? Is that a limitation in any way?

**9:08 Josh Grossman:** So, first of all, you can write pure AI rules in Agasti as well. You can also write pure static rules in Agasti as well. One of the things that I was doing already a year or so ago was building up a set of custom static rules to find particular issues that were specific to either a specific codebase or a specific client. And really, Agasti is just sort of an extension of that. It's taking a customer, it's taking a a custom problem, okay, how do we discover this custom problem? So you can write a static rule to speed up your analysis. You can write a purely AI-based prompt that will just only use the AI aspects. You're certainly not tied to the previous rules. The idea is to decide, okay, how would I want to solve this problem? Maybe almost as, like, a human approach, how would I solve this problem? Maybe I want to find all the functions in a particular application, or maybe I want to find all the the places where they expose an API endpoint, or maybe I want to find all the places where there's a database call, or maybe I want to find all the places where a particular custom function that's defined in my organization is used. And then I want to say, okay, well, are they using it right? Are they using it securely? So I'm almost applying the same approach here. I'm saying, well, I'll have a static rule that finds me all these places in the codebase, and then the AI prompt that actually does that analysis, are they using it correctly or not?

**10:23 Robert Hurlbut:** Okay. So, what about false positives then? That's been kind of the bane of SAST's existence, has been running the tool against a codebase and comes back with 10,000 results or 10,000 issues. And then developers look at that, throw up their hands and say, this is useless. Can't get anything good out of it.

**10:46 Josh Grossman:** How does using an LLM, does that help me in the false positive detection and perhaps So, first of all, you know, AGAST is obviously focused on particular problems and finding particular issues. So, it's already a lot more focused. It's not just sort of throwing a whole kitchen sink of rules or potential issues and pulling out every single thing. When you're using AGAST, you're trying to find a particular problem, and therefore, at least, you know your results are going to be constrained to that particular problem.

**11:19 Robert Hurlbut:** Yeah.

**11:22 Josh Grossman:** The other side of that is that, again, the non-determinism thing, there is a risk that the AI side of the analysis will not go through the code properly or have some sort of hiccup and not analyze it properly. And when I use a GAST, I try and refine the rules and test them a few times to make sure that the rule is catching the things it's supposed to catch and not catching the things it's not supposed to catch. And there is a certain element of refinement there, but it—

**11:49 Robert Hurlbut:** Yeah.

**11:49 Josh Grossman:** Certainly does improve the situation. One mode that I actually built into Agast, which is not exactly what it was designed for, but it was quite easy to add in, was a mode where you can give it a SARIF file of more traditional SAST findings and say, okay, analyze each one of these findings, look at the description, look at the code area, and tell me whether you think it's a false positive or not. So this is now addressing the false positive of a different tool by saying, well, let's have a second pass through. A lot of, you know, something maybe the enterprise SaaS vendors, I think, are adding more into their tools as they go along. But certainly if people have either done their own customizations or they're using some sort of open source tool to find particular issues and they're looking for a way, okay, well, how can I get more assurance over these results? Then they can feed them into Agastya and they can get this sort of extra opinion about as to whether these are full positives or not.

**12:42 Robert Hurlbut:** Yeah.

**12:46 Josh Grossman:** Okay.

**12:46 Robert Hurlbut:** So I want to spend some time understanding how you used Claude Code in the building of, of the tool here, because I think it's a, it's a challenge that everybody's trying to figure out right now. And a lot of folks that I've heard from in security are working with Claude Code. They're trying to figure it out. So I'd love for you to just walk us through how you use Claude Code in the, in the creation process here? Like, walk us through some of the steps you went through in using Claude Code, because obviously it was a successful usage because you ended up with a working piece of technology at the end of it, not just a, you know, spaghetti code, crazy piece of code that doesn't really do anything. So curious to see how that process kind of played out for you.

**13:33 Josh Grossman:** Yeah. I mean, also, This is the first time I built something on this scale before, so it was also sort of a learning experience going through that as well. I think that when you're using an AI coding assistant, it does force you to make sure you do think upfront, okay, how do we want this to work? How do we want this to be structured? What we started off with was basically a very large specification document where we'd sort of put down our thoughts onto paper. And that was if not generated by AI, then certainly formatted and structured. I have a habit of sort of dictating a stream of consciousness into speech-to-text and then say, okay, well, format this up for me, make this tight.

**14:14 Robert Hurlbut:** So you wrote that document though? You didn't have AI write the document and then you kind of reviewed it and you kind of, as the human in the loop, you went through and captured exactly what you wanted it to do?

**14:24 Josh Grossman:** So the idea was to, the ideas in the document and the contents of the document was sort of roughly dictated by me. And then I asked AI to tidy it up, and I started going through it and reviewing it, saying, well, what about this? What about that? And often, if there's a simple change, I can make it myself. But often I say, well, okay, well, we talk about this here, but then later on we talk about that. Maybe we need to make a change all the way through. So I've got Claude Code sitting on the document, working through it. I mean, Claude Code for individual features and sort of slightly more smaller-scale things, it has its own planning mode. And that was something I was using further on once I'd sort of gone past the spec. But certainly, for the first part, I wanted my vision to be in there to begin with. And then I worked with Claude to sort of iterate and refine that and come up with a well-structured document that a human could read through, but an LLM could also use to then develop out the features. Because then the next stage was saying, okay, now we've got the specification, let's break it into steps. And the idea was to have a series of steps. So, Most like implementation sprints. So, okay, very sort of agile style. We want a deliverable product at the end of each sprint. We want something that runs at the end of each sprint. We want unit testing, integration testing as part of this as well, to make sure that things are working. And then when we come to the next sprint, we haven't broken something from a previous sprint, because we know because the tests are still working. And basically going through this sort of iterative process of trying to sort of build it up gradually. Part of that was also making Claude, you've got their claude.md or agents.md, depending on what code assistant you're using, and making sure that that also contained important guidance. A lot of that was making sure that it would operate in a certain way, use feature branches, ask me before doing certain things, certain commands to use, certain commands not to use. And a lot of that was sort of trial and error as well.

**16:15 Robert Hurlbut:** Okay.

**16:16 Josh Grossman:** I can see it's struggled because it keeps trying to do this. I'm gonna give it an instruction Not to do that next time. It's certainly not 100%. I think the biggest lesson we see with pretty much every single use is that it doesn't matter how many times you ask an LLM to do something, you will never be 100% sure that it'll do it. But that means you have to keep an eye on certain things and make sure that it's not gonna end up doing things that you don't wanna do, or pushing code you don't wanna do, or suddenly changing major settings. But certainly, I found trying to build that into the config as well reduced the likelihood of that happening. Another thing I sort of started doing more of was building skills or commands for Claude as well, where I found myself doing something quite often. It's almost like a sub-prompt that says, okay, well, when I ask you to do this, here's the process you're gonna go through, here are the commands you're gonna run, here are the things you're gonna check. That's almost, you know, that's like a reusable reusable skill that I can then use the next time I need to solve that particular problem. So, for example, I've got an internal repo for Claude and an external— sorry, internal repo for Augusta and an external one just to try and, you know, I can sort of release features gradually and also not make a big mess of the public repo. So, I've got a skill that helps me synchronize between them and goes through that process. I've got another skill that helps with the release process. Part of the release process is in CI. Part of the release process requires some thought upfront. And so, having these skills, then that means I can just write a simple, okay, do this now. And it's already got the memory of how to do that. It's got the process of how to do that.

**17:48 Robert Hurlbut:** So, yeah, it sounds like it was a quite successful experiment as far as using Claude, yeah, using Claude with you, working alongside of you. So, how much code do you think you actually wrote? percentage-wise versus what Claude wrote?

**18:08 Josh Grossman:** I wrote very little of the code, but I think being able to understand the code and look at something and say, well, does that make sense? Does that seem right? Does that seem messy? And that's sort of an important skill. And I do sort of worry about sort of the, almost like the next generation of people who are coming up only doing this, not having necessarily had the experience of writing the code, therefore, whether they'll have that sort of knowledge of being able to look over the code and say, yes, this makes sense, this doesn't make sense, or being able to diagnose. People talk about just saying to Claude, okay, keep fixing this till it works, keep fixing this till it works. And to a certain extent, that works. And to a certain extent, you can tell Claude, okay, well, look at the errors from tests and fix them. But I think you also need to have that knowledge, okay, well, here's what it's actually doing. And therefore, if it's either going down a rabbit hole, or it's doing the wrong thing, or I suddenly see a command flash across I don't like the look of, I I've got that experience to say, no, stop, let's rethink this.

**19:05 Robert Hurlbut:** Seems like the future success in the future is going to be tied to having an architect's mindset and a product manager's mindset with enough knowledge of development to understand how the code works. It just, and just in my limited testing of Claude Code, it seems like that's the part, like my product management experience is what allows me to describe the thing. 'Cause I'm used to writing feature descriptions for developers to build for something that I think is cool. But I've found that being able to describe that well helps Claude code to be able to generate it. But to your point, like being, you really have to be able to do your own checks of what's coming out of it and understand, is there a, is it going in the right direction? Or is it completely going in a different direction from what a senior developer would do in that situation? Because it's not a senior developer at this point, right? Like, maybe we'll get there, but it's definitely not a senior dev.

**20:13 Josh Grossman:** Yeah. I think that also manifests when you, okay, I need to work on this particular problem, this particular feature. And if Claude only looks at part of the codebase and therefore it's doing things, it hasn't looked, okay, well, here's how it was done elsewhere, and having that knowledge, okay, well, I I remember how we did this a couple of weeks ago. I, as Josh, remember how we did this a couple of weeks ago, and I can say to Claude, well, you might wanna look over here, 'cause we did that in a similar way, or something. And I think that's also important, to have that awareness of what actually is there, what is in the codebase, and you can feed that context. In theory, Claude can consume the whole codebase each time, and figure out what's going on. But I think that does get harder as the codebase gets larger, and gets a little bit less realistic. And also, if you can tell it upfront, look here, find the, uh, an example of how to do this here, then it does, it's going to speed things up and it's going to, I guess, cut down on token burn.

**20:56 Robert Hurlbut:** Yeah. That seems to be the big problem these days for everybody. It's token burn.

**21:02 Josh Grossman:** Yes.

**21:03 Robert Hurlbut:** Who would've thought? Who would've thought that would be the word of 2026? Token burn.

**21:08 Josh Grossman:** I'm, I don't know. I'm, I'm wondering whether we're going to see a whole sort of industry pop up around sort of token economics in the same way that sort of cloud cost economics popped up. a few years ago, people trying to reduce their cloud costs. I think we're going to see more sort of structured approach to this because it's certainly costing huge amounts of money, and it's very easy to run up a massive bill. And I'm seeing individual things. I don't know if you saw the Caveman add-in for Claude.

**21:33 Robert Hurlbut:** Yeah, I've heard about it.

**21:35 Josh Grossman:** I saw something similar that's supposed to sort of proxy, you can proxy your command line commands through it, and it shortens the output to take up less context, like all these sort of like micro fixes. But I think we're almost certainly going to see more in that area because the sums of money being put into this are quite crazy.

**21:54 Robert Hurlbut:** That was going to be my new startup idea, creating a proxy. Your prompt goes through and then it just takes it down to nothing. Oh well, somebody already beat me to the punch.

**22:04 Josh Grossman:** Well, they beat you in one particular area. Like I said, I think there's room for something slightly more Let's say all-encompassing. Let's say.

**22:12 Robert Hurlbut:** That's true. Well, all you'd have to do then is go to Anthropic and negotiate a gigantic token purchase so that you can buy them cheaper. And then you don't have to be that optimized in the proxy. You're just a reseller of tokens and you happen to be getting a better price than they would get because you bought in bulk. I think people used to do this with phone cards. Remember those phone cards where people would use it to call around the world from payphones? I think I've really just reinvented the phone card scam. All right, I'm gonna, I'm gonna move on to a new idea. How about a, how about a real example of something? I'd love to hear something you found in a client engagement where you were able to use the tool to find a problem that off-the-shelf scanners wouldn't have found.

**22:54 Josh Grossman:** So I think a bunch of things come to mind. I think authorization issues, you know, all sorts of different authorization issues. I've got clients where they have, let's say, a more freeform approach to authorization, and therefore, it's not necessarily sort of one consistent way of doing things. It may be that their approach is, well, as long as we did authorization somewhere in the chain, we're okay. But actually tracking that, sometimes it'll be at the API endpoint, sometimes it'll be in the service that's actually processing the logic, sometimes it'll be at the data layer. And they're like, oh, we're fine because we've done authorization somewhere. But tracking that sort of thing and keeping on top of that sort of thing is very hard. So I've had a bunch of examples where by using this sort of approach, we can say, well, let's find examples where there isn't authorization anywhere in that chain. And that would maybe be theoretically possible by a custom static rule, but it'd be very, very complicated, very complicated to sort of set out, especially if they're using multiple different ways of doing authorization. Whereas an AI prompt can read through that chain and say, well, that is an authorization thing that I recognize, or I can see that they're some sort of authorization. So that's one big area that I've seen wins with this. Another area, I had a particular client who had a, let's say, how do you describe it? Not quite out-of-band login, but an authentication mechanism that bypassed the standard authentication process. So like, okay, well, we need to offer via a particular interface a way that will basically log the user in without having entered in their username and password. And I was like, okay, well, that sounds dangerous. We need to have a bunch of different controls in here. And so once I'd suggested to them, I said, okay, well, I'm also going to build a rule to make sure those controls stay in place. So that if you in the future say, oh, why are we doing this? And then you drop off one of the controls, the rule will pick that up. Now, you know, generic rule is never ever going to pick up on that sort of thing. And again, a custom rule You've got multiple different functions, you've got multiple different ways of doing the same particular check. Does this user belong in this? Is this user in this particular group? Has the assertion that we've been provided by the third party to let the user log in, does it contain these particular items? Again, if I try and do this with a static rule, even if it's a custom rule, becomes quite complicated. But again, having that AI element in there as well allows it to reason, allows it to figure out, have they missed this off? In their case, they actually added another endpoint to do something similar. And they added some of the checks, not all of the checks.

**25:24 Robert Hurlbut:** Sweet.

**25:25 Josh Grossman:** And that meant that, because it was another very similar endpoint, the check still picked up on it, and it flagged up, said, okay, well, we can see it's done this rule and this rule, but it's not checked this particular aspect. And it's those sort of issues that I think this is very powerful for, the sort of things where a generic tool is never, ever, ever gonna pick up on them, because they're very specific to how the organization works, or how the codebase works. But at the same time, even if you write a custom static rule, even if you can do it, it would be very, very complicated. Yeah.

**25:55 Robert Hurlbut:** It seems like one of the real sweet spots of LLMs being integrated with security tools is, and I think you alluded to it in your first example with authorization, it's really where there's more context that can be captured. You know, and I think about like business logic flaws. Static analysis has never been able to find business logic flaws because they're not predictable. They're just a, they're a, somebody had an idea in how they rolled out an implementation and they had a bad assumption or whatever, and it generated a way you can drop through the code that provides some insecure approach or some insecure setting or whatever. But it seems like LLMs are poised to be able to sniff out those type of problems in a way that a classic SAST would never be able to see. Is that kind of, am I landing in the right spot here?

**26:49 Chris Romeo:** Yeah.

**26:50 Josh Grossman:** Yeah, I mean, again, this is something I've been thinking about for a long time, just in terms of the custom rule perspective. You know, to the extent that one of my colleagues, Michal Kaminski, who used to work with me, she did a course on this at Black Hat a couple of years ago about, okay, building custom rules, both static and dynamic, to find these sort of issues. But again, these static rules have a limit, and once we bring in the AI aspect as well, that's where it becomes really, really powerful, because that's where you've got that additional reasoning, and you can pass them particular context. They can really provide a more reasoned example of, okay, is this an issue? Is this not an issue? But at the same time, again, I think there's a lot of— a bunch of tools I've seen will be like, okay, well, we'll find a particular issue, we'll spit out a markdown file, and someone will look at it. And one of the things that Agartha tries to do is to take that one step further and say, okay, well, it's nice to have a big text file with the result in, but we want to try and make this easier to build into processes. We want to try and incorporate this into the way that code analysis works to begin with. And so Ghast, by default, can spit out a SARIF file itself, which is JSON format, sort of well-understood format for a finding with code flow set out in a particular way that you can then use in other tools as well, or you can even use to feed into another tool, maybe some sort of remediation. But the idea is to make it easier to integrate into the overall process whilst getting this benefit of the this sort of AI ability.

**28:15 Robert Hurlbut:** Where do you see August kind of play out? Like, where do we integrate it? Is it a pipeline tool? Is it a command line tool that developers run? Does it run at code check-in? Like, where should it sit in your mind?

**28:33 Josh Grossman:** So the biggest challenge comes back to token economics, because once again, As soon as you're running an AI rule, you are using a certain number of tokens. And I think that AGAST does try and cut that down, but I think there's— I'd always be a bit scared to run it on every single pull request because it means you're burning tokens on every single pull request directly. Although again, one of the features I'm currently looking at is looking at focusing AGAST on diffs, on the diff of a pull request rather than everything. Again, I think that there is a risk there. You know, Ghast is designed to be used in CI and designed to be automatable very easily. You run it at the command line, pass it flags, pass it config files. You know, it's designed to be run in a non-interactive way that builds into existing automated processes, but you do have to think about the token cost as well. Although, like I say, you can also just run purely static rules in the Ghast as well. And that's something you can get out of the box straight away. And I think that you can get a lot of value just from static rules. Like I say, this client where they had a lot of authorization issues, some of the more basic ones I had static rules for, and I had a whole sort of suite of static rules that were, these rules run on this repository, these rules run on that repository, and out of the box, you can get that with a GAST as well. So you write the rules, and then you write the config that says what to run when. The only rule, and once you've done that, that's not using tokens. You can run that on every pull request. It doesn't take very long. You can run it.

**30:04 Robert Hurlbut:** Yeah.

**30:05 Josh Grossman:** periodically as well if you want. It's once you start using tokens, once you start using AI, you have to be a little bit more careful with that.

**30:11 Robert Hurlbut:** Is there a particular environment or code base or language or framework that Aghast is more in tune with, or are there things that it doesn't support today, or is it pretty much, it can ingest any code?

**30:29 Josh Grossman:** So, Currently, the sort of the primary discovery method for Ghast is Semgrep, just because I've used Semgrep a lot in the past. But even then, Semgrep has support for a lot of languages. So, you know, the static, you know, if you want to do the static discovery, you have to make sure the static discovery is going to be compatible with your, with the tool you're using to discover. And, you know, in this case, it's Semgrep. Semgrep supports a bunch of stuff. most common languages, so that should be fine. If you've got a language they don't support, it might get slightly more difficult. But other than that, I don't think there's any sort of particular preference or particular, you know, ultimately, if the AI provider can recognize it, the AI coding assistant recognizes it, then it knows how to navigate it, which I think that given the way these coding assistants are trained, I think that they're They'll always be more familiar with the more common languages, but I don't think that means they're unable to process other languages as well.

**31:31 Robert Hurlbut:** So I can run this with my COBOL, that's what you're telling me?

**31:36 Josh Grossman:** So you might have to do this discovery manually, but if you can feed it some COBOL, then I imagine the AI can provide some sensible—

**31:45 Robert Hurlbut:** If I could find something that I could run COBOL on, then I would go for it, but I don't even know where you would.

**31:52 Josh Grossman:** You have to ask Claude to build you an emulator for it.

**31:54 Robert Hurlbut:** Oh, now that's kind of cool. That could be my next startup idea. Everyone wants to do COBOL. Nobody does. Nobody wants to do COBOL.

**32:04 Josh Grossman:** It's, it's, hey, a lot of banks using COBOL or Fortran or both. I can't remember. You still hear stories about these.

**32:10 Robert Hurlbut:** Probably both, probably a hybrid of both integrated together just to make it more complicated. Uh, okay. So what's, what are you thinking about for the future with this? Like, what's your roadmap? Where are you gonna go?

**32:22 Josh Grossman:** So I think the— I've got a few sort of features on, sort of ideas. I think the diff feature is something I want to work on and try and get out soon, because I think it can make it a lot easier for those sort of pull request situations where you've only touched certain code, and you want to just get it to focus on that code, but also maybe focus on a little bit of the code in that call stack. So it's not literally just looking at lines of code that changed, but also potentially looking at lines of code that were impacted by the lines of code that changed. I want to look at other discovery methods as well. So like I say, the basic at the moment is Semgrep, but there are other ways you can do discovery as well, other static mechanisms I think could be valuable. And then I think one of the things we're talking about at the moment is whether we want to push this in as a WASP project, because right now it's open source, it's AGPL, because I don't want necessarily people building commercial tools around it, but That's still very compatible as an OWASP project, doesn't stop people taking it, using it in their own organizations, and benefiting from it. I think potentially OWASP is a good home for that. So I'm actually going to be at OWASP in Vienna next month, and I've got a training course I'm delivering there, which is one of the things it does is it uses August for part of that training course, because the course is about trying to get a handle on code security and sort of building a better process for code security, and August sort of fits into that stack as—

**33:44 Chris Romeo:** Yeah.

**33:44 Josh Grossman:** sort of from generic to custom static to custom AI. So Augusta is an important tool for that. And so it may be that by then I'll manage to speak to Oasman, see if we can get that set up. I think it's definitely valuable for the wider community. I also want to see the input from the wider community as well, see how people are using it, see what features other people are looking for. You know, I've got lots of ideas about how I might want to use it, but someone else might come along and say, you know what, like, I use this in my environment, it was really cool, but it'd be great if it did this. I'm hoping that if we move it to OWASP, we get better reach to actually get that sort of feedback and add features in that organizations need.

**34:20 Robert Hurlbut:** Yeah, yeah, definitely. Some of that user feedback is very important. Prevents going and building something that people don't actually use in the same way you envisioned they would use it. So I guess, what's a key takeaway or call to action for the audience here?

**34:39 Josh Grossman:** So I think I built Agast because, first of all, I could, and the inside code made it a lot easier, but also because it was a tool that I needed. It was a tool that I needed to feel confident about what was going on in the code bases that I needed to secure in a way that maybe certain commercial tools eventually will get to, but right now they're not there. And, you know, again, I've got specific problems that I'm worried about, and I wanted a way of actually making sure that those issues didn't occur, didn't recur, and I could feel confident about the security of the codebases I had to be responsible for. So, I think definitely, if you're responsible for code security in an organization, or you're worried about code security in an organization, or even if you have clients and you're auditing their code, and you want to try and find particular issues, then take a look at a GAFS, take a look at the way it works, figure out which sort of rule will work best for you, and try it out. You know, I don't think— there's nothing quite like it that's open source at the moment. There are things that are similar to it. Certainly Raptor and OpenAnt from Gadi Everon and Daniel Cuthbert, a few other guys, they're also doing similar things, but not quite the same. They're sort of looking at the wider codebase, whereas I'm saying for August, find targeted issues, find specific things that are worrying you, and make sure they don't recur. So I think definitely check out the tool. You can find the tool on GitHub. It's also available via npm, make it easier to install. There's a whole bunch of documentation. I recorded a whole bunch of videos showing walkthroughs as well. But I think we're seeing a lot more code being written, and we— I think that the idea of sort of shift left, I think there's certainly still a place for that, and figuring out how we make sure these coding assistants are building more securely. But right now, we've got a whole bunch of code, and we need to figure out how do we verify security within that code. And I think Agast is a very powerful way of doing that.

**36:34 Robert Hurlbut:** Very cool. Well, Josh, thanks for sharing this kind of update about Agast and walking us through how it works and everything. And yeah, hopefully our audience will go out, give it a test run, and give you some feedback. I think there are folks who aren't aren't shy with their feedback. So I think they'll check it out and give you some feedback.

**36:56 Josh Grossman:** Definitely, people who are interested in code security, like I say, apart from checking out the tool, I've got the course running at OWASP in Vienna in a couple months' time, and also slightly extended version of the course running in Black Hat in the summer. Again, about general code security, but the Ghast features as well, because it's part of that code security stack. So, if that's something that's particularly of interest to people, then definitely get in touch and look up those dates. Okay, cool.

**37:25 Robert Hurlbut:** All right, thanks, Josh. Cool.

**37:27 Josh Grossman:** Thanks so much. And yeah, great to be on again.

**37:29 Chris Romeo:** Now, we'll look at a video overview of AGAST that Josh put together.

**37:34 Josh Grossman:** Introducing AGAST, AI-guided hybrid application static testing. Agast is not designed to find vulnerabilities for you that you didn't know about. That means it's not really for beginners. It's designed to take your suspicions about potential vulnerabilities in your code and turn them into accurate validations. Agast lets you use a blend of static discovery and AI analysis to define your security checks, perform them in a way that is automatable, repeatable, and scalable, and get results in a structured and useful way. We built Agast because it was the tool that we needed to get our jobs done and the approach that we wanted to bring to the wider community. Let's see a quick example. In this example, I've defined a security check for an issue that concerns me, and I can run a Ghast to validate it. This particular check uses a Semgrep rule to find locations in the code that are potentially vulnerable and then uses an AI provider to validate whether the location is vulnerable or not. Now let's, let's speed up to the end. And in less than a minute, I've got my answers. I can now open up the SARIF file that's been created, look at the results that have come back from the AI provider of valid issues, and then go through the results in the code, taking into account the rationale the AI provider has given for why this particular location is vulnerable to the concern that I had. In future videos, I'll go into a lot more detail about how these checks work and the different types of check that you can prepare. In the meantime, there's plenty of documentation in the GitHub repository, including details of how the framework works and how you can use it, how to install and get started. There are a couple of tools you'll need to install depending on the types of checks you want to run, and you'll need an Anthropic API key as well for the AI-type checks. And most interestingly, we've got a bunch of example checks that you can use to try it out and understand how the different checks work. For now, you can find the tool at github.com/bounce-security/agaast, and you can install it directly from npm. Give it a try. Look out for future videos that go into more detail about how you different check types work.

---

Source: https://appsecpodcast.com/josh-grossman-ai-sast-is-it-a-match/