Adam Shostack -- Fast, cheap and good threat models

0:00Chris RomeoAdam Shostack is a leading expert on threat modeling and a consultant, expert witness, author, and game designer. He has decades of experience delivering security. His experience ranges across the business world from founding startups to nearly a decade at Microsoft. While not consulting or training, Shostack serves as an advisor to a variety of companies and academic institutions. Adam joins us to talk about fast, cheap, and good threat models. We discuss how Adam defines these categories, the weight of threat modeling, questionnaires and requirements, expertise, and how to make threat modeling conversational. We hope you enjoy this conversation with Adam Shostack. You're about to listen to AppSec Podcast.

0:44Adam ShostackWhen you're done with this, be sure to check out our other show, High Five.

0:47Chris RomeoHey folks, welcome to another episode of the Application Security Podcast. This is Chris Romeo, CEO of Security Journey and co-host of the podcast. I'm also joined joined by Robert Hurlbut. Hey, Robert.

1:01Robert HurlbutHey, Chris. Yeah, good to be here. Threat modeling architect and looking forward to our topic, which we've talked about before and we'll talk about again.

1:09Chris RomeoIt is the number one topic of all the topics that we've ever talked about, and that's because Robert and I get to choose the topics that we want to talk about. So, um, the— our regular listeners are like, yeah, we know it's going to be something about threat modeling, we can already tell. But that's okay. We're, we're joined by Adam Shostack, who most people— I'm gonna say everybody in the world of threat modeling has heard Adam's name before. This is Adam's 4th visit to the podcast. And so, Adam, as always, we are glad to have you here talking about threat modeling.

1:42Adam ShostackGreat to be back. Thank you.

1:44Chris RomeoSo I'm going to dive right in. And this topic that we're going to discuss today, fast, cheap, and good threat models. And so I don't know about all of our listeners out there, but I've been told my entire professional career that I can only have 2 out of those 3 things. And so, you're blowing my mind right from the very beginning of our conversation. So, set this up for us, Adam. Tell us what you mean when you say fast, cheap, and good threat models.

2:09Adam ShostackWell, I mean threat models that are fast, get them done quick, that are cheap, don't involve a lot of engineering work, and are good. And, and, you know, yeah, I named, I named the paper after the truism that you can only pick 2 of those 3. And I'll spoil the ending for everyone. The reason you can get all 3 is the comparison frame. So, if you're comparing fast, cheap, and good methods to no threat modeling whatsoever, You can get to all 3. And then as you want to get more out of your threat modeling, you get into those engineering trade-offs. Do we want it to be fast? Do we value fast more than we value good? And that's an okay trade-off to make. But so many people I talk to have never threat modeled their systems. And for them, you can get all 3. And I'm sure we'll talk about specific ways to do that.

3:15Chris RomeoYeah, I want to unpack these 3 words a little bit because when we think fast, cheap, and good, from a general engineering perspective, I think there's a common definition. But when we're talking about threat modeling, let's unpack each of these and say, okay, when you're talking about a fast threat model, what do you mean? Do you mean something that we can do in an hour or less? Is there an amount of time that you're assigning to something being fast? Like, how are you measuring Speed in this case?

3:47Adam ShostackWell, obviously in furlongs per fortnight, but when I say fast, I'm thinking about a methodology like just ask what can possibly go wrong, or just ask how would you hack this. That might be a 60-second conversation. That, that's how fast I think we can get. And I think we can get value out of having that 60-second conversation every time, right? Isar likes to— Isar Tyrendok likes to ask, threat model every story. An hour to threat model every story might feel too heavyweight to some people. And so instead of dropping down to 1 minute, they drop down to 0 seconds. I hate the dropping down to no threat modeling, and that's why I think fast and cheap can also be good.

4:47Chris RomeoSo when you're thinking cheap, are you thinking from a resources and time allocation perspective equating to dollars? And so it's cheap not because it doesn't have value, it's cheap because there's not a lot of effort that has to be put in to do it. Is that, is that where you're going with cheap?

5:05Adam ShostackExactly, exactly. People, everyone today, too many things to do, making trade-offs all the time about how do we get this feature, excuse me, how do we get this feature to customers faster? That's the metric is how much time does it take to do this?

5:25Chris RomeoAnd then the toughest one out of these 3, fast and cheap. I can very easily say, okay, well, we can do it in a minimal amount of time. And we can do it, you know, with a minimal amount of resources. But good, how do we measure good in this equation?

5:43Adam ShostackSo, so I'm going to quote Gary McGraw and say we, we use our badness-o-meter. We, we discover if we find bad things, and if we're finding some bad things that we wouldn't otherwise have found, then that's good. Right? That the value of getting started, of giving people a safe way to say, I'm concerned about this aspect of our design, rather than keeping that to themselves, that's good. Can we do better? We could probably do better than just giving them that space. We can use, you know, We can use structures like STRIDE, we can use structures like kill chain, we can get much better at it. But if all of that becomes heavyweight, then we never start and we miss out on any of the value that we aspire to.

6:45Chris RomeoI'd never heard that before. Certainly have heard plenty of things from Gary McGraw over the years. And, you know, as one of the— I mean, you can say— I mean, I'll say it, one of the founding members of the software security world, right? Like he was there right in the beginning. And so that's, that's interesting. It's almost like I could equate good in this case to a binary answer. It's either it's 1 or 0. Like if you, if you made some progress towards understanding your system and, and thinking of something that could go wrong, that's good. It doesn't mean you have to find 10 of those things or 100 of those things. Just one thing, just one slight movement is enough to say that something's good. Is that a fair assessment from what you're thinking?

7:27Adam ShostackYeah, yeah. Although I'd like to allocate more than 1 bit if we can, right? Maybe we can find 2 things and not overflow a buffer as we're doing it.

7:37Robert HurlbutSo we keep talking about and using that term heavyweight. Is threat modeling a heavyweight exercise or process? I know some folks think, well, it's just filling out a lot of forms and that takes so much time and so forth. How can we do all the things you're talking about? It really is heavyweight. What are your thoughts on that?

7:58Adam ShostackSo I don't think it really has to be heavyweight. I think it can be— and actually, let me just— let me pick up 2 words you just used. You used methodology and process, and I want to separate them just a little bit. So there's the methodology, which is what are we doing? It might be Stride. It might be data flow diagrams, it might be message sequence diagrams. And then there's the process, the, I have to create a diagram in Visio, I have to get it peer-reviewed, I have to check it in, I have to get it signed off on by security. And that's the process. And I think that where we often end up is that process designers observe failures in what's happening. And so they add steps to the process. Auditors say, but how do we know you did that? And so we add steps to the process. And so I think that oftentimes we end up with a lot of process that overwhelms the methodology and it overwhelms the value.

9:16Chris RomeoYeah, one of the things that I've said since I first got into threat modeling is one of our goals should be to put ourselves out of a job, right? Like, there's nothing— there's no better future that I can think of than the threat modeling process doesn't even exist anymore because it's just ingrained in how we build software. And 2 developers are sitting side by side, and one of them is like, hey, did you consider the threats of this? And the other person's like, Of course I did. That's how we build software here. Like, it's not that they needed a process and methodologies, and all those things are good, but it's really getting to the point where those are ingrained in their brains and just software is built and thought through and designed with security in mind. Now, I don't think we're there yet. I think we got a ways to go, but that's certainly in my mind, that's the utopian future where developers just understand security and think about it And we don't need process and methodology.

10:10Adam ShostackYou know, that's really interesting because I think that as these things evolve, we develop these ultra lightweight ways of doing things. So for example, it is now hard for me to write a bug that doesn't start with what I did, what I expected, what was the impact, and do I have thoughts on a fix? Right? And that's been pounded into me over the years. Similarly, you do a retro. What went well? What went poorly? What do we keep doing? What do we change? Where's the threat modeling version of that? So to go back to your little story, not just, of course I did, but yeah, I thought about this using this approach. I thought about the center of gravity, or I asked what's the worst thing that can happen. And super lightweight, super easy to do. But also that— Here's one more layer of what I did, because the way we build software here is always we file bugs that look like this. And I can tell the new hire because they haven't learned to file bugs that read like this.

11:34Chris RomeoSo what do you see then as the connection between kind of the example that you just shared about the bug, the bug kind of tracking and creating bugs, and then getting back to threat modeling? Like, connect the dots for me a little bit here.

11:49Adam ShostackSo I was really using that as just an example of how things become implicit, but I can see a threat bug being what I did, put a different machine name into the DNS. put RLO characters into the source code. And so what happened, the impact of the threat, what we're— what I think we should do about it is some mitigation. And so we can think about threats as bugs that haven't manifest yet.

12:23Chris RomeoYeah, I'm tracking with you now as well about the fact that, you know, you were tying the fact that, you know, opening a bug is something that is implicit in that it is, it's part of the developer's workflow. And maybe on day 1, you don't know how we do bugs here, but by day 30, you know how bugs get put into the system. It's built in. And so, a world where threat modeling is that same thing, you know, maybe you didn't know about threat modeling when you got there, but within 30 days, you're surrounded by people that are doing it. And so, you're learning by osmosis because you're just like, oh, wow, I'm I'm part of these reviews and people are asking questions about security and stuff like that. So yeah, I'm tracking with you now from that perspective. So let's— let me change our direction just a tiny bit. And so when we think about using a questionnaire and filtering what projects have to be threat modeled, is that something that we should be attempting to do? Should we be threat modeling everything? Should we be using some criteria that says we're going to threat model these things that are above the line and we're going to ignore everything below the line? Or what are your thoughts on that perspective? Yeah.

13:29Adam ShostackI don't want to— I don't want to argue with you, but I want to argue with you. And my argument here is that the questionnaire is a form of threat modeling. It is, what are you working on and what can go wrong? Right. And the idea is, for example, there's a line in the questionnaire that says, are you processing personally identifiable information? There's a line in there that says, are you processing credit card numbers? What are we working on and what can go wrong rolled into one question? And in the PCI case, what are we going to do about it? Oh, you're now in the PCI bucket of things.

14:11Chris RomeoRight.

14:12Adam ShostackAnd so if we think about method versus process, the questionnaire is a threat modeling process. And it's one that we all hate. So this is why I said I wanted to argue with you, is do we use a questionnaire to determine if we threat model? When you start thinking about a questionnaire as a threat model, you get into the space of let's now evaluate that method of threat modeling, that process for threat modeling. against other processes for threat modeling and ask, is it fast and cheap enough for the quality of results that we get? And I think a lot of times the reason we hate them is because those 3 knobs are out of whack relative to each other.

15:10Chris RomeoYou mean the 3 questions that are being asked? Is that what you mean by the 3 knobs?

15:17Adam ShostackWell, I mean, the fast, cheap, and good. It's not fast, it's not cheap, and it's not good.

15:22Chris RomeoTo use the questionnaire method. So, let me pose a slightly different thought about the questionnaire since we're kind of focusing in on that area. And you made that interesting point that the questionnaire itself is a threat model. So, when I've thought of questionnaires in the past, I think of questionnaires as being closer to a set of requirements. Because I'm asking a list of things like, you know, do you store personally identifiable information? And then usually the follow-on question is, if yes, comma, how do you properly protect that data at rest? And so when I think questionnaire, I think requirements. I don't think about a list of threats that somebody would have to consider. And it may be that I've just never seen anybody do it that way. So how do you react to that questionnaire being more of a list of requirements than a list of threats?

16:13Adam ShostackSo I think of, I think of threats and requirements as being facets of the same thing, right? So for example, when we talk about, are you processing personal information, the reason we're, we're talking about that question from security is because of the threat of disclosure or inappropriate use from a privacy perspective. And so we can frame it as, here's a requirements question. There's other things that will often go in, such as, are you using PHP? Are you using dynamic SQL? Which are more closely tied to vulnerability structures. So to me, it's we take the thing we're looking at, we look at it from different facets. If we want to call it requirements, I'm good with that. All models are wrong. Some models are useful.

17:24Chris RomeoYeah, and I'll give you, I'll give you kind of the example of how I got into thinking about requirements a little bit more. I've been doing some webinars, kind of just walking in more depth through the threat modeling process and trying to do examples. Like, we spend a lot of time as threat modeling teachers at events and things. We spend 30 or 40 minutes and all we do is introduce threat modeling. I'm like, I'm gonna start doing some things where I created a fictitious company and I just defined some features from the product manager's perspective and then showed, okay, as an engineering person, here's what I'm thinking.

18:03Robert HurlbutYeah.

18:03Chris Romeoyou know, with a security person sitting side by side with me. Here's the questions I'm asking when you show me this picture.

18:07Adam ShostackMm-hmm.

18:08Chris RomeoAnd so, I went through STRIDE. You know, we're all huge fans of STRIDE. But I also started thinking about and was doing a little bit more research on additional ways to consider what are the threats, what are the things you should consider after STRIDE. And I found some, and I don't even know who the original source of this is. I wish I did. I couldn't find it in the OWASP world, I found a link on a very old OWASP page where somebody was describing this idea of using requirements as an input into the threat modeling process. And so I started playing around with ASVS, Application Security Verification Standard, and saying, hey, we love STRIDE because STRIDE's a methodology, but it's a teaching methodology, and you should eclipse STRIDE. Like, it's a good thing when you're like, okay, STRIDE, I'm pushing you out of the way now because, you know, my brain is processing Stride, and I'm going to look at other things. And I said, putting ASVS as an input into that is just an interesting way, especially trying to solve the problem where you have developers who maybe don't have the security knowledge that those of us that are part of this interview do. Like, we've been doing this for a long time. We can look at a whiteboard picture and go, ooh, boom, 1, 2, 3, 4, 5. Oh, let's find some more.

19:19Robert HurlbutCome on, come on.

19:19Chris RomeoBut when you're a developer and you have Stride under your belt, like, ASVS is kind of that next level. And so, that's what's kind of gotten me thinking about requirements. And I'm curious what you guys think about that. if that's something you've ever used as an input into the way you approach teaching threat modeling.

19:36Adam ShostackLess on the teaching side, and I'm curious about Robert's experience teaching too, but I'll, I'm just going to jump in and say there's a, there's a chapter in my book, which is the requirements cookbook chapter, because I think that really getting to good requirements helps you figure out what threats matter. And it's— I've always found it hard to write good requirements. And so the idea of the cookbook is you have these slightly varied statements to show where the trade-offs might be in the hopes that that helps. And it's not a chapter I get a lot of feedback on, so I don't actually know if it's working for anybody. And I don't tend to teach requirements when I when I teach threat modeling, because I don't— because I feel that most organizations I'm working with don't have a strong requirements process. And so trying to impose one is sort of the opposite of this fast, cheap thinking. Robert, how about you? Do you teach requirements very much?

20:47Robert HurlbutOh, and I was thinking similar. I always think about requirements sort of after. Right? It'll lead me to the requirements. It'll help me understand what the requirements are. I mean, I might have an idea. I know I'm building something, and that's what we talk about in the first question in the 4-question— what are we working on? But beyond that, you know, knowing what the actual requirements are, that's what the threat model is going to help you with. So I, yeah, I don't really head that direction.

21:17Chris RomeoSo in regards to the requirements, I'm not as much teaching requirements from ASVS as saying, hey, ASVS is a source to think about other things. So, translating the ASVS requirement into a threat and saying, for those people that are new to this and they don't have the mental models of all the things that we've done where we've seen different threats, and so we can just see them jump off the page. It's almost like a— I don't know if it would be a good or an evil gift. I guess it's a little bit of both. But, you know, that's what I'm thinking is like, and because people have asked me, and I probably asked you guys the same thing, it's like, where do we, where do we go? Like, once we get Stride, like, what's, what's next? Like, what do we, what do we do with that? And I guess, Adam, how would you answer that question then, just to kind of close this one out and then we'll move on? But if somebody says to you, hey, our teams, we started with Stride, we loved it. Like, we feel like we understand it, we know it. But the devs are asking me like, What do we do next? What's the next body of knowledge that we should consume to be able to know more about threats?

22:20Adam ShostackMy usual answer is kill chains. Using a kill chain to think about, okay, as this goes operational, what's going to happen? And then overlay protect, detect, respond onto it as well. So for each stage of the kill chain, how do we protect against it? How would we detect it? How would we respond if it happens?

22:47Robert HurlbutOkay.

22:48Chris RomeoSo, is that— have you ever looked at like the ATT&CK methodology, for example, which I think kind of that— I'm not super familiar with ATT&CK. I think I probably know enough to be dangerous. Like, it is a representation of the kill chain though, right, with a number of other types of things. And so, does that— are you saying that— does that influence kind of the knowledge as well, or are you talking about just the kill chain at kind of like the high level? process.

23:15Adam ShostackLibraries like ATT&CK that make the kill chain concrete and provide variants are a great teaching tool, right? You've got to start, and you're doing this when you provide not just, hey, we're going to talk about threat modeling as this abstract set of things, these questions. Here's my company and here's threats to it, makes it Understandable. That waterfall of tactics within ATT&CK saying, here's how someone persists. They write to bashrc, they write to crontab, they put in a browser extension. All of those are persistence techniques. That's amazing teaching fodder. of here's the concrete thing, here's the concrete thing, here's the concrete thing, and they all fit into this bucket. Oh, what else fits into this bucket? What have you seen? What can you imagine?

24:21Robert HurlbutThat makes sense. But let's talk about what do you need to know to do threat modeling? And that sort of brings up the idea of expertise. Do you need to be an expert to do this stuff? I've heard that several times. Well, we can only do this when we have our expert here so we can get started, which I, you know, but what are your thoughts?

24:45Adam ShostackSo, so the whole idea of fast, cheap, and good is in opposition to that, right? Just ask what can go wrong. Play the fortunately, unfortunately game. Ask what's your center of gravity? None of these require an expert. You can get started, and sure, expertise helps in a lot of things, but hey, I read the news today and there's a lot of security stories in the news. Pick one of those up that you saw a headline— I hate to say it— pick one of those up where you saw a tweet, um, about repudiation attacks against Zelle, or elevation of privilege attacks by the NSO Group and run with it and say, what does that teach me about this system? Saying I must have an expert, I find it a little frustrating, honestly.

25:50Chris RomeoYeah. I mean, it's something that doesn't scale as well, but you mentioned the fortunately-unfortunately game, and then you did not define what that is. So please, please tell me what you mean by the fortunately-unfortunately game.

26:03Adam ShostackWell, let's be concrete. Unfortunately, I didn't tell you what it is. Fortunately, I can demonstrate it. Now, one of you want to pick up with an unfortunately?

26:17Chris RomeoI was giving Robert the opportunity since I asked the question.

26:20Robert HurlbutYeah.

26:23Chris RomeoI mean, we could do it at a super high level, right? Unfortunately, there are attackers all over the internet waking up and going to work today. But fortunately, we have threat modeling as a tool to be able to help us find design-related issues early in the process. So, okay, I got it now. That's cool. I see how that's a simple— like, that might be the most simple version of threat modeling. I mean, the 4 questions is simple. The 3 things, the fast, cheap, and good threat models you're talking about are good. But fortunately, unfortunately, that's almost like something you can kick off as a conversation over lunch when you're not— you're kind of teaching, but you're not teaching. Hey, let's play this game with you. That's an interesting thing that I don't know. I'd never heard it described in that way.

27:05Adam ShostackFast, cheap, good.

27:07Chris RomeoTo kind of bring all these things together, Adam, we, all of us have heard at different times where people will say this threat modeling thing, it's just, there's too much to it. It's too heavyweight. There's too many steps, too much process, too much methodology. And we've talked about that in kind of different facets of it, but let's really just bring it all back together. Should we be thinking about threat modeling from a kind of structured and ritualistic perspective of, you know, do these 12 things and have this thing spit out at the end and go through all these things? Or, you know, do we really need to be thinking about more of this conversational approach, whether that's the, you know, just asking the 4 questions over lunch, whether that's the, you know, playing the unfortunately-fortunately game or, you know, whatever? Like, what are your thoughts about— kind of how we bring this all together.

27:54Adam ShostackFortunately, we have these structures and methodologies like STRIDE or kill chains and the 4 questions that give us a lot of ways to be structured and systematic in how we threat model. Unfortunately, those make threat modeling heavyweight, slow, and expensive. Fortunately, we've got faster, cheaper ways that we can do it. And when we contrast the inhibition, right, that because, you know, for example, I teach threat modeling courses, some of them are multiple days. And so if people come away from that with this belief that to threat model, as Robert said, you need an expert and they have to have been to a multi-day class, what we end up with is no threat modeling.

28:49Chris RomeoRight.

28:50Adam ShostackAnd that's the most unfortunate thing, where our attempts to do good have led to this situation in which nothing happens because we scare people away. And so I don't want to give— I don't know what's right for our listeners' organizations, for their situations.

29:16Robert HurlbutYeah.

29:17Adam ShostackWhat I'm trying to do again with this white paper is give people another tool with which they can slice their threat modeling problems and develop ways of meeting the challenges in front of them.

29:34Chris RomeoSo what's the call to action then that you would offer to our listeners here? You know, we've been for a little more than 30 minutes talking about You know, fast, cheap, and good threat models. But what can our listeners do to put these things into action?

29:49Adam ShostackSo there's a white paper at showstack.org/whitepapers entitled Fast, Cheap, and Good. It's free. You don't even have to lie to me about who you are to download it. Grab the white paper, give it a read. More importantly, give it a try.

30:08Chris RomeoVery cool. And Adam, I have to— I'm always just amazed when I get a chance to speak to you because I feel like I'm somebody who studies this discipline and thinks about it a lot. And you always just blow my mind. Like, every time we talk, I'm like, I learn multiple things and learn different things that I hadn't even thought about. And so, I just always enjoy having you as a guest on the show and any chance I get to talk to you. So, thanks for being here again to talk about fast, cheap, and good threat models. And folks, go download— just download it. You don't have to put your information in. Just go to showstack.org.

30:38Robert HurlbutYeah.

30:41Chris RomeoDownload that, give it a read, and, uh, yeah, just put these things into action. And so, Adam, once again, thanks for being here with us.

30:48Adam ShostackAlways a pleasure.

30:51Chris RomeoThanks for listening to the Application Security Podcast. You'll find the show on Twitter @AppSecPodcast and on the web at www.securityjourney.com/resources/podcast. You can also find Chris on Twitter @edgeroute and Robert @roberthurlbut. Remember, with application security, there are many paths, but only one destination.