The Threat Modeling Manifesto – Part 2

0:02Chris RomeoThe Threat Modeling Manifesto took many hours to create. For almost 6 months, a group of experts met to debate and discuss. Their goal was to create a usable definition for threat modeling, one that actually worked. In the first episode, we heard clips taken from over 18 hours of recordings. Eventually, the team landed on a definition of threat modeling that everyone could agree on. What is threat modeling? Threat modeling is analyzing representations of a system to highlight concerns about security and privacy characteristics. At the highest levels, when we threat model, we ask 4 key questions. 1, what are we working on? 2, what can go wrong? 3, what are we going to do about it? And 4, did we do a good enough job? Next, the team moved on to the question of what the values and principles of the manifesto should be. Once again, our guide will be Chris Romeo, the co-founder of Security Journey and the host of the Application Security Podcast. Some of the members of the threat modeling group here that wrote this document have literally written multiple books on this same subject, hundreds and hundreds and hundreds of pages. So it would have been easy to say we have a 50-page limit and we're just going to keep writing and writing until we feel like we've, we've covered everything that we want to do. So it was definitely a challenge to say, How can we cut this thing down to the core, most important foundational things that matter and really make that a short resource? Because we realize in the modern world that we live in now, a 50-page document is not going to get a whole lot of attention. And just on that point, I think when Chris and I started the principle consolidation, it was really around duplication of wording, duplication of meaning. and eliminating extraneous sentences. Hopefully we didn't eliminate true meaning. We wanted to have the Threat Modeling Manifesto be something that could be printed on a poster, a one-page poster, and hung up on the wall inside of a company somewhere, because here's what we're gonna drive towards as we're doing threat modeling. If we had a 50-page document, we just wouldn't have that tight description of what we really value. What are the real principles that we think are fundamental that have to be there? And so yeah, it would be easy to write 50, 100, 250 pages on this same topic, but it was a challenge to, to get it down to the manageable size that it is now. But then, and that's a result of all of the effort and time the team put into discussing and debating and taking things out and arguing about putting things in and arguing about the words to get it down to what's the most simplest set of words that we can use to still get the most important points across. Irene Michelin.

2:48Robert HurlbutI found when I'm talking to teams, it helps when I make an analogy to cooking. So you teach Stripe as a sequence of steps and you say, this is a recipe. You follow it, guaranteed to get something edible and actually tasty in the end. Will you get better with practice? Will you learn to make it more delicious and whatever?

3:14Chris RomeoYes.

3:15Robert HurlbutBut at the moment, it's a recipe. You follow it, it will be good. So this is how I see it.

3:21Chris RomeoSo we are in the week of September 4th, and I would say that we are not at this point timing-wise, right? So we have not really finished the principles draft. I think we're set with the values, so I don't know that This date is probably going to be likely achievable. I just wanted to call that out. So today's the 4th, it's the holiday weekend. We need to finish up the principles, package it up, and then get the website updated, have everybody review it, get the final PR contacts and the pitch deck built once we have all the content together. So, you know, my, my first reaction is, is we're probably at least 2 weeks beyond the 18th. I don't know what everybody else is thinking, but it seems like we're a couple of weeks behind. Or now, again, I think, Brooke, you mentioned this way back. This was an aspirational date, right? And we were trying to hold ourselves to this. I just think that right now, you know, trying to rush to the 18th is probably not beneficial. And I just think that we're probably 2 weeks beyond that. Next, the team moved on to talking about measuring outcomes in threat modeling. And actually, not just gaps, if you think about it. Written material, documentation, source code, diagrams may not convey the correct or the consistent meaning, right? Somebody may look at a picture and derive 2 different things from it. So it isn't just gaps. It is to get a consistent understanding, which we have earlier, a common understanding.

5:04Robert HurlbutI think even if there's something doesn't actually exist in reality and the document's the only thing you have, I think the likelihood is that there are still going to be some discrepancies between what's in people's heads, either the author of the document plus key stakeholders in the document and the document itself. There's always, unless you spend all of your time writing documentation, there's always going to be a discrepancy between what's in the minds of the people and what's represented in some sort of artifact. And the quickest way to tease those out is by constantly updating documentation. Which I think goes back to Matt's assumptions.

5:42Chris RomeoOkay, well, I think we've got enough on this to work with, enough input from the team. Let's go ahead and move on to number 11, which is progress from threat modeling is measurable. Measure progress of threat modeling by counting the number of security and privacy bugs.

5:59Robert Hurlbutdownstream.

5:59Chris RomeoAnd then Fraser had— I made a sub-bullet here, which was Fraser's comment about an example that could potentially be part of the principal example of how this plays out in the real world. So thoughts on measurability?

6:10Robert HurlbutThere's the age-old problem of correlation and causation. If you have fewer bugs, if 50% of your teams are threat modeling, 50% aren't, and 50% of the teams that are threat modeling have fewer bugs, you can probably draw some sort of causation between that correlation. But Otherwise, there's a lot of that stuff, audit findings, bugs, all of that kind of stuff is a little bit tentative. But I don't know, there are very— threat modeling is very hard to measure in objective ways. Yeah, so from a privacy perspective, I wouldn't talk about bugs either. It would be like violations or something, not necessarily something that's wrong. Well, it's also partly wrong in the code, but it's more the end result. So if we can get like a more generic term, that would be great. What if we just generalize this completely and just suggest measuring threat modeling success with downstream metrics and just leave it at that? Do we need anything more specific than that? Because that's really what we're trying to say, I believe, right? Is just that, hey, you do threat modeling early on and you should be able to measure the impact of it by looking at how it how it has an effect on your overall development and the security posture of what you're deploying. So if we just, you know, one thing that I have learned just in recent research with vulnerability management metrics that I'm doing for another project is just how different the metrics that are tracked by different organizations are, and none are more or less valid than others. They're just very disparate. This is a recurring conversation I have at work is you come out of a process with 100 vulnerabilities, that's probably not great. If you come out of a threat modeling process with 100 threats, maybe you're just really good at finding threats and you've got a lot of stuff that you can do about it. You know, that's very positive. So it's really hard to sort of directly measure threat modeling outputs that way, whereas vulnerability management, all of that stuff in the pipeline is a hell of a lot simpler.

8:14Chris RomeoYeah, when I think about the return on investment for threat modeling, and that's, that's really one of the important things that we always have to consider, it's one of those situations where it's hard to prove a negative. So threat modeling, we know from an experienced perspective, those of us that have done a lot of threat modeling, we know that we have prevented vulnerabilities from making their way into production systems because we've threat modeled something and then we've pinpointed the fact that there's a gigantic security feature missing from this design. And without that security feature, this thing, when it's released into production, is going to be very weak and be very easy to knock over. So when you think about the return on investment of doing threat modeling, it's really preventing vulnerabilities in the future. And every company out there that is of any size can tell you how much a vulnerability costs for them, like what they have to pay each time they have a major vulnerability. And it doesn't take very long to get to the return on investment of one vulnerability with the time that you're gonna invest in teaching people about threat modeling and getting them to be passionate about this manifesto. And one of the other things to think about when we think about threat modeling and manifestos and things, we want this to be a guide that starts the conversation. We want companies and organizations to get to the point where 1 year, 2 years in the future, they're like, we don't even really— the manifesto is a document we looked at, it helped to guide us, but we've made threat modeling a core part of what we do to the point where we remember that was our lineage, that's where we got a lot of good ideas from. But it's not like they're using the Threat Modeling Manifesto on a day-to-day basis. They used it to guide their program. They taught people to threat model. People are doing threat modeling now. That's the future we're looking for.

9:49Robert HurlbutWe never stressed like the need for systematicity or structure, which I think is one of the main benefits of threat modeling. So I like to have those keywords in there somewhere, whether the description here is already something, well, it needs work. But, and also maybe you guys have a different opinion there because from an academic perspective, that structure and that systematicity really adds value and also helps in automation and so on, but maybe practitioners will say, well, yeah, it helps a bit, but we still need all that creativity around it, so it's not that important, maybe. Yeah, for me, a systematic approach is very useful, especially when you're getting started, but what it can do is amplify sort of biases and blind spots. So you need some ways in to provide sort of novel thinking that challenges, assumptions, and methodologies. I think it's certainly, from an maturity model perspective, a great starting point and a great backbone, but you then need flexibility around it. I think these all fit with our definition of principle that's up at the top of the document, too. If you just look through that, these are fundamental truths about threat modeling as we see them. The whole concept of a manifesto is, this is how we see the world. This is how we Feel it needs to be presented. The whole thing could ultimately be arguable. Yeah, but, but we believe it's a fundamental truth, so I see no problem with it. So maybe I should rephrase. They should be not arguable that they are inspiring. That would solve the problem. Like, and that way it will also solve the fact that if we go with an approach that is more prescriptive, we're writing a policy on, let's say, threat modeling at some point. It might have actually felt like that, and that's why I actually cringe when I look at the first write of what we did for the principles, because people just want to be free and thrive. If we come with something in the same vein of what Agile Manifesto did, I think we're not like those boomers anymore. We need to go with the new flow that goes with Let's just take this as its experience. It's like what we see as a group, you know, it's intelligence and experience that we give it to you and then you do whatever you want. But still, if you try to argue like the truthfulness behind this, like I think Irene just said, then it might be because you don't think it's expiring. And then that, that could be a like a way of reworking each point. And so, but at this point, I really also like the fact that if we steer away from what exactly the Agile Manifesto is, like, I think that's ironic because we're creating a manifesto and we're maybe afraid of not following what others have exactly done.

12:56Chris RomeoExactly.

12:56Robert HurlbutWhile at the same time, you should be free and do whatever the hell you want. So, I mean, at this point, let's just do whatever the hell We feel like is better.

13:05Chris RomeoIsar Terandosh.

13:06Robert HurlbutIt was a model of what good looks like. It's not a line-by-line thing that we have to follow. It's not a recipe. Yeah, yeah. But if you set people into arguing with you, then it becomes just rhetorical banter, you know? Like, if you come in— I know that because that's my problem in life. I always come— I always used to come way too strong. And so I would get in arguments just because I'm just coming in too strong.

13:34Chris RomeoSo, Alyssa Miller.

13:36Robert HurlbutI mean, at the end of the day, you have to have conviction. If we're going to do this, there's got to be conviction behind it. And if we're going to try to placate everybody, that, that will come through as a lack of that conviction. And then there's kind of like, what's the point? The fact of the matter is we've got, what, 13, 14, whatever our numbers are. Seriously smart individuals, plus me. I'm not sure why I belong here, but who all do this all the time, right? I mean, and we've all got very strong ideas for how to make this work. I don't, I don't care if we upset people. I don't want to set out to upset people. I want to set out to be genuine in what we're communicating, that it fits with what we truly believe. And if that ruffles people's feathers, Great, let's do it. But yeah, I you know, and that's I think the fine line between trying to you know be strong and be clear about what we believe and what we feel versus you know just kind of going attention grabbing and you know trying to market this thing, which really shouldn't be our goal.

14:43Chris RomeoIrene Michelin.

14:44Robert HurlbutOkay, the goal of the manifesto is to describe what we think threat modeling is, and you can describe things by explaining what it is. But also sometimes it's helpful to just to explain what it isn't. And what I hear is people will get offended not by specific words, but by us telling them what they do is not quite kosher threat modeling. Is that the main problem?

15:09Chris RomeoAlyssa Miller.

15:10Robert HurlbutThat's what I was hearing. I mean, I thought that was our concern, was that by saying threat modeling is not this, that people are going to get offended because that's exactly what they are doing. Well, I mean, you know, Communist Manifesto didn't apologize to people who were practicing other forms of socioeconomic government to— yeah, that's the whole point, is we're saying, you know, there's something wrong here. If there wasn't something wrong, if people weren't doing things we disagreed with, we wouldn't be writing a manifesto on the right way to do it.

15:45Chris RomeoWhat people might see and get offended by. because they see it as doing it wrong, but be nervous and confused by because they— because of what they've been taught or what they have come to understand is now wrong. And so that was my concern with the word hobbled. For those who are using adversaries in threat modeling, to say it's hobbled, they may be confused if they're not mature enough, as opposed to some who may be offended because their whole methodology relies on it.

16:12Robert HurlbutYeah, so I, I will vote against because I, I think that that reduces the, the power of the original principles and, and, and makes them nice-to-haves. But maybe that's because there are some of the principles in there that I kind of feel strongly about, and I, I would feel bad that they are not principles anymore. But, um, well, Kim, let me ask you this. Let's not say one or more.

16:39Chris RomeoAre there any in either of these statements that you feel we should tease out and put up in those principles stated in a different way?

16:47Robert HurlbutRemember, we, we have complete control here. We're not bound by this. This was just an idea. If there's something in there that you think we should say very definitely, this is the way it is, pull it out and stick it in the, in the principles. Grab any one of them.

17:03Chris RomeoI certainly learned a lot more about threat modeling along the way. Walking on this path with this collection of experts from all the different various backgrounds, academia, commercial trainers, authors, I learned a lot about the process. I also learned how to do something like this where there's a collaboration between a lot of really smart people. I learned how to do that in a way where you get to the end and everybody is happy with the final product, but also still friends at the end of the conversation. And so I'd say that's one of the big things I took away from this is You can work together with passionate technical people that want to work towards a common goal, and everyone's opinions and feedback and everything can be heard in that process. And you can come up with something really awesome at the end, and you can still be friends when you're done working on the project.

17:55Robert HurlbutThat's my point. We did that, and that were these principles. And now we remove them again in chunks and saying like, these are principles and the others are additional things that you should think about. So, so my feeling is that we are now seeing that the only the first, what is it, 4 or 5 are really essential and the rest is like, well, if you have time, maybe have a look at it. And just not to beat a dead horse, but what is wrong with having 9 principles or 10 principles instead of having 5 principles? Because we are losing also structure. So the last 4 are negative. We will really struggle to express them as principles. When we group them and say these are bad things, don't do them, then it's sort of one principle. We consider these to be bad things. And then you want a symmetry. So yes, the form was restricting us, but, but also it was the good logical form. But doesn't that mean that you can just have principles and anti-principles. Exactly. Anti-patterns. Why do we have these? I mean, you can negate the benefits or negate the hobolds things and turn them into benefits or the way around. Why do it this way? When I want to understand what this new thing is, it helps when it's explained what it is and also what it isn't.

19:28Chris RomeoYeah, I think we might be onto something here with the idea of patterns and anti-patterns. In that, so one proposal would be, we have this first section, we follow these guidelines or these principles. The second one is, we recommend, not even recommend, we endorse these patterns for threat modeling. And then the third one is we acknowledge these collection of anti-patterns, or we recommend you look at these closely or something like that. So that would be kind of one proposal. Another proposal would be, if you think about the hobbling ones, those aren't principles. So if we turn these, if we made these into a single list of principles, it's not gonna be 15, it's only gonna be 9 total. I don't know, again, I disagree with that.

20:16Robert HurlbutI don't think that this is speaking to methodology at all. This is speaking to the absolute core of what threat modeling even exists for. I mean, we're not telling them how to go about answering these questions or anything. We're just saying, I mean, like number 2, and Adam knows this, that, I mean, I use the Timmy Turner picture, what could possibly go wrong, in my conference talks when I talk about threat modeling. That's exactly what threat modeling is for. The goal of the manifesto was to become better threat modelers, and I was wondering whether that was really the end goal we had envisioned. So the input that we got was about securing systems and being more effective. So this was combined in this new sentence, but I think there are already some new comments about this sentence that I did not really look at in detail. So basically the question is, what is the goal of the manifesto? Why did we write it?

21:15Chris RomeoTo help people be more effective.

21:19Robert HurlbutThat's my goal.

21:20Chris RomeoTo encourage them to threat model at all.

21:24Robert Hurlbutand to share what we learned in the effort to do so. Yeah, there's a lot of—

21:29Chris RomeoI mean, Isar, that's an important point.

21:32Robert HurlbutThere's—

21:33Chris Romeowe say in the, you know, about the authors, there's a lot of experience here. There's a lot of experience encapsulated into this, into what works and what will kill things. Maybe that's important to highlight here. Fraser Scott.

21:49Robert HurlbutI think many, I mean, at least within our sort of walls, you know, we face a number of forks in the road about how to adopt threat modeling as an organization. And I think there are some very clear, distinct good ways and some very clear, distinct bad ways. And I think for me, this manifesto is highlighting the good ways about how you set that vision and how you shape your strategy, which is what I've blurbed about in that comment. Yeah.

22:16Chris RomeoSo here are the values of the Threat Modeling Manifesto.

22:20Robert HurlbutWe have come to value a culture of finding and fixing design issues over checkbox compliance.

22:28Chris RomeoPeople in collaboration over processes, methodologies, and tools. A journey of understanding over a security or privacy snapshot. Doing threat modeling over talking about it. Continuous refinement Over a single delivery.

22:42Robert HurlbutWe follow these principles: the best use of threat modeling is to improve the security and privacy of a system early and frequent analysis. Threat modeling must align with an organization's development practices and follow design changes in iterations that are each scoped to manageable portions of the system.

23:02Chris RomeoThe outcomes of threat modeling are meaningful when they are of value to stakeholders. Dialogue is key to establishing the common understandings that lead to value, while documents record those understandings and enable measurement. Anything else anybody wants to say before we wrap?

23:24Robert HurlbutJust thank you for me. Thank you for the opportunity. Thank you for all the great work. Yeah, it was really exciting. It was wonderful to meet all of you and work closely with all of you. I have a very bold, let's say, statement that I almost never say. I am satisfied with the result. Thank you, everybody. This happens once in a blue moon. That is legit. That's a huge achievement.

23:49Chris RomeoWe are honored, Jonathan. Yeah, I'm really pleased.

23:53Robert HurlbutI, I love the work that y'all have done. I can't believe we created something like this. It's really cool.

24:04Chris RomeoThanks so much for joining us. You can learn more at threatmodelingmanifesto.org, where you can read a full copy of the manifesto and learn more about each member of the team. The authors of the Threat Modeling Manifesto are Zoe Breiderman, Adam Shostak, Jonathan Marcel, Steven DeVries, Irene Michelin. Kim Vutz, Robert Hurlbut, Brooke Schoenfeld, Fraser Scott, Matthew Coles, Chris Romeo, Alyssa Miller, Isar Terindas, Avi Duglin, and Mark French. The working group would also like to thank Lauren Kohnfelder and Sheila Kamath for their technical edit.