The Threat Modeling Manifesto – Part 1

0:02Chris RomeoIn June of 2020, a group of experts began a weekly meeting online.

0:07Robert HurlbutTheir goal was to create a definition for threat modeling to encourage people to threat model and help them succeed.

0:13Chris RomeoAs online threats continue to multiply, designing for security has become increasingly important.

0:22Robert HurlbutBut while the term threat modeling has become in vogue, not everyone agrees on exactly what that means.

0:28Chris RomeoSo some of the heaviest hitters in the industry decided to change change that.

0:32Robert HurlbutThe Threat Modeling Manifesto Working Group contains some of the biggest names from the corporate and academic worlds. Some have literally written the book on threat modeling. So the challenge was to boil decades of combined experience down to a simple definition. Over these next 2 episodes, we're going to take you inside those meetings. We've taken clips from over 18 hours of recordings to give you a sense of the intense debate that occurred each week. Our guide will be co-founder of Security Journey and the co-host of the Application Security Podcast, Chris Romeo. Hi, my name is Chris Romeo, and my role in this process has been as a contributor. I was one of the early people that gathered together to start thinking through this whole idea of a Threat Modeling Manifesto, and then we got to see the group blossom into the large group that it became.

1:25Chris RomeoHey, Mark, how are you?

1:26Robert HurlbutNot too bad, how are you?

1:27Chris RomeoGood.

1:28Robert HurlbutHey, Matt.

1:28Chris RomeoHey, how's it going?

1:30Robert HurlbutI thought this call was actually yesterday, so I dialed in at noon yesterday and was like, why is there nobody else on? So did you get a lot done in that process? I did.

1:41Chris RomeoI had—

1:41Robert Hurlbutit was easy to make decisions. Everyone had a great opinion. So you're done? There was a small group of us that met in the beginning. It was Mark French, myself, and Adam Szostak. And then we reached out to Isar Tarendok and Matt Coles, who had also been thinking about this same idea about doing something with a threat modeling manifesto. And then what we did is we went through all of our networks and we said, how can we build a group of diverse individuals that have high knowledge and high levels of experience with threat modeling and bring them all together to try to achieve this common goal?

2:20Chris RomeoI think we have a quorum. I would suspect. So, indeed, why don't we get started? We have 2 pieces of old business.

2:27Robert HurlbutIn creating the Threat Modeling Manifesto, the problem that we as a group had all seen is that there was no good definition of what threat modeling actually is. There is no good definition of what are all of the things that are important to consider when someone is new to threat modeling. And so our goal here was to say, how can we provide a document that helps people to be able to understand threat modeling and then guides them in what they need to do to actually go after it.

2:57Chris RomeoThat's a good statement. That's a really solid statement. People are at the center of threat modeling. So as I look at this, and this is going to sound a little nitpicky, but it's not intended as such. When we say starting a threat model, we're implying that threat modeling is a deliverable rather than a journey. And so 2 things come out of that in my mind. The first is that I want to change the principles, the first 3 principles to be starting to threat model. But also I want to add a value, which is threat modeling as a journey rather than threat modeling as a deliverable.

3:42Robert HurlbutThe state of threat modeling, from what I see as I look across the industry, is there's a lot of different methodologies for how people are doing threat modeling. there's not really a good standard or a definition of what threat modeling is and what are all the pieces. So, when I think about the way different people approach threat modeling, I see some groups of people are doing threat modeling or they're claiming or saying they're doing threat modeling, but when I look at it, they're not going through and really getting to the results. They're not getting to the mitigations, the most important things that exist in threat modeling.

4:16Chris RomeoMatt Coles.

4:17Robert HurlbutThat notion that threat modeling is only done because there are attackers or because there are threats. Right, we do this as a proper— as a part of systems development and of secure— as an action under, I think, under an action under secure design as a means of validation, just like we do code analysis.

4:36Chris RomeoWe don't know that there are threats against code, but we do it for code quality and other reasons, right? That's my view, obviously.

4:46Robert HurlbutSo I would be careful about saying that until there are no—

4:51Chris Romeoyou know, we don't We only do— if we only do this because we know that there are threats, when threats go away, we would still be—

4:58Robert HurlbutI think we would still continue to do threat modeling.

4:59Chris RomeoAlyssa Miller. Yeah, I mean, I'm, I'm almost— this might be a bit inflammatory to say it this way, but I'm almost terrified at the perspective of making threat modeling so narrow that it only focuses on security implications. I mean, threat modeling is one of those things that has, you know, application from a development perspective. It has application from a business perspective. And that's where you get the value out of it, is that more holistic approach. So if we're going to say threat modeling as we're going to talk about it is only talking about security threats and nothing else, I think we're, we're really tying ourselves into a pretty tight scope there that the end result is whatever we create out of this, I don't think would be that valuable. I was going to say, I was going to go back to this timing issue because the problem I have with modeling— I mean, you want to start modeling before code or before implementation. That's true. But I have seen a lot of really successful modeling that benefits from agile iterative development. So you think you know what you're going to talk about, in your threat model, and then somebody starts implementing and says, wait a minute, that security thing won't work. But how about if we do it this way? And you change your threat model. So there's a natural interaction, and in DevOps, there should be a natural interaction between the creative processes involved in modeling and the creative processes involved in implementation. It's not one or the other. They're tied together, I hope.

6:38Robert HurlbutYeah, we wanted to be very careful to ensure that we had the best possible definition that the most people on Earth are going to be able to get behind. Because there is a lot of debate about what threat modeling is. There's a lot of different schools of thought that are driven by methodology. And so, our goal here was to say, okay, we have a very diverse group of individuals who are from academia, they're from the corporate world, they're trainers, they're people that have written books. We brought all these people together to say, Can we agree on a definition of what it is? And I can tell you, there was a lot of debate, there was a lot of discussion, there's a lot of back and forth about what threat modeling actually is. But we, as a diverse group of people, were able to come to a definition. And then from there, we were able to play into what are the values that are important for threat modeling? What are the principles that help you to do that? What are some of the anti-patterns we want to avoid? And so, by bringing this group of experts together from all the different backgrounds, we were able to drive towards a definition of threat modeling that we all were comfortable with, and we hope that that can then have a big impact on the industry.

7:49Chris RomeoTo query whether threat modeling is the right term that we want to use for this, my feelings about threat modeling are a bit mixed. Firstly, it's a known domain, it's a known area within security. I question how well known it is within engineering teams. And if we are targeting a very wide group of users, is there a different term that we could use to describe the same activity that doesn't sound so security?

8:21Robert HurlbutMatt Coles.

8:22Chris RomeoI'm finding also moving in that direction, threat modeling is easier to, easier to say.

8:28Robert HurlbutPeople recognize it more than architectural review or architectural threat assessment.

8:34Chris RomeoBut I'm not, I'm not sold on that, on that 2-term phrase. So, Fraser Scott.

8:39Robert HurlbutOne of the things we're seeing is talking to people who think they know what threat modeling is, and then it turns out that they're talking about threat intelligence, for example. So there's, there's often some sort of clashes of domain. What's the difference between threat modeling, threat analysis, and risk analysis and threat intelligence? People come to these conversations with a predetermined view of what that is, and if you don't tease that out early on, you end up with that sort of triangles versus squares challenge. Threat modeling is analyzing representations of a system to highlight concerns about security and privacy characteristics. When we think about all these systems that exist across the internet that are storing our data, these are systems that are crucial that they do security and privacy correctly. And so that's the real advantage of this Threat Modeling Manifesto is we as an industry group, we want to help to make threat modeling something that's happening everywhere, because the more threat modeling that occurs, the less vulnerabilities that exist in all of the systems that we all know and love and we've come to rely upon in our daily lives.

9:39Chris RomeoBrooke Schoenfeld. So here's the thing, you know, and this is built on having taught threat modeling to thousands of people and security architects, hundreds over the years. So it's, again, it's anecdotal and it's maybe really biased by me and what I do and how I work. But so take it with a grain of whatever you want to take it with. But I think the hardest piece about threat modeling for people to gain and why they need a security engineer there is attacks. What do you want to call it? Think like a hacker or think like attacker? I don't care. I, you know, I have no preference, um, because I think that's really hard to do if you have no concept, especially for developers. Developers, I find, want to think about how it should work, not how it shouldn't work. And that's one of the hardest pieces. I've spent a huge amount of time trying to figure out How do I help people walk through that particular piece? And, you know, that's what STRIDE is all about. It's not a method, it's a method for thinking about ATT&CKs. And I think that's the hardest part. Your whole continuous threat modeling checklist is a way of walking around that problem in a different way, and that's why I think it's brilliant and put it into every deck I have.

11:11Robert HurlbutYeah.

11:12Chris RomeoBut nevertheless, that's kind of the larger problem here, why the security engineer is needed in the room, because they can actually have an encyclopedia of attacks, at least the relevant ones, the threat library, if you will, in their heads.

11:28Robert HurlbutThinking like an attacker is really the primary thing that we want to drive into the threat modeling process. Because attackers are sitting out there and they're brainstorming and doing this same type of process as they prepare to try and break into various systems. So attackers use threat modeling already. And so what's important for us is we want to equal the playing field here. We want to see threat modeling become something that the defenders and the builders are using so that they can help to get ahead of where the attackers are going. And so that's one of the real big value propositions about threat modeling.

12:09Chris RomeoBrooke Schoenfield. Frankly, I don't think anyone's expecting anything. Most folks are just getting out and trying to do what they can. And then there's a bunch of people who are in this space of varying levels of skill and originality who are trying to make noise. So there's a lot of buzz around threat modeling. Our timing is good, I think. But, you know, the quality of the offerings vary tremendously, in my opinion, from, wow, we use Stride, which is, you know, by now fairly well-trod territory, to, you know, some really great offerings. So I don't think we have any expectation, frankly.

13:01Robert HurlbutYeah.

13:01Chris RomeoI think we can just stay silent until we have something, you know, to push. One observation I made when I was trying to draft the stuff for what the differences between threat modeling and things like threat assessments and threat intelligence is, it felt like the manifesto itself is almost in its entirety the sort of the definition because the values and the principles are sufficiently fuzzy to allow you to say, okay, something that looks more like this looks more like what we consider threat modelling. Something that's more over here is less like what we consider threat modelling. So I was kind of—

13:39Robert HurlbutI was in two minds about whether it was worth trying to put a cut between, okay, this is threat modelling, this is threat risk assessment, and this is threat intelligence.

13:46Chris RomeoI'll bring that to that discussion, but it might bring it on a bit of a tangent because Yeah, it feels like that sort of not being able to pin it down to a single definition is a feature of the manifesto as much as anything else.

14:01Robert HurlbutIsar Terandash.

14:03Chris RomeoSo one of the things that came out of the discussion of the definition was that we might end up having a short definition, the values and the principles, and then a longer definition if needed, if we couldn't agree to something. So I think that if we go with something of that structure, which to my mind makes sense, because before you start talking about values and principles, you want to define what you're talking about, then we can assume that it was defined previously and that kills that whole thing.

14:35Robert HurlbutFor me, I think just using the word threat modeling in the values, it's just possibly slightly redundant.

14:41Chris RomeoI mean, the subject is threat modeling. So just people over processes and tools doing it over talking about it, or doing over talking. It's just, we know we're talking about threat modeling because it's a threat modeling manifesto. So it's just, it might make the language a bit crisper and cleaner. There are, you know, there are several different reasons to threat model. One of them is around design. I forget who said that, but, and I'm primarily focused on that because that's my thing. But I have nothing against people who threat model just to identify risks. Penetration testers and other security analysis often employ very lightweight threat modeling in order to get started and understand where they are going. Do we want to include that? I do very much. I have written thousands of words about this. Whether they are good words or not is not a— that is another question, but I have thought about it a lot. And, and that's okay with me, even if I am focused on design. But do we want to focus on design, or do we want to open up the tent a bit, to use Adam's words, to make sure we're inclusive? I don't have an answer for that. I'm stumbling over that early for that reason. For me, there's a significant difference. Like, if you've got 10,000 software engineers churning out software every day and you've got 20 pentesters the impact of threat modeling could make as a discipline to 10,000 software engineers thinking about security is potentially much greater than giving the pen testers another tool.

16:16Robert HurlbutSo, like, the design aspect of it, I think, is critical, because that's probably where, as a cyber industry, is it—

16:25Chris Romeothat's where we're weakest.

16:26Robert HurlbutAlyssa Miller.

16:27Chris RomeoSo I'll go all middle of the road and really frustrate everybody. But, you know, it's I don't disagree with your definitions, Adam. The only thing is I think colloquially they don't fit, but I do think tools are more than software, right? I mean, a tool could be a spreadsheet, which the spreadsheet itself isn't really the software. Having a model like Stride, Stride is a tool. It's not software, but it's not necessarily intellect either. It's a little more tangible. And so that's, I think when we say tool in here, either people are gonna go directly to thinking software, you know, if I'm thinking about our audience, or they may consider also, you know, frameworks and things being a part of a toolset. But I don't think they're going to absorb that, the definition that you gave, Adam, unless we specifically call it out.

17:20Robert HurlbutThe people that we assembled here, all, all of us have, have known each other in different capacities over the years, and we've bumped to each, bumped into each other in threat modeling circles. And so, once we got into the discussion, this is one of the most orderly groups that really respected each other's opinions. And we argued and we debated, but we always did it from the idea of how are we going to make the Threat Modeling Manifesto better? There was no sense of ego. And we had some really heavy hitters in this room, in this virtual room, part of this process. And when you think about all the books that have been written by various members and all the conference talks, and all of the engagements, and training classes, and all the things that these folks have done. Yet, we were all able to check our egos at the door and say, we just want to make a really awesome threat modeling manifesto that'll change the world. By the way, I had one point that is very meta about the Agile Manifesto, and it goes over the principles and the values.

18:25Chris RomeoThat's the thing I actually spotted last month.

18:29Robert HurlbutLike, do we recognize that the Agile Manifesto exists in our universe? Because right now it seems that we can just do hashtag include Agile Manifesto and then we solve half of what we're saying. So me, I kind of have a problem of basically signing off on stuff that passes as a novelty while it's actually just a rehash because Agile Manifesto was made by really smart people, and I think we're really smart people. So when we dilute the principles to fundamentals, it's normal that we arrive at the same point. So I think I would like really to see this group leveraging what is unique about threat modeling in the message that we're going to do. Instead of just saying, oh, are we on par with the Agile Manifesto? I would like if we can just get rid of it and then say, oh, by the way, threat modeling is agile. Okay, thanks, bye, go see their manifesto. And then we concentrate on the essence. Like, when we did the whole brainstorming on stuff, I agree that it was okay to think about everything. But right now, it's like, really, for me as a developer, it's like we're releasing code that we just wrote, stdlib again for a few functions. And I'm like, Why not just include it and mostly concentrate on the novelty of what we're doing? We definitely looked at the Agile Manifesto early on as a reference, knowing that we were trying to have the same style of impact across an industry. We said, let's look and see what has worked in the past and see what we can learn from that. And so, we did start from the Agile Manifesto and studied it and then looked for ways, and you'll see a little bit of commonality between a few of the points there, but what we found over time is, as we got deeper and deeper into the debate and conversation, we really diverged from where the Agile Manifesto was going and the language that they were using there. And we landed on our own new thing that is the Threat Modeling Manifesto.

20:35Chris RomeoSo, I think that we are having a really good discussion in there.

20:41Robert HurlbutIt's pulling into all kinds of directions that I definitely did not think it would go in the beginning, but really good, really good, and the best discussion. And I think that the next step is really to start closing around it. And it seems that right now the biggest thing standing—

21:03Chris Romeoand please, people, correct me if I'm wrong— is how basically how long we want this to be. And there was at some point a discussion of having something shorter, smaller to start with that would be sort of an elevator pitch, and then something slightly more detailed that could be plugged in the, in the bottom. And I think that that's the next step, to converge towards using the short-long or just the medium.

21:37Robert Hurlbutthat we have in there.

21:40Chris RomeoI would like to close the discussion in about a week so that when we actually start cleaning the values and principles, we have that thing in our head.

21:49Robert HurlbutI don't think that it's going to be anything revolutionary.

21:52Chris RomeoIt's just basically words meeting now. We all know what we want to say. We know what we want to put out there. So yeah, no surprises. It's unfortunately jargon. That's one of the problems. It means something to everyone on this call, including me, don't get me wrong. I use this term all the time, but it doesn't mean the same thing to everyone, and that's my problem with it because it creates an inside and outside that I don't like. If there's some other way to say this, or— Yeah, emphasis on compliance isn't the same as checkbox where you have a list and you just go to the list. and do each thing on the list and don't do anything else. I don't know how to say that shortly other than just saying over-compliance. That may be enough, and then we have a principle later or something else to explain it, but it's a problem because it's jargon. Yeah, so from an academic perspective, we are always looking into how we can capture that creativity and that expertise as much as possible.

22:53Robert HurlbutI think tooling has the same question. So I don't have a suggestion to make this more feel like the direction I want to point to.

23:03Chris RomeoI think Isar already mentioned that he also objected a bit with the art term.

23:09Robert HurlbutI will think about it for a week, maybe something pops up.

23:13Chris RomeoBut we are trying to make it as much into a science and trying to capture that expertise in a way.

23:18Robert HurlbutAnd I do not want to have like in 5 years or 10 years the manifesto like thing, but we still cannot, I don't know, have it as a science.

23:29Chris RomeoIt should be a science, it should be engineering, and you can make it a bit better with creativity. Currently, you can make it a lot better with creativity, but I'm not sure yet.

23:38Robert HurlbutWhen we think about who should threat model and who this document's for, we can go directly to the document itself. And what we wrote there in the manifesto is, you, Everyone. Anyone who is concerned about the privacy, safety, and security of their system. And so, this document is written for everyone. We realize that different people are going to be coming to the manifesto with different technical levels, different experience with threat modeling, different experience with development and security and privacy and all these pieces. But we created this in such a way that it's valuable to anybody, whether it's a manager, whether it's a developer, whether it's a security engineer. Everybody can derive value from this because this is a collection of values and principles. These are the things that we value as a group that has done a lot of threat modeling in our careers and a lot of teaching about threat modeling and a lot of speaking about threat modeling. But it also has the principles that we call the fundamental truths of threat modeling that'll help you to get there. And so we see this as a document that's available for anybody, whether you're brand new or whether you're someone who's been doing threat modeling for 20 years, there is something for everyone in this document. In the next episode, the team moves from the definition of threat modeling to an even bigger challenge: deciding what the values and principles of threat modeling should be.

25:04Chris RomeoYou know, I see values, you define this is what we believe, and principles then are that, that next layer of detail of this is how we, how we, you know, live by those values or how we accomplish those values.

25:15Robert HurlbutThanks for listening.