Chris and Robert -- Security in the Methodology

0:05Chris RomeoThe Application Security Podcast. Here we go. Hi folks, Chris here. On this episode, Robert and I talk product development methodologies. We explore how to apply security activities to Waterfall and Agile. We had every intention of getting to DevOps in this episode, but we ran out of time. Look for a future episode where we dive deeply into the things of DevOps security. If you're enjoying the podcast, please visit iTunes and give us a rating. Our topic for today on the Application Security Podcast is application security as it relates to the different development methodologies. And when we, when we say development methodology, anyone who's building something, whether it's a web application or a router or a mobile phone or a mobile app, is following some type of methodology And the methodology is just the steps that they're gonna go through from the time that they get the light bulb over their head that says, we got this idea to create this information technology-related product, this application, all the way up until they release it, deliver that product to their end customer. So there's a couple of different methodologies that exist out there. One called Waterfall, there's another one called Agile, and then there is the latest craze of this idea called DevOps. And so our conversation today is going to be focused on, at a very high level, what are these development methodologies that you may meet up with in your development travels? And then we're going to focus in on how does application security play out in those? In a previous episode, we talked about the different activities that we have that are done in application security. Now it's time to focus in a little bit on how the methodology impacts those particular activities. So Robert, the first one we have here is, I'll call it the classical one, and that is the waterfall. Have you ever developed a product in the waterfall model?

2:33Robert HurlbutI have. I have. And I still see companies that are following a waterfall or mostly waterfall, partially from there's this way we've been doing it forever and we continue to do it. But there's some good reasons why, and we'll talk about that here in a moment too.

2:52Chris RomeoYeah, so when I think waterfall, I think this is a sequential design process. This is starting with my requirements, going to my development, going to My— or sorry, I got design in there first. Requirements, design, development, testing, and release. And each of these phases is very much— you can only be one place on that line at any given time. There is no jumping from the design phase to doing some type of testing. You finish design, then you start coding, then you start testing, and then you release to the customer. Is that consistent with your idea of what waterfall development methodology actually looks like?

3:33Robert HurlbutYes, and really the main reason that I've seen it or why most or some companies have followed it is because the requirements are really well known. They don't change. They, you know, we already know exactly what we're doing. We've already done this before, let's say, and it's really well known. So it's really easy to keep everything on track because we know exactly what at every stage is expected and we can, you know, do the work and no surprises.

4:07Chris RomeoYeah, and waterfall is really good at the documentation. So it's a documentation-heavy, it's a process-heavy process, but it is also documentation-heavy. It's easy to understand expectations of the system because you have a firm set of requirements written out. You can even have people leave the team. You can bring new people in and they can pick up right where the other people left off because there's so much documentation and so much planning that happens. Now, from a— that's the things that you can kind of think of as good or the pros from a waterfall perspective. But Robert, what are some of the cons then? What are some of the things that are bad about waterfall?

4:45Robert HurlbutWell, when things change, especially requirements.

4:49Chris RomeoOh, you mean some things are going to change in a technology product?

4:52Robert HurlbutI know, it's really strange. Why would that happen? But they do. And taking a step back is really expensive in time and resources and having to redesign, having to, you know, redo some— a lot of development, especially if we've already gone down a path for months at times and now have to kind of scrap it all and start over on something. It's just really expensive, really, really difficult to do in a waterfall process.

5:20Chris RomeoYeah, and I also think about Everything is so tied to your requirements. So if you don't have a great set of requirements that really truly capture what you're trying to ship at the end of this process, your project is doomed because you can't just iterate back in, in the waterfall model and quickly make a requirements change. You have to take the whole— you have to turn the whole battleship around, make a left turn, hit your left turn blinker on the battleship, make that turn, bring the whole thing back around to the beginning and start over again. And I use the battleship analogy because that's how the water— that's how clunky the waterfall methodology is from an application security implementation perspective.

6:01Robert HurlbutRight. And, you know, it makes sense. I mean, traditionally waterfall was actually the method you would use to build cars or to build, you know, other kinds of things in a factory because you absolutely need to have the requirements set early. It doesn't make sense to change things in the middle. But software, on the other hand, especially when you're working with customers, can change. And so that's again one of the drawbacks about the analogy between, you know, where waterfall came from and the software world is where it falls down many times is that things do change and are not always known at the beginning of the project.

6:39Chris RomeoYeah, and so let's transition a little bit into how application security plays out in this waterfall methodology. And I think from my perspective, waterfall is actually the easiest place to do secure development lifecycle, to do application security, because Microsoft created the first secure development lifecycle that was ever released and talked about to the world. And they were building in a waterfall methodology at that point. So most secure development lifecycles have that legacy back to the Microsoft model or process which was already set up for waterfall.

7:19Robert HurlbutRight, so at every point, the requirements design, the coding itself, the testing, I mean, those were all in separate buckets, and so it made it very easy to apply security in each of those buckets, if you will. Okay, I know I need to do some design. Okay, what's a secure design here? I need to do, of course, requirements. What are the security requirements? And so on. So it's very easy to apply those things at every point because we already have those well-defined in a waterfall methodology.

7:51Chris RomeoYeah, and I think about in the world of waterfall, it's very easy to target where each activity is going to happen. So threat modeling in a waterfall world happens during your design phase. In a perfect world, it doesn't always happen this way. But in the, in the best possible solution, your threat modeling is happening while you're doing your design and before you've written any code, which in the world of waterfall is actually pretty easy to pinpoint that moment because it's really not in flux. You've got a full schedule that tells you from when we start working on requirements till we release, that's 12 months total time. Design's going to be like month 2 and month 3 of the process. So, you know, you can focus your application security activities and tie them much closer to a schedule that exists.

8:40Robert HurlbutRight.

8:41Chris RomeoSo let's move in and look at Agile now. So Agile, you know, is what, 10 years old maybe?

8:49Robert HurlbutIs it that old? At least. At least. At least older.

8:52Chris RomeoSo now have you, Robert, have you been part of an Agile team?

8:55Robert HurlbutI have.

8:56Chris RomeoI have. Okay, now let me ask you this. Did you have to memorize the Agile Manifesto? to be part of that team?

9:02Robert HurlbutI did not, but I did know it, so—

9:05Chris RomeoOh, wow, okay, please. Yeah, I don't know, I don't think we need to recite it, but since you've worked on a team that's done the Agile methodology from a development perspective, give us just a high-level walkthrough of what that looks like.

9:19Robert HurlbutWell, just to say, traditionally, or actually it was an answer to the traditional waterfall methodology in the sense of requirements do change. That was the main idea or the reason for Agile is because requirements change, we need a way to address that. And so the first idea was this idea of incremental approach that we have sprints, we have shorter time between thinking about the design and the requirements developing and testing and then getting back those results to a customer to get some feedback as opposed to waiting months and months and months to actually finally go into the testing phase and into, you know, some kind of checking in with the customer and see, is this really what you want? Instead, try to shorten those times so that we're much quicker in getting back feedback rather So that we're not off track, you know, months and months later. Instead, you know, a typical sprint, for example, could be a week or a month or whatever.

10:25Chris RomeoAnd there's testing activities, right, that occur in each of those sprints. So you're not going to wait like in waterfall, you're going to wait till the end, towards the end of the process to do your testing. From an agile perspective, you've got development and testing activities happening all the time.

10:42Robert HurlbutCorrect, yeah, and that's always typically part of a sprint is that you determine what are called user stories, which essentially are kind of a replacement of the— all the documentation requirements and design and so forth that we talked about before in the waterfall. You're really thinking about, okay, here's a feature, here's what it needs to do, here's, you know, what we expect, and then you test against it. So that all becomes a part of a typical sprint.

11:10Chris RomeoAnother thing that I've heard about in the world of Agile is this idea of a user story. So is a user story equal to a requirement, or what is a user story?

11:22Robert HurlbutYeah, user story essentially is a— I mean, the way it came out is that you're thinking about users, or a process it could be, how do they see the system? And so try to get down into discrete units of functionality or features within the system. So A typical user story could be, as a user, I want to log into the system. That's a security feature, security requirement. What comes out of that then is, again, the user story that represents what is the user wanting to do, how would they do it, and then how do I know that I've completed it? Well, I should be able to, as a user, log into the system.

12:09Chris RomeoYeah.

12:09Robert HurlbutAnd I'm now there. So now that's a complete unit in itself of something of describing what I want. You know, the details about how that gets done, you know, may not necessarily be that important, but you could put that in somewhere in the description. But the key part of that is it's also testable, and that's another aspect of user story as well.

12:31Chris RomeoAnd it's focused on somebody actually using your system.

12:35Robert HurlbutCorrect.

12:36Chris RomeoSo it's not like a requirement where I say, the system must perform authentication, which seems like I'm pretty disconnected from the people actually using my system when I write stuff like that. But with a user story, I'm going to say a user is going to use their web browser to log into our application. And hidden inside of that will be some of the details about the authentication process and storage of passwords and other things that are traditionally security requirement related. But they're going to— it's going to be captured from the user's eyes, which I think gets people closer to building something that somebody might actually want to use.

13:16Robert HurlbutCorrect, correct.

13:18Chris RomeoSo a daily Scrum meeting is a thing. It's part of Agile is to get together and chat every day, right?

13:25Robert HurlbutRight. Just, you know, what did we do? What worked? What didn't? And what are we planning to do today? So it's just a recap of what happened, what do we plan to do, any barriers, any problems, let's deal with them for a few minutes.

13:41Chris RomeoOkay. Now, I hear this, I've heard of this term called continuous integration, and I'm just curious, is continuous integration required with Agile? And well, first, what is it? And second, is it required as part of Agile?

14:01Robert HurlbutWell, continuous integration is really that process of if I develop my code and let's say I check it in to some kind of source code repository, that I'm not done there, that I may have some kind of process that also takes the code, compiles it, makes sure, you know, everything is good, and may run some tests against it. So What you're trying to do is make sure that I don't just check in code and then at the very end somewhere down the line I may find out if my, my code works or not. But instead to try to build in a process of always checking the code matches my expectations. And those are essentially the tests that you might write, unit tests for example, that are testing expectations, again related to the user stories as we talked about before. So is—

15:00Chris Romeoso then is Agile required? I'm sorry, is continuous integration required by Agile, or is it just something that fits nicely with Agile?

15:08Robert HurlbutYeah, it's not necessarily a requirement per se, but it is certainly something that's kind of come along with— in order to confirm The other aspect of agile is this idea of getting to a point of we're done. How do you do that? And the best way to do that is that I have tests or I have some kind of verification that the user story has been completed as I requested or required. And so it's completing the loop. And continuous integration is a really great automated way of helping you do that.

15:45Chris RomeoOkay, so just to summarize what I, what I took away from, from the description you just provided of agile in general, some of the advantages to agile are that you can make changes. It's a lot, a lot easier to change the user stories or requirements or what you're focusing on. If a customer has a desire to send you in a different direction, you don't necessarily have to go all the way back to the beginning. You can, you can adapt As the airplane's flying, you don't have to land and make changes to it. Feedback seems to be very, a very important part of the process because you have your daily Scrum meetings you were talking about. I like the fact that testing is, is much more rolled into this process and the fact that the product could be shipped at any time. So when you finish a sprint, the product should be in a state where— If you wanted to, you could ship it to someone and let them actually, you know, use it. It might not be a fully functional product, but it would have enough functionality to do something.

16:49Robert HurlbutRight. So your goal is really as much as possible for these user stories, each of the user stories, if we call them done, at least that feature is done so that somebody could test it.

17:02Chris RomeoThat's the goal. Yeah. So I guess one of the potential downsides of Agile is it could be more expensive though. Over time. If you're not very careful with— you described the idea of definition of done as an agile concept to know when something's complete. If you're not careful with that, you could drift into budget overruns and things because you're just continuously searching for the perfect versus stopping at the good.

17:32Robert HurlbutRight.

17:34Chris RomeoFrom a development perspective.

17:36Robert HurlbutWell, that's true. I mean, the other thing that I've heard as well from, I guess, the horror stories of agile projects gone wrong is that even though you say, well, you know, we're kind of going along and we're doing this ad hoc, it's not really always completely ad hoc. You need to have at least some idea of where you're heading, and you'll— good agile groups I've seen self-correct as they go along and they understand, they're getting better and better at estimating, for example, they're getting better at understanding the system. If you're still ad hoc all the way through, yeah, you may never know when you're done. And of course, you know, that causes all kinds of problems.

18:19Chris RomeoYeah, so let's transition and talk now about how AppSec plays out in this agile world because it's definitely not as cut and dry as what we see in the world of waterfall. So the first piece of advice that I've heard and that I've shared with other people as well, it's important to embed security into each Scrum team. So what do you take— what's your take on that? Is that good advice?

18:47Robert HurlbutYeah, I definitely agree with that. You should have what I call a security champion or someone that essentially understands some of the security issues. It could be a developer. who just wants to know more about security, wants— has a concern about it, but somebody who is asking the right questions. You have a new feature, you have a user story, and they ask the right questions. You know, what is the security implication of this feature? We want to have a login. Well, what does that mean? How would we do that? And then they might detail out some information or some steps. And that could turn into another user story. So that's essentially, yeah, very important to have somebody who's involved who has some kind of security focus or perspective.

19:33Chris RomeoYeah, so then another thing that comes up is the idea of user stories as they relate to security. So I think of these as there's really 2 different ways you can approach the user stories for security. One is you should definitely understand what makes— what user stories are security sensitive. So some user stories are not going to have a whole lot of security features or functionality built into them. They're just going to be usability features. You know, we want to make the browser window light blue instead of purple. Now, I don't know what web application you're building that's going to have that as a user story, but there's not a lot of security. I'm not that interested in that user story from a security perspective. Right. security perspective. So I think you have to know a method to identify what user stories are security sensitive so that you can focus your security resources on those.

20:32Robert HurlbutRight. So, right, as you said, I mean, for example, access control is one that I see that comes up often. Well, how do we do that? And that may itself be a whole other story or set of stories to build out an authorization framework for your system. And so, you know, those are the kind of things that, you know, as you're going through, you identify here's a regular user story, but okay, well, this just kind of implies that we have some kind of role-based security somewhere in here. So then that becomes its own story.

21:08Chris RomeoSo that's where those kinds of things may come out as separate stories for Yeah, and I think I hear people talk about abuse cases or misuse cases as being a mechanism of using the user story to capture negative behavior, and then that negative behavior should result in some type of either assurance-based activity where you're going to do some tool to ensure you don't have SQL injection problems, or it may be creating some new security functionality in the design and then ultimately in the code to ensure that some vulnerability's been mitigated. So have you ever seen anybody use abuse cases before?

21:50Robert HurlbutYeah, I have. An example of that, going back to the access control, as a regular user, I should not be able to do certain admin functions. So admin may be accessed by let's say a tab that appears on a page or maybe a particular area or page itself within the website. And so an abuse case may be written, a user story written in such that, you know, as a regular user, if I navigate to the admin page, I will be redirected back to a login screen or a 404 page or whatever. That's an abuse case that you want to test, and you can write that as a story and test it and verify that absolutely we are handling this abuse case correctly.

22:41Chris RomeoThat makes good sense there. So when I think about AppSec in the agile world, I just think that the security activities just have to happen at a different frequency. They can't be scheduled in a linear fashion like they can be with waterfall. So, I think about breaking down the activities that need to be done into 4 different sets. So, the first set are the activities that are going to happen every time we check in a piece of code. This could be activities like the static analysis that runs in your IDE, so that when— or maybe even a minimal static analysis check that happens right when the code gets checked in. as part of the approval process. So that's the first place where you can have some level of activity happen. The second one is at the Scrum, for each individual Scrum or each individual sprint, what are the activities that you want to happen there? Do you want to schedule threat modeling so that some threat modeling occurs in every sprint? Or do you want to have a sprint where threat modeling is very well-focused? Do you want to have penetration testing happen every sprint? Or do you want to focus that at certain points in the process where features are really starting to come together and we need to take a holistic look across the entire application. And then there's also what activities are going to be done on a given customer when we get ready for the customer release, or the things that we want to do that are specifically at the end of the overall process. So that's kind of my— the way my brain is thinking about agile application security activities. Is that consistent with the agile experience that you've had in the past, or would you add anything into that?

24:25Robert HurlbutNo, that's, that's, that's pretty spot on.

24:28Chris RomeoOkay, so we talked about waterfall, we talked about agile, and one other thing I'll say about agile that I think is actually kind of funny— agile, when someone says agile, doesn't necessarily mean they're talking about agile. So I've seen different organizations Even different teams within the same organization where one person says Agile and they mean something completely different than the other person does. So, and I don't know, Robert, if that's been your experience as well in the Agile world. I don't see that as much from the waterfall development process. People seem to just kind of get what that is, but a lot of times I see Agile defined differently.

25:10Robert HurlbutIt is, it is. I mean, there's also, there's a hybrid. where you, you do a lot of upfront planning and you do some backend testing, and then in the middle you do your agile methodology. So I've seen that as well, where they, they do something I've heard called Water Scrum Fall.

25:31Chris RomeoThat must be— we'll have to do that in a whole other episode, the Water Scrum Fall methodology.

25:37Robert HurlbutRight.

25:38Chris RomeoYes, so I think that the, the important thing here Regardless of the methodology that you use, the important thing here is to understand what are the security activities that need to be done, what are the steps that exist in our methodology that our company is deciding to use or our team is deciding to use, and then adapting those security activities to provide the most, the highest return on investment. So when you're first starting and building an application security program, Some people like to look at Microsoft's SDL and say, oh, there's 27 things there on that list and we gotta do 'em all, which is really just not feasible when you're starting out from a secure development lifecycle or an AppSec program perspective. You need to pick certain activities from the list and start with those from a pilot perspective and develop your team and your developers' and testers' knowledge and understanding And then add more as they start to be able to deal with the pieces that you put in front of them. So we were gonna talk about DevOps, and we even mentioned it in our initial methodology discussion, but I'm gonna leave that, Robert, for a deeper dive because we have an upcoming episode where we'll dive very deeply into the world of DevOps and how security fits in there. And I see, when I look at DevOps, I see a lot of agile things. Agile and DevOps are closely related, and And there's some crossover in what happens conceptually there. But I'm gonna leave that DevOps conversation until we can dive really deeply into that because that's, that I think is gonna be the newest piece of methodology for any of our listeners that are out there.

27:21Robert HurlbutI agree. I agree.

27:23Chris RomeoSo folks, look for an upcoming episode where we dive deeply into security in DevOps. Thanks for listening to the Application Security Podcast. Our intro music is 8-Bit Kung Fu by Boring and TJ, and the outro is Southern Delight by Stefan Kartenberg. You can find us on Twitter @AppSecPodcast or on the web at www.appsecpodcast.org.