Eric Johnson -- Continuous Integration in .NET

0:05Eric JohnsonThe Application Security Podcast. Here we go.

0:09Chris RomeoHello all. Welcome back to season 2 of the Application Security Podcast. On this week's episode, we talk to Eric Johnson about static analysis, pen testing, continuous integration, and much more. As always, thanks for listening and let's get right to it.

0:44Eric JohnsonHello friends, this is Robert Hurlbut, and today I am joined with Eric Johnson. who is a software security consultant who is joining us. And Eric, would you please introduce yourself?

1:00Chris RomeoSure, absolutely. Thanks for having me aboard today. This will be a fun conversation. I have a passion for static analysis inherently. I started out in the development world right out of college. It just happened my first job was with a financial institution here in the US. and wrote internet-facing code at the age of, what, 22, where really security at that time, and from what I've heard, is still not actively being discussed in your standard computer science classes. So, out of the gate, I'm sure we were introducing tons and tons of security issues along the way. And as I went through an InfoSec program, it kind of brought to light a lot of these application security issues that are heavily discussed in a lot of the OWASP projects and things of that nature now, and kind of transitioned from my development space over to AppSec. We started writing a lot of core, you know, kind of, we'll call them like high-risk code sort of libraries that centralized our authentication, authorization, you know, session management, validation, things of that nature. And over time, as I'm sure, Robert, you're familiar with, you kind of stop writing as much code when you get too close to those things. And we started running, you know, commercial static tools against our codebases, doing audits, dispositioning results, fine-tuning rules. And I did that for about 10 years or so before I decided to kind of branch out and go do these same sorts of things with small to medium-sized businesses, also get into pen testing and mobile testing and things like that along the way. So it's been a fun journey up to this point, and definitely no end of opportunities to keep learning stuff, which is the funnest part.

2:53Eric JohnsonAbsolutely. So you told me a little bit about your story, and we usually start with a superhero story. Tell me a little bit about how you got interested in security in particular. I think you— if I remember, you mentioned you did some development, or you told us you've done some development. What was it about security that attracted you?

3:12Chris RomeoYou know, I think it was that aura of, oh man, you've got these special powers where you can go, you know, break into some website or, you know, make this software do something that maybe we did not really intend for it to do. So, I think it was always that draw of really just wanting to understand, you know, how these things were happening and why we were able to make those things work. So, it kind of all lended itself towards me just being curious by nature as to how these things kind of happen. And of course, at that point, you want to know how to fix it at the same time. So, it kind of goes hand in hand.

3:51Eric JohnsonSo, when you're trying to figure out things, you're tearing them down, you're breaking them apart, trying to figure out how they work as well?

4:00Chris RomeoAbsolutely. Absolutely. So, it helps. We always kind of say it, most of the people that work with us or for us, if you can't write code, it's really hard to understand both the red team and the blue team side of things. So, most of the folks that we talk to all do come from a development background, which is really weird in the InfoSec world, I think. there's probably only about 5% of us InfoSec pros that actually came out of a more dev-focused organization. So we're kind of rare breeds, I guess I'll say.

4:34Eric JohnsonRight. Well, it does help. I agree. In my own career, my own experience, I've really enjoyed the background that I've had in development and pushing into security and understanding both sides. And it helps me to communicate also with developers as well. And if I understand correctly, you also do some training of developers in those areas as well for developer security.

4:57Chris RomeoI do, yeah. I've been with the SANS Institute as an instructor since about 2012. A former colleague of mine, James Jardine, who I think you're pretty familiar with, I think— were they on your podcast here a couple weeks ago? I think I saw.

5:12Eric JohnsonThey may be. I have not seen all of our season 2 yet, but I believe that's right.

5:18Chris RomeoYeah, so James and I both kind of had a heavy hand in a lot of .NET security work with an organization, and at the time James was authoring and maintaining and teaching their Secure .NET course. It's basically developing defensible applications. So he kind of brought me in and kind of got me started in the teaching realm, and what I found is, as you said, I actually really enjoy it. It's refreshing to hear about, you know, just the struggles that regular software engineers go through as they try to adapt and integrate security into their everyday work. And, you know, a lot of these folks get this bad rap, or, oh, we're out there creating these, you know, insecure applications, but it's just truly the demand on them is on features and speed and making businesses money. So, it's really hard to justify security in a lot of these situations. So, it kind of keeps my perspective fresh so I don't turn into one of those angry InfoSec folks that just likes to point their finger and kind of yell at people type of thing.

6:22Eric JohnsonSure. So, when you're training developers, what are the typical things that come up in terms of questions or in terms of things that they're trying to figure out? What do you see typically?

6:35Chris RomeoYou know, a lot of it is just kind of understanding why we should care I feel like a lot of the challenge. I think I spend the whole first day of this class just talking about, look, here's the attacks, here's this hostile production environment your applications live in. Whether you want to believe it or not, there are people that are out there always poking and prodding at your applications. It's kind of building that mindset that, you know, we actually do have important things to defend. And giving them that, uh, we kind of do some very light pen testing just so they can understand how easy some of of this stuff is to accomplish. And then that tends to kind of say, oh, man, yeah, we better actually get better at this or we're going to be in trouble.

7:22Eric JohnsonRight. Excellent. So, some other things I know that you've been involved in are static analysis, continuous integration. So, tell me a little bit about that. First of all, what is continuous integration?

7:35Chris RomeoYou know, CI is kind of an open-ended term, in my opinion. It's really this ability to integrate all of these tools that, if you think 10 years ago, we all ran 10 different tools independently. You know, they're all their own executables or, you know, packages or whatever, depending on your OS. And it's this ability to kind of commingle all of them into one centralized environment to accomplish all these things that used to, you know, kind of work separately from one another. And now they can kind of all happen inline in one interface. So we don't have to bounce around and kind of lose productivity that way. That's kind of how I how I've always viewed CI.

8:17Eric JohnsonOkay, that makes sense. So instead of just keeping all these things separate and having the user push a button, push the next button, push the other, you try to keep it all together so that when you— let's say you do a kickoff, a build, it's calling multiple things at the same time, testing and so forth, if I understand correctly?

8:38Chris RomeoAbsolutely. You know, and Jenkins is probably one of the most popular CI servers out there. And if you think about everything that's capable of doing, you know, it can pull from your Git repo now, and now it can then launch a build, and we can evaluate the results of the build within Jenkins. We can also wire in static analysis scans, our unit tests, our functional tests, and we can expand on those and start, you know, kicking off ZAP scans, or we can kick off Arachne scans. We can integrate, you know, QA test plans. We can even start deploying code out to our, you know, test, dev, UAT, production-style environments, and even start running automated smoke tests against production, do some monitoring. It's just the, uh, the world is endless on the amount of things that we can kind of integrate into this one CI type of environment.

9:32Eric JohnsonRight. So that way, like we said, you're not having to wait on all these things, or perhaps they don't get done. Now they can be built into the pipeline, if you will, so that especially the security tests, which I like as well, not just running unit tests but also running— or integration tests, running security tests. So that kind of jumps into this idea of static analysis. So, you know, many times I might run an application separately to check on my code, but Tell me about static analysis as part of CI.

10:05Chris RomeoYeah, so, I mean, this has been around for a bit. Your big commercial scanners have started to roll out an IDE plugin here and there, or a Jenkins integration. Some of them have had, on the Java side, Maven and Ant build integration for quite a while. And it was always that, but we still have to go check the results, and we need to go see what there was. And with this kind of adaption, or adoption of CI in the software engineering world, these tools are being used to build and deploy applications, whether security is on board with it or not. So it's this concept of, well, if we as security folks can start to understand the development tools that are actually being used and say, look at this huge opportunity that CI is offering us to enforce that scans are actually run, and evaluate the health of those scan results. And potentially, in some scenarios, if something bad enough is found, we can even fail a build and stop some vulnerability from going out to production. That's kind of the goal that we set out with as we started writing some of these static analysis tools to say, let's try to stop that OWASP Top 10, you know, kind of super low-hanging fruit that always seems to end up in production and then land some organization in the news. What if we could identify that the second it's committed to Git, for example? You know, that's a huge win for security in our eyes.

11:36Eric JohnsonOh yeah, definitely. So what you're talking about is being able to, let's say for example, catch SQL injection. You would see strings concatenated and sent into the database and catch that as something to say, hey, we can't allow that out into production, we've got to flag it. So that's what we're talking about with the static analysis, is catching it in the code and then as part of the CI pipeline, stopping that build or stopping the results of that build going out further if we have those kinds of vulnerabilities.

12:07Chris RomeoYes, absolutely. And that's always kind of the one that we use as an example. And, you know, I've read some articles lately that say, hey, SQL injection's going away, we've got all these ORM frameworks. And I'm not kidding you, last week in a shiny new Node.js application that's supposed to be impossible to write SQL injectable code, 10 to 15 lines into a class, look at it, oh, there's some SQL injection because we're calling into some underlying Microsoft SQL Server database using this ORM framework that's got this dynamic query method that allows you to just execute arbitrary SQL statements. So, it's definitely not going away, and it's the idea if you write that and if you commit that, the build health is immediately flagged as unstable, and we've got to go fix that. before we can push it into an environment.

12:59Eric JohnsonRight. Wow. Excellent. So in terms of being able to do this within certain environments, I think you've been focusing on .NET, correct, with the latest tools?

13:12Chris RomeoCorrect. And this kind of came about— I've worked with several organizations that are very large .NET shops. And, you know, coming from my background where that was kind of all I worked in for the first 3 years writing software, And then was kind of my niche focus within AppSec for several years, just from a static analysis perspective. It's kind of always been my, we'll call it my passion, is just the .NET Framework in general. I've followed it since the late 1.0 days and kind of ran into somebody who was saying, hey, have you heard of this new Roslyn thing? And of course, being busy with all this other stuff, I'm like, no, who's Roslyn? Thinking it's a person type of thing. And he's like, no, this is this new .NET compiler API, and you can query all of the information that's available in the compiler. He was talking about it from a kind of a style-checking perspective. Oh, we can enforce our coding guidelines and all this stuff. Of course, my evil mind, static analysis mind, is heading directly towards, wow, I bet I could find a lot of cool security problems with this using the same functionality.

14:22Eric JohnsonYeah, definitely. I remember the same thing when I heard about Roslyn. I thought the same thing. Hey, how can we do this with security? Because we can do all the style checking, we can do all the— are you putting your semicolons in the right place, or, you know, opening and closing your braces and all that stuff? Great. But can we check for some security issues? And so that's— I also thought the same thing. Roslyn, it would be a great tool to do that. That's some of those things that you've done, right? You've taken some of the checking the code itself, checking for those particular issues and being able to find them and report on them. That's essentially in a nutshell what you're doing with Roslyn or allowing you to do. Tell us a little bit more, maybe a little bit more deeper than that on what you're doing with Roslyn today.

15:13Chris RomeoAbsolutely. So, it kind of all started with that conversation. And walking away from that, I went home and I Googled Roslyn, of course, and I find this MSDN article. And it's— I can't remember exactly who wrote it, but it's basically like, here's how to build your first Roslyn rule. So, I read through it, checked it out. Okay, walk through. Here's the dependencies you need to install. You got to go put the compiler API in Visual Studio and all that stuff. All right. So I wrote this first rule and it found a regular expression, you know, or looked for invalid regular expressions. And I'm like, okay, that was actually pretty easy. They, you know, Visual Studio has this syntax visualizer. It's a little side window, and as you click on a code block or a method or an object creation expression or whatever it is, it tells you exactly what type it is and what kind it is. as far as what the compiler's interpreting that as, which then you can turn around and say, okay, I want to write a rule that finds all object creation expressions. And then you can say, what type is this, and what namespace does this object live in? So, you know, last week I was on a flight out to a conference, and I sat there on the plane and said, you know what, I'm going to write a rule to find all instantiations of DES, because those just, you know, maybe shouldn't be in the code anymore. And it's like, okay, find all object creation expressions. If the class name is DESCryptoServiceProvider and it lives in the System.Security namespace, we'll flag it. And you can raise a diagnostic, and it gives you the little green squiggles in Visual Studio, and then it adds a warning to, to the compiler. So you kind of get that data in 2 places. Number one, the engineers see it in the IDE, And 2, it'll show up in an MSBuild task as you push that out into your CI server, whether it's Jenkins or, you know, TFS or whatever. That warning is sitting there in those build results now, so you can evaluate them and kind of evaluate the health of your build based on how many of these things might exist. So that was my first introduction to it, and when I got this proof of concept done, I thought, oh, this might be a fun talk at at AppSec USA to give, which this was probably about, I want to say, March of 2016. So it was right when the CFP calls were coming out for AppSec USA in DC last year. So I submitted an abstract and kind of sat on it, and then of course they picked the talk up, and I said, well, all right, I need to, I need to get to work here and build out, you know, some actual, uh, at least some additional rules to kind of build this thing out. And that's kind of where this little PumaScan tool that we're calling it was born. So we've been working on this for about a year and a half, myself and another gentleman named Eric Mead, and we've probably got about 50 to 60 rules put into this open-source repository at this time, just looking for all sorts of variants of cross-site scripting, SQL injection, password management, certificate validation. The crypto rules are in flight, so it's, it's been a fun little side project to work on.

18:26Eric JohnsonYeah, it looks nice. I actually took a look at it myself. It looks like it runs for Visual Studio 2015 and 2017. It's been updated for 2017, looks like.

18:38Chris RomeoCorrect. Yeah, I just did that a couple of weeks ago. I'm always kind of a slow adopter as far as picking up the latest and greatest version of the IDE is, but I think they released the preview and someone opened a Git issue pretty quick and said, hey, you need to support this thing. And I'm like, all right, everyone settle down. It's not even an official release candidate yet. But yeah, we just got that in there within the last month or so, I think.

19:00Eric JohnsonOkay, great, great. One thing you mentioned, going back to what another thing that Roslyn does for you, which I also like, is the in-place compiler. So it's always telling you, hey, if this thing was completely compiled, because it does it in the background all the time, it's going to tell you with a warning to a developer, hey, are you sure you want to use this? Is this the right thing? In this place. And so I like that in-place code information for developers. So it's not just at the end when you run the scan to tell them that there's a problem, but as they're writing the code, if they're watching for those things— and I know for myself, for years I've used tools like ReSharper and others that will tell me, hey, this is what's going on in the code, here's a problem, you may need to think about it differently, and so on. With a tool like this or using Roslyn, you can tell a developer, hey, there are some security issues in your decisions that you're making right now that you can change before it even gets to the point of a build, before it even gets to a point of a CI integration.

20:07Chris RomeoAbsolutely. And I consider that to honestly be the biggest strength of this mechanism. If you think about a lot of the other commercial scanners, it's okay, right-click on the solution and go scan this thing, and then you might get the results back an hour or 2 later, which any software engineer has already committed the code and moved on to their next backlog ticket at that point. They've already kind of forgotten the context around what they've written, and they've moved on to the next thing. This is instantaneous feedback. And if you talk to really any kind of secure DevOps person out there, this is the absolute earliest that we can wire security into an application, is the second they write it. bam, they've got this feedback that says that was not actually a very secure line of code. We should maybe consider doing it this way.

20:57Eric JohnsonExcellent. Yeah, I think that's a great win, especially for developers who are aware of that or using it. And it just gives them that extra help if they need it as they go along and try to understand, to make the right decisions, really. So I like that. Right now it's just a .NET tool. Do you know of others that are similar for other environments, frameworks, Java, and so on out there?

21:25Chris RomeoYeah, absolutely. I mean, the whole reason I kind of wanted this, and so I have at the end of these training sessions I do on the last day, we've got this static analysis section and there's this slide that says, hey, here's your free and open-source versions of .NET security scanners, and the slide was always pretty bare. You know, FxCop has a couple of security rules, and Sonar's got a couple of security rules, and, you know, we've had cat.net that kind of got pushed out there in 2009, but honestly hasn't been updated since then. So, it was always these, there's a few things, but they're not really that good. And it always kind of irritated me because Java's got Find Security Bugs that's pretty well maintained, and that plugs into Security Bugs in Java, and it does a pretty good job of finding, you know, 80-ish bug patterns, and Ruby's got Breakman that's maintained by Justin Collins. So, there's all these other platforms that have decent ones, and it just kind of always irritated me that .NET didn't have a good one. So, that was really my driving motivation for kind of filling that hole and giving us something to use without spending the big bucks on the commercial tools.

22:37Eric JohnsonGotcha. Okay. So, just to kind of wrap up a little bit, a couple more things. So let's say a company doesn't have this in place. They don't have continuous integration. They don't have anything. Where would they get started? What should they be thinking about if they want to start doing these kinds of things and put these kinds of things in place in their own build pipeline and production release of code?

23:02Chris RomeoYeah, I mean, it's always baby steps at this point. In a lot of organizations, some of the smaller to medium-sized companies that we work with, this is the exact scenario that we often find them in, and I'm sure you've experienced kind of the same thing. So, you know, specifically with this scanner, if you're writing .NET code, this installation is as easy as possible. It's honestly, you can download a Visix file and just double-click on it and install it in your desktop, or you can install the NuGet package into each project or the solution itself, and then that will guarantee that these analyzers live with that project forever, essentially. So if you're scaling this across 10 different workstations, all of those analyzers will always run in every single IDE they're presented in. So from a static analysis perspective, that's about as easy as it gets, and that's probably a really good place to start if you've got absolutely nothing going on. Because that will at least ensure that those results are being presented to the people that are the closest to the codebase, which I think is always very important from a security perspective.

24:18Mm-hmm.

24:19Chris RomeoMaybe step 2 might be getting your security folks involved, and if you don't have a build pipeline, it's setting something like that up. You know, we've used Jenkins almost primarily. I've seen a lot of TFS instances, but there, of course, are other options out there. And then it's at that point, let's start getting nice nightly builds going through some sort of a pipeline, and they don't even have to do security to start with. But once we get that in place, then it offers a huge opportunity for application security to look for out-of-date dependencies, do static scanning, dynamic scanning. It's really kind of— it opens up the world of security if you can at least get that pipeline in place.

24:59Eric JohnsonSo, in terms of getting security people involved, then, They could be working with developers, working with teams, helping them understand how the tool could be installed pretty easy, how to get that integrated into their CI pipeline if they have one, or if they don't have one, how they can maybe think about putting that in place and have these kinds of security scans happening for their code before it goes to production. It sounds like some ways to get involved.

25:30Chris RomeoAbsolutely. And you nailed it with security should be sitting with the software engineers during a lot of this process anyway. You know, if we just kind of did that on a weekly basis, I think a lot of things would probably be caught earlier in the process rather than after we've pushed it to production. But, you know, it's something as simple as creating our security, you know, user stories or abuse cases and burning those into unit and functional tests would offer a tremendous amount of security that doesn't take place today.

26:01Eric JohnsonOkay, excellent. Well, Eric, I wondered if you have any other last comments about, you know, for groups that are trying to understand better how to write secure code, how to make sure it's secure, things like that. Any other comments or thoughts?

26:16Chris RomeoI mean, the easiest one is honestly, it's just awareness. So, you know, just if it's something you're interested in, dive in and Do weekly lunch and learns or something within your organization. That's kind of the easiest way to raise that sort of awareness to all of the folks working around you is just to kind of just dive in, take that on, don't be afraid to fail. You know, you should always be learning something new anyway. I do every week. I see something I haven't seen before. So, it's kind of that don't be afraid type of deal and just go out and try to disseminate the information as much as you can.

26:50Eric JohnsonExcellent, excellent. Yeah, I like that idea. Don't be afraid to fail. I like that a lot.

26:56Chris RomeoYeah, we all do, whether we want to admit to it or not.

26:59Eric JohnsonThat's for sure. All right, Eric, well, we thank you again and appreciate you being with us this week. And, uh, thanks again.

27:06Chris RomeoSure thing. Thanks, Robert.

27:10Thanks for listening to the Application Security Podcast. Our intro music is 8-Bit Kung Fu by Boring and TJ. And the outro is Southern Delight by Stefan Cartenberg. You can find us on Twitter @AppSecPodcast or on the web at www.appsecpodcast.org.