---
title: "Jay Beale -- Docker Security and AppSec"
url: https://appsecpodcast.com/jay-beale-docker-security-and-appsec/
date: 2017-08-22
duration_seconds: 2686
guests: ["Jay Beale"]
topics: ["Software Supply Chain", "Cloud and Infrastructure"]
audio: https://www.buzzsprout.com/1730684/episodes/8122711-jay-beale-docker-security-and-appsec.mp3
transcript: true
---

# Jay Beale -- Docker Security and AppSec

*August 22, 2017 · 45 min*

with [Jay Beale](https://appsecpodcast.com/guests/jay-beale/)

on [Software Supply Chain](https://appsecpodcast.com/topics/supply-chain/), [Cloud and Infrastructure](https://appsecpodcast.com/topics/cloud-and-infrastructure/)

[Audio](https://www.buzzsprout.com/1730684/episodes/8122711-jay-beale-docker-security-and-appsec.mp3)

## Show notes

Does moving an application into Docker make it safer, or simply change the risks you need to manage? Jay Beale of InGuardians joins Robert to explain containers, how they differ from virtual machines, and why sharing a Linux kernel matters for security. He examines container breakouts, the danger of assuming isolation is stronger than it is, and practical defenses using Linux security controls. The conversation then turns to patching: teams need to rebuild and replace images, track the software they contain, and test updates before deployment. Jay connects those responsibilities to third-party application libraries and explains why container convenience does not remove operational accountability. He closes with resources for keeping pace with Docker and Linux security.

The Application Security Podcast is brought to you by [Security Journey](https://www.securityjourney.com/).

About Security Journey  
Security Journey provides application security education for developers and everyone in the software development lifecycle.  
→ [Learn more about Security Journey](https://www.securityjourney.com/)

Connect with Jay Beale:  
→ [Jay Beale on LinkedIn](https://www.linkedin.com/in/jaybeale)  
→ [InGuardians](https://www.inguardians.com/)

Mentioned in this episode:  
→ [Docker](https://www.docker.com/)  
→ [Kubernetes](https://kubernetes.io/)  
→ [Linux kernel](https://www.kernel.org)  
→ [LXC](https://linuxcontainers.org/)

Chapters:  
00:00 Docker security with Jay Beale  
01:50 From Linux hardening to security consulting  
05:36 Containers compared with virtual machines  
10:35 Sharing the Linux kernel  
14:35 Why Docker changed the developer workflow  
16:08 Container breakouts and new security risks  
21:53 Controls that reduce breakout risk  
26:51 Rebuilding images and keeping containers patched  
28:29 Finding vulnerable software in containers  
32:53 Third-party libraries and application security  
36:31 Testing updates before deployment  
39:09 Where to keep learning about Docker security  
42:55 Final advice and training resources

## Transcript

*7,156 words · assemblyai*

**0:05 Chris Romeo:** The Application Security Podcast. Here we go. Hey folks, Chris here. On this episode of the AppSec Podcast, we explore the world of Docker security. So we got to this conversation because we had a listener reach out and say, hey, what's a recommendation you would have about a podcast I can listen to or a blog post I can read on the topic of Docker security? And we took a look around and we couldn't find anything. And so we went to the source. We reached out to an industry expert, Jay Beale, from from Enguardians and asked him to come and enlighten us on this topic of Docker security. And we'll also get into how does Docker security impact the world of AppSec. So thanks, and we hope you enjoy.

**1:04 Robert Hurlbut:** Okay, recording now. Hello friends, this is Robert and I'm with the Application Security Podcast, another episode. And today is going to be an interesting and special broadcast in terms of looking at Docker security. We had a question from a listener indicating that we had never covered that topic before. And so in that vein, we decided to ask our guest today, Jay Beale, to join us and talk about it. Hi, Jay. Thanks for joining us.

**1:46 Jay Beale:** Hi. Thanks a lot for having me. Absolutely.

**1:50 Robert Hurlbut:** And so, to get started, typically what we do when we have a podcast and have somebody on our podcast, we ask that person if they could give us your superhero security origin story. How did you get started in this?

**2:03 Jay Beale:** Oh, wonderful. Well, let's see. So, I started out a long, long time ago. I don't look this old, but I started out back in 1999, and the university I worked at had had a hacker running around and I had a lot of interest once we were sure that we'd booted the hacker in doing something proactively to make our systems a lot harder to break into. And so among other things, I started writing a Linux lockdown script that would make it— that would tune a system and make it a heck of a lot harder to break into. And that script eventually became the Bastille Linux program. And Bastille Linux in its heyday, and it's been some time since we've updated it, so it's not really maintained at this point, although I keep swearing I'm gonna start it back up again. But Bastille Linux had millions of people downloading it, got used throughout companies and governments and people at home. And that honestly got me a lot more opportunities to do work and to do different things in our field. So I got to write or edit or or team write so many books, I've kind of lost count. I think it might be 9. Some of those were in my Open Source Security Series and some of those were the Stealing the Network, what we called technical works of fiction where you can look at these and you can read these books as if you're reading a novel, but you'll see a whole bunch of— you'll see a whole bunch of typing this command and typing that command and those commands will all be—

**3:48 Robert Hurlbut:** Yeah.

**3:50 Jay Beale:** will all be, you know, very workable and you'll have explanations, you get explanations on what's going on and you're kind of reading a novel at the same time. So anyway, I wrote some books and I started giving lots of talks around Bastille Linux and other proactive Linux security tools or techniques and I just got a number of, you know, offers to do consulting work and I had some peers, I had some friends who were in similar situations where they were teaching at SANS or they were building tools and they were also getting a bunch of consulting offers. Eventually, we just joined up and we created Enguardians. Back then, it was Intel Guardians. We created a consulting company. Nowadays, that consulting company has been around now for 14 years and we have a bunch of employees who do just amazing work. And it's been really great. We get to do everything from the Linux defense work that I really love and specialized in early to penetration testing and red teaming kind of work, that attack-centered work that I've been doing at work most of the time for almost 20 years in my career. So at InGuardians, we have lots and lots of fun. I work with a bunch of crazy and crazy good people. And honestly, it's been a blast. I had no idea when I left my math PhD program that I was going to have so much more fun in IT security, but it's been a heck of a good time. Excellent.

**5:36 Robert Hurlbut:** Excellent. Well, thank you. So, I have a question about— let's just get started in Docker. What is Docker? I've heard the term. I know Some of our listeners have heard it, I'm sure, or maybe even use it, but what is it?

**5:47 Jay Beale:** Sure, good question. So Docker works— the easiest way to think about Docker for me is to start by thinking about virtual machines. So we've got— when I first started out in IT, it's been so long that we didn't have virtual machines. So we had these big computers and we ran a bunch of programs on them and we tried to get everybody on the security side, we tried to say, hey, could you please only put one big program per computer? So, run a SQL Server on that computer and the web app and your web application server on another computer, and then the web server front end on a whole other computer. So, those would all be running on 3 different pieces of bare metal if it was best practice. So, eventually, we got an old concept, got commercialized on on Intel hardware, and that was by VMware. And that's this idea of a virtual machine, of a hypervisor that takes the machine and splits it up, in essence, and kind of, let's say, splits it up into a bunch of virtualized hardware. And so each of those virtual machines gets its own view of the machine. It thinks it's on its own machine, and that own machine has whatever number of processors, processors and whatever amount of memory and whatever hard disk space you define. That's kind of your way of splitting up a big server into smaller ones. That's kind of one thing— that's been the paradigm for a while. What happened next was that people who were setting up what we'll call platform as a service, This is one of the kinds of, you know, whenever we talk about, I run my program in the cloud, right? What are we talking about? Well, if we— we're usually either talking about infrastructure as a service, where you basically tell Amazon or Microsoft or another, you know, or DigitalOcean or Rackspace that you'd like, you know, that you'd like to buy or rent one or more virtual machines. So that's one way. The other way is platform as a service, where you're going to put your program on, say, Red Hat or Amazon or somebody else's platform, and you can write your program in certain languages, and then your program is going to run on that machine, and it's going to run on that machine separate from the other programs, but not getting its own hypervisor instantiated virtual machine. It's not going to get a VM of its own. It's just going to run on there under certain controls. And all this platform as a service stuff was facilitated, you know, needed some way of having not just security separation, but also some way of separating the resources. So instead of VMs, they're using bare metal hardware because it's so much more efficient to do it that way. And the reason it's more efficient is the virtual machines, every single person's virtual machine gets its own full operating system. That operating system has a kernel, that operating system has whatever the system logger is, whether it's syslog or whether it's the Windows event log. Every single one of those gets every single part of the operating system. And so with a given piece of hardware, you could say, okay, we can put about 10 VMs on this. Or alternatively, in the platform as a service place, they used containers. And that's what we're about to get into here.

**9:29 Robert Hurlbut:** Okay.

**9:29 Jay Beale:** They used containers and these containers, instead of having 10 on a machine, it might be 100. Figuring out the number would depend on, you know, would depend in essence on what each container needed and, you know, and resourcing. But you could put a heck of a lot more containers on. You could put a heck of a lot more programs on if you didn't give each one its own hypervisor, its own virtual machine. So, Docker has taken this Linux containerization that I'm just starting to introduce, and it is one of the tools that's competed at creating containers, and it's made these containers so easy and so much more honestly fun to work with as a developer that it's just taken the world by storm. Just to highlight, the difference between a virtual machine and a container, or what a container is. A container, we're going to call for the rest of this interview, let's call a container a lightweight virtual machine. That's what I was going to say.

**10:35 Robert Hurlbut:** Okay.

**10:35 Jay Beale:** Yeah, and what we mean by that is we're running on the Linux kernel. We're just— you've got the Linux kernel running, and the Linux kernel is going to be shared between all the containers, and it's going to own the hardware instead of a hypervisor, and it's going to provide all of the security separation, all of the resource confinement, all of the, like, you only get to have this much of the CPU and this much memory and this much disk space and so on. So the Linux kernel is now gonna be providing that. And if anybody on here has heard of KVM, KVM's a hypervisor like VMware that's built into Linux now, We're not talking about that. We're talking about just the Linux kernel going and taking some— and taking a few kinds of 2 concepts. One is namespaces and the other is cgroups or control groups. And it's basically gonna use those 2 things to let you put a whole lot more stuff on the machine. And so I guess unless you ask me, I won't try to go too deeply into that, but I will say the control groups part is what says, Okay, this one program or this container can have this much, gets this much priority, CPU priority, or this much memory, or this much hard disk space in the system. Namespaces are what makes it look to each container that they're on their own darn computer. They each get their own version of the file system, they get their own version of the IP address in the machine and the hostname on the machine. They get their own view of memory and of the process ID space. So that's basically how the containers work. So Docker creates containers. Containers are being mediated by the kernel, and the Docker daemon is basically telling the kernel what to do. And the containers are using control group— are basically segmented from each other by control groups.

**12:47 Robert Hurlbut:** Okay.

**12:47 Jay Beale:** and namespaces, control groups. Okay. So I think I'm going to try to re-summarize it. I think I'll stop there. Thank you.

**12:54 Robert Hurlbut:** That sounds good. It sounds like, and from what I understand, I mean, essentially what you're looking at doing here is, is because of VMs, I mean, it was a, it was a great improvement over bare metal and not having to have so many separate real machines. Now you could put a few on a real machine, VMs, virtual machines, Now Docker with the containers allows you to put even more than that. As you said, your example of 10 versus 100 versus even more. And so it's another way of utilizing resources to be able to multiply the effectiveness. But as you mentioned, it sounds like you're still running against— you're running against the kernel instead of having a completely separate— all your resources as you would with a virtual machine. So those are some advantages but also some drawbacks, it sounds like, with Docker and containers. Okay, very good.

**13:47 Jay Beale:** Yeah, to get a sense for the efficiency, I have a Raspberry Pi. The Raspberry Pi won't, you know, the Raspberry Pi, it would be— I don't think there's even a way to do KVM on it, but, you know, I don't think there's a way to have anything like VMware in a Raspberry Pi, but I can comfortably run 4 containers on a Raspberry Pi in Docker. So it's just, it takes a lot less, it takes a lot less resources to be able to do that, to be able to do the same kind of virtual machining as it were, the separation of one piece of hardware into multiple virtual hosts. That's probably a good way to say it, virtual hosts here. Okay.

**14:35 Robert Hurlbut:** And not only the number of of things that you can run, containers that you can run. If I understand correctly, it's also how quickly you can spin these up and tear them down. That's also another big win for why you want to take a look at this as well, correct?

**14:49 Jay Beale:** Oh, certainly. So, you know, first, a container starts in seconds versus a virtual machine starts in whatever time it takes you to boot on the system. But the other part is honestly what What Docker has done that Linux containers really hadn't done so well before Docker came along is made it— is just created a whole bunch of really easy and fun and capable and effective tooling to be able to quickly create a container, tear it down, duplicate, you know, like bring it back up, bring up a container that was just like the previous one but with just one change and, you know, and and all of it and just kind of throw containers away as you like, create new ones and work with them just really easily. I'll tell you, it's just a blast. And I don't mean to sound like too much of a Docker fanboy, but it's really quite fun.

**15:46 Robert Hurlbut:** Well, it certainly has taken the application security world or application development world by storm. I've seen it. A lot of shops nowadays are moving towards Docker. using containers a lot more than they did in the past few years. Certainly it seems like it's taken by storm, and so it sounds like a good reason.

**16:06 Jay Beale:** Certainly.

**16:08 Robert Hurlbut:** Okay, well, if we could just shift a bit, we've talked about Docker and containers and what they are and how they're different slightly than virtual machines, but let's talk about the security of that. And first of all, are there any threats that may be introduced by now pulling in Docker and containers into our environment?

**16:28 Jay Beale:** Sure. I think I'd like to think of 2 of the major threats. And the first one is that I talked about VMware and other kinds of virtual machine hypervisors, like KVM was, you know, KVM is another one, Xen Server is another one. With— in hypervisors, there have been hypervisor breakouts where you've got a program that's running in one virtual machine and an attacker takes it over, or maybe an attacker was the one who put the program there in the first place, and the attacker is able to either break from the one virtual machine into the others, or the attacker is able to break from the one virtual machine into the hypervisor, into the host that's running all the VMs, and thus start contaminating the other virtual machines.

**17:19 Robert Hurlbut:** Right.

**17:21 Jay Beale:** virtual machines. So that kind of breakout risk has been present on hypervisors like VMware, but the nice thing about it is that hypervisors are such a mature technology that people have been breaking out of them for a long time. And so people who write those hypervisors have been putting a lot of effort into making them much harder to break out of. I like to say anytime I'm talking about this that nGuardians made the first hypervisor, the first ESXi breakout that was publicly known. And it took us quite a lot of effort, but once we found it, there were a number of other people who found similar things. Nowadays, that doesn't happen anywhere near that often because hypervisors are pretty mature from a security perspective. Containers, on the other hand, are really, really a lot newer. And I guess that's part of the reason we're talking about them, you know, that we're talking about Docker and containers right now.

**18:17 Robert Hurlbut:** Yeah.

**18:19 Jay Beale:** on a podcast because it's still that new exciting technology that— I mean, while some form of containers have been around for quite a while, they've become so much more popular now that they're finally— they're just in the last few years starting to get a lot of security research. And so, the first threat that I think of when you're using Docker is if you're using Docker and you're doing anything like multi-tenancy where you've got Let's say first either where you've got multiple customers and each one of them is in a different container and that's on the same either metal or they're in the same VM, but they're just separated by the container. Container breakouts are going to happen and they're going to happen more commonly, I suspect, than hypervisor breakouts for a while until this gets more mature. So that's our first risk. Now, I did just say multi-tenancy. The other kind of thing with multi-tenancy is basically, do you have both a public web server and then some kind of internal, you know, let's call it more sensitive data stored in another container running on that, you know, running right next to that web server? And if you do, the breakout might not be between 2 customers. It might just be that someone hacks the web server, and then from the web server, they break out into that—

**19:45 Chris Romeo:** Yeah.

**19:46 Jay Beale:** into that sensitive data, whether that data is something that you might think of as protected health information or credit card numbers or really sensitive intellectual property or what have you. So, if Docker were to introduce any threats, the first one would be that if you use it, would be that people may think that it provides just as much protection as either a either a hypervisor like VMware KVM, or that it provides as much protection as having separate bare metal. And that just isn't the case right now. We can talk if you like about the kinds of countermeasures you can use to make a breakout much less effective or to break a breakout. But that first risk is really the risk of breakout. Honestly, the other threat that Docker brings, and if the Docker folks were listening, I'd want to make sure that I was really careful in my language here, is that when you create containers, a lot of people, they spin up containers and they don't think about the lifecycle of the container. So, is that I got this container, I pulled it down from a registry like Docker Hub, and And now I've got this container and I'm going to use it for a little while. Is it going to be around for— we don't really think about patching too much. I mean, containers don't really have— you're not shipping full operating systems usually in a container, especially in a Docker container. And so, you have that question of, well, are you going to patch it or replace it? Are you going to make sure to do that often enough? So, the other threat is that there are a lot of stale containers out there that people started and they didn't think through how they're going to patch or how they're going to replace them whenever patches come out. So I guess that's the other major threat to consider. The former is kind of baked in, the latter is just a matter of practice. The nice thing is you can do something about both of those.

**21:53 Robert Hurlbut:** Okay, and so that actually leads into my next question is, okay, now we know there are some threats and risks and running containers, what are some ways that we can deal with those?

**22:02 Jay Beale:** Sure. Well, let's— I've got a couple. So first, on that breakout concern, there's some really cool things I've seen. One of the things you can do— Red Hat, if you're using CentOS or Red Hat Enterprise, by default, Red Hat's actually done a really good job with SELinux where if you run Docker, each of your containers gets a different compartmentalized— gets a different MCS or compartment label in SELinux. So if you're not running on one of those, you certainly want to think about using SELinux to separate the containers. If you are running on something that has SELinux, you know, that's doing that already, you can look at using SELinux. And if you like AppArmor, or one of the other alternatives like SMAC or Tomoyo. But, you know, first and foremost, use a Linux security module that can provide even more separation between the containers so that if a process was able to see from its container into the host or into other containers, SELinux would be stopping that or providing a lot more restriction. The next thing along the lines of container breakout is, and this one's probably the absolute most important, is you have to use an unprivileged container. So that's basically a container that doesn't have root, that's running as a non-root user. And I should say that that's honestly so critical that one of the other container projects Canonical's LXC says that they won't issue a CVE if you come up with a way to break out of the container if it was a privileged container. So unless you're using an unprivileged container, they won't even treat a breakout as a bug that they really want to spend time handling and getting a patch out for. So you can also use 2 other things. The first is if you've got a root container, you can use what's called capability stripping. And that's where the— that's where if you have root privilege on a Linux system, you get a whole bunch of special powers. And all those special powers, like the ability to write to any file even if it wasn't owned by you, or the ability to listen to a port under 1024, that's— those powers of root are actually all split up and enumerated. And they're enumerated as capabilities. And so, you can say, okay, my container only needs one root-level capability. It only needs the ability to bind to ports under 1024. So, getting your container, if it does have to run, to either not run as root, be unprivileged, or run as root, but then you strip out all the capabilities except for the one or two you need, that does a lot towards breaking breakout. And then, the last one is, and Docker has a pretty decent default for this. There's something called seccomp, and seccomp takes the syscalls that underlie what your program does on the file system and on the network and kind of interacting with any of the computer's resources, and it makes a blacklist or a whitelist of what syscalls that program is allowed to use. And that's what seccomp does. Docker ships with a pretty good seccomp profile. If you are really, really interested in getting incredibly strong security around your program, you can create a custom seccomp profile that's even tighter than Docker's. And so, if you take those 4 techniques I just went through, the Linux security module, the unprivileged container if you can, the capability stripping if you can't, and then seccomp either way, you end up with a lot of protection against breakout. and a lot of— and a lot better security. I mean, this is so much— I teach a training class at Black Hat that I've been teaching various forms of it at Black Hat and other conferences since 2001. And I teach containers in my Linux lockdown class specifically so we can just say, okay, you've got programs running on the system. Can you do better than the old chroot or chroot jails and put those programs, you know, put the most sensitive network-attached programs into a container?

**26:32 Chris Romeo:** Yeah.

**26:33 Jay Beale:** And then, you know, wrap that with Linux security module and end up with something where if an attacker can try to exploit the program, his exploit either doesn't work or gives him, say, a shell that doesn't do anything. So that's what I do on the breakout side.

**26:50 Robert Hurlbut:** Okay.

**26:51 Jay Beale:** On the other— but for the other threat we were talking about, you've got, you know, that question of how do you keep these things patched? That is less technically involved than avoiding the breakout, but it's certainly the one that takes some thinking. The best practice here isn't to try to start patching your containers, although that can be what you do in the short term, but it's basically you start watching for the security vulnerabilities that come out in the software that's in your container. Whenever you have one of those vulnerabilities come out, you just rebuild the container with a fresher patched version of the image. Got it. And the awesome thing about doing that is the containers generally rebuild so darn fast. And given that when you're using containers, you usually have, you know, you've got one container image that's hosting, that you're using to host 10 or 20 or 100 copies of a program, you know, you get, instead of patching 100 virtual machines, you've patched one image, or rather, you've, you know, you've recreated one image And that one image is now patched across. So, you can get much faster patching as long as you make that part of your process. And, you know, I've seen companies make it so that their containers last them for a day or a week. I've seen them get— honestly, there are companies where their containers last them only a few hours. And I can imagine a day where, you know, your container might last you for one transaction and then you kill it off and you're just constantly you're constantly rebuilding the containers and using more recent images.

**28:29 Robert Hurlbut:** Okay. So in terms of trying to find out, are they patched or not patched, are you running different scans? Are you keeping up to date on what vulnerabilities that may be found or available so that you can verify, do I have that in my container or not? I mean, is that how you're keeping track of that? Is that one way of doing that?

**28:49 Jay Beale:** Yeah, it's a really good question. There are tools out there now that will investigate a container that you have and look at the different software that's in there and basically tell you whether you've got vulnerabilities that need to be patched. I'm not sure if there are any of those that aren't commercial, but the other part that you can do honestly is basically have an inventory automatically created of what software is in each container. And one of the goals of containers is that they're supposed to be as minimal as possible. So in Docker, by default, you start up a container and you start up that container with one program. That program might run some others, but a container generally, like, you don't put an SSH daemon in each container. You don't put a system logging program in each container. You don't generally even run something like systemd or that first init program on a Unix box in the container. Your container is kind of— this container runs an Apache web server, so it has that first Apache web server process that started along with all of its little workers, all of its copies of itself. So if you start with Docker, if you're starting with Docker container, the containers that you get are kind of stripped-down versions of an operating system. But I'd recommend even from there, once you've got something that you're going to go towards production with, stripping it further and saying, okay, what are my program's real dependencies? And stripping that down to such a small number that you won't have to patch all that often. You won't have to rebuild or update all that often. And because you're not going to have that, oh darn it, I've got vulnerable code moment coming up anywhere near as often. But I will say this, inventorying the software that you're running with a program, whether it's on a virtual machine, a computer, a container, or what have you, it's kind of been a pain in the butt. It's not an easy thing. You can script it to make it easier, but we're constantly realizing when you deal with web applications, most web applications are pulling in JavaScript from across the web, or they're pulling in their own local copies. And so, even when you've got a package manager for the container, you have to start realizing, well, wait, am I also using a package manager for my web app framework, for Ruby on Rails, or my package manager for Python, or my package manager, you know, npm for JavaScript, and trying to track all of those? So, it's It's kind of— it's a little bit similar in containers to just on normal computers. The nice thing about a container is you get to have a much smaller system in there, and so it's a bit easier.

**31:51 Robert Hurlbut:** Sure, but there's still some management, it sounds like. I mean, you're not escaping that. You still have to manage what is it that I'm running, what are the things I'm running running, and so on and so on. you make sure that everything is as secure as possible and up to date with patches and so forth, it sounds like.

**32:08 Jay Beale:** Oh, certainly. It's the— I mean, the thing over 20 years in this field that I've noticed is that anytime someone says, okay, and once we do this, that problem is no longer a concern, those are the places as a security professional that I know to then look. Wait, is our understanding of that's no longer a concern, correct? Or is it— have we kind of glossed over something? You know, maybe we were reading marketing material or hearing somebody else's talk or what have you, and you just dive in and learn a little bit. I'd love to, if you'd like, I'd love to talk about a few other, you know, a few other kinds of the protections or attacks that are here, that are present in this space.

**32:53 Robert Hurlbut:** Well, I was going to say, if we could maybe shift slightly to application security, Sure. And at the end, I want to ask some further ideas and research and information if we can, if you can help us with that. But in particular with application security, you did touch on the different platforms that you might use, web applications and so forth, environments. Are there some other things in regards to application security to think about when you're building a an application within a container and thinking about Docker security in general and related to those?

**33:30 Jay Beale:** Sure. Well, so one of the— I think it's a decent segue. We were kind of just talking about all, you know, when you have an application, you've got a whole bunch of, we'll call them third-party libraries, or just to generalize, third-party code you're pulling in. All of those, you know, all those different JavaScript files that are coming into your web application or all of the— whatever all those kinds of— I just think of them as include files. All the dependencies that your application needs, you want to be making sure— so first, you want to be making sure that you're getting— that you know what your dependencies are and you're keeping track of what their vulnerabilities have been, that you're choosing those dependencies well.

**34:20 Robert Hurlbut:** Yeah.

**34:24 Jay Beale:** well, and that you're tracking them over time. For me, one of those is thinking about rather than pulling a library down, rather than pulling a static version of a library down from a site and including that in your page, in your web page or what have you, are you packaging that library? And now if you're packaging that library, are you making sure that that you're repackaging whenever that library gets updated. And that question in particular, honestly, I've seen a few talks by people who've done research into this. The number of times where your application doesn't have a vulnerability itself, but one of the libraries, some of the code that it depends on does, and you don't realize it, you're shipping with, you're downloading and including or you're having a browser include vulnerable code, that just happens so much that it feels incredibly difficult to deal with. The other part of that for me is when we're dealing with containers, are you going to remote include libraries, which on the security side we never like, or are you going to pull it down in the container? If you're going to pull it down in the container, you need to make sure that, let's say, your Dockerfiles, which are like a Makefile in Docker that that tells Docker how to build your container or how to build the image that your container rests on, you've got to make sure that your build process is pulling down the newest copy or is pulling down a safe copy when it's building the container instead of just the one you pulled down when you designed this application and you haven't gotten to the point where you've thought about the maintenance of that as that library changes over time. I think that's probably the biggest thing that I would hit from an application-specific side. Do you have anything that you would throw in there, Robert?

**36:31 Robert Hurlbut:** Well, I like the idea of getting the latest, but I also think about testing, making sure that anytime you do that, that you still run your regression tests, integration tests, all the other tests, security tests to verify that now that I've got a new version, are there any incompatibilities? Because obviously just, you know, pulling down the latest, I may have developed the code, I may have gone through all the aspects of making sure that everything runs with my version of the code and my version of the libraries, but then when you deploy and you pull down the latest, well, now there might be some inconsistencies. Yeah. And so I think that needs to be factored in as well in terms of application development, but also application security, because you may also inadvertently open a potential problem within your code now that it's relying on an older version of the library. And now you've just pulled down a new version of the library. Does that potentially open something else that maybe there's a library— or function, rather, within the library that's no longer valid or something like that. Who knows?

**37:37 Jay Beale:** Oh, yeah.

**37:38 Robert Hurlbut:** I would add that in as well, just to make sure that you retest before you deploy with a new version of the library to make sure that everything is still working correctly and tests correctly.

**37:49 Jay Beale:** Yeah, I have to admit, when I said newest version, the words came out of my mouth and the second thoughts in my head popped up and started saying, hey, wait a second. Newest? Are you sure you're going to You're going to use the newest version without looking into it? I mean, we've— I've certainly seen situations where you pull down a new version of the library and, you know, it's worse than just a function no longer existing or a function being, you know, function being deprecated. But, you know, I've seen situations where the function's arguments change.

**38:18 Robert Hurlbut:** Sure.

**38:18 Jay Beale:** And not in a way that get immediately caught. It's, you know, now the -c flag or the, you know, or the var2 flag is going to mean this. And you're like, wait, wait, that's huge. And the library developer just figures you're just sitting around every day watching their GitHub or watching their documentation, looking for subtle changes, and that's all we have to do all day. But yeah, that's certainly critical. I think you're totally right, and we need to just think of that. I mean, the biggest thing for me to say is that Docker isn't going to change some of the fundamental problems that you have or some of the fundamental problems you've had to solve in application security. Like exactly that.

**39:09 Robert Hurlbut:** Yeah, that's a really good point. So it's not going to do everything for you. You still have to think about what you're doing and apply still some really good principles and ways of doing, you know, how we've been writing secure code or trying to write secure code for a long time. It's not going to solve all those things. You still have to think through those problems and solve them yourself as well. Okay, great. One last thing as we finish out here, and it's been great to talk to you today about this topic, but in particular for our listeners, they may want to know where can they learn more about Docker, Docker containers, and security related to Docker. Where are some good places that they might look for information?

**39:50 Jay Beale:** Sure, that's a good question. So honestly, I would— Docker's got some— Docker has some really good documentation and blog posts, and I think honestly just following the blog posts from the Docker team, from the Docker maintainers, have been really useful over time. There are, you know, you'll start to notice as you read the blog posts which of the Docker maintainers are most into security security and are most interested in security. And I'd go to watch their blogs and honestly even more than that, watch their GitHubs for the projects that they've been trying to solve. There's a Docker maintainer who's now switched over and is working on Kubernetes. Her name is Jessica Frizzell and she gives some good talks and she's also got some good blog posts and also some just projects in her GitHub. And I read just yesterday, I read a tweet from a kernel hacker over at Stripe who was saying, you know, each time I try to get into another, you know, each time I'm wrestling with a new problem in container security and I start to work on it to create something, I go and I do a search to see if anybody's already been there and it turns out, you know, Jess Frizzell was there 2 years ago, and thank you. But look at the other people that are Docker maintainers and look at what they're doing in the security space. I just mentioned Kubernetes, and that's another good one, is basically watch the Kubernetes team and what they're doing in the security space. I think you'll learn a lot. I mean, I like to recommend books and I love books, but to be honest, this Docker is— Docker containers, Kubernetes, they're all moving so fast that if you read the books to get yourself a good baseline, and Docker Up and Running is a good one from O'Reilly, you're going to find that you really need to get on the web.

**42:06 Chris Romeo:** Right.

**42:07 Jay Beale:** to keep current enough.

**42:09 Robert Hurlbut:** Right. Yeah, it'll be out of date by the time you're finished, and, and now they're already beyond a little bit. So that makes sense.

**42:15 Jay Beale:** Yeah. That said, I do, I do think there's a lot of value in the books. If you think about it, you know, they say, you know, um, by 5 years in practice, what the, uh, what the average doctor learned in medical school, a tremendous amount of it's out of date. And yet you wouldn't want a doctor who hadn't gone to medical school. Um, you, you just, you know, you want them to have Started with a solid foundation and then added on to it. So that would be the route I'd take is take a look at a Docker book, get into practice. They're great. There's great documentation, so look at the documentation. But then to go that next step, start reading the blog posts, start looking at GitHub for the maintainers, and stay on the edge.

**42:55 Robert Hurlbut:** Okay, great. Well, Jay, thank you again. We really appreciate you coming and speaking to us today about Docker security. Are there any last thoughts that you might want to share with our listeners?

**43:05 Jay Beale:** Let's see. I think I'll throw in a plug. I do Linux security training and that includes some work with containers and Linux security modules. So watch out for my training classes. Watch InGuardians blog because we do a lot. We've been doing a lot of posts lately on Linux security and that's just @inguardians. And I'm @jbeale on Twitter.

**43:31 Robert Hurlbut:** Okay.

**43:32 Jay Beale:** So if I were to think if I could throw in something else that's of value, it would be no matter what you do, look at the— if you get into containers enough, you're going to end up with an orchestration platform of some sort. And so when you do, some of the biggest things to do are authentication and authorization, and then network. So basically, Think 2-factor auth on authentication. Think about role-based access control. What does everyone need to be able to do? Who needs to be able to use this? And then network management. Make sure that the containers can never— make sure the containers can't reach the host and they can't reach each other.

**44:14 Robert Hurlbut:** Okay, excellent. Well, great advice, and thanks again, Jay. Appreciate it.

**44:19 Jay Beale:** Sure thing. Thanks, Robert.

**44:20 Chris Romeo:** You bet.

**44:21 Robert Hurlbut:** And to our listeners, have a great day.

**44:22 Chris Romeo:** Thanks.

**44:23 Jay Beale:** Thank you. Thanks for listening to the Application Security Podcast. Our intro music is 8-Bit Kung Fu by Boring and TJ, and the outro is Southern Delight by Stefan Cartenberg.

**44:37 Robert Hurlbut:** You can find us on Twitter @appsecpodcast or on the web at www.appsecpodcast.org.

---

Source: https://appsecpodcast.com/jay-beale-docker-security-and-appsec/