Jannik Hollenbach — Multijuicer: JuiceShop with a side of Kubernetes

0:00Chris RomeoYannick Hollenbach is a security automation engineer at Idera Tech GmbH, working on and with open-source security testing tools to continuously detect security vulnerabilities in the company's software and systems. He's also a member of the OWASP Juice Shop project team. Yannick joins us to discuss MultiJuicer, or how to run Juice Shop in a Kubernetes cluster with a separate Juice Shop instance for each user. For season 7 and beyond, we've launched our YouTube channel, Application Security Podcast, where we post the video feeds for all our episodes. You'll want to check it out, as many interviews now have demos included where we capture screen during the interview. We hope you enjoy this conversation with Yannick Hollenbach. At Security Journey, we believe security is every developer's job. We work with our customers to help them build long-term sustainable security culture amongst all their developers. Our approach is to provide security education that's conversational. Conversational, quick, hands-on, and fun. We don't do lectures. Instead, we let the experts talk about what's important. Modules are quick, 10 to 20 minutes in length. We believe in hands-on experiments, builder and breaker style, that allow your developers to put what they learned into action. And lastly, fun. Training doesn't have to be boring. We make it engaging and fun for the developers. Visit www.securityjourney.com to sign up for a free trial of the Security Dojo. Hey folks, welcome to this episode of the Application Security Podcast. This is Chris Romeo, CEO of Security Journey and co-host of the podcast. I'm also joined by Robert Hurlbut. Hey Robert.

1:49Hey Chris, yeah, it's Robert Hurlbut, Threat Modeling Architect. Good to be here.

1:53Chris RomeoAnd we are joined by our first ever video-based guest here. This is Yannick Hollenbach. And Yannick, the way we like to start these conversations is we want to dive right into what is your security origin story? How did you get into this crazy world of application security?

2:11Jannik HollenbachYeah, I think I have a pretty normal background, I would just say, because my security background basically started in university. Where I took a security class and it was a really nice security class because we were actually using OWASP Juice Shop in the class to learn about security vulnerabilities and testing different security vulnerabilities out. And it was a really hands-on class. So the instructor, Timo Pagel, who's also a member of the OWASP Juice Shop team, did a lot of walking around helping us with the challenges, instructing us how to do certain things. And it was really, really cool and it got me really hooked on security. And I basically, after the class, I had to do an internship and I just walked up to Timo and said, hey, I would like to do more on that. And he recommended me a company to work for. And it's basically how I got to my current company because Timo knew that they were also using G-Shop for their security trainings and said, okay, that sounds like a good match. And that's how I started the current company doing various security topics and also got me to work more on GShop because I'm now like a member of the GShop team.

3:22Chris RomeoSo when you started your college career, then you were planning to be a developer, just be someone who writes code, or what was your original plan?

3:33Jannik HollenbachYeah, my original plan was just to be basically a normal developer. And my current plan hasn't really changed that because I still want to be at least some form of a developer. But I currently have like a role where I do both. I do security and I also do development. It's something which I'm really happy with because I can do both things which make fun to me.

3:55Chris RomeoCool, cool. So it's exciting that you got your introduction to application security through Juice Shop. We've had Björn on the show a number of times and we're big fans, both Robert and I, big fans of Juice Shop and all the things that it can do. And so it's pretty cool to hear somebody like, you use Juice Shop and then you're like, hey, I think I can make a career out of this.

4:15Jannik HollenbachYeah, I think that's gonna be like a trend in security that Juice Shop is gonna be the introduction to security for a lot of folks because it's very accessible for new people wanting to begin with security and want to learn stuff.

4:28Chris RomeoYeah, and there's a lot of cool things happening in the Juice Shop front as well that's gonna make it even more usable from the perspective of somebody who's new. So Juice Shop, we've talked to Bjorn, like I said, a number of times. The topic we want to discuss today is this project that you're working on called MultiJuicer, which once again, great name, fits into the whole Juice Shop kind of scheme. MultiJuicer. I wanted to hear though, what's the origin story of MultiJuicer? Because whenever we talk to OWASP project people, we always ask this question because I find it fascinating. What made you want to build this project and what was the story for how that happened?

5:06Jannik HollenbachIt's basically, it basically started from a problem we had at our company because we're running our security trainings with Juice Shop. And we are basically doing like 1-day trainings to get the developers up and running and teach them the basics about security. And we just noticed that we were spending a lot of our time on the trainings to help people set up Juice Shops on their own machine. So whenever we did a 1-day training, the first 2 hours would be just troubleshooting problems people would have with Juice Shops on their machine. And Juice Shop is a really, really easy-to-install application, at least in most cases. But no matter how easy the application is to install, something can always go wrong. And we would see that it just happens. And we basically asked ourselves, okay, how can we just provide the Jupyter instances to the trainees? And we came to the conclusion that the only good way to do that is if we host all the Jupyter instances ourselves. And the problem with Jupyter is basically that Jupyter is a single-user application.

6:07Chris RomeoRight.

6:08Jannik HollenbachIt's not really intended to be run by multiple users at a time. And our solution was basically, okay, we have to— if you have a training for 20 people, we have to run 20 JuShop instances. And we need to find a way to expose these instances to the users. And that's how we landed at the current architecture of MultiJuicer. MultiJuicer is basically a wrapper around JuShop and it's basically a load balancer Balancing traffic to individual Juicer instances.

6:39Chris RomeoIs— now, is MultiJuicer tied into Kubernetes, or can I use other orchestration platforms, or kind of what's the tie-in here between MultiJuicer and orchestration?

6:52Jannik HollenbachYeah, we had a few iterations of MultiJuicer before we open-sourced it, and our current version of MultiJuicer is basically really tied to Kubernetes. So the basic flow is a user for the training goes to the MultiJuicer page, which is basically just a registration form where they can put in their names and say, okay, I want to have a Jupyter instance. And they click on the join button or the start hacking button, I think it's called. And what's happening in the background is that MultiJuicer tells Kubernetes, okay, Kubernetes, please spin up a new Jupyter instance for me. And we will then wait until Kubernetes signals us that the Jupyter instance is ready. And we can then start— we can then say to the participant that the Jupyter instance is ready. And they can basically use Jupyter just like they would on their normal machine. They wouldn't have to go around the hassle of setting it up on their machine themselves, which basically meant that our trainings now have like 2 additional hours because we don't have to work around troubleshooting the instances of the people.

7:57So does each person then have their own instance? If I have 10 people connecting, you have 10 Jupyter instances running, for example?

8:07Jannik HollenbachThat depends a little bit because normally if you have like trainings where every participant just works for themselves, then we have 10 people equals 10 Jupyter instances. And MultiJupyter will just go ahead and spin them up by itself. So we don't have like to have an administrator to spin up the instances by hand because it just happens automatically in the background. But we also have like the concept of teams. So when you join a team or when you basically start MultiJuicer, you type in a team name. And if somebody else wants to join the same team, then they could put in the same team name and the PIN code which the person who originally created the team got. And so that they can work on the same Juiceshop instance and collaborate on there.

8:50Okay, so you can have a team of however many that's connecting to one Juiceshop instance.

8:55Jannik HollenbachOkay.

8:56Chris RomeoIs there reporting based on that then? So like, is there points that the team earns together or anything like that, like a leaderboard?

9:02Jannik HollenbachCurrently there's no like, no such thing like an automatic leaderboard. It's something where we really want to go in the future. And what you can always do is G-Shock has like this automatic, well, maybe not automatic, but they have like a CTF mode where solving a challenge gives you a flag which you can then put into a CTF leaderboard like CTFD. Or I think Facebook Capture the Flag, or I think there was like a third one which just got integrated into G-Shark. And, but there are different options at the moment, but it's basically like a, not like a manual procedure. So the participant would have to go copy that flag and paste it into the CTF platform.

9:44Chris RomeoSo with Multijuicer then, what do I get when I download Multijuicer? Is it a Docker container which then runs Kubernetes in a Docker container, or what is, what are the components Yeah, basically, it works a little bit different because Multijuicer doesn't really handle the Kubernetes cluster itself.

10:05Jannik HollenbachSo we assume that you already got a Kubernetes cluster up and running.

10:09Chris RomeoYeah.

10:10Jannik HollenbachBut we also have like a few guides to help you go through and create Kubernetes cluster on different cloud environments. So we have like a guide for AWS. We have a guide for Azure, I think, one for DigitalOcean. which walks you through how to create a Kubernetes cluster specifically for MultiJuicer and then also how to install MultiJuicer onto the Kubernetes cluster. And the way we package up our application or MultiJuicer in specific, we have like a Helm chart and Helm is like a package manager for Kubernetes. So as soon as— as long as you have like a Kubernetes cluster up and running, you can basically just run in your command line Helm install MultiJuicer. And that's basically all you have to do and Helm will then install Multijuicer on the cluster. And then you have Multijuicer up and running and can start hacking on the Multijuicer instances.

10:57Chris RomeoSo do I get upgrades through Helm then on a given cluster when you release a new version of either Multijuicer or Ju Shop?

11:07Jannik HollenbachYeah, Multijuicer basically has like a fixed Ju Shop version it is using. The current Multijuicer version is 3.1, I think, and it comes with Ju Shop 10.1, which is the most recent Ju Shop version. release. And if you want to do an upgrade to a newer Juju version, you would have to run Helm upgrade to upgrade to the newer Multijuicer version and the package upgraded Juju version.

11:35Chris RomeoOkay. So sounds like the most complicated thing in Multijuicer is probably gonna be setting up the cluster.

11:42Jannik HollenbachYeah. That's also something which we would like to like to help people who are just starting with Kubernetes a little bit more. We're currently thinking about and planning a little bit to provide Terraform modules.

11:59Okay.

12:00Jannik HollenbachWhich would basically give us a way to just say, okay, if you want to run Multijuse on AWS, you can just say Terraform up and then include the, Multijuicer Terraform module for AWS and it will set up, I think they're called EKS clusters, Elastic Kubernetes Service on AWS.

12:21Chris RomeoYeah.

12:22Jannik HollenbachIt would basically include everything from the box and would give you like a recommended Kubernetes cluster for Multijuicer.

12:29Chris RomeoWhat's the largest number of Juice Shop instances you've seen anybody spin up inside of Multijuicer? I mean, are we talking 10,000 Juice Shops?

12:37Jannik HollenbachI haven't seen 10,000 Juice Shops yet. The problem is I heard a few rumors of people using it for at least 100 to 200 Juju instances in parallel. But I'm not sure. I don't really know the numbers because I wasn't involved in setting these instances up myself. Yeah, also pretty cool that as soon as we put it open source, some, some larger organization already used MultiJuicer to run larger trainings than we ever did internally. And it seems to have worked out nicely because I never got any any cries for help that anything broke. I'm currently talking with somebody who's planning to use MultiJuicer for trainings with up to 1,000 people at a time, which I'm pretty excited about because it's like an order of magnitude higher than I've seen it before.

13:29Chris RomeoYeah, you're certainly solving a problem that is real. So I did an in-person training with our OWASP meetup here in the Triangle of in Raleigh, North Carolina. And what I did is I just went to Heroku and used the Juice Shop deploy button and just hit it a bunch of times in a row. So I ended up spinning up like 10 instances of Juice Shop in my free Heroku instance area.

13:54Right.

13:55Chris RomeoBut it was— yeah, I mean, so I mean, but it was, it was kludgy in how I had to do that. And so I love the fact that you're giving people something that's more of a systematic approach And we have some cool things you're gonna be able to do in the future because you've got everybody connected through a single cluster.

14:11Jannik HollenbachYeah, that's also something which was pretty important to us, that you have the ability to run all the GShop instances under the same address. If you spin up 10 Heroku instances, you probably have like 10 different addresses you then have to hand out to the participants.

14:25Chris RomeoI did.

14:26Jannik HollenbachI did.

14:27Chris RomeoI ran around table to table and I was throwing out pieces of paper like, here, this is your instance, this is This is yours. It was messy.

14:33Jannik HollenbachYeah, that's just a hassle we would like to avoid in the future. And that's basically one of the reasons why we started MultiJuicer.

14:39Chris RomeoSo what have you seen from the adoption perspective? You mentioned in our pre-discussion about adoption of MultiJuicer since it went open source. What is that? What have you seen there?

14:51Jannik HollenbachYeah, it's pretty interesting because it's something which a lot of companies are like, a bit trying to hide if they're using it or not, or at least are not like announcing it out in the wild that we're using this. But we've seen a few companies which are a few larger companies and also some smaller companies starting to use JuSHoP in their own trainings, which I'm also pretty excited about. The first, at least I think I know of 2 universities which started using MultiJuicer or are currently evaluating MultiJuicer for their security classes to provide G# instances for the students, which is basically like a nice tie back to my startings of my security career with G#, using G# in classes.

15:38Chris RomeoWhen you think about the future then of MultiJuicer, what are some of the features and things that you have on the roadmap that we'll be able to do in coming revisions, coming versions?

15:49Jannik HollenbachOne feature I'm really looking forward to is like a dashboard view to see who or which teams have solved which challenges. To give trainees or trainers more— mostly trainers— a better overview to how far the participants got along, which challenges did they solve, which challenges are they having problems with right now. And just also for instances of MultiJuicer where we're not like catered to like a 2-day or 1-day trainings. But basically like long-running Multijuicer instances where you just say, okay, we at company XY have like this Multijuicer installation and whenever you want to learn something about security, you can just go to Multijuicer, go onto your G# instances and learn something and give like the security teams a better insight about what challenges individual developers have already solved and a better insight to how far the security trainings got along.

16:48Chris RomeoSo if somebody wants to provide input or feedback on MultiJuicer, what's the best avenue for them to interact with you and with the project?

16:56Jannik HollenbachThe very best way for at least some, like, easy-to-solve questions or just general feedback would be the Project Juicer channel in the OWASP Slack space. We are pretty active there, both me and Björn, in providing feedback there. So if somebody has a question and wants an easy answer, We are pretty— we respond pretty quickly on the questions. And if somebody has like a bigger problem where they need to like include a stack trace or whatever, then the best thing is to open up an issue on the GitHub page.

17:32Chris RomeoGreat. Yannick, any final words you would leave for our audience? Advice as somebody who's spent a lot of time with JuShop here so far? What's kind of your concluding remarks?

17:47Jannik HollenbachYeah, definitely try out Multijuicer. And also always feel free to give us feedback, both positive or negative, on how you're using it. Also, maybe if you're a company who is using Multijuicer and I don't know about yet, maybe we should start something like a, like a section in the README of companies who are already using it and can provide some feedback on how they're using it. And also we are always open to new pull requests, both Juiceshop and Multijuicer. to new features or customizations they want to do. That would be really cool.

18:21Chris RomeoWell, Yannick, thank you for all the efforts you're putting into building this project. You know, most of our audience knows that people aren't getting paid in the OWASP universe. I mean, it's, it's volunteers that are making this stuff come together because you love security, you love helping other people learn and help other people get better. So thank you for all the efforts you're putting into this, and we look forward to future versions of MultiJuicer As you get that dashboard, you get other things in here that's exciting for us as well as practitioners. So thank you very much for sharing with us today.

18:53Jannik HollenbachYeah, thank you for having me. It was a really nice time. Thank you.

18:56Chris RomeoThanks for listening to the Application Security Podcast. You'll find the show on Twitter @AppSecPodcast or on the web at www.securityjourney.com/application-security-podcast. You can also find Chris on Twitter @edgeroute And Robert, @RobertHurlbut. Remember, security is a journey, not a destination.