Robyn Lundin -- Planning & organizing a penetration test as an AppSec team

0:01Chris RomeoRobyn Lundin started working in tech after a coding bootcamp as a developer for a small startup. She then discovered her passion for security and pivoted into pen testing for the NCC Group and now works as a Senior Product Security Engineer for Slack. Robyn joins us to discuss the role of penetration testing within the application security realm. Robyn provides actionable guidance that you can apply directly to your application pen testing program. We hope you enjoy this conversation with Robyn London.

0:33Robyn LundinThe Application Security Podcast is brought to you by Security Journey. We help enterprises reduce vulnerabilities through application security education for developers and everyone in the SDLC. Learn more at securityjourney.com.

0:46Chris RomeoHey folks, welcome to another episode of the Application Security Podcast. This is Chris Romeo. I'm the Chief Security Officer at Security Journey. and also co-host of said podcast. Also joined by my good friend, Robert Hurlbut. Hey, Robert.

1:04Robert HurlbutHey, Chris. Yeah, it's Robert, Principal Application Security Architect at Acquia, and focused on threat modeling, and of course, always, always interested in talking about application security topics, as we will today again.

1:18Chris RomeoYeah, that's where we're going to go today. We're going to talk about the intersection of penetration testing and application security. But before we get there, Robyn is our guest for today. And Robyn, we always, we throw our guests right into the deep end. We don't believe in giving you time to, you know, tell us what you had for breakfast or anything. It's like, let's go right to your security origin story and tell our listeners, how'd you get into security? What was your path?

1:43Robyn LundinYeah, yeah, I can definitely talk about that. So, I'm Robyn. Right now, I work in product security at Slack. But security origin story, I originally chose a career in government. I was working for the Social Security Administration, working in claims. Got really just bored and frustrated with the bureaucracy of working in that career. So I just kind of left without a plan. Luckily, I had life circumstances that allowed me to do that. I started my own company. It was a subscription box company for coffee and chocolate and whatnot. And did that for about a year. Realized that shipping is really expensive and subscription boxes don't really make you much money. It's hard, hard to do, especially with the competition of Amazon. From there, I went to a coding bootcamp and then at my coding bootcamp, NCC Group did a presentation on penetration testing and security. And I just was completely blown away. You know, they were doing like this live hacking demo and I was like, What just happened? Being brand new to tech, being brand new to security, it was just kind of my mind blew up. Um, and that got me really interested in learning more about security on my own. Like, my coding boot camp didn't really cover it much, so did a lot of self-learning, um, you know, online courses, a lot of reading, a lot of, um, GOAT web applications, things like that. And then eventually applied to work for NCC Group. 'Cause I was like, oh, you guys are doing what I wanna do. I don't know, work here. Made my way in. And then from there, decided I wanted to see more of the side of internal AppSec. So, I had a friend who was working at Slack and knew of an opening, and I just kind of got lucky at the right time. And now, here I am.

3:36Chris RomeoVery, very cool. And that's, you know, one of the things that I see a lot of times is people go, you know, the coding bootcamps have become a really big, a really big thing for people to provide kind of an alternative route to tech versus going, you know, through the 4-year university system and everything else from that perspective. And so, I guess, how did you make that? Like, what got you to that, to being ready to make that transition point to that first job with the NCC Group? Only because I think we might have a lot of people listening that are in that same boat, right? Like, they're like, hey, I'm doing a coding bootcamp and I want to be in security. That's my big life dream. And you've kind of, you found the path through there. And so I'd just love to know a little bit more about how did you make that happen so that maybe we can encourage some other folks out there like, hey, there's a path you can get to security the same way Robyn did here is a path you can follow.

4:31Robyn LundinYeah, absolutely. Part of it was the luck of being married to someone that's a security nerd. So when I told him I was like interested in security, he was able to point me toward, okay, This is the stuff that you need to look at. So if you don't happen to be married to a security nerd, maybe like make good friends with a security nerd, someone in the community that can mentor you a little bit and give you guidance on, you know, there are so, so many things that you could focus your attention on and you're never going to learn everything. But if you get down XYZ basics, you can at least put your foot in the door. There is so much good online content out there. I think the thing that taught me the most was picking out intentionally vulnerable applications like OWASP Juice Shop, doing things like Over the Wire, getting my hands on actual how do I hack this thing really helped me absorb, you know, how, how the attackers think and what these vulnerabilities actually mean. You know, you can talk about the SQL injection all day, but if you don't understand why is this bad or why would somebody exploit this? What's the motivation behind somebody that would attack your system? Then it's really hard to be able to talk about vulnerabilities in a meaningful manner. So, yeah, I would say mentorship and hands-on hacking. Those were my top 2.

5:58Chris RomeoVery cool. And mentorship is such an important thing for us as an industry. And we've talked about it a number of times here on the podcast. And, you know, Robert and I have both been in security for long enough to have gray hair. But one of the things I get the most reward in my— that I've had in my entire career is really mentoring people, helping them. Because I had some people that mentored me when I was a, you know, 21 or 22-year-old kid who had no idea. I had no idea how famous these people really were. But they mentored me and they answered every question I could ask them. And now I look back and I'm like, yeah, I asked them some stupid questions and they were so nice because they didn't tell me they were stupid questions. They just took the time to explain it. So I think that's a call for all of us in the industry. If you're not mentoring somebody right now, why not? Like, there's a need and, you know, that's, it's just something that we can all be a part of.

6:49Robyn LundinYeah, absolutely. And, you know, now that I've been in the industry for a while, I've had the chance to offer some mentorship to earlier career associates that I work with, and that's been super rewarding. So highly recommend anyone in the community that's looking to mentor people, The coding boot camps are a good place to start for that because they need mentors too. And yeah, they'll be very receptive to any, any help you can give.

7:16Robert HurlbutGreat advice. So Robyn, as we get started today in talking about our topic on penetration testing and AppSec, before that, I think you have an interesting story about a time where you accidentally started physical penetration testing long before you got into security. Can you tell us about that?

7:39Robyn LundinYeah, yeah. So I mentioned my husband. So when he and I first started dating, I was still working for Social Security Administration. So not in security yet. And when I would go like to meet him and, you know, go over to his apartment to meet up and go out to dinner, Um, I would just kind of like walk into the building and everybody would just let me in, or, you know, like sneak in behind people. And then I would like walk up to his front door of his apartment. He'd be like, how did you get in here? I don't know, people just trust me, you know. I, I don't look like I'm about to hurt people or, um, steal stuff. So I always thought I would be a really good physical pen tester because of that. I think I think just my general demeanor is somewhat disarming, but I never got into it because I've heard that the travel schedule can be just grueling and you're just always on the road. So, I don't know if I could take it.

8:40Chris RomeoJust use those powers for good, right? Like, instead of using them for consulting, you just use them, you know, when you need to get into a building or when you want to go somewhere.

8:50Robert HurlbutRight.

8:51Robyn LundinRight, exactly. You know, that's good stuff.

8:54Chris RomeoSo, to kind of dive into the deep end of our penetration testing conversation, I wanted to help set the stage a little bit for our audience. And really, the question I have right now is, when you think about pen testing, so how does pen testing fit into the secure development lifecycle from your perspective?

9:14Robyn LundinYeah, so from my perspective, it should really be on the tail end of your secure development lifecycle. So like, as you're going through different tasks of threat modeling or, you know, helping the eng teams with design review and secure architecture and all that, like that's your early phase of your secure development lifecycle. And then, you know, once everything is kind of built functional and it's, it's in a spot where you actually can meaningfully penetration test it. I think that's kind of where it comes in is we've built this system and we think it works and we're about to release it. Let's get some second opinions and see if we missed anything and if these you know, people that are coming in with a completely different perspective on what we built, if they can stump us at all. And if, if they've, you know, figured something out that, that we didn't. Right. So different perspective and probably toward the end. I've seen, like, when I was doing penetration testing myself, I've had companies that would hire us to, you know, run an application pen test on something that was maybe half-built, half-baked, and like, okay, cool. Yeah, come pen test this thing that barely works and that is going to completely change before we release it. That's a waste of money.

10:57Robert HurlbutSo there's got to be something there. Like I said, half-baked is the results you get are going to change. And so it may not be much value at all.

11:06Robyn LundinRight, exactly.

11:09Robert HurlbutSo what's the difference between an AppSec pen test and maybe what we call a system-level pen test?

11:18Robyn LundinSure. Yeah. So system-level pen test, that makes me think mostly like a network pen test or something where you have people coming in and running, say, an Nessus scan and then handing you results of Hey, we scanned your environment and we found all of these CVEs that may or may not be meaningful to you. An application-level pen test, if it's done well, typically you're getting consultants who are actually looking at what APIs you've built, the surface of your application, and actually poking at it. When, when I was doing application pen testing for NCC Group, typically I would start with just using that app as if I was a normal user and walking through the paths and seeing like, what's here? You know, what, what would I do if I was just trying to use this app as a normal human being? And then intercepting all those, those requests and looking at the APIs that are under the surface, even just walking through as a normal user, a lot of the time you can find interesting low-hanging fruit that, you know, there will be secrets exposed in an API request that no one thinks that you can see. Um, so that would be step one. And then, uh, from there it would be how, how would I abuse this if I was a malicious person? Um, so I think there's a lot more manual work and there's a lot more, um, thought that goes into given the way that this application is laid out, what are the potential abuse paths and why would I do this? And what will be the motivation of the attacker rather than just kind of throwing a scanning tool or, you know, looking at the system or the network without, I don't know if there's a good word for this, without any personality to it, right? Like, your application has a personality, and who would attack it has a personality, too. So I think there's more of a human context.

13:27Chris RomeoYeah. So you mentioned low-hanging fruit, and I guess that's always an interest— or I'm not allowed to use the word interesting anymore in life, I decided. So that can potentially be a challenging piece of the pen testing process because I've heard so many stories about people, they pay all this money to do a pen test, and then the pen tester completely destroys the thing they built on the first day. the first hour of the day, and now what are we going to do with the other, you know, 3 weeks of time that we had allocated to this? Because we have full access and have downloaded the entire contents of your database. So what do you tell teams to— what do teams have to do in the SDL process so that they don't leave any low-hanging fruit behind? Because I have the same thing with bug bounty too. People always like, they jump into bug bounty and then they pay $10,000 'cause someone found a cross-site scripting that, 'cause somebody scanned it with a web scanner and found a cross-site scripting and they paid 10 grand for that. Like, you could have just paid, you could just use that web scanner and given me the 10 grand. But from your perspective, like, how do we prevent that being one of the downsides of a pen test?

14:36Robyn LundinYeah. Ooh, there's a lot. There's a lot that you can do for sure. I think one of the somewhat neglected parts of the SDLC can be developer education. making sure that, that your devs can kind of threat model themselves and that they know the basics of what's available to them to prevent them from getting in hot water. And then point 2 would be building them tools to help them keep from getting in that hot water. Like, to be a developer, you shouldn't have to be a security expert or a security engineer yourself. That's like 2 different jobs, 2 different functions, and that's a lot to carry in your head. So building the tools that allow those devs to not have to carry all that security knowledge in their head all the time is great. Like putting things in your CI/CD pipeline like static code analysis to help them make good decisions and not accidentally Choose a function that's vulnerable to RCE. Things like secret scanning are great. And then threat modeling, I think, is probably one of the most valuable parts of the secure development lifecycle process. I think initially it's good to have security engineers be involved in that process just to introduce the developers to, hey, this is how we threat model. this is what thought process you wanna go through when you are picking out what is a possible threat in your environment. And then eventually allowing them to do it themselves can save your security team a lot of time. So yeah, well, if we covered, we covered threat modeling, tooling, education, those are 3 really good starts, I think, for, for picking out your low-hanging fruit.

16:37Chris RomeoAnd those set us up so that when we do get to the pen test, at least they're not finding, like you mentioned, that example of you scan the app as a user and you see a secret right away in one of the transactions that's going back and forth. Like, these types of things you're talking about are going to prevent against a secret being there because your secret scanner is going to hopefully detect something that's, you know, being served up in the source code specifically.

17:06Robyn LundinYep, exactly. Yeah, I've had those pen tests happen where you get, you know, system admin access on day 1, and then you're like, oop, well, I did it. And you keep going, you keep providing information, um, you know, you keep poking around, but day 1 should not be I completely, absolutely destroyed your application. Like, that means that there hasn't been quite enough security diligence done before the pen test. Yeah.

17:37Chris RomeoI've been away from pen testing for a long, long, long time, but the last big pen test that I worked on, and I may have told this story before, so I'll make it quick, was this company flew us into Boulder, Colorado, and took us to their offices, gave us a cube, and said, okay, let's see what you can do. The test was supposed to be like 3 days. This is about 9 o'clock in the morning. About 10:30, we have full access of the entire domain. We've just used a simple domain admin exploit. Boom, we were in, and we had full access to the entire thing. And so we had to go walk. It was almost like the walk of shame down to the person's office, but we shouldn't have been because we're like, we took this place out in an hour and a half or whatever. But so we had to go have a quick briefing with them like, oh, I guess it's time to go home. So back to the airport we went, and that was my last foray into pen testing. Like I said, Oh, wow. That was a long, long time ago when the years began with 19. So AppSec Podcasts then, what's the purpose at the end of the day? If we had to create a very tight kind of summary of what's the purpose of an AppSec pen test, like what are your thoughts on that, Robyn?

18:41Robyn LundinYeah, there's the purpose that a lot of people are after and then there's the purpose that I think they should be after. So I think the purpose that a lot of the time happens is we need to check a compliance checkbox. And that, I think, misses the point. The actual purpose of an application-level pen test, I think, is getting people looking at your application from the perspective of an attacker that don't work for your company, at least If we're talking about external pen test, I think having a different perspective and a different opinion and getting outside of your own assumptions is really, really helpful. I think the most valuable pen tests that I've been on either side of, it's— you have those conversations at the end when you're doing your readout where the developer gets the vulnerability reported to them and they go, oh wow, I didn't even think of that. Thank you.

19:45Chris RomeoYeah.

19:46Robyn LundinSo getting someone who thinks differently from you, I think, is, is incredibly valuable. How do you create security champions? Security Journey brings together 2 powerful approaches to provide application security education to help developers become security champions and produce safer applications. Security Journey training content extends beyond developers to reach the entire SDLC, creating a security-first organization. Learn more about our enterprise security training at securityjourney.com.

20:16Robert HurlbutWell, let's talk about frequency. So let's say we have 1,000 apps. Do we test those, all of those quarterly or yearly or maybe other frequency timeframes? What are your thoughts?

20:36Robyn LundinYeah. I think at least part of that is definitely gonna be dependent on your budget. But budget aside, it's— I think you have to do a risk assessment of what is in your environment. Like, are you— are your 1,000 applications all at the same level of potential risk and potential for being a high-profile target for attackers? Or is it like one of your applications is for taking employees' lunch orders, but another one of your applications is for hosting a bunch of valuable financial data? So those two have totally different risk profiles. Maybe the lunch order application really doesn't need a pen test unless it's storing, you know, PII or something. But your financial data application, that one Probably is a pretty high-profile target, especially if you have high-profile customers. So with that one, if you're if you're adding to it really frequently and changing things really frequently, quarterly might be a really good idea. But I think a lot of companies default to annual. Maybe that's fine. I don't know that there's a one-size-fits-all solution to this. I think it's very dependent on how frequently is this changing. And how much do I expect people are going to be motivated to attack it? Hmm.

22:07Chris RomeoYeah, and just to let the record show, I think we need to protect the lunch app because that's where I get my lunch from. And this is crucial to me as an employee of any company that my lunch is available at the right time. How about different groups I can work with in the world of pentesting? So I know you came from the consulting background, so you got some perspective on that, but I think about, like, When I think pen testing, I've got pen testing as a service kind of providers, and we're not gonna talk about any particular— we're talking generically about an industry here, not about any given company that's involved here. But I know, 'cause I know we got a few of 'em that would say they're pen testing as a service. So between, you know, pen testing as a service, security consulting companies, maybe internal red team resources, developers themselves, like who's the best? Like what are your thoughts on these different options as far as an AppSec team that's out there? Maybe they are newer to pen testing, and they're trying to figure out how to plug this into their AppSec approach and their program. What are your thoughts on those different parties that can play in this process?

23:10Robyn LundinYeah, yeah, yeah. They're all valuable in their own way. But I think if you are a newer AppSec team that doesn't have a whole lot of pen testing expertise internally, it can definitely be valuable to hire out a consultancy that knows what they're looking for. And then they can— you can learn from their process to be able to do it more effectively internally if you, if you want to. A lot of the time, the AppSec team isn't going to have like the, the time or the resourcing to really conduct a thorough penetration test, or maybe not even resourcing, but just the baseline of I've done pen tests before and I know how to approach them in an efficient manner. I think that's where consultancies can really come in handy. And they definitely do have like varying levels of diligence. So I think it's important to be selective about who you're going to work with. My favorite consultancies to work with are ones that are domain experts in a specific area. So I won't talk about the specific consultancy, but Slack recently worked with one that was an expert in testing and developing Electron apps, and Slack's an Electron app. So having a lot of knowledge about that specific technology was extremely helpful to us, and we learned a ton and they found really interesting findings. So yeah, I think that's very valuable.

24:57Robert HurlbutWould that also apply to, so if we had internal resources and you wanna hire folks on your red team, is that what you're looking for as well? Either they have some background or willing to learn domain as well?

25:11Robyn LundinYeah, I think if you're gonna do it internally, you either need to hire someone who does have that background Or really give them the time and the resources to be able to learn it themselves. That can be kind of a downfall of just taking somebody who's brand new to something and eager and willing to learn and then going like, great, you're going to do awesome, throwing you in the deep end, not giving you any time to figure this out. I think that's a recipe for failure.

25:39Robert HurlbutYeah. So maybe when I, in my own experience, well, in general, just whether it's a large enterprise or sometimes even a midsize or small, take some time to learn the organization, how it works and what they're doing and the products they have and so on. So I've done that as a developer, as an architect, as a threat modeler. I think it makes sense for a red teamer as well, or pen tester as well.

26:08Robyn LundinYeah, absolutely. And I think whether you're hiring people to do internal or external pen testing, that time that it takes for somebody to learn the environment well enough that they're going to be able to simulate an attack that is meaningful to your organization, you've gotta budget for that time. You can't just say like, well, I think it's gonna take you a week to test this application without thinking through, well, you've never touched this application in your life. You have no idea what my business model is. You don't know my customers. All of that's really important information.

26:48Robert HurlbutVery good. So if you would, Robyn, just sort of wrapping up here, what are some key takeaways and maybe a call to action for our viewers and listeners to the podcast today?

27:02Robyn LundinYeah, key takeaways. I think I mentioned earlier, try not to approach a pen test as if it's just a compliance checkbox. Like you can get value out of hiring pen testers, whether they're internal or external. And it's very valuable to have your AppSec team spend time thinking through what am I trying to get out of this penetration test and how can I set up the people that are conducting this test for success? How can I work with them and provide them as much information as possible so that the attack that they simulate matters to our organization? There's nothing worse than getting a pen test report that's full of false positives and findings that make you go, well, this is there on purpose. What did you just do? You just gave me a finding for something that I built in as a feature. Yeah, if you take the time to set people up to really test your application well, then you won't be getting those disappointing reports all the time.

28:12Chris RomeoVery cool. Well, Robyn, thank you so much for sharing your pentesting and AppSec knowledge, as well as your story from coding bootcamp to, you know, making a name for yourself across the tech industry. And that's, you know, that's how we found you for this interview. I saw your talk at AppSec Village and I was like, Like at DEF CON, I was like, oh cool, this is, this is a topic that we need to learn more about and have more conversations about. So thank you for sharing that expertise with our audience. And we look forward to another future conversation about something else, about whatever you're thinking about next year.

28:48Robyn LundinAwesome. Sounds great. Yeah, it was really nice talking to both of you and thanks for having me. None of the top 50 university programs teach secure coding in their curriculum. At Security Journey, we help enterprises reduce vulnerabilities through application security education for developers and everyone in the SDLC. With over 400 up-to-date lessons created by industry-leading security experts and a programmatic approach that creates security champions, our program has increased AppSec knowledge as much as 85%. Visit securityjourney.com to try our training today.