Erez Yalon — The OWASP API Security Project

0:00Chris RomeoErez Yalon heads the security research group at Checkmarx. With vast defender and attacker experience, and as an independent security researcher, he brings invaluable knowledge and skills to the table. Erez joins us to speak about the new OWASP API Security Project, and more specifically, the new API Security Top 10. We hope you enjoy this conversation with Erez Yalon. At Security Journey, we believe security is every developer's job. We work with our customers to help them build long-term, sustainable, security culture amongst all their developers. Our approach is to provide security education that's conversational, quick, hands-on, and fun. We don't do lectures. Instead, we let the experts talk about what's important. Modules are quick, 10 to 20 minutes in length. We believe in hands-on experiments, builder and breaker style, that allow your developers to put what they learned into action. And lastly, fun. Training doesn't have to be boring. We make it engaging and fun for the developers. Visit www.securityjourney.com to sign up for a free trial of the Security Dojo. Hey, folks, welcome to this episode of the Application Security Podcast. This is Chris Romeo, CEO of Security Journey, and also co-host of said podcast. I'm joined today by Robert. Hey, Robert.

1:31Hey, Chris. Yeah, it's Robert Hurlbut, Threat Modeling Architect. Good to be here.

1:35Chris RomeoAnd we're joined by a guest who has been with us before in the past. Erez Yalon was featured with Leora Herman in an episode that we did about the Application Security Village at DEF CON. But we have invited Erez back to talk about something else that he's working on. And so we're gonna jump right in with this. The topic today is API and API security. And so, Erez, let's just dive right into the deep end of API. Well, I guess it's the shallow end of API by starting with, what is API?

2:10Erez YalonIt is the shallow end, but it's a ground for many religious wars, I think, to define what API is. So, let's tread carefully here. API, we all know, it stands for Application Programming Interface. And without going very deep into what it means, it's basically just an interface or communication protocol between some sort of client and on the other end a server. The intention is to simplify the building of client-side software, and it has some sort of a contract nature between the client side and the server side where everyone knows what to expect from each other.

2:57Chris RomeoThis is basically a very generalized idea of Now, when you say a contract nature, I'm starting to think about a protocol. So, what's the difference then between a protocol and an API?

3:12Erez YalonSo, a protocol would define probably many aspects of a specific process, while an API is usually a very specific call. that will tell you what you're going to get and what you're expected to give back. Again, we're talking about something very, very general here. It can be something in the operating system, even lower than that, or just in the programming language. But the meaning of API security here is not really about specifically the API, but what can be done with it.

3:50Chris RomeoOkay. Yeah, the impact. And so, when we think about— so, a protocol, is normally something that has a standard that's backing it up. It's like a bunch of people have agreed, hey, here's how we're gonna communicate. An API is gonna sit on top of that. API is going to use various protocols and standards to be successful in fulfilling those contracts between client and server. Okay. So, when we think about API, then what's kind of the issue from a security perspective?

4:23Erez YalonOkay, so let's start with how common it is. Who uses APIs these days? I think that everyone who is developing for mobile, IoT, B2B, cloud or serverless, single-page application. I'm not sure if I missed any buzzword, but any of these technologies or architectures or basically every modern application is based on APIs. We love APIs and with good reason. APIs make our lives very, very simple sometimes and let us concentrate on the business issues, on the logic issues of the software we write. So basically every modern application uses APIs, and when I'm referring to API security, what I'm actually referring to is API-based apps or modern apps security. It's what you can do and how you can abuse this new way of communicating between modules and nodes in the flow of a very generalized and very flexible mesh of software today.

5:38Chris RomeoSo, when I think about kind of the— I don't even know if I wanna call it the old web, but I'm gonna call it the old web anyway.

5:48Erez YalonI'm going to use the word traditional.

5:50Chris RomeoTraditional. Okay. That sounds so much better, the traditional web. But when I think about the traditional web and web applications, and then I think about API, there are a lot of similarities, though, right? Because kind of like the classic monolithic applications had your frontend and your backend all together in one big fat application, and now with API, you're kind of separating it. So, is there a lot of commonality between the challenges of monolithic monolithic or traditional and the API?

6:22Erez YalonSo, that's a really good question to look at and to think what exactly is different about API-based apps. Okay. This is something that we put our minds into it because every time I talk about API security, then people tell me apps are apps. What's the difference? I mean, it's very transparent to the user what's going on behind it, and it really doesn't really matter. So with API-based apps, we know that— or modern apps, we know that client devices are becoming varied and stronger. We used to have one specific client in the past. It was our browser. Now we know that it can be any IoT or smart toaster, or as I said, B2B. Or a smartwatch or an iPad or a mobile phone. It can be Android, it can be anything else. It can be web-based, it can be some sort of other software. So it's very, very different. And also the computing power of the client side is stronger than in the past. So the process we see here is that logic moves from the backend in a way to the front end. So if in the past we saw the similarities of traditional and modern applications are with the communication of the user with the client, and also another similarity is between the server and the database because this is where things stay the same. In modern application now First of all, we see very many different kinds of clients, as we said, but there is also a difference of where the logic happens. In traditional applications, the client would send a GET request to a server, the server would get the information, the logic would run there, and the server would return to the client some sort of a fully rendered HTML page to be presented. These days it's not really like that because each client is, let's say, in charge of presenting the information. So the client would send a series of API GETs to a server, the server would be getting the information from databases more like a proxy and will return raw data to the client. And on the side of the client, there will be some logic to render and to create the end result that the user sees. So the logic is a bit different, and this is a lot of— this is the source of most of the issues we see.

9:25Chris RomeoYeah, so it sounds like while there is some commonality, there's also some unique things that API security is going to bring to the table, unique things that we need to consider?

9:36Erez YalonYeah, so since the server is used more as a proxy and the rendering component is the client, the clients consume raw data, the APIs expose the underlying implementations of the apps. Even the user states, we can see that, are usually now maintained and monitored by the client as more parameters are being sent in each HTTP request because it's not filtered anymore. The client is supposed to filter it. So you can see a lot of object IDs and values and filters and so on and so forth. And the attacker can get a lot of endpoints. The attack surface is growing all the time and the endpoints are numerous.

10:24Chris RomeoYeah, and this has actually been very helpful. I had never thought about the server as proxy. in the new kind of generation of applications. But yeah, I mean, it's effectively what it is. It's just serving up data and kicking it back to the client for rendering.

10:41Erez YalonYeah, now it is the same. And also, we have our big wave of focus moving towards DevOps and DevSecOps. Even that is becoming very easy. APIs change all the time now with the CI/CD. It takes really just a few clicks to spin up a new API host with new Kubernetes and Dockers and everything that is cloud. APIs become very, very easy to spin, just to spin up, and sometimes it's too fast for our own good. It becomes very hard to track. It creates some sort of shadow API that you don't remember you ever put on, or all the exposed APIs that someone forgot to delete. So, even the DevOps part of API management kind of contributes to these issues.

11:43Chris RomeoWell, while all our listeners might be thinking, how could we possibly understand all the different API security challenges? All hope is not lost because Erez has been working on a new project called the OWASP API Security Project, which is educating all of us and making us aware about the issues that we have to be considering from the API perspective. And so, Erez, give us just a kind of a high-level summary about this project. Talk a little bit about the team that's delivering it. I love for other OWASP folks to get recognition as well because I know there are a lot of folks that are probably working behind the scenes on this thing. Give us a little bit of perspective on Okay.

12:26Erez YalonSo yeah, as you said, the first thing we wanted to do in order to start treating API security as it should be treated is to bridge the gap, the knowledge gap and the awareness. A lot of users, a lot of developers, and even architects are not really aware of the problems, and you cannot really fix anything or remediate anything until you're aware of the situation. So as a proud OWASP member, I decided to stop waiting for someone to create the project and decided to go for it together with a friend, Ynon Shkedi. Also, he has a lot of experience with pentesting I would call him an API specialist. This is what he is doing now. So together with other friends, we kind of went for it and started to try and dig out information. We decided to start with an API security top 10 because this would probably be the best awareness document we can create. just to put a very clear picture of what we're missing in front of people. And I think it's working very well already. We'll probably discuss that a bit later. The work was kind of— it took a lot of time. We started at the beginning of the year, but we're now ready to release the OWASP API Security top 10 list any day now.

14:14Great.

14:16Chris RomeoWell, one thing I'd love for us to do now is just walk through each of these items in the top 10. And we're gonna— I wish I had a timer noise going in the background. We're gonna keep ourselves to about 2 minutes per item just because we want to introduce each of these. Some of them are going to be familiar for those that have looked closely at the kind of primary OWASP top 10 from more of the web application perspective. But there are a couple of things on here that are different, you won't see on that list. And so, let's jump right in here, Rez, with number 1, broken object-level authorization.

14:50Okay.

14:52Erez YalonSo, this is something that many people, when I explain to them, tell me, oh, isn't it IDOR, insecure direct object reference? And we decided not to use that name but to use another specific name, Broken Object Level Authorization. The reason is because that the IDOR name is not very accurate. We don't think that the direct issue is problematic, and I'll explain what I mean here. When you have an object that you wish to give access to, let's say a specific document, you would not want anyone to get access to this document. I think it's a very, very clear architecture that several users can access specific objects and others cannot. So if you ask to get your own document, you should get access to it. And if you ask to get a document that you're not allowed to, you're not supposed to get it. This is— it sounds very clear, but— Yeah. We can see many times that when implementing API access to objects, something gets lost on the way. And it's very common because the attack surface is much wider. APIs receive more IDs because clients maintain the user state, as we said. And there is no magic bullet. There's no security solution that solves the problem, and it can be handled in many different layers. And sometimes when you create a layer, you think that someone else is taking care of it. So just accessing an object that is not yours, or you're not allowed to access it, is the main problem. And we see it all the time. It happened to Verizon lately, and they had to— they actually managed to— Fix it. To fix it, but a researcher found a way to access 2 million Verizon monthly contracts just because he could access any object he wanted. So this is just an example. The researcher was— the researcher's name is Dali Bey, and this is practically— we consider it the main issue because it's really hard to address, basically.

17:22Okay, next is number 2, broken authentication. Tell us about that.

17:26Erez YalonYeah, so broken authentication, I think the name here is clear. It kind of explains what we're talking about. So let's start with why is it so common in APIs. Again, authentication endpoints are exposed to anyone by design. These are— this is why they're called endpoints. We have a lot of misconceptions with software security engineers about authentications. Sometimes there is a lot of confusion in authorization and authentication, which leaves the authentication broken. And sometimes they use very clear methods that are just wrong. It's either lack of protection or misimplementation. These are 2 different ways that authentication can be broken. Lack of protection is obviously something that is just missing. For example, if you don't put in a lockout mechanism or CAPTCHA or something like that, you will be able to be— your authentication is broken because it allows credential stuffing and brute forcing. Sometimes you do implement something But for example, we see a lot of people using JWT now, but they use as the algorithm, they use the value none, which means that there is no encryption. So there is either lack of protection or misimplementation in authentication. There are many things that can go wrong with authentication.

19:03Chris RomeoOkay, the third one takes us to excessive data exposure. Which generally, like many things on this list, sounds like a bad idea.

19:12Erez YalonYeah. So this is very much an API issue. APIs expose sensitive data of other users by design sometimes. It's very common because the REST standard and API economy in general, they encourage developers to implement APIs in a very generic way. It's very comfortable, it's very nice, and if everyone do it the same way, there is a very short onboarding for every new process. The problem begins when you use very generic functions from some sort of model or ORM without thinking about who's the consumer. So if you will send all the data in a specific object to the client and hoping that the client will filter it out for you, you are probably sending too much data. For example, if you want to send someone the, let's say, the name and the hobbies of a specific person, you have on file also their address and other data, but you're not supposed to send it because this is private information. By sending just a standard object, you might send some data that you're not willing to do or you're not supposed to.

20:42Okay, so let's see, number 4, lack of resources and rate limiting.

20:47Erez YalonYeah, so I don't think we need to go very deep into that. If you don't limit your access to your resources, you will probably be very open to brute force attacks and denial of service attacks. I think this is kind of easy to understand because there should be a limit because there is a physical limit. So it should probably limit it. Any resource that can be accessed should be limited.

21:21Chris RomeoOkay, that takes us to number 5. And I'm noticing a little similarity with number 5 in that it says broken function level authorization. yet number 1 was broken object-level authorization. So, what's going on here with 5?

21:34Erez YalonRight. They are, we call it, brother items or sister items. It's indeed similar, but we felt that it's important enough to get its own separate item. Broken function-level authorization does not talk really about the object, the access to the object, but the function that can be done to it. If I will explain, just give an example, for example, a regular user can access through API some sort of object and can watch it and maybe even edit it, but only an admin can access it and delete it. So the action, the function is defined And the function should be only allowed according to a specific role and not to anyone. And again, this is very common in APIs, the issue of missing that. It's because the function-level authorization can be implemented in several different ways. It can be in the code, in the configuration, in the API gateway. Also, it's really easier to detect and exploit in APIs because the endpoints are more predictable. The URL or the API endpoint can be very, very similar. You just change a word, let's say from GET to DELETE or something like that, and it's very, very easy to try and enumerate these words. It's very predictable.

23:16Okay, and next is something I've seen in the wild a few times, mass assignment, number 6.

23:21Erez YalonYeah, so mass assignment is kind of the reversed excessive data exposure. If in item 3, API 3, we gave too much information, here we take too much information. We just take the, the parts of the object that are legit, but we need to make sure that we're not getting any malicious things. For example, a request can get a username and password, which is fine and legit, but we need to make sure that the user is not sending me also his role. If I would send my username and then my password and then my role as admin, I must make sure that I will not assign that into the object unless the user is supposed to.

24:11Chris RomeoNumber 7, security misconfiguration.

24:15Erez YalonYeah, this is kind of a general thing. It's, it's more of a bucket of bad things. I know that everything is misconfiguration these days, but still we decided to put a bit more emphasis about these things. It's a long list. It's— you will find there weak encryptions and unnecessary exposed HTTP methods and Sometimes no CSRF protection and detailed errors and improper CORS. This is— these are all things that should be configured right and hardened and to make sure that the security configuration is correct.

24:55Okay, and number 8, injection.

24:59Erez YalonSo injection, I'm hoping now that everyone knows what injection is. I think the SQL injection is celebrating about 20 years now or even more in the OWASP Top 10. So I don't feel I need to expand, but I will explain why we lowered it so much from A1 to API8. I think that the main reason that injection is currently number 1 in the OWASP Top 10 is because of SQL injections. But SQL injections are getting not very common in modern APIs. It's because of the use of ORMs and increasing use of NoSQL. I know some people will say NoSQL injection exists. I know NoSQL injections are a thing, but they are usually not as common or severe. So I think that as soon as we take this out of the equation and hopefully it would be a thing of the past. I think that injection issues are much lower in our list, but as you can see, they're still there.

26:10Chris RomeoYeah, I was just thinking, should we be having, like, a party for SQL injection for, like, 20 years? But then I'm thinking, no, we should not celebrate this thing as a challenge. So, the party is canceled.

26:24Erez YalonThe party is canceled.

26:26Chris RomeoHopefully, someday, we'll eradicate injection from every list. That takes us to number 9, improper assets management.

26:34Erez YalonSo, the management part is the part I used to hate most as a developer, and I'm pretty sure I'm not alone. The management part has— well, it's 2 different housekeeping issues, as I see it. We're talking about either lack of documentation, which means that really no one remembers now what legacy XYZ export all users API means at the moment, because really no one actually documented it. Maybe it was for QA, maybe it's doing nothing, maybe it will send all the names and the credit cards of the users. You don't really know anymore. Because there is no documentation. The other side is similar but different. It's more of exposed risky APIs. If you used to have a version 1 or version 2 of the API and now you move to version 3, please make sure that version 1 and 2 are either monitored or deprecated because if they are still exposed, it might be very risky, especially if you combine them and they are undocumented. So if they're just lying there, open API endpoints lying in the shadows, I can promise you that the hacker will find them.

27:59Okay, and finally, number 10, insufficient logging and monitoring.

28:03Erez YalonOkay, so this is actually the exact same as, as A10. We believe that logging should be part of any healthy system and app. So, it is still there in the same place.

28:19Chris RomeoWhy does— I'm just curious, Erez, kind of your perspective on this, because I always chuckle when, like, every list it seems like we have out there, logging and monitoring always show up, and they're always in the number 10 spot regardless. So, what's your take on that? Why do you think that is?

28:36Erez YalonI think that was added Only in the last 2017 list of the OWASP Top 10. I don't really know how to feel about that because it is not really a weakness that will help a hacker go into your code or your assets, but it might help him get lost easier and hide easier.

29:05Yeah.

29:05Erez YalonAnd not allow you to actually understand that something happened though. So I think it's not the classic weakness we think about. Probably some of us are not really sure it's supposed to be though. But this is also why it got the number 10, just to be on the safe side.

29:28Chris RomeoYeah, it's good that it's there. I just, it's, I don't know, Maybe I, I chuckle at strange things, but I just always— it always—

29:36Erez YalonNo, I, I completely agree with you. And, and the, the first thing I, I thought when I saw that was, okay, so let's add lack of fire extinguisher in the, in the server room, right? Let's just be safe in general. But it makes sense. It makes sense because the hacking part is not just being able to go in. It has a lot of of consequences and, and the way to cover up your tracks after that. And logging is important.

30:05Chris RomeoYeah, I mean, maybe there is a— maybe there's a new number 10 that's more focused on general safety and general application safety that includes things like logging and fire extinguishers both. Um, so where you got— where are you going with this project? Like, it seems like you have, based on the way you described this kind of in the beginning of the interview, you talked— it sounded like you had more of a bigger vision for maybe this is the first piece on a series of projects or things that are gonna focus on API?

30:39Yeah.

30:40Erez YalonSo, the beginning was just to see where we stand and to understand the gap in knowledge. I work at Checkmarx, and we see a lot of code of our customers. We see a lot of mistakes out there. And we slowly see the mistakes migrating from the traditional ones to the modern ones, to the API ones. And the gap was very clear in front of us, and we decided to do something and to start the project. We started, as I said, with the Project Top 10, and it got immediate attention from everyone with full house when we talked about it in the OWASP events, and people contacting us, they want to help, and a lot of attention from media. And also, I already got conversations from people telling me they're actually using what we released so far, which is the release candidate. We had some, a few versions of release candidates in the past. They're already using it in their organizations. So, it's very clear to me that the need is real, and I'm very happy we took that step. As soon as we're going to finish and release the final version of the of the API Security Top 10. We're going to move forward with the project, which is going to include some sort of cheat sheet because we want to help developers and security people to understand a bit more than just the top 10 and also give them tools to remediate and maybe understand how it looks like. Because the top 10, as I said, is an awareness document It has some details in it but not enough. So the cheat sheet would be able to help developers to actually improve their coding and their design techniques. And this would be the next step. In addition, since I think that you cannot really learn how to protect without breaking, we're going to create a vulnerable API project. It's got the name CRAPI, C-R-A-P-I. Yeah, which stands for Completely Ridiculous API. We're going to start with the top 10, obviously, but then with the help of the public, maybe we will get more and more into that. I think that seeing things break really helps you build them stronger next time.

33:20Chris RomeoYeah, I agree. I love the name Crappy. That just taps into my middle school sense of humor.

33:27Erez YalonYeah, I'm still there.

33:30Chris RomeoWell, so, um, you mentioned also, uh, previously something about a call for contributions. And so are you looking to collect data for future versions of this top 10 from, from those folks that are out in the industry like the other top 10 projects doing?

33:46Erez YalonWe did not achieve a real contribution this time. I think we maybe we'd not reached far enough, so we used our own, our own ways into looking at open bounty programs and things like that. For the next versions, we will surely use some more data-driven decisions, and I'm hoping that we will get a lot of information when we call when we publish some sort of call for data. We currently, though, have opened the call for discussions and call for contributions. It's a bit late for the, for the top 10 document, but it's exactly the right time for the next cheat sheet and crappie. You can find all the details and join our mailing list if you go to our OWASP webpage. So you will find it at the OWASP API Security Project, and we would love to hear from everyone and get contributions. And basically it's done for, for the public, so knowing that the public is actually using it already, and we have a great community in OWASP, and the AppSec community is amazing, and we love to hear each and every one of you and to get any contribution possible. Obviously, the project is run on GitHub, so everything is open for everyone to comment and even suggest edits.

35:17Chris RomeoAwesome. Well, that's for our audience there. You heard, if you wanna participate in this process and provide data or just jump in and look for a way to help Erez and the rest of the team, this is OWASP, so there's always an opportunity. There's always a need for more volunteers and more people to help. And so—

35:37Erez YalonI don't think I said no. no to anyone who suggested help so far.

35:42Chris RomeoThat's great. Ares, thank you, uh, to you and the team as well who've been working on this. I know this is a labor of love, and there was lots and lots of hours, many, many more than anyone will ever realize, went into creating this document. And so, we definitely appreciate all the efforts, and the document's great. Um, just from, from what I'm seeing, it looks like something that's very easily used to teach developers about what they need to care about from an API security perspective. So, we look forward to seeing the cheat sheet and hearing about CRAPPY and everything that goes with that. Thanks for being with us today, and really appreciate all the efforts you're putting forward.

36:21Thank you.

36:24Chris RomeoThanks for listening to the Application Security Podcast. You'll find the show on Twitter @AppSecPodcast or on the web at www.securityjourney.com/appsecpodcast. application-security-podcast. You can also find Chris on Twitter @edgeroute and Robert @roberthurlbut. Remember, security is a journey, not a destination.