--- title: "Ken Toler -- Blockchain, Cloud, and #AppSec" url: https://appsecpodcast.com/ken-toler-blockchain-cloud-and-appsec/ date: 2022-01-18 duration_seconds: 2699 guests: ["Ken Toler"] topics: ["Secure Development", "Security Testing", "Software Supply Chain", "Privacy and Compliance"] audio: https://www.buzzsprout.com/1730684/episodes/9907022-ken-toler-blockchain-cloud-and-appsec.mp3 video: https://www.youtube.com/watch?v=K-Ws08s8eo4 transcript: true --- # Ken Toler -- Blockchain, Cloud, and #AppSec *January 18, 2022 · 45 min* with [Ken Toler](https://appsecpodcast.com/guests/ken-toler/) on [Secure Development](https://appsecpodcast.com/topics/secure-development/), [Security Testing](https://appsecpodcast.com/topics/security-testing/), [Software Supply Chain](https://appsecpodcast.com/topics/supply-chain/), [Privacy and Compliance](https://appsecpodcast.com/topics/privacy-and-compliance/) [Audio](https://www.buzzsprout.com/1730684/episodes/9907022-ken-toler-blockchain-cloud-and-appsec.mp3) · [Video](https://www.youtube.com/watch?v=K-Ws08s8eo4) ## Show notes Ken Toler is a principal consultant at Kudelski Security and is passionate about building and optimizing application security programs that stick through strong adoption and ease of use. Ken has spent considerable time on all sides of the security aisle from playing defense and managing security teams to offense by breaking applications and reviewing code. Ken is also the host and creator of the Relating to DevSecOps podcast that focuses on forging strong relationships between engineers, operations, and security through collaboration, understanding, skill-sharing, and healthy debate. Ken joins us to talk all things Blockchain and AppSec. We define Blockchain, discuss the connections between cloud, appsec, and blockchain, common architecture failures, pen testing, and even dive into smart contracts. The Application Security Podcast is brought to you by [Security Journey](https://www.securityjourney.com/). About Security Journey Ken Toler is a principal consultant at Katelski Security and is passionate about building and optimizing AppSec programs that stick through strong adoption and ease of use. → [Learn more about Security Journey](https://www.securityjourney.com/) Connect with Ken Toler: → [Secureum Videos](https://www.youtube.com/c/SecureumVideos/videos) → [The Rust Programming Language](https://doc.rust-lang.org/book/) Mentioned in this episode: → [Secureum Videos](https://www.youtube.com/c/SecureumVideos/videos) → [The Rust Programming Language](https://doc.rust-lang.org/book/) → [Blockchain Security @ Kudelski](https://kudelskisecurity.com/services/applied-security/blockchain-security/) → [Blockchain Security A Need For Todays Businesses Complete Guide For Beginners](https://www.blockchain-council.org/blockchain/blockchain-security-a-need-for-todays-businesses-complete-guide-for-beginners/) → [Rust](https://www.rust-lang.org/) Chapters: 00:00 Meet Ken Toler: Blockchain, Cloud, and AppSec 02:06 It was a place in Texas. And Ken and I happened 05:00 It's a term that we heard. I mean, certainly in the 06:31 At the end of the day, It's an immutable ledger of 12:05 That's helpful because I hadn't really thought of it. I guess 15:20 Yeah, that's, that's very helpful. And before we transition into thinking 18:53 Yeah. To draw a parallel from my AppSec history and experience 21:59 They can decide who they want to go after first. It's 24:45 I want to get into what kinds of surprising mistakes you've 26:52 Yeah, definitely a different set of skills than our classic, you 29:05 Can you pen test a blockchain 32:13 Smart contracts and secure coding, how these things fit together. When 34:14 Yeah, I think that's a good definition that takes away from 36:42 We have a smart contract that is written onto a blockchain 38:58 You look at the code 42:53 Yeah. Very cool. So what would be your key takeaway then ## Transcript *7,867 words · assemblyai* **0:00 Chris Romeo:** Ken Toler is a principal consultant at Katelski Security and is passionate about building and optimizing AppSec programs that stick through strong adoption and ease of use. Ken spent considerable time on all sides of the security aisle, from playing defense and managing security teams to offense by breaking apps and reviewing code. Ken is also the host and creator of the Relating to DevSecOps podcast that focuses on forging strong relationships between engineers, operations, and security through collaboration, understanding, skill sharing, and healthy debate. Ken joins us to talk about all things blockchain and application security. We start by defining blockchain. We discuss the connections between cloud, AppSec, and blockchain, common architectural failures, pen testing, and even dive into smart contracts. We hope you enjoy this conversation on all things blockchain and security with— **0:54 Ken Toler:** You're about to listen to AppSec Podcast. When you're done with this, be sure to check out our other show, High Five. **1:04 Robert Hurlbut:** Hey folks, welcome to this episode of the Application Security Podcast. This is Chris Romeo. I'm the CEO of Security Journey and co-host of the podcast. I'm joined also by Robert Hurlbut. Hey, Robert. Hey, Chris. Yeah, Robert Hurlbut, Threat Modeling Architect. Good to be here. It's a new year actually for us. It is. This is the first recording of 2022. And with that, we're excited to be talking about something that a lot of people are going to continue to be thinking about and considering in this new year, and that's blockchain. So I have to tell a quick origin story for this conversation. Normally, we jump right into our guest's origin story, but our guest today is Ken Toler. And this conversation actually began at all places, of all places, the LASKON 2021 speaker dinner in Austin, Texas. I think I was in Austin, Texas. **2:05 Ken Toler:** I don't know. **2:06 Robert Hurlbut:** It was a place in Texas. And Ken and I happened to start chatting over the dinner talking about blockchain and AppSec and the intersections. And it was such a fascinating conversation that I said, hey, we got to get Ken on the podcast to tell some of these stories and talk about these things. for our audience. So Ken, let's jump right in with your security origin story. Our audience wants to know, how did you get into this crazy world of AppSec? **2:27 Ken Toler:** Yeah, I mean, I, you know, I came to AppSec from help desk, actually. I mean, I was working at a company that maybe some of the younger folks in the audience may not even know anymore, but LivingSocial. I was basically a music major that was paying for school with IT jobs and Eventually, I was doing an antivirus deployment and worked really closely with the security team, and we were all working pretty closely together, and I made the switch to InfoSec and had this grand vision of hackers and breaking things and all that, like most of us do. It's like when you get into software development, you want to go make games. When you get into security, you usually want to break stuff or hack into things. And so when I left there, I did some odd jobs consulting in InfoSec and got a really amazing opportunity to work with some senior folks at like a small AppSec security firm. I was growing with them and did a bunch of work for government and breaking into web apps and all that. And I think, you know, the rest is history. I've just been learning and picking up things as I've been going. And, you know, there's every year it's something new and just keeping up with it. And a lot of it is just diving in, reading, researching. And I've been doing that since day one, I think, ever since I made that switch. And I haven't looked back since. **3:47 Robert Hurlbut:** How'd you get into blockchain specifically then? Because I haven't seen blockchain and AppSec. I haven't seen a lot of intersection of these 2 topics yet. And so I'm just curious, like, what's the, what's the point where you realize blockchain's a thing and start to consider the security and ultimately AppSec, how they fit together? **4:06 Ken Toler:** Well, it came through engagements just in consulting. You know, we were running an application security services team, and we had a lot of requests come in. I mean, just the intersection of crypto and AppSec, and we were performing a lot of the same services for these code reviews for things like smart contract audits. And the techniques, the ways that we approached, you know, the methodology of assessing source code really intersected well, and it just kind of happened organically within the organization as we started to take on more and more of these engagements. And it's become its own service. And it's another rabbit hole that, you know, we just dove into, but it's all code. So it really fit really well together there. But it was, again, on the job, just kind of got into it. I've been investing in cryptocurrency here and there, but it hasn't really, you know, that was, that was the main thing was just a work-associated thing. **5:00 Robert Hurlbut:** So it's a term that we heard. I mean, certainly in the last few years, blockchain and cryptocurrency and so forth. But what is blockchain? Could you define that for us? **5:09 Ken Toler:** Sure. I mean, blockchain is, I mean, what it's, it's what a loaded question, right? I mean, it's kind of like asking someone what's a pen test or what's DevSecOps, but I'll do my best. I mean, at its core, blockchain is an immutable ledger. It's a list of records, a list of transactions that we are, and if you talk to some software developers, they're like, what's the big deal about this ledger? We've been doing this forever. But we're trying to find a way to make something that's efficient, fair, real-time, functional, reliable, secure mechanism to ensure that all the transactions that you're talking about are genuine. And because we have this big decentralized mechanism, we're looking for a way to make sure that all of these participants that are working inside of this chain are agreeing on the status of this ledger. So at its core, it is an immutable ledger. That's what the goal of blockchain is. But I think when we talk about blockchain and blockchain security, just like when we talk about pen tests and DevSecOps, it kind of inherently bundles all these things and industries and markets and technologies like decentralized finance and cryptocurrency and DAOs and supply chains and NFTs. And so it all sort of becomes part of what people think about when they talk about blockchain. And now thanks to some of the larger corporations, we're now, now it's part of metaverse. I mean, there's all these things that we think about when we think about blockchain. **6:30 Robert Hurlbut:** But at the end of the day, It's an immutable ledger of transactions, immutable being it's unchangeable. Once a transaction is written and accepted on the blockchain, it's there forever. And you think about something that seems oh so simple, but yet can have so many different use cases. And as we start to unpack this, so many different potential areas of challenge in the future. This is one of those areas that it's kind of, it's an interesting time to watch on Twitter all the conversations in regards to blockchain, because there are some people that hate this. They hate the whole idea of, I think it's really cryptocurrency is what's driving their hatred. And this whole idea of, you know, Web3 being this new generation of companies and new technologies that are going to do things using the blockchain as the kind of backbone, you know, versus Web2 where companies like social networks and things, these companies own all the data, they get rich off all of the content and stuff that you post. And Web3 being this idea where, you know, you're in charge of your own data because you have some further ability to own it, control who can access it and whatnot. And so I just find it funny that there's some people that just have such a hatred. They're like, this isn't even a thing. It's just, you know, a bunch of made-up stuff for people to try to rip other people off. And I don't feel like they've gone and really understood the essence of what can happen here with this powerful of a technology. So let's see, Ken, can you tie together blockchain, cloud, and application security and draw some connections for us there? Because most of our listeners are going to be very familiar with AppSec. They're going to be very familiar with cloud and blockchain is going to be the new thing. It's going to be the one thing on this list that doesn't fit. And so how do we, what's the intersection between blockchain, cloud, and AppSec? Sure. **8:25 Ken Toler:** So I think the best way to look at this is to think about blockchain exactly as you described it, which is a technology, as a tool, as a platform to build interesting things on in the same way that, you know, I always say like a decade ago, but probably like 5 or 6 years ago, we talked about during cloud migrations, it was cloud as a tool for you to move your workloads. I like to think of blockchain in a similar way because we're talking about a lot of the same When we're coming up with ideas, we're looking at ways to use this technology to make it better or easier for humans to interact or safer or more secure. And I think security is constantly bundled into blockchain and it's almost made it this inaccessible idea because everyone thinks of it as something that if they don't understand the tech of it or they don't understand it in depth, that they can't even start to approach like, well, addressing security on it or thinking about it in that way. But if you look at a lot of the organizations like these, where you buy your cryptocurrency or some of the projects that you're seeing, I mean, I've seen everything from decentralized exchanges to dating applications being developed on the blockchain, but they're still applications. And one of the common misconceptions is that if something goes onto the blockchain, that it's inherently secure because it's built on the blockchain. And I think what commonly sort of gets maybe not swept under the rug, but avoided is that there are centralized components to a lot of these, these products and these projects, whether that's internal applications and APIs, or whether it's someone creating maybe a centralized front end. You mentioned Web3, there are client components to this. So there are, there are standard applications built on these SDKs that are, that run in the browser and interact with the chain directly, that is all application security. And weaknesses manifest in the same way that we see them in every other JavaScript application or, you know, React application or things like that. Those are going to also manifest inside of these client-side applications. There's also cloud infrastructure that goes into this. There are folks that are running Validators or mining rigs inside of clouds. Organizations hold these in order to sort of have some some heavy weight inside of the the market, whether they're doing proof of stake or something along those lines. Like they have some sort of infrastructure that they need to protect, and a lot of these applications that we're talking about, you know, when you look at your NFT marketplaces, some of them you can connect with just a wallet, and we have all of the. Decentralization, but that's just a browser plugin. You know, there's an application security component there. And some of them, you actually have to sign up as a user with your email address and whatever. So that anonymous assumption that everything that you're doing is with a wallet is not correct in every implementation of using a blockchain or using a, you know, one of these public blockchains, because those users have to be stored somewhere. So just because blockchain is a part of the tech stack doesn't mean that it's all of the tech stack. And so there are still areas where applications are being used, APIs are being used, third-party integrations are being used, cloud infrastructure is being used, all of your containerization, all the things that we typically think of. It's just now that there's this extra technology component that we need to consider when we're looking at the entire threat model. **12:04 Robert Hurlbut:** That's helpful because I hadn't really thought of it. I guess I had this kind of maybe naive view of blockchain-related applications and things being just things that are like, you know, smart contracts or smart or distributed applications, right? It's another thing, dApps. And they get written to, they have some component on the Ethereum blockchain where the application itself can ride on that. And so I was kind of thinking maybe naively that, you know, the, the, blockchain-related applications are things that are just on the chain, but it sounds like what you're saying is there's just a lot of other, there's a lot of other what we'll call classic web infrastructure, cloud infrastructure, other types of applications and browser plugins and lots of different pieces that are, the blockchain is really just acting like a database in that component with all these other components that we need to worry about. **12:58 Ken Toler:** Yeah, there's a bunch of angry faces happening right now calling the blockchain a database. I can hear it in my ears. I can see it in your mind. Yeah, but it, to a degree, yes, you can store data on the blockchain, but it, I think the way that I like to look at it is when we started to move into app, like modular applications and working with multiple APIs and moving in that direction away from and into microservices, away from monolithic applications. **13:31 Robert Hurlbut:** Yeah. **13:32 Ken Toler:** We had this, we have a lot of these technologies that have come into play that we don't really think about as much anymore, like app-to-app authorization and all of these different security components that allowed us to securely and safely talk between these APIs that served a specific function. I think we're still kind of figuring that out to a degree. And so when you talk about smart contracts, smart contracts, some of them are very large, some of them are very small. But usually you're interacting with them by providing an instruction or a transaction on the chain. That smart contract takes it, does something with the logic, it interacts with other smart contracts. So it's very much like a— it's similar to a microservices architecture. But how that instruction gets there can come from a variety of different areas, whether that's from one of these client-side applications with direct user input, Or it can come from like an organization wants to send this automatically or use an API. So these all interact and have this interplay together. And I think that we need to treat it in a very similar way because the application itself doesn't solely exist on the chain. It's just one component of it exists on the chain. And so you'll see a lot of this happen. Like we had some of the larger attacks and weaknesses and And things were around, you know, cross-contract weaknesses or authorization or the wrong, you know, they didn't check that the correct owner was allowed to run a particular function within that contract. These are all authorization vulnerabilities and weaknesses that happen that we would, we would consider very similar to an unprotected or unauthorized API, right? If it was just a public API, we had no authorization structure, like we'd have a very similar attack. It's just that now we can see it all happen on the chain. **15:20 Robert Hurlbut:** Yeah, that's, that's very helpful. And before we transition into thinking about different failures, I want to get your take on one other thing. So when I think about blockchain and AppSec, cryptography is a big part of blockchain. Like cryptography provides the foundational strength that allows blockchain to be provable and to go through the operations there. Now, as somebody, you know, you're somebody who's thought about this a lot more and has a lot more, a lot of experience in this. Do you consider cryptography to be part of application security in regards to blockchain? Or do you just— have we reached a point where we're just like, oh, the crypto is all good? It's been checked by lots of cryptographers and everybody believes it's good and there is no security component of crypto when we think about AppSec and blockchain? **16:16 Ken Toler:** Um, I don't think so. I think that there are still cryptographic weaknesses because there are— so there are a couple of— the way that we approach this, like, from the— on the consulting side is, you know, I am— I'm not a cryptographer by trade. You know, I've got, um, some, like, some crypto chops, but I don't spend a lot of my time doing cryptographic assessments. And, you know, we have a team that does cryptographic assessments, uh, that are looking at new and novel cryptography. And that's part of, you know, those are part of the services that we offer. And, you know, I come out of those conversations always feeling stupid when I'm talking to these folks, right? But there are new chains being developed all the time. Like we talk about Ethereum and we talk about Bitcoin and we talk about these main blockchains, but there are still new and novel chains being developed that need new and novel crypto or are trying to implement a particular type of crypto in a different chain or in a different language. And so I think that there's still a security evaluation that needs to happen for those. I also think that we have the same issues with cryptography in writing contracts, right? The correct implementation of cryptography and the correct implementation of the libraries that are known and evaluated by these professionals. But I don't think that we're at the point where we can just Just call it good. And in fact, one, I read a Reddit post recently that was like, and I know already I'm like getting, you know, just like going to the wrong resources to find information. But I read this Reddit post and essentially it was like, you know, security on this particular chain is, you know, thinking about it is stupid because it's, you know, one of the best chains and da da da da da. And there's still this mindset of exactly what you said, which is, you know, the crypto is strong. Like the whole point of blockchain is to be secure. And so we don't have to worry about it, just build the application. And there are still people in the blockchain community that believe that if something happens to you, it's the user's fault, right? **18:24 Robert Hurlbut:** Mm-hmm. **18:26 Ken Toler:** That the chain is secure. So whatever you did, you lost, you didn't handle your wallet correctly or you know, they wrote a bad smart contract, but if they had written it correctly, then it would've been fine. But I think that there's part of our issue in blockchain in the community is this assumption of security because it's blockchain. And that's just not, that's just not the case. I think it, it, we still have to continue to be mindful of things that are changing. **18:52 Robert Hurlbut:** Yeah. To draw a parallel from my AppSec history and experience, and, and you guys will, this will probably resonate with both of you as well. There was a period in time, I wanna say, I'm going back 10 years where people, developers would say, you know, you'd say, well, you know, how's the security of your application? Oh, it's great. We have TLS. We implemented TLS. Like, yes, that's a great thing, but all you've done is encrypted the channel that the attacker can use to send a SQL injection inbound to your application. So we had this false sense of security that was brought upon by using strong cryptography. Like, it was a good thing what they were doing, but they had this idea of like, I don't need to worry about SQL injection, I got TLS. Well, no, that's, that's, that's a— you're protecting the tunnel, not the things that go through the tunnel. It sounds like there might be a similar challenge right now in the blockchain world where people are like, of course the chain's good, it's, it's, it's cryptographically strong. **19:51 Ken Toler:** Exactly. You're, you're exactly right. And to, to sort of add on to that point, the idea that these client-side applications don't need to be considered in security evaluations. Like many of the security audits that are requested are for smart contracts, and there are organizations that ignore their client side completely because they believe that, well, it's, you know, that's for the user. Like we don't have any servers hosting this or whatever, so there's no responsibility for us on the client-side application. It's just interacting with the blockchain. So we just want to cover the smart contract. And that's like going into an application and only looking at the controllers. You know, it's like there's this whole other side of it. And so then you have to have the conversation with that organization that is, well, do you care if your user, you know, is affected by a malicious user outside of your control? And do you want to protect them against that? Like, does that have an impact on your brand at all? Like, you know, what does that look like for you? And I think that that is a conversation we're going to continually have as these applications become more and more complex. It's almost a dirty word to talk about centralization in the blockchain community, but I think it's an important conversation to have because there are some problems that are being solved with centralization in the decentralized space. And it really depends on how an organization or project or person wants to use this technology. So if you're thinking about the crypto space and specific things like decentralized exchanges, sure, but there's still a market for centralized exchanges like your, your Coinbases of the world that, that are going to provide a centralized place for you to go and use your credit card. Well, if you're using your credit card, I mean, you know, that's not going to the blockchain, you know, that has to go to some payment processor elsewhere. And so there is a centralized component to that. **21:42 Robert Hurlbut:** Yeah, you thought I was going to get in trouble for saying the blockchain's a database. You just suggested in a world of decentralization that security might lead us down a centralized path. **21:53 Ken Toler:** So yeah, I know, I'm starting people's New Year's off with a lot of like red faces and anger, I'm sure. **21:59 Robert Hurlbut:** Well, they can decide who they want to go after first. It's me, the blockchain is database, or you, the centralized frontman for decentralization. There you go. Well, as Chris mentioned a moment ago about failures, and I think maybe you've alluded to some potential, but what are some of the common architectural failures that you've seen? **22:20 Ken Toler:** I think that we sort of discussed them, but just to summarize, it's kind of like, I think it's the ignoring of these centralized components. I think the— when you're looking, when we're doing an architecture review, I think there's a very large percentage of folks that come to ask about security that focus solely on smart contracts and the blockchain, which is important, and ignore the rest of their infrastructure or don't have an idea that, you know, many of these projects exist open source. It's super— I mean, I love where the community is in terms of transparency of their code, transparency of their security audits, and everything that goes into trying to be transparent with the folks that are using your product and your platform. But there are organizations that have these, you know, cloud infrastructures and just don't worry about them. And so I think that as far as the failures there, it's ignoring the security in your in the rest of your product is like the biggest failure. So your user management, your 2FA, your input validation, your trust of the user, the trust of wallets, even the trust of the contributors to your project. You know, just watching as, you know, these open source projects accept commits and, you know, they get pulled in and it's thousands or hundreds of thousands of people committing code, university students, professionals, non-professionals, folks that have just learned this language are making contributions, which is great, but offers the same problems that we have with open source. And so, you know, how, how much backing does the project have? You know, how, how much has security looked at this? Who is looking at it? All those things are, are weighing in just like what weighs in with open source. But now, you know, you have something that's handling money, currency. And that now that is like open source. I mean, it just, it has a bunch of potential problems, which I think is why larger organizations with sort of bigger wallets and bigger ideas tend to have some centralized component of it because they want to control a piece of this. **24:45 Robert Hurlbut:** So I want to get into what kinds of surprising mistakes you've seen in blockchain security. But first, as I'm thinking about everything that you've said so far and about the fact that there's organizations who are doing things in the blockchain space, who's responsible for blockchain security inside the average organization? Is there a title of blockchain security engineer that you've seen, or who's doing it? Who's responsible? **25:11 Ken Toler:** Yeah, definitely. There are, there. So I think you have to look at how big some of these, I mean, we say organization, but projects, how big these projects are, because when just What I think the really powerful thing and the great thing that blockchain is offering and the cryptocurrency market is offering is like, if you have an idea, you can create that idea or that application, you can deploy it yourself on the blockchain, and you can be handling and making money off of that idea very quickly based on just the economics of the blockchain that you're working in. And some of these organizations are 1 to 2 people. Right, that come in and do this. So in those organizations, no, you're not going to have a blockchain security engineer, but you might have some folks that are interested in helping you with your security, or you might be able to send crypto to someone to do a security audit from, you know, your, your favorite firm. And you can go onto GitHub and look at a bunch of public, you know, security reports that, that show you, um, you know, how good these folks are, what, whether they're looking for, you know, what your what you're interested in. And you have conferences just like we have for, you know, security where you can go and meet these, these folks that are doing these security audits. But there are, there's definitely room for those people. I think it's just that finding those people is really hard because it's so new. And there are just, there's a, there's a huge learning curve to being able to conduct an audit in this space because of lack of tools, lack of expertise, lack of knowledge. lack of finance knowledge, right? Just having that under your belt. **26:51 Robert Hurlbut:** Yeah, definitely a different set of skills than our classic, you know, people that take a— follow a path to a security engineer. You know, I think blockchain security engineer, you got to have a certain amount of crypto knowledge, meaning when I say crypto, I don't mean crypto coin, I mean crypto cryptography. You know, you got to have that foundational understanding for how things are working under the hood. It's the only way you can know if they're working correctly is if you know how they're supposed to work. And then you've got, you know, components like we said earlier of cloud and AppSec and It's really a, it's got to be a brand new area, but a lot of opportunity for people to be successful there. So surprising mistakes. So what surprising mistakes have you seen in blockchain security? **27:34 Ken Toler:** I think repetitive mistakes. I think, you know, one of the interesting things is that you have, especially in more well-known well-known chains, you have things that already, that have been looked at and reviewed and exist, and they have these best practices, but they continue to sort of manifest in other ways because the same things that we have in application security, your StackOverflows and copypastas and everything. So that's super surprising. I think the other surprise, which I mentioned earlier, is that we are repeating history again and again and again with just like the trust and all of that being, seeing the same sort of pattern that we saw with microservices security, mobile security manifest in blockchain security in this. I think I keep harping on it, but just the ignorance of, or not ignorance, but ignoring of sort of the other components of an application. Just being too laser-focused in one particular area. I would say that that is the biggest thing I would want to take away in terms of common mistakes that I'm seeing is just like, take a step back from the project and look at the bigger picture. And that is like what manifests, I think, from like the threat models that we do or the architecture reviews that we do. **29:05 Robert Hurlbut:** Can you pen test a blockchain? **29:12 Ken Toler:** Yes. Yes, you can. I think it depends on what your definition of a pen test is. Just like we were talking about before, it's— I like to look— so when you ask somebody like, hey, we want to pen test, I mean, you have probably seen this before. Some folks sort of look at a pen test as like a web application assessment or a dynamic assessment. Some of them will look at it as like an adversarial simulation. **29:40 Chris Romeo:** Right. **29:41 Ken Toler:** Some of them will look at it as like an audit, or, you know, they think of the final report that they're going to get every year. It's required, so they just ask for a pen test because that's what they know. And so some folks will say you can't pen test a blockchain, but I am of the opinion that you can pen test a blockchain and The way that I answer that is I think that anything that you can attack as a quote-unquote hacker or attacker, you can pen test. And so what I think we don't have right now that's consistent are the tools that we have to conduct other pen tests. So a lot of the pen testing that we do, if we think of pen testing the way that I think of it, is like an adversarial simulation. You're looking at it as an attacker. you have to write your own clients and write your own way of interacting with the chain and tamper with your own data because there is no Burp Suite or Metasploit or any of this that you can use these tools. So you're doing a lot of this manually. So there's just a general lack of tooling. Now, I will say that when we get a request for a pen test, often what we're trying to do is sort of set the terminology for security within blockchain conversations because we are saying, Well, do you want us to do a source code review or are you looking for a smart contract audit or what is the actual thing that you are wanting to get out of this? And a lot of folks in the industry or in the blockchain communities just don't know what they want. They see that, they know that their users, the people that are using their product or their project want some security assurance and they don't necessarily know how to give that to them because it's still an industry that we're figuring out in terms of what folks want. And usually that's very different. So you can pen test it. I think the method of doing that is just not 100% clear to everyone. And what folks define as a pen test is also not 100% clear, but you certainly can, especially if you're looking at it from the project level. From the blockchain, if you're talking about like just the chain itself, I think that you could still do that too. I mean, you're looking at, in that case, you might be operating on the, you know, the sort of the consensus layer or networking layer and trying to tamper with that as an attacker. Or maybe you're standing up a, you know, malicious miner, or you're trying to control transactions that happen on the chain. You know, that would be pen testing a blockchain, in my opinion. **32:12 Robert Hurlbut:** So I want to talk about smart contracts and secure coding, how these things fit together. When I first heard the term smart contracts a few years ago, Like, I didn't really grasp exactly what the power of a smart contract is. I had this idea of like, I don't know, a real estate transaction being done digitally. Like, that was what I thought a smart contract was. I just, I didn't realize that a smart contract would be something you could have code associated with that could be run multiple times over and over again. So when— Give us a definition of a smart contract before we start. Before we get into even secure coding and smart contracts, like set the stage for us. When you hear smart contract, like what's the— or someone says, hey, Ken, what's a smart contract? What's your definition? **33:01 Ken Toler:** I'm going to go completely off book here and say that I look at smart contracts as something that you put input into and generate output from. That's like how I try to look at it. Because the capabilities of languages in smart contracts on whatever chain you're talking about seems to change pretty frequently. And now we have like these very powerful languages like Solidity where you can do almost anything in a smart contract that you'd be able to do logically in any other, you know, high-level language like Python or whatever else. So if you can, if you think of it as like if I were to write a you know, from the application side, if I were to write a function and I'm taking input into that function as parameters, anything that you can do in that function, you could probably do in a smart contract. Now, the difference is you're going to have to pay for that compute power. And one of the things that we look at, you know, it's the operations that happen on the chain. So that's, I don't know if that helps. I'm trying to like sort of step outside of the contract word and think of it more as as logic that's executing in a shared compute environment. **34:14 Robert Hurlbut:** Yeah, I think that's a good definition that takes away from contract is such a loaded term. And that's what I was hung up on for a number of years until I went on this recent trip into blockchain and all the things that are part of it. But so let's talk secure coding now. You mentioned Solidity. What, you know, is secure coding a thing? **34:39 Ken Toler:** Absolutely, absolutely, 100%. It's just really difficult to find guidance for secure coding in these ecosystems because it means a bunch of different things. I think when you, when you look at secure coding now, you have great resources from OWASP and you have books and you've got all ways of like how to code securely and common attacks and things to avoid and the top 10, all these kinds of things. Yeah, yeah. Of things. But with the exception of Solidity, Solidity has— there are some projects out there that talk about common weaknesses in Solidity and things of that nature. But most of the other sort of low-level languages that these things are written in, like Rust, when people think of secure coding in those, it usually means they're still thinking on the language level and not necessarily on the smart contract development level. And so the business logic considerations that you make are changing. And I think what's becoming more, or what I'm realizing more and more, is just the importance of threat modeling in these exercises, because you really have to think through the business logic of what this smart contract is trying to do, what inputs and outputs are going into this function that you're executing, because most of the attacks that people are going to care about are not going to be I mean, they're definitely going to be your underflows and overflows and things of that nature that are these low-level sort of weaknesses, all the same things that we talk about, like race conditions or these sort of what I call low-level language-specific sort of weaknesses. That business logic stuff are things that you have to think through, and most of them are taking advantage of the financial logic or the math that happens within that function that you're going to have to think through. And I think in order to like be a great security auditor of a smart contract at that level, you really have to understand the finance side of things as well, the fraud, the sort of research into that and how the money moves and what the math looks like. **36:42 Robert Hurlbut:** When we have a smart contract that is written onto a blockchain, Is that— can I just go download it and read that? Like, is that then— is that code then available to anybody, or does it get compiled into some way that it's like Java bytecode and I can try to decompile it but I can't see all the variable names? Like, or is it just flat out once it's there, it's there and anybody can read it and search for vulnerabilities? **37:12 Ken Toler:** So it, it is compiled, uh, in, in most cases, but it really depends on the, on the chain that you're using. So in the case of Ethereum, you know, it is compiled like you were talking about. You can see it on the chain and, you know, there, when you're looking at the Etherscan, you know, you're looking at sort of the decompiled or the decompiled contract, but you don't really have a way to say, yes, this is 100% what was deployed on the chain unless the author goes on there and uploads the corresponding code. **37:43 Robert Hurlbut:** Yeah. **37:45 Ken Toler:** We can also say that most of these projects being open source and those open source projects being available and the code being available, you can audit that code, but you have no real way of determining whether or not what was in the GitHub repo is actually what was deployed to the chain. So there are, to a degree, yes, you can see the code that's there and it's all open. What's really open is the ledger. Right? What was the input and output of that particular transaction, but not necessarily 100% whether it's what the code that you're looking at is exactly what was uploaded to the chain. But again, that also depends on which chain you're talking about. And these are some of the things that, the questions that you're having, why new chains are developed. Well, you know, we can't see all the code. So, you know, we don't, maybe we go and that's a feature of this new chain. The other things that are coming out are projects that are, you know, working through SDKs and ways to develop contracts that allow you to have a consistent way of developing these contracts rather than developing them from scratch. So it's semi-transparent from the code perspective, but for the most part, yeah, you can look at the code. **38:57 Robert Hurlbut:** So do you look at the code? Like if I write some code in Solidity, do I have to validate the input that's coming inbound. Like when I, when, when someone asks me like, what's the, what's the root of the OWASP Top 10 issues? It's, it's input validation. Like if you could only solve one thing, if you could only take one mitigation from the OWASP Top 10, for me it'd be input validation all day long because I can, I can protect against so many other things, whether it's log injection, whether it's cross-site scripting, SQL injection, and any other injection. Is that, is that something I have to worry about in a smart contract where I first have to validate my input, and if I don't, then that could be the source of a giant vulnerability in my contract? **39:41 Ken Toler:** Absolutely. I mean, there are— one is if you're taking, if you're taking input into a contract, you should be validating it. You have similar protections around, you know, static typing or, um, or some sort of built-in validations in some of these languages. But you may not be able to cover all cases unless you, unless you understand your own business logic and validate what's expected. This is also the source of a lot of the ownership issues that we were talking about, or the authorization issues between contracts, because it wasn't validated against the owner of the contract. So when you deploy something in, depending on which chain you're talking about, you obviously don't want everyone to be able to execute administrative functions on the contract that you're deploying. You want to be able to control that in some way. Well, the only way that you control that on the blockchain is by verifying the private key, you know, the key relationship to the owner or the authorized party on that contract. So if you don't validate that and you just let everyone access these functions, you know, that authorization control isn't there. Comparing values is also similar. The other thing that I, you know, sort of harp on in terms of like sending things back out into the world from the chain, is that the outputs of these contracts eventually end up in the hands of like a client-side application. So all of your sort of browser-based attacks are still valid as well inside of the smart contract because the output that you're generating eventually ends up on the client side, well, in some cases. **41:10 Robert Hurlbut:** I'm realizing we could literally talk about this and I could ask you probably 100 more questions and we still wouldn't capture all of the All of the intricacies of blockchain and security. So I want to ask one question before I get to your key takeaway, just to point our listeners in a direction if they want to go look at some other resources and stuff. Like, what would you recommend? Are there any books or podcasts or blogs or sources of information that you would say, hey, for somebody who just wants to dive into the intersection of security and blockchain, where would you send them? **41:49 Ken Toler:** So there, I'll give you some links to maybe drop in the show, but there are a lot of resources on Solidity. There's the Blockchain Council. YouTube is a great place to go for a lot of this stuff. The, you know, on my side, from just like the Kudelski Security blog has some things around blockchain there that you can read. As far as books, I would definitely, if you're looking at some of the low-level languages, you know, read up on Rust and Python and things of that nature. Definitely JavaScript for like WebJ3. There is a great collaboration on just blockchain and cryptocurrency and security from Securium that I'll send you as well. It's just a list of videos. They did a great job putting a bunch of educational material together if you're just looking to, you know, dive straight in. But I'll definitely give you a whole bunch of lists. We need more people in the space, so I'm happy to point everyone in the right direction. **42:53 Robert Hurlbut:** Yeah. Very cool. So what would be your key takeaway then? You know, as we, as we wrap up our conversation, when you're thinking about, you know, what's the call to action for our listeners here? What do you want them to go do? as a result of our conversation today? **43:07 Ken Toler:** I don't want folks to be afraid of security and blockchain. If you think— if you're listening to this podcast, there is something that you can contribute to blockchain security, whether that's architecture reviews or just being involved in the conversation or thinking through business logic flaws or whatever it may be. I think that the biggest takeaway is that blockchain is not a silo. that you can definitely 100% be involved in blockchain security with your current application security skill set. And in some cases, you may not need too much ramp-up depending on what that engagement or conversation might be. It's not as, I guess, unapproachable as I think the majority of security folks think it is. **43:53 Robert Hurlbut:** Well, Ken, thank you for taking the time to educate us and our listeners about blockchain security and AppSec and cloud and, you know, how all these things fit together. I know I took away a lot of different things and I literally have hundreds more questions. So we'll do a follow-up in a few months. We'll do another one of these because I want to, I want to ask more questions and learn more from you about this. So thank you for sharing your, your knowledge and experience with our audience. And I look forward to a future conversation in the coming months. **44:26 Ken Toler:** Sounds good. No, I really appreciate you having me on. I love talking about this stuff, so I'm always happy to chat through it. Thanks for bringing me on. **44:34 Chris Romeo:** Thanks for listening to the Application Security Podcast. You'll find the show on Twitter @AppSecPodcast and on the web at www.securityjourney.com/resources/podcast. You can also find Chris on Twitter @edgeroute and Robert @RobertHurlbut. Remember, with application security, there are many paths, but only one destination. --- Source: https://appsecpodcast.com/ken-toler-blockchain-cloud-and-appsec/