Frank Rietta — The convergence of Ruby on Rails and #AppSec

0:00Chris RomeoFrank Rietta is the CEO of rieta.com, a security-focused web application firm. He's a web application security architect, expert witness, author, and speaker. Frank joins us to discuss secure coding with Ruby on Rails. We get into a discussion about Ruby on Rails versus other languages, primary threats, counters to those threats, and tools available for the Ruby on Rails developer to assist with security. We hope you enjoy this conversation with Frank Rietta. At Security Journey, we believe security is every developer's job. We work with our customers to help them build long-term, sustainable security culture amongst all their developers. Our approach is to provide security education that's conversational, quick, hands-on, and fun.

0:49Robert HurlbutWe don't do lectures.

0:50Chris RomeoInstead, we let the experts talk about what's important. Modules are quick, 10 to 20 minutes in length. We believe in hands-on experiments, build-or-breaker style that allow your developers to put what they learned into action. And lastly, fun. Training doesn't have to be boring. We make it engaging and fun for the developers. Visit www.securityjourney.com to sign up for a free trial of the Security Dojo.

1:14Robert HurlbutHey folks, welcome to this episode of the Application Security Podcast. This is Chris Romeo, one of the hosts and also CEO of Security Journey. Robert's not with us today, but I am happy to be joined by Frank, who is going to talk to us about Ruby on Rails security. But before we get there, we have to ask the question that everybody is sitting Maybe they're driving in a car, so they're not sitting on the edge of their seat. Maybe they're sitting on the edge of the car seat, whatever, wherever they are. But Frank, how did you get into this crazy, wacky world that we know of as application security?

2:05Frank RiettaOkay, Chris, thanks for having me. So my name is Frank Rietta, and I have been self-employed for 21 years, which is not— maybe doesn't sound that impressive, except I'm only 38 years old. So this has been a good part, in fact, my entire career. So how did I get into security? How did I get into owning a web development firm that focuses on security is an interesting story. So I have been programming every single day. I mean, there might have been a day here or there that I wasn't feeling well, but, you know, for the most part, every day since I was 14. I just fell in love with coding. At the time, I was actually on like an Epson Equality 2, which was a DOS machine from the '80s, because even though my family had a 486 with Windows 3.1, I wasn't allowed to really mess with it. So I spent all my time on the computer I could mess with. So coding, you know, popping some assembler and stuff, learning a lot of those skills fairly early on, self-taught from books. Fast forward a little bit, and this was the late '90s, like '98, '99, and the web was really starting to take off. And being that I was into programming and a little web dev, I started a web hosting company. First as a reseller, and then I went to my mom. Again, I was under 18 years old, so I couldn't do this on my own. And my mom, for some reason, let me charge a Dell PowerEdge server on her credit card. And so I started racking and stacking equipment in a local data center and marketing hosting services. And it— and so this was before I went to college. And turns to find out that if you take Red Hat 6.2 and put it on the internet, it was compromised pretty quickly. Okay, so I cleaned that up. I start learning how to lock down Red Hat Linux. And so I ran that company for all the way finishing out high school, undergraduate Georgia Tech. I did study computer science. And I sold it right before I graduated in 2005. I sold the web hosting business atlantawebhost.com. But in that time of running that, I got to see like the dirty underbelly of society, you know, carding, hacking, you know, constantly attacking the servers I was running and my customers. And it quite frankly made me mad. And so since I had just graduated Georgia Tech and I had just sold a business, which, you know, wasn't a tremendous sum of money by, you know, my standards now, you know, it was more money than I'd had at one time for like ever up to that point. So I went ahead and went to grad school and I did Georgia Tech's Information Security program, GTISC. It's now Security and Privacy. And so I studied— I have a master's in information security and I went the policy track on that. And this was early. So Georgia Tech got certified by the NSA as being an academic center of excellence. And, and they had a lot of a very traditional network security approach. So when I came out of that, The career opportunities that were kind of natural progression were to, you know, go work for the government. And trust me, these are great jobs. Life would have been different as an FBI agent or a CIA analyst or whatever that career path would have been. But I didn't choose that path. And the other was to go work in, like, you know, the ISSA has a lot of members that are, you know, stodgy, well-heeled corporate enterprise, but I was a Linux dude. I'd been using Linux and FreeBSD, and so the thought of going and adminning Windows networks for security was just like death. That really did not appeal to me. I ended up falling back into web dev, picked up some PHP jobs, started doing more and more, and ultimately built a business on the theory that you can't bolt security on at the end. And since the, the decisions that lead to insecurity are made when the company that has them is in its infancy, when the first code is being written. And so my theory was, well, if I learn how startups work and get in very early in doing development, then I can move the needle a lot for these companies before they even have a problem. It turns out that that's nice in theory. In practice, there's time to market and cost-benefit analysis, but I still believe firmly you have to build security in. You can't bolt it on at the end. So yeah, that propelled into the last 9 years of owning a firm that does a lot of Ruby on Rails development and security auditing and consulting now too.

7:13Robert HurlbutSo when you think about the knowledge that you acquired on the application security side, Sounds like you had a lot of almost school of hard knocks type of approach to learning by, you know, starting the web hosting company, getting things hacked, which is, I mean, in all actuality, a great way to learn. I had a similar experience when I was in college and my first Linux server was hacked and I got to experience incident response in the early '90s when nobody really knew kind of what that was. But that's a story for a different day. But when you think about kind of the knowledge that you have acquired in the world of application security. What's the primary way that you've done that? What are the, what are the types of resources and things you went after in case somebody's listening right now and they're thinking, well, I have a similar story here, you know, I'm a dev, I was a developer for a long time and now I want to get into security. What were some of the things that you did to be successful?

8:05Frank RiettaYou know, when I was starting this stuff, OWASP was just in its infancy, right? I mean, the, I think the first top 10 was around 2003. And a lot of that was, you know, buffer overflows and stuff, and that shifted. And so in my early days, a lot of it was— there weren't the resources there are today. And if there were the resources, they were very expensive. And I think early on when I was at the Georgia Tech program, we had these capture the flag events. It was, I think, UC Santa Barbara and a bunch of universities ran a VPN together CTF. And one of the things I learned early on was, you know, definitely from the web hosting days a little bit earlier was don't assume everyone who uses your application is going to be for good. I think as developers, it's very easy to focus on the happy path, and as organizations, when you're writing like feature requests and stuff, it's very easy to ask, but why would anyone ever do that? And so having that mindset is the first one. And so anyone who's listening here probably already has checked that box of having a security mindset. But, but today is not like the mid-2000s. Right now we have the OWASP We have the OWASP Top 10, we have the ASVS, we have the cheat sheets. Like, all these are great resources for understanding not only common vulnerabilities, but also like patterns that you can do in your work to harden your application against these threats. And so I would say avail yourself of the open materials. There are others. I don't wanna list off names. I'm not casting disparages against some of the more enterprise-focused tools. But I like open source. I like OWASP and the resources that the organization has been able to make public. And so I'd say start there would be my answer to your question.

10:16Robert HurlbutYeah, I think that's great advice for somebody who's brand new, who may be coming to this podcast interview via the development path and is thinking about, hey, this security thing seems kind of cool. You've given us some great guidance on where they can go. So let's get to our primary topic.

10:33Frank RiettaOne step I will say, Like I did an informal survey about 2 years ago on the Tech404 Slack community, which is a kind of Atlanta area. So these are paid professional developers, like they're paid to write code. I don't have the graph in front of me, but it was depressing. It was like maybe half of them had even heard of the OWASP Top 10.

10:54Robert HurlbutYeah, that is a pretty depressing—

10:55Frank RiettaSo there's a lot of just simply having that knowledge is already taking you to the right path.

11:00Robert HurlbutWell, let's get into specifically Ruby on Rails here, and I'm gonna pretend that we have some people that are listening that may not be as familiar with Ruby on Rails as I am, and I'm gonna just ask you that kind of general question. What is Ruby on Rails?

11:18Frank RiettaSo Ruby itself, I'm gonna answer it in 2 parts, one Ruby and then Ruby on Rails. Ruby is a beautiful programming language, and I'm saying this as a computer scientist. Ruby is a fully object-oriented programming language on the order of Smalltalk. Everything is an object. And that is a paradigm that we lost for a little while. From the '80s into the '90s, we started having a lot of— the C-style languages started to really take over. And as computers got faster and Ruby has been around since the '90s, and it became a popular scripting language on the Unix side. If you ever use FreeBSD, you may remember the ports collection with the port upgrade utilities were written in Ruby. If you work in security and you know of Metasploit, you may think that Metasploit is written in Ruby. It is. And so it's this beautiful programming language, purely object-oriented. The number 4 is an object. It responds to messages. You can override them. Everything is a message. The divide character is a message. There's no— there's hardly any language special features, and that has lots of interesting properties as a programmer. Rails, Ruby on Rails, was invented originally by a gentleman of the name of David Heinemeier Hansson, and it was written for Basecamp. What's now called Basecamp was built on top of it, and it has evolved along that, and it created a platform to build web applications that standardized and solved many of the common plumbing that is necessary to have a web application so that as a developer, you don't have to focus on it at all. And at the time, and it had a very strong model-view-controller approach, which is an excellent object-oriented design pattern for creating software. Now, you say, but .NET has MVC. But .NET MVC came after Ruby on Rails. There is CakePHP, or there are several PHP ones, but those came after Ruby on Rails. Ruby on Rails really redefined what it means to have a web application framework for productive creation of web applications, and a lot of those benefits have filtered into languages since then, have copied it in a sense. Not necessarily bad copy, right? But were so inspired by the benefits that their platforms have taken that approach as well.

14:01Robert HurlbutAnd when you think about Ruby on Rails, a lot of people will say, ah, that's just for early-stage startups. It's something they use to be able to build and deploy apps quickly. What's kind of your response to that? Is Ruby on Rails just for early-stage startups?

14:20Frank RiettaI would say no, but I want to go back to another lesson. If you're a student of computing and you go back to the '80s, Microsoft Excel was not the spreadsheet of the day. VisiCalc was, but then Lotus 1-2-3 was super strong, but then Excel took over. And the story goes, if I recall correctly, was Lotus was in the middle of a rewrite of their application for various reasons. And, uh, rewrites never happen as fast as you think. And in that time, Microsoft slipped in and gained market share and Lotus never recovered. So rewriting your software as a mid-stage or growth-stage startup can be a very bad idea, um, for all sorts of reasons. So if you start on Ruby on Rails, you're very likely to wisely choose to stay on Ruby on Rails unless your scaling issue is really unique. Twitter famously started on Ruby on Rails, but then got off of Ruby on Rails because— not because Ruby on Rails was slow, but it turns out you can't run Twitter at scale as a web application backed by a relational database. They had a very, very niche-specific communication platform need that had a different engineering approach needed. There are other companies. Basecamp, obviously, is on Ruby on Rails, and others that they've created. Square, the credit card startup, is on Ruby on Rails. Shopify is on Ruby on Rails. I mean, you can go down this list. I don't know exhaustively off the top of my head, but these are some massive companies doing a lot of business, continuing to run on Ruby on Rails, and that is not an issue for them. It's still a good technology choice. I am— I can't name names, but I am directly aware of enterprises that have Ruby on Rails as part of their infrastructure. It may not be that they have a ton of stuff written, but some of their web services were written in Ruby on Rails. And so those are something they have to support as part of their portfolio of applications. I am aware we have on our client base several government agencies, state government agencies that had custom software written in Ruby on Rails, and now their agencies utilize this software heavily. And so supporting it over long term is something that we help them with to make sure that it's kept up to date. that it's secured, that it continues to function as new Rails versions come out. So it's all over the place. I don't have market share figures in front of me, but it's out there and it's only growing.

17:15Robert HurlbutI think it's more popular than most people think. I think people have a perspective that Ruby on Rails is that early-stage startup thing that I mentioned. And I think I've heard the same thing that you're sharing here, that there's a lot of big names that are using it, and so it's gonna only become more prevalent. into the future. And so here's the million-dollar question on the Application Security Podcast. Do you, as a practitioner, developer, and AppSec person who focuses on Ruby on Rails, do you consider Ruby on Rails to be secure?

17:51Frank RiettaYes, and I only hesitate for a moment because like any platform, you can make insecure software. Quite easily. But Ruby on Rails as a platform has a lot of things going for it. One, Rails itself has a lot of convention over configuration, and it comes with some really good defaults that have good security. Number 2, the Ruby ecosystem has embraced test-driven development in a significant way. And that means that the software tends to be very well tested, which means that it can be updated very quickly. So one of the problems in other languages, other systems, is it takes us a long time to update software when you're afraid that it's going to break if you make any changes. That is less of the case within a Ruby on Rails application that has a lot of automated test coverage. The gems— there are a lot of very mature gems. So for example, if you reach and grab devise to do your user authentication, then it's going to come with a lot of good defaults, and it's going to come with bcrypt. And so you may not, as a Rails developer, you may have never heard of bcrypt and why that's how you should hash your passwords, but your passwords were hashed with bcrypt. You never had the option of using SHA-256 unless you just really reached into the innards and changed all the configurations and said, I want to go the stupid path. You have to work at making it insecure in that way.

19:27Robert HurlbutYeah.

19:28Frank RiettaSimilarly, you get ActiveRecord by default. When I do audits of Rails applications, I fairly seldom see SQL injection issues. I'm not saying you can't make a SQL injection, of course you can, but the default is so strong at steering you down the path of doing it correctly that you just don't see that as often as you see in some other platforms. So I'm going to go with a yes. A Ruby on Rails by default is one of the most practically secure platforms from the point of view of the OWASP Top 10. I definitely see more like— when I see issues, it tends to be like an access— a business logic flaw or an access control issue, not a fundamental framework didn't know how to reject cross-site request forgery, because by default it even has cross-site request forgery protection built in. You have to fight it to turn it off.

20:25Robert HurlbutYeah, that's, that's always been the dream of the framework that we've all kind of wanted to see happen, in that you end up with a framework and the framework locks you into deep secure defaults that you really have to wiggle to get yourself out of. And I would say that's like the goal of like Spring. With Java. I don't think they're— I think they've done a lot of good stuff, but I don't think they're there. I don't, I don't think Spring doesn't lock you into that same scenario you described with password hashing, because you still have more, more freedom to be able to do that in like with Spring.

21:03Frank RiettaI mean, don't get me wrong, you can do it. But like Ruby, it's not locking you in like here are handcuffs. You can do a lot of things with Ruby. But it's just like the culture and the tooling is so much, but here's the happy path. Don't you want to stay on the happy path? And that really does steer devs in a good direction.

21:23Robert HurlbutI mean, just with the example with password hashing, like that, that is something that if I'm looking at that as a developer and I'm saying, well, this devise gem kind of package is, is out of the box handling password storage for me. I would have to be crazy to say, I'm gonna— I think I can do this better. Like, I want to move on and do something else. Like, we— I have other features that I need to deploy. I don't need to solve password hashing if you're giving it to me in a default secure manner. And so that's one of the things that I think also is really cool about Ruby on Rails is that, is that that thought has gone into it. And it's like, let's lock people in. Yes, they're not handcuffs, but you'd have to work— you'd have to work to, to work your way around that secure default, which is how all secure defaults should be. You know, if somebody chooses to turn it off, but it should be turned on by default. If they choose to turn it off, that's their— that's on them. That's their risk management process that's going to eat them for lunch at some point in the next 12 months.

22:22Frank RiettaYep.

22:23Robert HurlbutSo what about secure coding guidance? So when I think about like the C language, we have the CERT C Guide, which is so well known across the industry. Like if you say, you know, where do I go to learn more about secure coding with C, everybody points you to that CERT website. Does Ruby on Rails have anything similar that's kind of like a de facto place you would send people?

22:42Frank RiettaSo if you Google Ruby on Rails security, there is a security.html guide for on Rails that introduces you to a lot of the security countermeasures and stuff. That is a good place to start. And then from there, there's a lot of material you can find on Rails security. I've started to see some more training materials coming out on it. But if you start at that guide and you run some tools, there's a tool, it's a gem called Breakman, which will scan your Rails application and flag— it's a static analysis tool, so it will flag a lot of vulnerable patterns like, you know, you're using eval, you really shouldn't be using eval. You know, you're taking a string and you're smashing it in the middle of another string that's passed to a SQL database. Like, that's really a bad idea. You know, it's going to flag a lot of those. And so I would start with the guide and then kind of go from there. And, and then run— set up your pipeline to run, you know, Breakman, to run Bundler Audit. to flag a lot of stuff. But the number one thing that I would teach, and this is different than a guide, this is a slightly different question, is really embrace writing automated tests as you go about doing your work. And so one of the nice things about that is when I write a test against a web application, we often write the happy path. But what I encourage you to do Is to think about the things that could go wrong and to write a test for that. So a common example is, let's say I write a—I have a controller action that gives access, and there's an access control. So there's a concept of accounts. So always write the test that says that a user B tries to. edit user A's document and specifically gets an HTTP status error that it is unauthorized. Write that test. So read the guides, but start thinking about how you test security failures, especially those business logic and those, those authentication errors that the framework and the scanners can't always detect for you.

25:20Robert HurlbutSo I've read a couple of articles recently that have been talking about the fact that TDD or test-driven development seems to be kind of on the way out from the perspective of development at a high level looking down. And so, you know, you mentioned some other languages as well, and we think about, you know, with Java, C#, you certainly can do TDD in that world. It's not as prevalent as what you're describing in Ruby on Rails. I mean, I would say you can, you can take that approach with any language, but from what I've seen, people are starting to grumble that TDD is on the way out. Do you think that is going to impact Ruby on Rails in any way, or do you think it's so ingrained in how Rails works that it'll just continue?

26:05Frank RiettaI have a different opinion. I disagree. I think it is short-sighted and foolish for industry to throw the baby out with the bathwater with testing. So I'm putting back on, and I know this isn't exactly your question. Hopefully we'll get back to it, but I want to share this. If you look at the history of computing and the trends that we're going, we are increasingly dependent upon web services and, and dependencies, our dependency graphs are only growing, they're not shrinking. And that means that with every dependency added, you add brittleness, and you add another source of failure, sometimes that are outside of your control. And so software systems are becoming harder to reason about, not easier. That DOS programming environment I was talking about learning to code on, it was really easy relatively to read the code and understand what was going on. A modern web application is very hard to know all the bits and pieces that can cause it to break. And so someone who advocates for the elimination of writing tests that can provide some confidence that the software continues to function In the face of change, I think, is making an ill-advised trade-off for short-term gain for massive unsustainable technical debt payments in the future. Better would be to educate developers and to work on the culture to ask what's better, what's the most important things to test, but, but we do need to have a way of knowing the software continues to work. Rails itself, I don't think will change in that regard because I think the culture is pretty well set. We can dig more into that, but I'll get off my soapbox. I am not one of these people that are like, how dare you not write the test before you write the code? I can say, this is a better way because the best way to have software that continues to work in the face of changes, to have sufficient automated tests to have confidence that it works now and will work tomorrow and will work when a dependency's dependency is updated and we need to find out if we can deploy again. I can say that the best way to do that is to practice test-driven development, and the arguments are sound. But, you know, I can say if you're like, well, I wrote the functionality and then I wrote a characterization test to show that given this input, I get this output, that's sensible. That's not test-driven development, but that's sensible. But what definitely does not pass muster is to say, eh, we're not going to test. It's too hard, because that is very problematic.

29:13Robert HurlbutYeah. So when you think about security threats, then, the primary security threats that face a Ruby on Rails application, what's kind of on your short list of things that you're thinking about?

29:27Frank RiettaOh, well, is it a web application? Most Rails applications are web, so all the threats that are typical amongst web apps. So just, you know, pull out your OWASP Top 10 and start there. Now, of course, if you are fine with all those, that does not mean that you don't have some exotic threat against your system. That is true. So you start doing more sophisticated things like threat modeling and and more advanced testing, or maybe you engage a pen test. Code review before pen test. If the first time that you are thinking about security is when you hire an external pen test firm, you are doing it wrong.

30:08Robert HurlbutWell, this is— you're on my soapbox now, Frank. Come on. This is, you know, my big statement I keep saying. I've been saying this whole year is you can't hack yourself secure. So same exact mindset that you're coming from here. So we share a soapbox.

30:22Frank RiettaThere we go. But back to your actual question. I look at the OWASP Top 10, and I definitely am concerned about those. I'm concerned about password security. So even though we use bcrypt by default, you know, detecting password stuffing attacks, you know, is a technical challenge at some scale. You know, knowing to go look into the NIST— what is it— 863B standards and start implementing. There's a gem. Oh yeah, there's a gem for that. There's a— if you're familiar with Have I Been Pwned and the work that Troy Hunt has done to distribute checking for known vulnerable passwords, known compromised passwords, You can actually use that K— what is it— K-anonymity method for checking if a user who's presenting a password is presenting one that is in that corpus. And you can do that without transmitting the password itself to the network. And you don't have to— unlike the first time that came out, and we actually imported all of Troy's SHA-1 hashes into the PostgreSQL database and checked that way. This lets you utilize as a web service call. And so that'd be a very good idea because I've— more compromises of web applications come as a result of credential misuse than all of the sophisticated compromise vectors that are so much more fun to think about. But at the end of the day, if you don't handle the password issue, you don't have to worry about all that other stuff because you're going to get pwned anyway.

32:14Robert HurlbutOkay, and so software supply chain. So you were, um, in our kind of lead-up to this interview, you were— you shared a story about some of the influence you've had on helping the software supply chain, the gem supply chain, uh, from a typosquatting perspective. So, uh, tell us that story.

32:36Frank RiettaSure. So rubygems.org is not the only way that gems Gems can be distributed, but it's the primary way that gems are distributed publicly. And so it's an open repository. It's run by volunteers, and hosting resources are donated by various sponsor companies, and you can go to rubygems.org and see who the sponsors are. And so Turns out that there was several instances of attacks against Ruby gems in the last couple years to distribute malicious software. There's a few categories of these attacks. One type of attack that, that has happened and was covered on our blog was something like the REST client gem, which is a known, valid, well-used gem. One of the maintainers' credentials were compromised, and someone took that to publish a malicious version of the gem, presumably with the hope that users of REST Client would update to the malicious version, and then there would be a backdoor to steal things. Usually, these payloads do something like take a cookie and then read it, and then if it has certain aspects, to eval that code so you can actually pop a remote code execution on the vulnerable web application. Another type of attack is called a typosquat attack. And that's just like domain typosquatting, where you, you don't have the maintainer's credentials for the legitimate gem, but you make a confusingly similar gem in the hopes that someone searching, because the main way to discover gems is to search RubyGems, or let's be real, to search Google, which then had indexed the RubyGem page. Oh, that's the one I want, and then you copy and paste it. By having a confusingly similar gem name, you could trick people into using yours instead of the legitimate one. It turns out, so I started looking at this about a year ago, and it turns out that there's a lot of legitimate typo squats, meaning that most of the time that someone took a gem and added the number 2 at the end of it or changed a dash to an underscore, there was no obvious malicious intent. They were just like, well, they haven't updated that gem in a while. I want to contribute it, but to publish it, I have to change the name. I'll just change this little aspect of it. But there were malicious ones that got caught by eagle-eyed volunteers said, wait, why did this change? And then, you know, The alarm is raised and you get the articles that come out about it and the gem, malicious gem gets yanked. And for a while, RubyGems, for the very popular gems, like we're talking like ActiveRecord, right? Like Rails. To protect those from being typosquatted, introduced a typosquat protection technique based on the Levenshtein distance, which if you know how that works, it calculates the number of insertions in a string necessary to morph one string to the other. And so this is used in spell checks, algorithms, and other things. And the challenge with it is that it has a lot of false positives, especially for short names. So like if you have a 5-character string and you have a different 5— 4 or 5-character string, then the distance can be very low. So it's hard to set that threshold. Even if you set it to 1 or 2, you're gonna have a lot of false positives. Well, the way around that was to only check for, like, gems that had just, like, tons and tons of downloads. So there was a threshold here, a very high threshold. Well, then another round of— and that did work, but there was a lot of false positives. And back in April, there was a major attack where something like 750, off the top of my head, I don't remember the exact number, gems were all by the same bad guy They were all squatted and with a malicious crypto-related payload. And, and a lot of them had well under that threshold. So if you looked at the compromised gems, if we call it that, the victim gems, the data analysis showed that none of them were protected. So the protection worked. If they tried, but presumably, but I mean, there was hundreds of them that were under that threshold. And so I started looking at, is there a better approach? And basically would download the nightly PostgreSQL dumps from rubygems.org. I had a bunch of techniques experimenting with different ways of doing the queries, but I wanted a very fast query that could, that could check. And the technique that worked looking at the data was if a new gem cannot be the same name as an existing gem except for a variation of the dash and underscore characters. Does not catch everything, but it catches the vast majority of them, and it can be indexed, which means it can be fast with PostgreSQL. So we can actually now check every gem against every other gem, and there's no longer that threshold that has to be part of it. I worked with the maintainers of RubyGems.org, and within the last month, actually had a pull request merged in, and it's now part of RubyGems.org, this protection that I worked on. It replaced that previous Levenshtein distance method that was so problematic that they actually disabled it after this attack. There was a couple months where there was no protection at all until the one I worked on got merged in.

38:32Robert HurlbutVery cool. So when you think about the tools that are in your Ruby on Rails tool belt to help from an application security perspective, you mentioned Breakman, you mentioned Bundler Audit, which we need to— we should talk about those a little bit more. But just walk us through kind of what tools are in that tool belt and kind of a high-level description of what each tool does.

38:55Frank RiettaOkay. So I'm going to focus just on what we use, you know, bread and butter daily work. There's lots of tools out there. Um, I would say that the most important thing is every single one of my projects that's of any commercial importance is in CI. And so in my business, we use a lot of GitHub. There are other tools. There's GitLab, there's, you know, Bitbucket, all these tools have it. GitHub Actions will take the code and build it and go ahead and run that Bundler audit against it. I actually have that as a nightly security job. If Bundler audit ever— there's a gem. What Bundler audit does is it checks our gemfile, which is like our package.json in Ruby land, our gemfile. It checks not only our dependencies, but our dependencies' dependencies. For known vulnerable versions. So not just, hey, they released a new version, but there is a CVE against this gem. And so that is a flag. And so not only do we run this tool, but I have my CI configured to run it automatically every night. That's a big help. Breakman is a static analysis tool. There's a Breakman Pro, but There are a few restrictions on running that in, say, as a service, like they want you to buy Breakman Pro, but as an individual, you can run it locally or in your CI, and so that will actually scan the Ruby on Rails codebase for common coding errors that lead to security problems, so that's our static analysis tool. I feel like in a lot of other platforms, you have to reach for something like SonarQube or some commercial offering to get what we get basically for free and mature in the open-source community in the Ruby on Rails ecosystem. Those are the bread and butter of our dependency and scanning. Most Rails applications have a strong JavaScript component, especially if you're doing a lot of fancy client-end stuff. Yarn and/or npm are both very common now, and so running yarn audit or npm audit on a nightly basis along with Bundler audit, that helps give us coverage in the JavaScript side. Then we, on GitHub, we use a tool they provide. This is an open source. It's a Sass tool— not Sass. I mean, software as a service that runs on their servers called Dependabot. What Dependabot does is it looks at our source code repository, and it kind of does a little bit what Bundler audit does, but it actually creates pending pull requests that change to the newer version of the gem, and then that automatically kicks off the CI process. Every morning, one of our devs checks in and takes a look at Dependabot recommended pull requests, and if all the tests pass, we actually can merge and deploy that change within that day, which, compared to an industry average of 38 days to make any update of any consequence to production, is really remarkable speed-up. And then sometimes Dependabot, in our Slack channel, we, you know, it's the little engine that could, so it will try to upgrade to Rails 6, and the test will fail, and we like virtually pat Dependabot on the shoulder and say, that's so cute. Thanks for trying. You know, but, but those are some of the tools I use on a regular basis. But, but the biggest takeaway I want you to hear is that we don't just run one tool. We have a system in place to continuously integrate and run the tests, and those tests passing is what lets us know we can ship to production. The dream would be— not everyone's here yet, but the dream would be if one of your dependencies has a critical security vulnerability and there's a public vuln, that you would wake up to a notification on your phone that says— from the computer that says, this update was available. I ran all the tests. They all passed, and I deployed to production. Like before— so your first notice is that production's been updated. And as a preview of this, one of my aforementioned state government clients the other day in a meeting said, Frank, you know what my favorite part about working with y'all is? When you email me and say there was this security vulnerability and it's already been patched, and then I see the news items about it. That's cool. And so we're, we're not quite turned the keys over to the machine yet, but that's the direction we're going. And there is one more tip I have. If you're working in Rails, join the security mailing list. And very common, that early warning stuff. One of the main volunteers in the community is called— his name is Aaron Patterson. And Aaron is one of the core team members for Ruby on Rails, and he will— is often the one who's point for fixing security vulnerabilities and will release the patch. Multiple times, an email from Aaron to that list kicks us off to doing updates, and then Bundler Audit and Dependabot get the news the next day. when the actual CVE is published. And so that's a good way of staying ahead by at least a day before, uh, the, uh, the news fully breaks.

45:00Robert HurlbutYeah, that's a good tip to get a little bit of, uh, early intel in the process about, uh, potential vulnerabilities as they come out.

45:09Frank RiettaAbsolutely.

45:11Robert HurlbutSo key takeaway then, We've got— we've gone through a lot of different things in regards to Ruby on Rails, what it is, how to use some of the secure coding principles and things. And we've talked about the benefits of the framework, kind of wrapping this all up into, you know, kind of some simple takeaways. Like, what are the things that you would leave our audience with as far as the key takeaways, and then maybe even a call to action, something they could do? with this info?

45:42Frank RiettaWell, I would say if you're a Rails developer, no, you absolutely have a leg up over people who've come from other programming disciplines in that the tools are right at your fingertips to do this extraordinarily well. Embrace writing effective tests. I would say, um, a wonderful book to look at is called 99 Bottles by Sandy Metz and Katrina Owens. There's a Ruby version and there's now a JavaScript version. I highly recommend that because it helps learn the thought processes of effectively writing tests and effectively refactoring software. With the aid of tests, so I guess that's for everyone because lots of people write JavaScript too. For people who are coming from an enterprise and think that Ruby is just some toy that's going around away, it's going away. I respectfully disagree and think that you should. Consider why you think that and look at the enterprises that are built on top of it. They never had— this is an open-source platform written by people who got benefit out of it and contributed and built it. There's no marketing budget behind it. So if you think that the latest .NET is technologically superior just because you hear about it, I don't want to say anything bad about .NET. Don't hear me say that. But I'm saying, as a computer scientist, look at the merits of the individual systems, not just the companies that have a financial interest in promoting their product as an enterprise solution. So Rails is here to stay, and it can be used to make very secure web applications. And then finally, if you look at the OWASP Top 10 and some of the fundamental issues, Ruby on Rails have some of the best secure defaults that you can find, meaning it's way harder to create an insecure line-of-business CRUD application in Rails than it is in other platforms that don't provide that same level of convention over configuration.

48:25Robert HurlbutFrank, thank you so much for sharing your knowledge and experience with Ruby on Rails. And you gave us a lot of really cool things to consider from the overall perspective of kind of what, you know, all the different languages and how Ruby on Rails fits in. Also some tactical things we can do from tools and from perspectives of of thinking about Ruby on Rails. So, thanks for sharing that knowledge with us. And our audience, I think, is gonna get a lot out of this. So, thanks very much for your time.

48:54Frank RiettaThank you. And I enjoyed our time together.

48:57Chris RomeoThanks for listening to the Application Security Podcast. You'll find the show on Twitter @AppSecPodcast or on the web at www.securityjourney.com/application-security. security-podcast. You can also find Chris on Twitter @edgeroute and Robert @roberthurlbunt. Remember, security is a journey, not a destination.