Kalyani Pawar -- Shaping AppSec at Startups

0:00Chris RomeoTalyani Puar is an application security expert in Silicon Valley, known for building security programs from scratch for startups of all sizes. She advises early-stage security ventures on product-market fit, has spoken at conferences like DEF CON and Day of Security, and serves on reviewer boards for DEF CON, GHC, YSYS, and multiple BSides chapters. Talyani also mentors aspiring AppSec pros, giving back to the community she's helped shape. Kalyani joins us to unpack how AppSec can work within even the smallest of startups. There's much to learn from the experience shared for organizations of every size.

0:38Kalyani PawarThe Application Security Podcast is brought to you by Security Journey. Our training includes theory and immersive learning that teaches the skills and knowledge needed to create a security-first mindset across your organization. Learn more at securityjourney.com.

0:59Chris RomeoHey folks, welcome to another episode of the Application Security Podcast. This is Chris Romeo, CEO of Devichi, the threat modeling company, joined by my good friend, Robert Hurlbut. Hey, Robert.

1:13Robert HurlbutHey, Chris. Yeah, Robert Hurlbut, principal application security architect and threat modeling lead at Acquia. And as always, excited to be here to talk about AppSec.

1:23Chris RomeoYeah, we're going to talk about AppSec and how it intersects in the world of startups, which is something that a lot of folks don't usually think about, but I think we're going to learn why they should have thought about it before, but at least they can start now. But before we get to that, Kalyani, if you could share your security origin story, or how did you get into this world of cybersecurity?

1:49Kalyani PawarThanks for having me, first of all. Thank you. I am someone I would like to call an accidental security engineer. I did not— I was— this was not a direction I thought I would take. Even in high school, I thought I'd be a surgeon, and surgeon is far off from what a security engineer is. But what happened was my grandma, she owns a small business, and it got hacked, and we didn't know what to do, and she was really upset about it. And we happened to recover a lot of it. She got ransomwared at that time, and it was like, it's a small business, honestly, and it's like in a small town of India. Nobody talks about ransomware there. It's not a known term. Forget ransomware, nobody talks about security much in small towns.

2:33Chris RomeoHmm.

2:33Kalyani PawarAnd then we somehow paid the ransom at that time, which was certainly a small amount, but we got out of it. We were able to recover her accounts and everything that was on it. But that somehow made me think like, why would someone want to do something this evil, right? Like why? Like it's just, it's a small business. What are you gonna get out of it? And then I soon moved forward and realized that this is like actually a whole different world out there. There's like a lot of threats that are happening around. We were talking about a small number, but I see this happening for like millions of dollars out there. And then, It just, it sparked my interest. I was like, how does this happen? And then I started learning about something called the bug bounty programs as I was looking at it. And I, when I was, I was good at math. And so everybody was like, you should probably pursue engineering. And I took, I went to the engineering school and there I realized I'm not a great coder. So I was like, did I choose the wrong career or what? Like, is this a wrong choice or what? But I figured what I was good at was telling that something is wrong with the program. This is vulnerable and I can make this break. And that's when I was like, oh, bug bounty seems like an interesting career to also pursue. And after that, I started chasing small bugs here and there. And then I finally figured out how to find out different vulnerabilities. And now that's the past. And now I'm out here securing different applications and protecting them from all kinds of hackers. But yeah, that's, that's my origin story.

4:07Robert HurlbutVery cool. Very cool. So, today we're going to talk about startups and AppSec. So, I have a question. Is AppSec something that a startup should invest in and why?

4:21Kalyani PawarThey should totally invest in if they want to have a successful MVP and also not have a lawsuit on their hands. Well, This, again, it's an industry-specific investment. Again, if I was making a financial software, I would definitely want to involve AppSec early on because I don't wanna be leaking PIIs. But if I was making a focus app with 25 minutes of focus, so I'm just gonna, you know, give myself a 25-minute timer and a 5-minute break, I don't think AppSec is that big a concern there.

4:55Chris RomeoYeah.

4:58Kalyani Pawarunless your application is making some, is allotting some permissions that it should not be having. For example, like locking your phone altogether. I think that if you were giving that kind of privilege to your application, AppSec should enter early on. I have seen a bunch of very intelligent programmers making really amazing applications, but oftentimes, we are always talking about velocity and delivering features, and then we're also leveraging the, should we push this feature and should we make it more securely? And there's always a debate going on in that sector. I think the conversation more so boils down to what is the impact that's gonna happen if this feature is delivered maybe a little later or in an insecure manner, right?

5:44Chris RomeoAnd then that's a conversation to be had, but you should absolutely invest in the So when you think about the stage of startups, where's the right place for AppSec to enter? Because I'm, I guess I'm assuming that when we say AppSec, we mean a person attached to that function.

6:09Kalyani PawarYes.

6:10Chris RomeoBut I guess, could there be AppSec without an AppSec person, or do you really have to have an AppSec person?

6:17Kalyani PawarTotally. So here's my take on it. Once you deliver a successful MVP and are able to get some amount of funding, that's when you start hiring your heads of different things, right? You have a head of, I don't know, operations and then finance, and then you have a CTO. I think once you start scaling to maybe like 30 engineers, it's about time that you start considering AppSec. or security in general as a feature. Again, like I said, if it was fintech, do it since day one, or at least have engineers who are like hyper-aware about security since day one. But if it is not something that is that consequential, maybe sometime around like 30 people mark is a good time to start involving security. And it would start with like a generic security, you know, jack of all trades who would know basic things like how can I set up an IDS for my application? Or how should I put it behind a firewall? Like these basic things, like have MFA. I think that is like really basic. I was just talking to a startup 2 or 3 weeks ago, and they were literally authenticating all of their suppliers with username and password. And I was like, wow, that's such a nightmare. Forget password complexity, that's another conversation to have for some other day. But they were dealing with like entering credit card information and all of that. And I was like, wow, you only use username and password? So that you might think that, you know, at this time in this world, we are talking about MFA, reading about all of these, you know, breaches online, and so people might be aware about it, but certainly not. And so I think you need to have that awareness, and I think it would start with one single person coming on your security team, on your technology, on your— Who reports to the VP of tech, VP of engineering? that one person could build that culture. And I also feel like the earlier you bake in that culture, the easier it is to build on it. For example, if you think about security only when you wanna get a SOC 2 report, I think that's a little late because now you're gonna have to press the brakes on velocity and then try to bring all of these things that fit in the audit, right? But if you do that since the beginning, and then you are already meeting a bar higher from whatever's required for your SOC 2 report, and so you're already in a good enough place. I've also heard of nightmares where people have lost sales because they didn't have SOC 2 reports. And so, I guess, better late than never.

8:47Chris RomeoI'm going to— I'll hold my opinion of SOC 2 as far as— it's just, it's very easy to get if you have to. Yeah. In this day and age, it's not a major mark of security excellence. It's not ISO 27001. Like, if you go through ISO 27001, you felt some pain.

9:09Kalyani PawarYeah.

9:10Chris RomeoBecause that's, that's the, that's the results of that. But coming back around to the startup side, I'm curious about what you've seen. Because when I think about first security hire inside of a startup, it's often more of an information security person. It's kind of like they're part IT and they're part engineering. And they start with those simple things you were talking about, but they, I very seldom have seen them referred to as AppSec. Now, has that been your experience as well? Or, and do you recommend a different path? Should we just forget InfoSec in this 30-person startup? And, or should we get, or should we, and start with that AppSec person and just call them AppSec and have them do AppSec-y stuff?

9:56Kalyani PawarI think this part, IT part, engineering person, somewhere along the line, when you see your codebase growing in huge number and you see your infrastructure also bloating up in size, because I'm guessing you're gonna want to sign up more clients, you want to invest in R&D, right? Because you want to scale a startup. If you want to get funding, if you want to have, you want to be able to show revenue for all of those things, right? That person at some time, at a certain point in time, should realize that this is not scalable, and there is an aspect of governance to it, and the more this keeps bloating up in size, the lesser governance, lesser control you're gonna have it. I think the easiest indication of we need to get an AppSec person right now is when you see your AWS bill skyrocketing, you know, like, and infrastructure security, infrastructure, even DevOps people are security aware, But they also, when you are maybe deploying just certain workflows, because everybody's an admin initially, and then everybody just like spins up whatever instances they want to, you know, spin up. I think once you start seeing that change in the bill, I think that's also a good indicator that, okay, I need some governance, I need to have certain controls in place so that nobody can just spin up some EC2 instance and nobody can just download files out of the S3 files that I'm storing or something like that. And also GitHub, especially if you're storing your code on GitHub, or I think even GitLab does this, they do have these innate features like secret detection, and sometimes there's like code scanning features to it too, right? Those, I think any security-aware part engineering person should know about those, at least in the beginning. Those are like the basic hygiene for AppSec. I think once you've passed the basic hygiene for AppSec, like, oh, I have these secret policy in places, and what should I do once I leak a secret? I think that's at that point when you need to have more structured policies of like, is my CI/CD pipeline secure, or am I spilling secrets just in runtime or something like that? That's exactly when you don't have, when you need that specific AppSec expertise. And once you've hit that mark, that's when you should get a specific AppSec person to take care of this. Also, like, you cannot be an octopus. You're not an octopus with 8 hands. Even if I had all the time in the world, I don't think I, while I'm also maintaining my IT infrastructure, I'm also, it is just impossible. I'm not an octopus. Like, think about 8 different aspects at a single given time. It's just too much for one person, right? So, yeah, I think that's a good marker to involve AppSec.

12:50Robert HurlbutWell, one of the things about startups, and we kind of hinted at this, but they prioritize speed and innovation. How do you balance that need for rapid development with a robust application security program?

13:09Chris RomeoHmm.

13:09Kalyani PawarDon't be a difficult person to work with, first of all. I think I learned this very early on, and thanks to my mentors for teaching this, that you have to befriend your dev friends. I think I figured this out from all my startup experiences is that you have limited bandwidth and you have a lot to deliver. If I'm able to show metrics maybe to leadership, but if the metrics make no sense to them, for example, if I just go saying like 100 of our machines are obsolete, that makes no sense. What does that metric even mean, right? Like, what is the founder gonna do with this metric, right? Or what does an executive do with this metric? But instead, if I say like, oh, if these obsolete machines are being targeted by some CNC server, I think, and if I can explain what a CNC server does, what a botnet does, maybe they will understand the severity of it, right? Something similar, maybe not metrics, metrics might not be the best way to talk to devs, but explaining the impact of something. For example, if I'm running something in production, but I've left debug mode enabled, I think that's a big risk because I'm leaving it open to the outside world to just like, experiment with whatever they want with my application and then make whatever changes they want to make, right? I think that's a big impact. If in terms of, say, if I can say in a finance software, if I was able to say that if we were able to push this feature, we're gonna print out all the SSNs, I don't think anybody wants that, right? I think at that point, people are willing to work with you on this. Like, and I, in my experience, I've seen many friendly people.

14:53Chris RomeoYeah.

14:54Kalyani PawarI have hardly had a time where I made a suggestion on something that was severe and they were like, I'm not gonna do this. I feel like software is like a programmer's baby and nobody wants to hurt their baby in a sense, right?

15:07Robert HurlbutYeah.

15:08Kalyani PawarYeah, so I've hardly, I think it's about explaining and always finding that balance. And also I feel like the more you start seeing the growth in your startup, the more you understand what features are important or not. you sort of develop this compass over time, which makes you understand what needs, you know, alarming, what needs escalation. Yeah, that's the long and short of it.

15:34Chris RomeoSo, as the startup matures over time, we kind of, you took us to this point already where you're 30 people or whatever, and you're kind of You cross over based on the amount of infrastructure and things you have from having maybe one InfoSec person to having an InfoSec person and an AppSec person. How do you see AppSec grow as the startup matures? Let's think of a startup that's heading towards like a Series C, and maybe they're 2, I don't know, Series C always seems like they're 250 people or something.

16:13Kalyani PawarYeah.

16:13Chris RomeoLike, how does AppSec change? And then what about the next level when you go to 500 people?

16:20Robert Hurlbutat the startup.

16:20Chris RomeoIt's not really a startup anymore.

16:21Kalyani PawarIt's not really a startup anymore, exactly. I think a nice ratio to have, and I've read this somewhere or heard it from someone, I forget to quote them, but a nice ratio to have would be 10 engineers to 1 AppSec person. Now, obviously we're scaling and we keep talking about automation all the time. Earlier, when it's really nascent, it's just like sending a Slack message to someone and being like, I'm building out this feature and can you help me? you know, threat model it or something. But over time, say if you were pushing a whole feature out there, a whole workflow out in your AWS, and I'm pretty sure you should have an admin who is, or someone who is, you know, at least giving it a green light or a red light, right? I think like a checkpoint, like a gate, essentially. That gate should also have a security gate to it. Only if the security gate has been approved of, then you should let this whole feature go to place. Now, when I'm talking about a feature, I'm talking about something really vast. Like, I'm not talking about like a red button or blue button. I'm talking about something like adding extra, you know, levels of authentication to something or changing the whole authorization model. That has to be changed early on, right?

17:36Chris RomeoRight.

17:37Kalyani PawarI cannot just be like, oh, it's gonna go into production tomorrow, so let me sit and review this tonight. That's, that's I think absolutely someone should take a stand on something like this and be like, this is not okay long-term. And so adding these bottlenecks or gates at the right, in the right workflows through the right processes, that definitely helps with scaling. Like I think earlier you receive a lot of DMs and suddenly you start receiving a lot of Jira tickets that that could also be a cumbersome point. Then another, to address that, you know, this flood of Jira tickets, there's another part of like just baking in security from the beginning. Maybe implement, like we call, like we say, secure libraries. There's a set of secure packages and libraries that you can use. If that is predefined, you don't have to do the whole work from the beginning itself, right? If I know that, oh, they're gonna use this secure package, then I can just, be like, yeah, that's okay, you should go ahead with it, or not allowing use of certain tools that are not vetted before. So I think it's using the right gates at the right time in the whole process, which will help with scaling, say, 250 people to 500 people. And like I said, again, building that security-aware culture is a very hard needle to move, but if you bake it in earlier, it does not seem like a cumbersome task. And I would like to think that it is a top-down approach more than a bottom-up approach, essentially, because then it feels like a uniform guideline to follow rather than me and another engineer just quarreling on what needs to be delivered correctly. So, yeah.

19:21Robert HurlbutOkay. So, let's say now you've got an AppSec team at the startup, What should be their primary focus, or maybe several things that they're focusing on?

19:33Kalyani PawarWhat should be their primary focus? First of all, you cannot secure what you cannot see. So, try to see everything that you have. Try talking to as many, say, if it was my day one at a certain startup as an AppSec engineer, my first line of action would be to talk to all the developers and ask them about an architecture diagram for whatever has been built. Start with looking for what has already been implemented, and then you will be able to find out the gaps of, you know, what needs more work. We spoke about secret detection and things like that before, but then we also need something for code scanning, right? Not all code built is secure code. And so there are so many tools out there which are like simple rule-based mechanisms to like look for vulnerable, you know, code blocks that. Another thing I also usually suggest is if you've never had a pen test, you should at least have a pen test when you have the first hire. They should push for a pen test externally where you get a contract with some external vendor and ask them to, you know, like, just pen test your product and see what comes out of it. So pen testing too. Policy work, again, when I think there's a lot of latitude of freedom initially where you can start building policies, you do have that freedom. It just has to be implemented correctly where you're not breaking workflows, where you're not taking up too much space, if I can put it that way. If it is seamlessly integrated, that policy work can go much smoother than otherwise, than having no security at all. So, I think these would be my top 3 things to look at.

21:22Chris RomeoSo, we've talked about culture a few times. It's kind of come up as a thread that's running through a lot of the conversation here. But if we stop for a second and think specifically about security culture in the startup world, Like, how do you, how do you build that security culture? And I know you already, you talked about just having conversations with everybody as one way, which is great because you want to be, you want to be known as the helpful security person, not the person who stands in our way of everything cool we want to do, which that goes back to the beginning of time. Uh, our, our challenge as a security team, right, is that we, we, we block, tried to block everybody from doing cool stuff.

22:05Robert HurlbutYeah.

22:06Chris RomeoAnd at a startup, they'll just step on you if you try to block them from doing cool stuff. But what else can you do beyond just getting to know everybody to really build that culture of security inside the startup?

22:17Kalyani PawarSo, I've been toying with this term for some time called culture is code. When I say culture is code, it could be, like I was talking about kits earlier, it could be something like your PR. talking about, has this been security reviewed or not, or is this a security-relevant feature? If you have like a checklist of what feature are we exactly touching, like maybe is it anything to do with encryption or authentication or something like that? I think once you're able to define these checkpoints uniformly, I think it's easier to build on that culture. And I think that pretty much falls under the bracket of culture is code, because there is some amount of automation that you can apply on all of these things. Besides making friends amongst developers. Also, another thing is rewarding. I think if you, who does not like being celebrated? We all like being celebrated, right? If someone is able to deliver a change that has improved the security of your software, you should definitely celebrate it and talk about it.

23:28Robert HurlbutThat's true.

23:29Kalyani PawarAnd the positive change about this I've seen is that earlier security is not an item on the roadmap. And then eventually in the next 2 years, once you start seeing this trend of celebration, you start seeing, oh, we need to be able to make a more secure software as an item on the roadmap, as like a, you know, like an SLA of its own, which I think that is a change in culture, right? Because we were not talking about security and now we're celebrating all of these achievements. We're talking about it in our all-hands meetings or what have you, all meetings, and giving props. And then now people care about it and wanting to actually work on it. So that, and your security champions, that's a thankless job. Finding security champions, I think that happens more so from conversations, obviously.

24:17Chris RomeoYeah.

24:17Kalyani PawarBut also making, also, sometimes you have to rely on them to educate the fellow team members working on that feature, right? I have found security champions very helpful in times where I could not see things, but they could see it before I could. So, security champions, if you can scale that program, you should totally do it.

24:44Chris RomeoYeah. I want to double-click on this culture as code, because I've been sitting here pondering this idea, like, Because I've started talking about security culture probably 10+ years ago and watched it grow as a term and the security awareness vendors grab ahold of it and steal it and make it their own so that it's not even really thought about. When you say security culture, it's not even really thought about development anymore. It's thought about as how secure are your people when you send them phishing emails and stuff, which is too bad. But Culture, like, what are you codifying when you say culture is code? So, like, you mentioned, like, PR checks, security architecture, you know, security reviews or something like that. But what's the code? Like, what am I putting that somehow into code? Or, like, what's the— or is it just the fact that we have code, that it's surrounding the code, that you're thinking of it as culture is code?

25:43Kalyani PawarNo, well, you have something within the code and that is propagating our culture, if I can put it that way.

25:51Chris RomeoGot it.

25:52Kalyani PawarYeah. If it's maybe like a simple linter even where you're like, check on the buffer size or something like that, I think that is the aspect of code. And if every time I make a PR and that's checking, if I've overflowed my buffer or something. I think if you make an error, obviously this PR is not gonna make it, it's not gonna be merged, right? And so I think that is somewhat like a subconscious thought, but then it becomes a conscious thought because you see your error being caught somewhere. And so eventually you start building culture as code, like you're implementing mechanisms throughout the code base to propagate.

26:33Chris RomeoSo it's an automation, it's really ultimately an automation play.

26:37Kalyani PawarYeah.

26:38Chris RomeoBecause if I pull on the thread of SQL injection, right? How does culture as code, the concept of culture as code, say I just have a bunch of SQL injection problems inside of my startup and people are creating them left and right. By basing your definition, culture as code would be where we would just integrate some type of a solution to check for them and—

27:03Robert HurlbutA guardrail.

27:03Chris Romeoguardrail and block the PR.

27:05Kalyani PawarYeah.

27:06Chris RomeoSo, it's kind of like— so, I like that you said guardrail, because that kind of brings a bunch of things together. So, culture as code is a companion to guardrails, or guardrails are a companion to culture as code.

27:18Kalyani PawarExactly. It's more of a top-level umbrella, like a root, and then we are following all of these things under it.

27:24Chris RomeoOkay. Makes sense. I just like—

27:28Kalyani Pawarit's a fancy term, I guess. And it's kind of relevant in the startup world because we're always talking about code and we're always talking about features. And, um, problem is I want to talk about culture, and so I'm going to find ways to bake that culture.

27:43Chris RomeoAnd the advantage in the startup world is you're starting from scratch for the most part. So, you don't have the decades and decades of bad decisions that enterprises are keeping hidden in closets around the world called data centers, right?

27:57Kalyani PawarYeah.

27:57Chris Romeoyou don't have that. You can, you really can embrace a new culture. And so I like that idea though. I could see a t-shirt, culture is code.

28:05Kalyani PawarCulture is code, yeah. Maybe we should talk more about it at conferences.

28:08Chris RomeoYou should, yeah, you should definitely put together a talk on culture is code, because I think that's, I think it's just a, like I said, I've been thinking about culture for 10 years. It's just a new way of spinning it, a new way of thinking about it.

28:20Robert HurlbutYeah.

28:21Chris RomeoThat brings it back into the development. Nobody's going to say culture is code, that's security awareness. That's not going to happen.

28:26Kalyani PawarThat's not going to happen. Right.

28:29Robert HurlbutAll right. Well, here's another aspect to have when you have a startup, but what role do third-party tools, open-source solutions, or external partners play in establishing and maintaining a strong AppSec program for startups, for that startup?

28:46Kalyani PawarSo, when we're talking about third-party tools, we often overlook the aspect of vendor security when I feel like every time we onboard new tools, there has to be some kind of security vetting process in place. Maybe it could be like, I don't know, reviewing their posture, going over their pentest report, anything, any, what have you, whatever works for you. So third-party tools should be vetted before a contract is signed with them, because once a contract is signed with them, it's more so like, okay, now Now it's a bigger problem because now I have to think what I have to gate before I share with this tool or not. Open source, I, well, as much as I would love to adopt open source because there's no way I can gate that at all, it would just severely block the ability to create something creative. More so if you were able to use like a very popular package, the chances of that having vulnerabilities are lesser than something that's just, I don't know, obscure. Maybe like one person uses it. It's just something you found off the internet. And then also when you have your dependency scanning in place, I think that's also a step further to secure your code. So that's with the open source part. But yeah, I think what I want to pretty much say is that in startups, you should review a lot of things. There's obviously a lot of latitude, a lot of freedom to build security, right? But also there's a lot of latitude to build product, right? And I think if you find a way to make that go hand in hand, it just gets— it just makes the future simpler. For example, now, if I'm ever going to be talking to like a 10-year-old company who has legacy code and then just legacy infrastructure. And my first, the first thing I think of is like, can we just isolate this? And like, can we please document this? Because I'm not, I don't understand what's going on here. But with the startups, it's a different case. There's just a lot of freedom. And I think awareness about reviewing things, review, having someone at least look at it before it goes live, I think. That, that really helps, uh, in the long run.

31:13Robert HurlbutVery cool.

31:16Chris RomeoWell, this has been a fun, uh, exploration of the startup ecosystem in regards to application security. And I love that culture is code. That's going to stick with me for a while. I'm going to have to think about that some more, but we can't let you go till after you've answered the now infamous, not famous, infamous Robert's Lightning Round.

31:37Robert HurlbutHere we go. All right, so we've got 3 questions, Kalyani. The first one is, what's your most controversial opinion on application security and why do you hold this view?

31:48Kalyani PawarI'm averse of having too many security tools at the same time. And also, I'm just bored of seeing a lot of tools together because all I see is graphs and pie charts and Well, like, I was just working with a really popular vendor the other day, and you see like this startup where I was consulting them on like what needs to be their top priority, and they were talking about like this really amazing vendor and all of these amazing features in the tool and everything, and I was like, you are a startup, you are small in size, you only have 3 people working on this right now, what are you going to do with these 10,000 things that this pie chart and graphs are showing you. It's, you don't even have bandwidth for this right now. And so while it's great that there's so much visibility, I think most of them are not great with contextualization. And so I hate that, that, you know, sometimes we have, when I, and I see this too, when startups get a lot of funding, they just get fat with a lot of money and then they keep getting a lot of tools. And I'm just like, you don't need these many tools though. Just save it for a bad day.

32:57Robert HurlbutI don't know.

32:57Kalyani PawarAnd yeah, it's, you don't need this. Why do you need this? First, you gotta solve like the bigger problems, the bigger issues, and these tools are not giving you that. Like, there's hardly anything you can make off this finding if it's not even contextually appropriate to you. So you need one whole, you know, human being, engineering hours allotted to make sense of what these findings are, right? And I think—

33:23Chris RomeoRight.

33:23Kalyani PawarI don't know, people, the cybersecurity tooling companies are gonna hate me for this, but that's, yeah, it's always my, you know, honest opinion is like, before you get any tool, please, please, you have a lot of low-hanging fruit to chase. Let's just focus on that before, you know, following the graphs and price charts. And it's really, it's just gross to see them now. I just get so bored of seeing them. It's the same light theme, dark theme. It's, oh my God.

33:51Robert HurlbutYeah. All right. So, the second question, we may already have said it, but I'm gonna just give it a shot here and see. If you could display a single message on a billboard at the RSA or Black Hat conference, what would it say?

34:07Kalyani PawarWe're not aware enough. I was talking about this the other day on another podcast, and that's just like, we're not aware enough. We have a lot to do. It's just that, Also, don't get burned out when I say this, but we're not aware enough in the sense that we keep falling victims to phishing scams day in, day out, and that means there is a lack of awareness even amongst general public, and I think the day some grandma doesn't get hacked, I think I will believe that we are safe and we are aware enough, but until then, we're just not aware enough. Yeah.

34:44Robert HurlbutThat's a good gauge, yeah. And the third question is, what's your top book recommendation? Doesn't have to be security, but, and why do you find it valuable?

34:54Kalyani PawarI've been reading this book called The Alignment Problem. It's a very interesting book that ties machine learning with human values and how, you know, we as humans need to start also wondering about the problems of using AI responsibly. Yeah, there's a very beautiful example that they give in the book where like older generation AIs, if you give them something like doctor minus man plus woman, it will give something called nurse. Why not a doctor? That's just a gender bias in the whole dataset, right? I think it's about time we also start thinking about responsible AI. Obviously it's gotten better. It's not saying nurse anymore, but I was myself experimenting with ChatGPT the other day and I was like, with all the information you know about me, create a picture of me. And then it created a picture of a man who's reading something on his laptop and doing something with code. And I was like, but you knew I was a woman, so why did you make this picture? And so I think, yeah, about time we have these conversations.

36:00Chris RomeoHmm. Very cool. Good book to challenge the thought process because that's definitely a challenge. There's a challenge in, we've heard lots of stories about it, you know, from a lot of different angles as far as how AI is being improperly trained because that's the thing I love to always remind our audience and everybody else on Earth. This thing's a parrot.

36:28Kalyani PawarIt is.

36:29Chris RomeoThat's all it is, is a fancy parrot. It just, it's, it's only spitting back the things that it's already heard. It's like a 3-year-old who, when you say a naughty word and you're like, oh boy, I'm hearing this again.

36:39Kalyani PawarEvery time I, um, you know, see some text that is generated by AI, I just feel like it's like a room where the lights are on, but nobody's home. You know, it just, it sounds so dead. lights are off, nobody's home. That's just how dead the text sounds to me. It's just lifeless. And every time, like, I also review submissions to conferences and every time I see something like that, I just, I, it just puts me off. I don't want to read the rest of it no matter how beautiful it is. It's just, you're not presenting it well. You're just gutting it.

37:11Chris RomeoYeah. There is some good AI stuff. I review submissions for conferences as well. And occasionally I've found a diamond in the rough. So I keep reading them. But I'm in the same boat you are. I'm like, oh, another AI talk on the OWASP top 10 for AI.

37:27Kalyani PawarYeah.

37:27Chris RomeoWhere they've taken OWASP's work and just tried to reflect it. I saw a couple submissions for that and I was like, oh, come on, at various conferences.

37:34Kalyani PawarAnd I feel like it's, it should come from actual people who made the submissions rather than people parroting off of it. I think if you wanted to talk about it, maybe talk about like an interesting way in which you jailbreak something. I guess that I would definitely love to see that or, or show it to the audiences. Like, how do you jailbreak something? Who doesn't want to see that, right?

37:54Chris RomeoYeah, people always, always got to focus on the original, right? That's part of the success there. So, Kalyani, how about a key takeaway or a call to action for our audience? What would you like to leave the audience with?

38:06Kalyani PawarWhile scaling startups seems like a huge dragon to control, it's not intimidating at all as long as you get your hands dirty. and get into the weeds with like the people building the product. You will definitely see gaps where simple solutions can be put in place. It's not as daunting as it seems. And I always say this to everyone trying to enter into security is that don't chase money. There's always 3 things to every job, work, people, money. Find great work. The day you find 3 on 3, you've hit jackpot. If you find 2 and 3, you're in a good place. 1 and 3, go, run. But find work that makes you happy and makes you feel like you are making an impact. And I think startups give that to you, a lot of freedom.

38:57Chris RomeoYeah, I'm gonna remember that too. Work, people, money. 2 out of 3, you're in a good place. 3 out of 3, you've hit the jackpot. So that's a good way to think about finding the right opportunity.

39:09Kalyani PawarYeah, about your career.

39:10Robert HurlbutYeah.

39:11Kalyani PawarEspecially, yeah. In this day and age, sure.

39:15Robert HurlbutYeah.

39:15Chris RomeoDefinitely. Well, Kalyani, thank you for sharing this wisdom that you've accumulated in regards to startups. It was very helpful. And like I said, culture is code. That's sticking with me. I'm gonna noodle on that some more. Are you gonna— I'm gonna be thinking about that, trying to continue to codify it. But I think you should definitely do something with that in public. Do it, take it to a conference and—

39:35Kalyani PawarThank you.

39:35Chris RomeoLet's see. I'd love to see you flesh that out as into a whole talk as far as the moving pieces of culture is code and how you change culture using code and everything. I think there's some good stuff. So, thanks for being a guest here on the Application Security Podcast.

39:49Kalyani PawarThank you so much. Thanks for having me.