SIEM Modernization? Is That a Thing?

Chuvakin: Hi there, welcome to the Cloud Security Podcast. Thanks for joining us today. Your host here are, as usual, Tim Peacock, the Product Manager for Threat Detection here at Google. And me, Anton Chuvakin, reformed analyst and member of the Cloud Security team here at Google. You can find this podcast wherever podcasts are distributed and at our website. Yes, we will have a website. Yes, it's coming. You can also follow us on Twitter @twitter.com/cloudsecpodcast. Our guest today is Eric Foster, the president of CYDERES, a Fishtech company. And the topic of the conversation is of course (SIEM), Security Information and Event management. And of course, some of the listeners were asking us, "When are you gonna talk about (SIEM)?" And you know, today is the day. So welcome, Eric. I wanna start from the first question we have, and it's kinda definition question. In many cases, we talk to clients and between ourselves and they kinda say, “Do you have a modern (SIEM)? Like do you have a (SIEM) that fits these requirements?" So, Eric, what's your take on that? How do you define modern (SIEM)? What are the modern (SIEM) components and requirements perhaps?

Eric: Anton and Tim thank you so much for having me on the show, big fans, big fans of the show, and I gotta say it's a distinct honor to be discussing (SIEM) with the person who's been called the godfather of (SIEM). So how do I define modern (SIEM)? I mean, to me, when I think modern (SIEM), I really think a (SIEM) that is capable of dealing with the big data problem that information security has proven to be. In fact, I would say, information security was proven to be a big data a problem a long time ago. It's just that a lot of haven't realized or haven't considered that point or caught that to that point if you will. I think modern (SIEM) really means two different potential architectural approaches. I think it's either running a "cyber data leak," I know how much you love that terminology, Anton, but you know. Swimming and drowning in the cyber data leak and then running an analytic solution on top that is "(SIEM)." Or having a modern solution that can do both the data leak and the (SIEM) functionality in one stack. So if I said a modern (SIEM) solution--I like to go use cases, right. I think there are three sort of property use cases for modern (SIEM). First, it's something that can support the minimum data retention that's necessary for modern investigations. And do I mean by that? I think it's about a year of data, not searchable at a minimum. You know, I think solar winds really proves how important--the solar winds really prove how important it is to have 9, 10, 12, months of data not searchable on stand-by. Use case number two for me is something that provides what I would describe as business reasonable performance, meaning that your analytical team aren’t dealing with the historical coffee break query, you know, to do the day-to-day work. They don't have to run a query, go get a cup of coffee and wait for those versus results to come back. And used case number three is something that provides business reasonable costs. So is a Ludacris cost to accomplish used cases [inaudible]

Tim: I like that. The concept of the coffee break query. I was at a startup for a while where we did the back-of-the-envelope math, and we realized we had a very performant data leak. But every query cost a cup of coffee. So you can either have your coffee break query, or your cup of coffee query. You call Anton, the godfather of (SIEM). I just wanna know whose bed* he left the horse's head at.
Eric: Tim, you really don't wanna answer that question, my friend.
Tim: No, I suspect I don't. I wanna pick up this modern (SIEM) concept a little more. Does that--does that require it to be a SaaS (SIEM)? Is there a future for the on-prem (SIEM)?
Eric: You know, I personally believe that On-Premises everything is dead. I just think a lot of people haven't caught up with that either.
Tim: I believe people's fleets of laptops and workstations are gonna disagree with you on that one.
Eric: Well, sure, when I say on-prem, I don't mean that there's gonna be a laptop or desktop someplace. But you know, from a zero-trust architecture perspective, which I think is very much the future of networking in architecture, that those laptops and desktops and endpoints should be synonymous wherever they exist, whether that's sitting in a Starbucks or, you know, sitting in somebody's office building. But, you know, when I say on-premises data, I mean, on an On-Premises data center. You know, I think the cloud service providers have very clearly proven that it is, not only much more costly but much more secure and better in literally every way to run things in the cloud. Although I will say everything in it and security seems to be so cyclical that I'm very much expecting, like, mainframes to come back any day now. So who knows?
Chuvakin: Yeah. And of course, there's those Europeans who may have a bit of a beef with that argument, right? But everything's gonna be in the cloud. Because last time I checked, the only cloud providers are either in the U.S. or in China, right? So if you're not U.S., or China, you're gonna use either a foreign cloud provider or you're gonna use--well, not the cloud provider. So here's the question that I get asked a lot, as, should I really say it with a straight face potential godfather of (SIEM)? “What are your top three root causes for (SIEM) deployment failures?” Like, I get the failure question a lot. I had a presentation many years ago about diverse practices of (SIEM) How to engineer failure in your (SIEM) deployment? I really had collected a lot of failure tips for (SIEM). So what are your favorite top three, how to really ruin your safe deployment, what caused the biggest failures?
Eric: Top three root causes for (SIEM) deployment failures? I would have to say, first and foremost, it's lack of resources. And then resources can be resource time, it can be a resource of expertise, it can be focus, it can be dollars, can be dollars to architect the system, you know. Ultimately, this all comes together in, you know, if we talk about those three use cases that are provided above the concept of being able to retain the data that's necessary for modern investigations, to provide business reasonable performance and provide business reasonable cost. I mean, to me, resources and resource availability comes in as the number one root cause. Number two, I would say, you know, and this is much more once the (SIEM) is actually deployed, often by, you know, under-resourced deployments, or whatever else, but then its lack of content, or lack of tuning, which I guess you could argue anything is a function of resources. But this is such a big failure point for me, you know, we'll see people put in a (SIEM), put in a bunch of effort, and then either go turn on 1,000 rules and have them generate just a ton of alerts. And then that is where it ultimately blows up. Or, you know, they'll turn something on and turn it on with 30 out-of-the-box rules, and not spend the time to develop the content to actually detect it, you know. It's never that place in the middle of a sweet spot of content and tuning. And then I guess, third, I would say, maybe bad architecture from the beginning, and especially trying to adapt the legacy (SIEM) technology to a modern cloud-based high-volume world. You know, we see people stand up a (SIEM) project and then go, “Okay, you know, we've moved X amount of our workloads to these cloud service providers, and now all we have to do is eagers, all this cloud log data back onto our premises back feed it into the (SIEM).” What do you mean, the (SIEM) is tipping over day three? Or, you know, we just stood up this really great EDR product, and we're gonna throw those logs at our (SIEM), right? What do you mean, we're out of luck capacity already? And, you know, day five of our (SIEM) deployment?
Tim: That's a great question right there is, you know, we've got modern (SIEM)s, you've got old school (SIEM)s, we've got AV and thanks to Anton of all people, we've got this new word EDR. We talked about this on a previous episode, the loyal listeners, what's the future of (SIEM)s and EDR as working together? Are they gonna kill each other? Are they gonna find a way to live together harmoniously? What's the--what's the path there?
Eric: I think they're absolutely gonna find a way to live together harmoniously. I mean, I think EDR is probably the single most important technology in your defensive detection prevention stack. Oh, yeah. I mean, hands down, you know, would be the first thing I implemented at any company if I'm buying something to actually protect myself.
Tim: Before you buy a (SIEM)?
Anton: Not uncommon nowadays, yeah, he’s right. My mind blew up when I first heard this two, three, four years ago, and I wanted to argue with this really a lot. But ultimately, I also agreed that the EDR centric SOX do exist where (SIEM) or log analysis and auxiliary. For me, that was a big moment in my life, because apparently, something very important to me turned out to be not true anymore. Yeah, that you don't log a log analysis first. Now that's legit.
Eric: Don't get me wrong. I mean, I'm gonna want those EDR logs into my (SIEM), and I'm gonna want to correlate them with other telemetry sources and everything else. But I truly believe, you know, you can stand up that EDR solution. You can start logging that data. You can start protecting your environment right away. I mean, there's no question, like, one of the first things we do, you know, I think you guys both know, but we do a lot of incident response, right for a lot of customers. And you know, not just those of our managed services, but for a lot of other customers. And the first thing that we do when we roll into an incident response for a non-CYDERES customer is deploying EDR you know, and in fact, even sometimes, if they've got an existing EDR, we're gonna deploy our top three chosen EDR into their environment because it's so valuable and so important and lets us gather so much important telemetry. So I definitely believe that EDR and (SIEM) is gonna coexist. And in fact, I think it's one of the core technologies that is driving that concept of security being a big data problem. You know, I will tell you, for example, you know, I don't wanna beat the solar winds sunburst drum too much, but I think that's just a perfect story of how modern information security with the right preventative and detective technologies, we're in position to respond to this kind of very serious, very pervasive threat. And you know, those CYDERES customers that had modern EDR and had a modern (SIEM) in place, were able to know by literally 8 am on Monday, the day that that broke whether or not they were impacted by sunburst, whether or not they had the stage one payload, whether or not they had the stage two payload, because we had all that telemetry. The moment those IOCs became available on the internet, you could go back in time and go pay to not have this. And the people who didn't, even people who had EDR, but didn't have the EDR logs, or didn't have a way to access those logs in any sort of easily effective way, we're spending weeks, if not months, trying to figure out if their organization was compromised by this very insidious threat. And that's just a huge gulf to me between the security mhaves and security have nots in terms of how much time I'm gonna spend an incident response in trying to help clean up my organization.
Anton: Yeah, funnily enough, this was one of the first lines I think that I've written into the first EDR paper back in 2013, was exactly that tasks that would take months otherwise take minutes with EDR was mostly around endpoint investigations was literally months became minutes for some people. So admittedly, that's quite cool. We had another question about what inputs would be most useful for modern threats. And I think it's very clear to me that you'd highlight EDR as one of the key data sources into your (SIEM) or into your analysis platform. So any other favorite data sources, or maybe favorite features, to deal with this type of modern possibly top-tier threats? Like what would you really like to have if you are facing an elite attacker and a modern environment to defend?
Eric: Oh, absolutely. And in fact, I'll go as far to say, I believe you can get 80 to 90 percent of the value in a list of five technologies of all security, detection, and response,
Anton: Cool.
Eric: EDR, DNS, proxy, DHCP, to know who something was at what point in time, and then I will go past that to go deception tech.
Tim: Deception tech?
Eric: I believe, deception tech.
Tim: No way.
Eric: Big shout out to Thinkst Canary, ridiculously affordable, ridiculously valuable, probably the single highest ROI product and security today.
Anton: Well, given that the cost of water, a couple of 100 bucks per thing, it's not a credit card price. It's like your kid's credit card price as far as I recall.
Tim: It's like your coffee budget price.
Eric: It literally is your coffee budget price, and it's ridiculously effective. But there are a lot of great deception technologies out there. There are a lot of enterprise budget deception technologies, where you can spend a lot of money and have a really robust deception platform and everything else. But you know, things I would argue is very robust and very valuable and is that exact coffee budget. But quite honestly, those five technology sources, and guess what? DNS, DHCP, everybody's got those already. It's just grabbing that data. Proxy, almost everybody's gonna have some kind of proxy, whether that's your firewall or something else that's proxying your connections to go get that. So you're really looking at EDR, which a lot of people still don't have, but you know, some modern EDR tech and deception tech. And I believe that combined with an ability to store that data and analyze that data and that's the vast majority of what people need to be able to respond to this. Now, I will say the one thing that I'm not touching in there is phishing-related email-specific data. But I will say for a lot of people, a lot of email-based stuff has gone past, like, the ability to analyze. And so it's really more emails become like a really advanced technology to just block stuff as opposed to be like a log source. And that's why I would not label it that. I will absolutely say, you know, email security products, absolutely critical in that. But I actually don't find them to be a super telemetry source for detection and response. It's much more of a put-in email tech to knock down the bad stuff before it ever gets there. And then I'm gonna use more proxy and DHCP, DNS lookups to find who actually did something with that "fish.”
Anton: By the way as a final interruption before Eric goes into the answer. When I started looking at this back in the garden, the days, I think it's maybe around 2012, we realized that for some cloud providers, you have to call them and say, “Can I have logs, please?” And they would, like, ship them. So this is like in the early era of the IR in the cloud, then you have to call a human and say, “Can I get my logs?” And they're like, “Yeah, sure. We're gonna drop them in the envelope and FedEx them or something.” I don't know how they send them. It--we went a long way from that time. That's what I'm saying. So there's a lot more telemetry now.
Tim: I love the idea of FedExing logs.
Eric: Oh, absolutely. Never underestimate the bandwidth of a station wagon loaded with backup tapes, right.
Tim: So let's shift gears a little. This is the thing I love talking about on the show. And I talked about it with users all the time. And you'll think about what I do for a living and find this unsurprising. But let me ask what's different about threat detection in a cloud environment? What's different about investigation there? What's different about response there? What changes with the move to cloud? We've talked a lot about on-prem. What about cloud?
Eric: Big question here, right, you know, cloud and cloud sources. What is detection and response in the cloud? In general, I have kind of a really specific take on this, which, you know, while ultimately a lot of people will say, you know, the cloud is just somebody else's computer. Sure, there's a way to look at that. I will say, cloud is not just somebody else's computer, maybe the concept of the cloud is. But the biggest thing is, the cloud service providers have vastly different services and service architecture than just your traditional data center. And in fact, it's the people who look up the cloud as, “Oh, this is just somebody else's computer. And I'm just running Windows VM and a cloud service provider” that are doing the cloud wrong. So while I would say threats in the cloud are not typically that different from on-premise threats, I mean, it's still user-based attacks. It's still I am attacks* It's still resource attacks in the cloud. It's a malware. It's still compromised of administrator credentials, and other things like that. The architecture, the services, even quite honestly, the acronyms are so vastly different. And in fact, it's one of the reasons, you know, we like to say, sadly, that most of the other service providers can barely spell cloud, let alone know how to actually secure it. We take these sort of traditional security guys, and you put them in front of a lot of cloud logs coming out of the various cloud service providers. It's all things that you're not traditionally familiar with. And it does definitely require a different level of expertise. I will say, you know, for sure the cloud has its own attack surface. We are very big fans of looking at a cloud service holistically, you know, whether that's with a cloud management platform. You know, something like Palo Alto Prisma Cloud, or, you know, each of the cloud service providers have their own cloud-centric tools. And I'm really hoping, quite honestly, that Google does a lot more with their cloud-centric security tools to make them more and more multi-cloud.
Tim: Oh, yeah.
Eric: Oh, absolutely. Palo Alto Prisma Cloud is basically the 800-pound gorilla in the space, mostly because they end up buying everybody who remotely competes with them.
Tim: They have had an open pocketbook.
Eric: Yeah. So it'd be really interesting to see a more agnostic solution approach to cloud management platforms. You know, although a lot of people would argue the CMP tools are more, like, vulnerability management and compliance, best practice and other things like that, then specific detection and response. But I kinda look at that as a very complimentary thing. You know, if you have a misconfiguration in your cloud environment, or your attack surfaces widen because you've misconfigured something. You know, that is gonna fall into detection and response and is something that can very much inform that.
Anton: You know, which reminds me of the Orc episode, right? That same exact theme came up on our previous episode with the team from Orca, right? The cloud security provider, right, exactly the same with--
Tim: You got the same question of left hand and right hand, and how do we, as a cloud provider, help them know what the other is doing?
Eric: That's probably my single most concise answer, though, is the, you know, in terms of what's different about investigation and response in the cloud, is it's understanding those kinda services and acronyms and cloud-specific architectures that maybe your traditional security or IR teams might not understand. And I--I guess, to even give you a sad story around that, you know, like, the obvious easy example is, you know, somebody's running a microservices first environment is obviously very, very different than a traditional on-prem, network environment, you know, in point-based environment. But still, a lot of people don't even understand that. I mean, we fought and fought and fought with somebody who was absolutely convinced that they needed a way to put modern EDR on their microservices. We’re standing up these Kubernetes things and we need CrowdStrike installed on them, obviously, because, you know, policy is we installed CrowdStrike whenever computer and so like, how do we install CrowdStrike as part of our CI CD pipeline was like. What? No.
Anton: That's where I fill everything seals basically. That debate is where you really--I’m not gonna say anymore.
Tim: I would end up with a lot of holes in the walls of my apartment. So here's the fun one. That's another question that I get asked occasionally, and I usually cringe before I answer. So let's see if Eric would cringe or not. What is your view of the current and ongoing frenzy about AI and I'm gonna air quote, “AI/ML” for security, give us your short but hopefully, final take on the role for AI and ML in security?
Eric: I love this, Anton. And it's--it's definitely a cringe question. But I--I will say it's much more of the cringe of every time I sit through yet another vendor presentation that has all this fancy AI/ML unicorn, what I would say has historically been very big snake oil in so much of this that it's become a bad word. AI and ML is not snake oil. But I would say most vendors use of the terms AI and ML are. ML is really good for certain use cases, where it's really mostly statistical analysis. But people wanna change that statistical analysis buzzword to be machine learning. Fine, right, you know, I'm doing volumetric analysis of the thing. Sure, that's machine learning, but it's really statistics. You know, trained models can be very valuable if you put in the time to train them. And the model is straightforward enough, you know, we've got an insider threat-based solution that adds on to our service offering and adds on to our (SIEM) that uses Bayesian inference and build those kinds of models, train those kinds of models. You can do a lot with them. But historically, unfortunately, most of this in security today is snake oil. But I guess I also have that rather, curmudgeon approach that I'm maybe I'm famous for that I would say sad, too high a percentage of security products fall into that same bucket as well.
Anton: But we also probably agree that the long-term future on this is positive. Like, we--we’re all hopeful, then by 2030, we will have effective AI/ML for many security problems. It’s just that today's stuff tends to be in this category. Right? Like, we're not arguing that the long-term trajectory is pointing to that. We are getting it today state is not magical.
Tim: So we really hate leaving listeners empty-handed. What would you point people to to learn more about, you know, trends in (SIEM), manage DNR cloud security? And the important caveat is that Anton's blog gets mentioned, and obviously, you’d get zero points if you mentioned Anton's blog.
Eric: That's funny. Honestly, my biggest single source these days is the collective hive mind of infosec Twitter. It's how I tend to use Twitter the most as, you know, as a kind of news feed aggregator opinion aggregator, everything else. There are a couple really good infosec lists on Twitter of, you know, infosec hive mind if you will. I don't wanna shout out any one specific list quite honestly, because I believe the way to do Twitter, right is to kind of curate your own list based on some stuff you see. I will always encourage people to follow some people you might not like, or you might disagree with. You know you create too much of an echo chamber otherwise. But quite honestly, rather than point anybody to, you know, the obvious Anton's blog or some specific blog that I think everybody probably already listens to or follows, I would really encourage you to take an hour, go to Twitter, punch in a couple of different hashtags, follow a couple of different people and look at who those people follow. I would not throw myself up as the bastion* of like, thought leadership on Twitter by any means. But I like my follower list. So feel free to come--like, check my list as a place to start or check somebody else that you respect. And, yeah.
Tim: And Eric, what is your handle?
Eric: I'm performa Ify. So perform and then ify. So like modify, but with perform as the base of the word.
Tim: With that, I think we're just at time. So listeners thank you so much for joining us today. It's a pleasure to have you on the show Eric. Performa Ify is that Twitter handle. You can follow this podcast @twitter.com/cloudsecpodcast, my co-host at @anton_chuvakin, and myself @_TimPeacock. Tweet at us, email us argue with us. If we like or hate what we hear, definitely invite you on the next episode. I promise joining the episode is more fun than fighting on Twitter. And so with that, thank you so much for joining us today. And we'll see you next time on the Cloud Security Podcast.