Get Cyber Resilient Ep 94 | SIEM detection and the best use of threat intel - with Neil Clausen, Regional CISO, Mimecast

Neil is seasoned security practitioner, who along with his leadership role at Mimecast lectures at Northeastern University College onDatabase Management, Security, and other IT-related courses. He’s also been on advisory boards for McAfee and Cisco and has built and managed SOC functions.

The Get Cyber Resilient Show Episode #94 Transcript

Garrett O’Hara: Welcome to the Get Cyber Resilient podcast. I'm Garrett O’Hara. Today, Neil Clauson, Regional CISO from Mimecast joins us. Neil is very much a security practitioner. He lectures at Northeastern University College on Database Management, Security, and other IT-related courses. He's been on advisory boards for McAfee and Cisco, and has built and managed SOC functions. In this conversation, Neil takes us through SIEM detection strategies, the best use of threat intel, running tabletop exercises, and purple teaming over the conversation. Welcome to the Get Cyber Resilient podcast. I'm Garrett O’Hara. Today, I'm joined by Neil Clauson, our regional CISO and director of security operations in the past. How you doing today, Neil?

Neil Clauson: Doing well. Thanks.

Garrett O’Hara: Great to have you on board. You're over in the East Coast of the US, right?

Neil Clauson: I am out of Boston, so I may talk fast. I apologise in advance.

Garrett O’Hara: Well, also I'm Irish, so I'm, I'm gonna match your pace and then sieve. [laughs].

Neil Clauson: Excellent.

Garrett O’Hara: Good to, uh, good to have you on. Neil, the first question we pretty much always ask is, um, how did you get to where you are today? Obviously you're the regional CISO for, uh, Mimecast, and, um, you've got a, quite a rich background. I was looking at LinkedIn and, um, great to hear, yeah, how you, how you wind your way to regional CISO for Mimecast.

Neil Clauson: Yeah. I mean, I guess I've been a lifelong technologist ever since I was a kid, always enjoying, um, actually being a hacker, before that was a bad term, I guess, to just figuring things out how the old, uh, you know, pre-internet worked and, uh, then just working for small companies and enterprises all the way through till now, helping them, you know, meet their IT infrastructure, business transformation type needs and innovation, just having a lot of fun along the process.

Garrett O’Hara: Happy days, happy days. Um, we're gonna get straight into it. Um, the first thing we wanted to kinda talk about today, just given your background and your, your sort of hands on and practitioner experience is SIEM, and, uh, sort of maybe specifically the de- detection strategies in there. And I know, like, I dunno if it's the same in the US, a lot of organisations in Australia and APAC generally are kinda looking at, or have already invested in a, a SIEM platform of some sorts. And one of the consistent complaints or problems I hear about is, well, first of all, how expensive it is to store data or how expensive it can be to store data, um, it'd be great to hear how you've kind of approached that and the, the SOCs that you've built and managed.

Neil Clauson:  Yeah. Yeah. I mean, I've been using SIEMs for over 15 years now, right? And we've, we've all seen them mature, and yeah, add features and functionality over time, but that also, uh, it's just become a data warehouse or a data pit of despair sometimes, right? Where it's just throw all the logs in there and hope for the best. Um, and that, that is a strategy. It's one that, as you said, you know, it, it doesn't scale, and it becomes, uh, costly and kind of hard to get that value out. So, I mean, in general, I'm a huge fan of shifting to the right, leveraging IT in cloud services and other people to manage that underlying infrastructure, let your security engineers do security engineering, not infrastructure engineering. Um, by engaging with those folks who are typically the, the biggest producers of those logs, now you get them as an active stakeholder.

So, um, again, I'm a fan of, again, Wardley mapping is the concept, but it's moving to the right of that value chain, and leveraging and understanding where MSPs or XDRs, which I know we'll talk about, or other third parties can augment the need for those logs, so, maybe I don't need to consume every single event on every single endpoint, I can roll those up to an XDR and then just consume the aggregated alerts. So just, and I think it's, it's about being smart, and, and, and figuring out what as, like, and as SIEMs have matured, figuring out where, um, its use cases and where other solutions might be, uh, covering that use case more effectively. Uh, but ultimately in most orgs, somebody has to be running some type of data warehouse, there's lots of open-source tools, so this doesn't need to be hard.

There's no excuse not to have a SIEM, but I'm a big fan of having process, right? So the, so how do you make sure it just doesn't become that, uh, log everything all the time. I like the idea of a log ingestion process. So, even for the logs you already have, but especially as you're bringing them on board, you should have a workflow where you're looking and doing some upfront, uh, tuning and pruning. Do I need every single event type, uh, can I tune it to source? That's a huge, uh, benefit. Like, the less I can send to the SIEM in the first place, that's fewer events per second, fewer things I need to, uh, unparse and unwind, um, so if you can, tune at the source and make sure the things that you're ingesting map to an existing use case, that's alignment to NIST, could also be alignment to MITRE ATT&CK, um, that's a, a great way to understand, uh, what your threat vectors are, and do I have, do I have ways to detect them?

So, there's, um, kind of a spinoff of ATT&CK called DETTECK, D-E-T-T-E-C-K, which is just, just take all those top 20 TTPs you're looking at, and on a scale of one to four, one being, I have no visibility, four being, I have great visibility, map those out. That would then drive your strategy for, do, you know, if all the logs I have provide zero visibility into my top 20 TTPs, you know, why am I bothering to log those?

Garrett O’Hara: Yeah.

Neil Clauson:  So it kind of helps justify it. Or, hey, this one log source, which is very voluminous and a lot of, uh, you know, uh, data, but that covers half my TTP use cases. That's a great way to give you that transparency, measurability, and alignment that I always talk about.

Garrett O’Hara: What was the, you called it DETTECK, what was the spinoff?

Neil Clauson:  It's, uh, so that's, so ATT&CK, MITRE ATT&CK, is, I think, something most folks are aware, but DETTECK, D-E-I, D-E-T-T-E-C-K, essentially, it's just, um, leveraging the preexisting, um, you know, JSON format to then put a score and a color coding that then gives you some visual representation of your ability to detect the top 20 TTPs that are relevant in general, or to you, right? So, um, right? Being able to identify those, we'll talk about that during threat intelligence.

Garrett O’Hara: Yeah. One, one of the things I've heard over here actually is that, um, even when it, it comes to kind of feeding into a SIEM from our platform that people sometimes won't use the out of the box integration because, to your point, they wanna kinda tune down the amount of stuff that they're getting sent over from Mimecast and, um, they'll actually go and roll their own rather than use the out the box integration, even though that would save time, uh, for that exact reason, they just don't wanna store or ingest as much of the, the log data as, as-

Neil Clauson:  Well, I would say, I would suggest investing in, uh, vendors like us, who now you can, um, be, uh, carve and slice and dice those logs. So, in, on the Mimecast platform now for some logs, versus you can say, I only wanna see high-level events. So, for the maybe, uh, newer security teams or ones that are already overburdened, you don't need to consume every single event, you can consume the ones that are just relevant to your most important detections, and then over time, spin those up. I, again, I'm a big fan of tiering your logs, right? So understanding and ranking them. So I, so we call them like, a tier one log either has a high level of importance or value in those detections, maybe has some audit, audit or regulatory compliance, or just some sensitivity. Those tier ones should have an entry in a Wiki page where the ownership, and the sources, and the dependencies, and the failure scenarios, if that log source goes down, right?

So, it's just about being mindful and saying, not everything could be in that tier one log, but the ones that are, I'm gonna make sure I have full log assurance, I'd call it, right? A hundred percent of the logs, a hundred percent of the time because of that sensitivity. So, maybe those tier three logs, ah, they just go to, you know, cold storage, or they, uh, maybe just don't get processed as, as hard as the tier one logs, or followed up on when maybe there's the inevitable blips in infrastructure. That's the real, um, bane, and the life cycle of this is making sure you, you're always consistent, which is gonna be really difficult.

Garrett O’Hara: Yeah, definitely get that. The, the, look, the other complaints are the quite often the, the, um, challenge with SIEM is actually then identifying the events, and you've kind of touched on this a little bit, but that kind of correlation of data to actually find, or a threat has been found. Um, obviously with less data that presumably becomes easy, but can you talk us through that problem, and then ideally, give us a, a sort of understanding of how you've tackled that problem in the past.

Neil Clauson: Yeah, absolutely. I mean, I think that this is the, the bane of, of SIEM vendors is the, you go in and you flip on every single rule and it lights up like a Christmas tree, um, but doesn't actually tell you anything of value. I think there's probably that additional 80/20 rule, where 20% of those logs are, or those events correlation rules are pretty standard. Tell, tell me about Brute-force SSH, tell me about new additions to domain events, right? There's that core, uh, core blocking and tackling, we, we call it, things that are just, uh, obvious, but then otherwise it's like finding a needle in a stack of needles, right? It's like, how do I understand what's unique to me? At that volume everything starts to look similar, um, right? Events that maybe just are suspicious, an aggregate don't necessarily warrant, uh, you know, red alarm on their own.

I, I like those high fidelity correlation rules. There's a thing called Sigma now, S-I-G-M-A. So it's the Rosetta stone, if you will, for, um, SIEM rule creation, where I can write it once in the Sigma rule concept, and then parsers will flip that out to different SIEM vendors, QRadar, and Elastic, and Splunk. So, I really like that idea as a way to share and develop your, uh, correlation rules in a way that, um, if you have to move SIEMs, okay, well, it's not, it's not a full rip and replace, or if I, you know, if you're collaborating as you should be with your trusted circles and your, you know, IT ISACs, and your other, uh, third parties, that's a great way to kind of start sharing, uh, that detection roll around.

But ultimately, I think, um, in terms of, uh, purple teaming, which I know we'd love to talk about, and, and we, we use it a as a way to, um, drive that process, understanding what threats you're subject to, uh, and then seeing, can I detect that? Can I respond to that? Can I test my control set and see if I have the ability to detect that? Um, and if not, then building out those correlations. So, I think it's really about having a good life cycle, right? And being able to, um, connect the dots, right? And, and connect that dots in a way that gives everybody on the same page, that is related to your specific, uh, threat scenarios, use cases, your geo, your customers, um, and the TTPs that are most relevant to you.

Garrett O’Hara: Yeah. It definitely seems like having consistency, uh, between the different vendors is gonna be key to this stuff going forward, right? I mean, if, if there was consistency in, in the logs, the formats, the data field representations, um, you know, everyone's life gets that much simpler. I've certainly heard that, uh, you know, one vendor represents something one way, and, and the other in a slightly different way, but it's enough to kind of break any of those kind of mappings.

Neil Clauson: Yeah. Yeah, exactly. And that's where things like, um, XDR are starting to help, right? Where they're starting to, there's more open, uh, platforms for that, there's more consistency. Uh, it's, it's been a long road especially for somebody who's been doing this for a while, and how, how challenging it can be, um, to actually get that, we call it, is the juice worth the squeeze, right? And there's a lot of squeezing for sometimes not a lot of juice. Um, but that's where, I think, as an industry, as, uh, it's a team effort between all of us, security practitioners, we're starting to make some progress and starting to be able to, uh, have some consistency with those platforms, and consistent outcomes is what really our leadership are looking for.

Garrett O’Hara: Yeah. You, you sounded like you, you've, you've seen some things just then, you know, [laughs].

Neil Clauson: Yeah, yeah.

Garrett O’Hara: The fatigue and the exhaustion [laughs].

Neil Clauson: Exactly. And one thing I would say for that, fatigue comes from our poorly defined rules or a poor life cycle around the development of those loose rules. So, create a Wiki, and test them. Don't just put a rule into production unless you know what you're going to do when it fires, right? Especially if it's, if you've got a global team, somebody might have created a rule with the best of intentions, but if they're asleep, when the fire is on the other side of the world, then it's, it's useless, or if they don't know what do. So, I use the cliché, an incident is a surprise, but our response shouldn't be, right?

So meaning, hey, this, these attacks that, that, that might, may or may not affect us, do we have the ability to detect it? If so, who, what, what happens then? What systems are you gonna trigger automatically, versus reactively, versus, you know, require investigation first? Where, what third parties do you have covering that? So, it just takes thoughtful process, um, to make sure it works. And then I'm a big fan of threat hunting as well. So leveraging, uh, that tuning to say, well, you know what, this, this rule, it's kind of false positively, it doesn't really always get to a, a conviction, but it's worth having around. Okay, that might be a threat hunt versus a correlation rule that is, you know, on the menu, that's gonna be, uh, get run, um, every day, all day.

Garrett O’Hara: Yeah. No, definitely. You sort, you started edging into this, so I might, might kind of go there now, but like, the, the SIEM versus XDR conversation, um, you know, in, in the industry there's a sort of fair bit of conversation around XDR, and obviously there's a bit of overlap there in terms of utility, but I'd love to get your thoughts on SIEM versus XDR, or what's the future of both, and which one's gonna win? Is this Betamax and VHS, or is it something completely different?

Neil Clauson: No, I think there's room, room for both. Uh, I'll use an analogy, and I love to cook, so it's like the restaurant analogy. We're, we're a bunch of security chefs, right? And we're working in the "back of the house" and we're consuming a whole bunch of raw ingredients, all those logs. And we're trying to prep those ingredients, parsing, aggregation, correlation rules, ultimately to create up recipes, right? A recipe that create, that correlates to a detection. So ultimately, that's what the people at the front of the house, our customers, are expecting from us in the kitchen. We, and each kitchen's gonna be different. You're gonna have different equipment, different chefs, or different experiences, different recipes that people expect, but also, to each their own.

Um, however, I think that as companies are maturing, realizing the challenges of SIEMs, and, and some of the benefits of moving to the right, having those, um, those third parties that are doing their XDR and foc- hyper focusing on being really good at one thing, expanding that to other use cases, I think the two can work hand in hand. I think the SIEM can really help cover and ingest those secondary or tier two use cases in a way that, yeah, it might be a little more difficult to do, uh, uh, automatic correlation rule from them. However, those tier one XDR managed alerts are really gonna give me that, um, you know, that, that, uh, initial trigger, the initial, um, threat to pull on to then go investigate in my SIEM, to pull out those logs and kinda do, correlate both. So yeah, I'm a fan of, I think I'm a fan of both.

Garrett O’Hara: Yep. That's uh, nicely on the fence there [laughs].

Neil Clauson:  Yeah. Well, I mean, so again, good, better, best, right? So companies, if you're just starting out, and you don't have a big budget, and you don't have a strong team, XDR is gonna cover those most common use cases. So, it's a, it's qualified, it depends, but just the idea of standing up a SIEM and throwing logs in it, assuming that some magic's gonna pop out the other end is probably not like-

Garrett O’Hara: Not, not gonna lead to anything other than disappointment. On, on the response side of things, obviously, is the X- XDR response is the last letter there, be very keen to get your thoughts on like, how you see that evolving, um, in terms of, you know, automation, um, how effective it can be, just sort of the gotchas that you have seen in the past. I mean, some of the things I've heard is that you can really only automate things that are well documented, well defined, you know, it's not a magic bullet, that you really need to know what you're going to, to automate. Um, you know, the danger being, you hit the big red respond button, and it breaks stuff because you haven't really planned it or thought it through. Um, and also that you don't need a platform. You made a good point actually, when, when we were prepping for this, that response and automation doesn't have to be some expensive platform, it could be a really good Python script that somebody's put together being... Um, anyway, that's a very open question, but yeah, I'd just love to get a brain dump on, on, on, response from you.

Neil Clauson:  Yeah. I mean, again, that, that incidence surprise response shouldn't be, so a Python script just perfectly matches that good category. The only challenge is that quite frequently in orgs there's some friction between IT and security, right? Especially at enterprise levels, where they have slightly different mandates and missions of availability and uptime versus security and resilience, and sometimes security's best intentions might conflict with that availability and uptime. Um, so yes, a good, uh, team sport eff- effort helps here as well as that it's not just about technology, it's about the people in process side as well. So, uh, yeah, that's where the, the, the more visual SOAR tools and playbooks allow you to give you that transparency, hey, this is what we're gonna do, these are the outcomes.

In general, I like, again, using that recipe analogy is, what are the outcomes I'm going to try to achieve? Reset an active directory account, block a port on a firewall, right? And what are the things, what are the failure scenarios around that? So, by going to your IT team and saying, "We've thought this through, this outcome is important to us, we all have this shared goal, but the reality is in these circumstances, we might need a second set of eyes. Are you okay with that or not?" Right? But the, the visual aspect really allows that transparency, that review, so it's not just some Python script running in, in the back that you never know when it's gonna trigger or not. Um, and then getting those stakeholders on board, going, "All right, we understand you're not okay with us resetting a hundred AD accounts at once, can we do one? You know, can..." Good, good negotiation and soft skills, I think, should be part of every security person's, um, training and effort, right? 'Cause it's how do you influence without authority? It's how do you, uh, show partnership and engagement? And I think that's really where, uh, to me, the response part comes in for enterprises is it's just showing, hey, we can do this, and we can do this effectively, especially with these, these next gen set of tools, and IT and infrastructure are, are key stakeholders in making sure that happens.

Garrett O’Hara: As you say that, so, you know, with IT and infrastructure being key stakeholders, when it comes to response, um, you know, when you think of SOAR, in general, it's the playbooks, you know, the visuals dropping in actions and, and sort of logic and whatnot. Um, but obviously they, they have a collaboration component also, you know, it's sort of the war room, if you're responding to a, into an incident. In, in your experience, do you involve or make available to the IT or infrastructure teams, the SOAR tools so they can see what's going on, and, and potentially even collaborate, as you said, through a SOAR tool, or is it much more of an informal, you know, human to human type thing?

Neil Clauson:  No, I mean, I think it's both, right?

Garrett O’Hara: Yeah.

Neil Clauson:  I think, again, it depends on the, the type of restaurant that you're running. But the, uh, more mature orgs will, will have the IT and security teams leveraging different playbooks in that same platform, because ultimately what you want, the hardest part is standing up those endpoints, the, the authentication and access to those endpoints that are gonna do the outcomes you want, right? Like block a port on a firewall, or, you know, uh, take a system offline, um, that kind of thing, nuke it from orbit to use an aliens reference. Um, it's the only way to be sure. Uh, so the, uh, so I think that by, in that good, better, best methodology, lots of times IT will have their own automation platforms, which might not match to the security use cases, but the more you can say, hey, for these endpoints, this is the platform of choice, so that the tier-one type endpoints that really need to make sure that happen and are up all the time can, um, you can achieve those outcomes. So, uh, but I think, and lots of folks have Slack and other ways of ticketing that kind of thing. Ultimately, the enrichment aspect of the SOAR can be just as beneficial as the response.

Garrett O’Hara: Yeah. Let's, uh, let's pivot a little bit and, um, jump of and talk about threat intel, and, and certainly it's, it's a term that gets used a lot in, uh, in the industry, and I think it kinda means different things to, to different people, and probably has different levels of quality if we're honest. Um, and you know, I've heard folks on, on from Recorded Future, um, Kendal Watt, who, uh, you know, kind of talks through this in a fair amount of detail about, you know, the operational and technical sort of intel versus strategic threat intel. And, and like I said, that variance in quality can be dramatic, um, depending on where it's coming from and, and what it is exactly. Um, obviously you've got fairly deep practitioner experience, it'd be very good to get your thoughts on how you see threat intelligence best used.

Neil Clauson: Yeah, absolutely. I mean, I remember 10-plus years ago, trying to get some kind of value out of my Cisco IPS solution by just uploading lists of, of bad IP addresses back when you could just block a whole bunch of IPS out, right? So, yeah, I, and I love the differentiation between, um, operational or technical threat intel, and then that strategic, or, you know, tactical threat intel. The, there's use cases for both. And again, just started, it was just, hey, here's a whole bunch of IP addresses. The, uh, so, I think that there's a great use case for that. Obviously the best source there are the things that are attacking or hitting you directly. So, pulling that from your, your logs, from your email stream, you know, shameless plug from Mimecast, right? That, it truly is understanding what your, the threats that are coming at you, uh, and being able to, um, to understand and derive what the top techniques they're trying to exploit through those, what the top, uh, people they're trying to exploit, what the, uh, you know, just the methodology, and then also leveraging that, uh, technical intel from your other trusted circles through those XDR ecosystems, like I mentioned, right?

Like, it should be relatively easy now, given the breadths and depths of tools that are out there for relatively free, um, to be able to, uh, cross pollinate, um, from, threat intelligence from, in the tactical stuff that, hey, if your email saw this bad hash, have your, make sure your endpoint blocks it as well. That's, I think, uh, basic table stakes. The, the more advanced part is when you're using that tactical threat intel, the, the keeping your finger on the pulse for the different malware families, the different, uh, things that may or may not affect you. So, that's a part of a more advanced program, but one that you really need if you are facing those threats. And, uh, to me, that means having, um, a dedicated person that takes your requirements, they're called PIRs, your, your priority intelligence requirements. So understanding, what are the things that we need to keep our pulse on, and having somebody do that actively day in and day out.

Um, I would say threat intel also would mean things like, what's the most commonly exploited vulnerability right now in the world, and then pivoting and saying, do we have that internally, right? So that's, to me, the tactical execution of making sure that you are situationally aware, and that you have really fast OODA loops that you're observing what's happening in the worlds, both in general, and to you, you're making quick decisions, and that you're able to act and close those loops, close that, uh, reduce that attack surface area as close as possible. It's, it's just the day in day out, you know, activity that, that should be informed with that, uh, tactical threat intelligence.

Garrett O’Hara: And is there some forms of threat intel that are kind of more valuable than others, or some sources of threat intel that are more valuable than others? You sort of touched in this, I think, maybe, but, um, yeah, maybe more explicitly?

Neil Clauson: Yeah. I mean, ultimately, it's the, it's the ones that you're acting on, right? You could be told the number one IP of the, the Death Star and still not, you know, if you don't do something about it [laughs], then what's, you know, what's good knowing about it? So, um, I think, right, so just, it's, it's that, it's operationalizing it, it's the people, process, and technology aspect, and that's where you have to get, um, those relationships built ahead of time with your firewall team to say, can I just have a rule that's empty, but if I needed to, I would put an IP address in there that's gonna get blocked across the entire company. Here are the scenarios where I would use it, because we're actively under attack or [inaudible 00:22:25], right? So, you know, that proper planning, that, to me, is where the value of the threat intelligence, is, this type of scenario is likely to affect us or already has, and, and, and we'll hopefully talk about tabletop exercises, but as a result of these failure or pre-mortem analyses, or this thing called red team thinking, we think this is the most likely, uh, place for this threat to become realized. Great. That's, that's really where, um, it's, the best threat intel is the one that actually drives change.

Garrett O’Hara: Definitely. Well, look, you've, you've just mentioned it, so let's, let's go there, let's talk about tabletop exercises, and, um, and kinda get into that. Uh, well actually, it's Nick Abrahams, who's a, a local, um, sort of Futurist, uh, speaker, a very cool guy, we were talking about Web3, and you know, fairly, fairly cool stuff. But, um, he's also heavily involved in, uh, cybersecurity and technology even more generally, but he has spent a bit of time working with boards, and uses simulations or tabletop exercises, um, with them to kinda get them understanding where they'd sit in terms of the pay or don't pay when it comes to ransom payments. Um, obviously, you, in your background, and you've just been mentioned them, so you've clearly, you've spent time, um, and probably run plenty of tabletop exercises. Can you talk us through what you've done, and then, you know, ultimately what's the value your teams have gotten from those exercises or the business more broadly?

Neil Clauson: Yeah. I mean, again, this is the, um, the train, like you fight and fight like you train cliche, um, which means getting all, everybody, all of your stakeholders, not just the people on the front lines, your analysts and engineers, but your, your legal, you know, your PR team. So, uh, ultimately, yeah, we've done a lot of exercises, a lot of workshops, and they can be quick. So, by, my general recommendation is, don't make this be, you know, uh, I remember playing Dungeons & Dragons as a kid, and there'd be like eight-hour long campaigns, it's kind of like that, but it should just be a real quick hit that's, um, good and consistent. We set up some workflows using our Wiki and ticket management system so we can create a backlog just like kinda software development, just a backlog of in-scope exercises based off of the events that we're seeing, based off of our company initiatives or projects, or just output from the previous training, "Hey, we did okay last time, but we need to maybe run these, these different, uh, variations." So really, uh, repeatability is the key.

And having those templates, um, just Word and PowerPoint templates have the, um, the guide for your coordinator, have your audience participation guide, right? So, it shouldn't just be that, oh, no only one person can do it, anybody should just be able to jump in, have those templates, run through the scenarios, keep people on their toes, and again, just engaging early and often, it doesn't have to be, people don't have all day to run these, uh, 30, 45 minutes. And then using that ticket system really also helps with tracking with auditing. So getting, you know, the, the people who are looking for that assurance that you're, you're actually training your teams, audit, uh, loves the evidence they'll see from a whole bunch of closed tickets in a, like a tool, tool like Jira.

Uh, but then also using that ticketing system to tie in the tasks and improvement efforts that you've, that have popped out of those, um, training exercises, right? So if you can really, again, this is about OODA loops, it's about observing how your team and your people, process, and technology would respond, uh, in a certain threat scenario or failure scenario. Um, and also like, just not boiling the ocean. This doesn't have... just showing a little bit of progress each time, I think is probably the biggest, um, thing that I've seen. It's like going to the gym, right? You don't lift a thousand pounds in one day. You, you lift a thousand pounds over a couple of days, and it's about repeating that over and over again so you build up that muscle, and making sure that, um, all the folks know it can be fun, it's a good way to kinda, you know, um, test out some new scenarios, and also for the folks who are having to go through them drive, uh, evidence and observability up to those, you know, leaderships, meaning, "Hey, we lack a complete set of controls across this, this threat scenario, right?" So you can use it to get funding, um, and otherwise get the resources you need when there's gaps.

Garrett O’Hara: What's the cadence? So you, you know, you mentioned you can lift, uh, a thousand pounds, I don't know what that is in kilos for the, uh, non-imperial, uh, [laughs] weight measure audience. But, um, spot on, like, that's one of the things we hear all the time. It's like repetition is mastery, but also you can have fatigue if you're doing it all the time. What have you found is a good cadence for the kind of, uh, exercises or tabletop exercises that you're running?

Neil Clauson:  Yeah. I mean, that could, that's gonna depend on the, the day-to-day workload of the team and how much gaps you have for that. And so, sometimes you have to push, and sometimes you have to step back a little bit if there's a lot of activity. I would say, you know, once a week is probably too much with one team that's already, um, got a lot going on. I would say no less than once a month, and certainly more than once a quarter. Um, and that, again, could be once every six months we do a full on exercise, it involves all stakeholders, but once a quarter we do one each, right? And then once a month we'd maybe bring in different, um, uh, new scenarios or new third parties situations.

Garrett O’Hara: Yeah. No, definitely. And, and, you know, in terms of situations or scenarios, uh, like do all scenarios kinda lend themselves to tabletop exercises, or are there some that you've just found actually these work really well, you know, this particular type of scenario works really well for tabletop?

Neil Clauson: Uh, I mean, again, especially if you're just starting out, you wanna do the ones that are gonna be the, the, the most likely from, from a FAIR risk assessment, the ones that either you have the biggest threat event frequency, and/or the weakest, uh, control coverage, right? So, tell me about the things that are gonna bite me in the butt and, and, and do something about that. Because again, sometimes look at the tabletops not just as the tabletop themselves, it's that meta point? It's the reflection up to leadership like, "Hey, we need help here." And it's not just the technical side, it's, we don't know what PR would do, we don't know how to engage with legal if we had a ransomware event. So the, I think, it's the ones that are going to, um, be your, specific to you. I think the harder ones, again, are where you doing that, do need that multi-team effort, um, and the ones that are harder to either wrap technology around, or wrap, wrap process around.

Garrett O’Hara: Yeah, definitely. And for those who are interested, uh, Neil, just mentioned FAIR. We had Marco here on a couple of weeks ago and, um, we spoke a length of it, FAIR. So, if you're looking for a, a brief primer on that, um, you gotta jump back in and get onto Marco's episode. Um, in terms of the tabletop exercises, Neil, uh, like, who do you involve? You mentioned PR just then, but what's the sweet spot in terms of number of people taking part? I mean, I know that part of what you would do as a sort of leader and manager in security is that you've gotta make the case for pulling people out of their day-to-day jobs and, you know, potentially involving other, uh, teams. And there's a cost involved with that, that you've gotta justify at a time. Like, how have you navigated that one? And, and, you know, what's the sweet spot in terms of the teams that are involved, the number of people?

Neil Clauson: Yeah, absolutely. I mean, I'm a big fan, again, of that, um, Tree MORT and MORT failure scenario analysis. Um, and so, those teams, I would say, this is part of their day job, right? They're managing and, uh, and monitoring the infrastructure that provides the assurances both to us internally and to our customers. And so, um, this is that, I love how Netflix does their Chaos Monkey, right? They run tools that break stuff in the environment all the time, because things need to be designed with resiliency. And so, I feel like, um, engaging with the IT, technical, cloud, and other third party infrastructure teams is critical. The, uh, other folks, again, would, as a separate bucket would be the legal, your PR, your customer comms, people who are gonna kind of be your voice, um, while your heads down, uh, responding to the incident. So, that's, I think those are two, the, the key teams, um, and making sure that they kinda understand what they, what they need to do when that big red button happens.

Garrett O’Hara: Yep. No, definitely. And it's, it's sort of adjacent to this, but, um, you know, purple teaming is the, the kind of next, um, it was next topic that I was keen to kinda touch on with you today. And I think, you know, we've probably got enough time to have reasonable conversation. We, we may, um, I was chatting to Neil before we hit the big record button, uh, because there's so much to cover here, we may end up, you know, splitting this into two episodes, but let's, let's see how we go. Um, you know, purple teaming is, is one of those things that I think many people are starting to get familiar with. People will definitely know red teaming and blue teaming, and, uh, but can, kind of give us a level set on what purple teaming is, um, from your perspective?

Neil Clauson: Sure. One thing, thankfully it's not yet is a product, right? One thing we're, we're great-

Garrett O’Hara: Give it time. It'll be in a brochure sooner.

Neil Clauson: Yes, yes. We should go register some domain names. Um, yeah, so it's, it's, I think it's a threat intellig- threat intelligence driven and coordinated approach to evaluating risk exposure. That's kind of the, the bottom line, but ultimately it's part of that good and comprehensive test and validate strategy that's augmenting and, and, and balancing your detect and response. Not everybody's gonna have a internal red team, uh, but it's a concept of leveraging some type of adversary simulation tool to then figure out, where do I have gaps? What, how many detections can I create? How quickly can I respond when those detections occur? So, I think in terms of purple teaming, kind of supporting three objectives from a tactical perspective allows you to concentrate your resources on those prioritized threats so that you can provide that timely event, uh, timely advice to decision makers. Um, and that calls back to that threat and tell, right? It's a, it's, it's, it's, it's, uh, a threat on one side, you test it, and then you come back saying, yes, we do, or do not have the capability to either detect or respond to this actively going ongoing threat.

Um, ultimately, again, it's about unmitigated risk. If I have unmitigated risk, does it exceed my threat and risk appetite? Purple teaming will help you understand that in, in a quantitative way. Um, and really understanding, do my controls work as expected? That's, that's what I look at, 'cause lots of times blue team will defend us, like, I've got a SIEM, I'm storing all these log, yay, life is good. Yeah, but do any of the rules actually fire? And if they do do, is there enough evidence and does it make sense, or you're just generating false positives, right? So, the tactical aspect of purple teaming really should be, uh, to, to make this be a team sport and make sure that everybody is focused on actual risks and actual detections, not just perceptions. Um, and then from an operational perspective, purple teaming really kind of helps ensure the security team is working as a whole, that everyone's involved and aware, that they know what are the threats, how they may affect the org, and really most importantly, what are we doing about them, right?

It's really, if you have a purple team, it means you care, and that you are, uh, trying to drive your risk exposure down as much as possible. At a strategic level, it's really that data fusion, it's growing that situational awareness to close the loops, again, to reduce that attack surface area. So, um, purple teaming is, it doesn't necessarily have to be a bunch of people, there are, from an automated perception there, perspective there's breach and attack simulation tools, there's, there's open source tools like CALDERA, and then full-pay vendors out there that you can run, or you could engage with your MSP to say, "Hey, yeah. Instead of only once a year bringing in red team once a quarter, come in with this breach attack, and run it through my endpoint and my firewall solution, and tell me how effective they are aligned against those minor TTPs. So give me some quantitative evidence." That's a real huge value. And then as you go down that stack or up the stack in terms of less automated and more custom, you have your TTXs, you have your vulnerability management, and you have your, you know, threat, threat hunting and purple teaming activities. So, it's kind of a, a mindset, I guess, to look at it

Garrett O’Hara: And, and does, like, as you're describing that, does that feed into any of the kind of lower tiers of the FAIR model? Like things like response capability. Like, you're starting to gather data where you can, you know, you get a gauge on how well you can respond to things and ultimately figure out risk at a higher level, like, is that part of this as well?

Neil Clauson: Absolutely. Yeah. I mean, that's the, the best thing about FAIR is that it really, it benefits from having quantitative or at least semi-quantitative finger guesses, as opposed to just, I, I think, right? Like, uh, somebody, I know, um, her name is Renee Murphy from Forrester, uh, has a quote, which is, "Trust is not a control and luck is not a strategy." And I've always lived that kind of mindset of just don't, don't just assume that your controls are doing what they, what you say they're doing, what a vendor says they're doing, you actually kick in the tires and see.

Garrett O’Hara: And try them. I reckon this business, um, and somebody's gonna do this where they're gonna set up a t-shirt company for cyber security, and 'cause there's a bunch of t-shirt quotes that, uh, I reckon, there's a huge amount money. Yeah.

Neil Clauson: Yeah.

Garrett O’Hara: Next cyber security conference in Black Hat could be the, could be the market-

Neil Clauson: Exactly. There you go, innovation.

Garrett O’Hara: Yeah, absolutely. Um, in terms of purple teaming, and this is probably the last question we've got time for today, um, what, what are the gotchas? Yeah, I mean, it, you know, it sounds wonderful and, you know, gives you data, gives you all the things that you've just described there, but any, any kind of gotchas or things that people may need to be kind of mindful of as they start out on this?

Neil Clauson: Absolutely. I think like anything in security, there's a lot of, uh, initial excitement, um, and it can become easy, uh, to become too distracted. I always say my ADHD keeps my OCD in check, right? But it's [laughs], but it's about staying focused on the, the most important risks and their relevant pathways. It's not a, it's about not being distracted by an unlikely tactic or a really weird edge case or vector if the front door is wide open, right? If the front door is wide open, then stop worrying about these very small, um, possible, but focus on the probable. Um, and so, that context, that collaboration and the tempo is king, so don't bite off too much at once, right? Like say, we're just gonna focus on this, run that to ground, don't try to spider off too many different activities 'cause then you'll add burnout to the burnout.

Um, and again, you don't have to invest, uh, a lot of dollars into the tools, it's about just that process. Um, it's a way of working, and it's about, um, you know, thinking smarter and being able to provide some quantitative evidence, that's really where a program starts to, to shine, and show maturities when you can be quantitative, uh, and not just wave your hands saying, "We, uh, I think we're secure." Right? Being able to, and being able to hold your vendors accountable. If your endpoint vendor is failing all your breach and attack simulation tools, or the most common, or most, uh, you know, newsworthy events, you can go back and, you know, shake that cage a little better, maybe change it out or get a better vendor, get a better price, but be able to be quantitative.

Garrett O’Hara: So, as we kind of finish out, I think a lot of what you've described today is, is helping people ask or sort of answer that question that I think is very frustrating for security leaders when, you know, board, or the business leadership ask, are we secure? And, you know, the... what's the answer to that, but, uh, yeah, at least you can start to, to sort of build some data or, you know, the foundations for a response to that massive question.

Neil Clauson: Well, I always say, you don't need a metrics program, which kinda gets, raises my eyebrows. There's a concept called Goal Question Metric. You really need a goals program. What are the goals that we're trying to achieve at a high level? What are the questions that myself and my leadership will ask on whether or not we're meeting those goals? Then find the metrics that help you answer those questions, right? So don't just do this shotgun approach, where like, "Oh, we're gonna collect all these stats and percentages and whatnot," like, be careful what you measure, um, and only measure the things that are gonna help you drive your program, uh, or show you evidence of that, you know, leading or lagging indicators. I think in terms of, uh, the quality, quantity, and consistency of that program, the quality, quantity, and consistency of the alerts, of the events, of my ability to detect, of my time to respond, of my coverage against TTPs.

Garrett O’Hara: Neil, it's been an absolute pleasure, uh, talking to you today. There's definitely an episode two, three, probably four, or five, who knows? Um, there is so much to talk about here. Um, really grateful for your time today. Thank you so much for joining us.

Neil Clauson: Absolutely. Thank you.

Garrett O’Hara: Thanks so much to Neil for joining us in. As always, thank you for listening to the Get Cyber Resilient podcast. Jump into our back catalog of episodes, and like, subscribe, and please do leave us a review. For now, stay safe, and I look forward to catching you on the next episode.