While the staffing crisis is real, our global MDR SOCs are thriving with top-notch analysts, DFIR talent, and no revolving doors (they like it here). In a high-pressure, high-stakes business, these are our lessons learned.
Measure your staffing performance meticulously and publicly
In an industry plagued by burnout, churn, and open jobs everywhere, be obsessed with your metrics to retain top talent. We do.
Last year, we grew our global Managed Detection and Response (MDR) teams by 68%
Our voluntary attrition for SOC analysts is under 5%
Since the start of Rapid7 MDR seven years ago, we’ve only lost about one to two analysts per year (as competition for cybersecurity talent went white-hot)
Rapid7 recruits talent from all over the world to join us in our state-of-the-art SOC locations. Each SOC has incredibly high retention rates.
We prioritize investments in training, competitive pay, project work and extracurricular activities, and ensuring analysts are doing the work they enjoy. The leadership team is in tune with job satisfaction and directly attacks any aspect of the analyst duties that causes friction.
Peter Drucker said it best: “Culture eats strategy for breakfast.”
According to a survey by Mimecast, 84% of security professionals are experiencing burnout due to the constant barrage of threats, the talent shortage, and other employees’ mistakes (as a result of burnout). And, while everyone battles “The Great Resignation” and our collective 5-year skills crisis, ZDNet reports it’s going to get worse. Nearly a third of the global cybersecurity workforce plans to leave the industry—not their jobs, but the entire industry—within two years.
To prevent burnout, we encourage a culture of friendship and after-hours socialization. People who work alongside friends help more and perform better. They trust one another. Like just about anyone in our line of work, Rapid7 MDR employees know they can go anywhere and do what they do. They also know we greatly appreciate the fact they choose to do it here.
A member of one of our SOC had his car in the shop for far too long due to a supply chain shortage of the missing part. There was only one thing to do for April Fool’s day:
As one member of the team stated, “we work at a place that crowdsourced a $700 prank!”
You don’t need budget for team-building consultants and “trust exercises.” Camaraderie is created in Slack channels and karaoke nights at the bar on the first floor of the Rapid7 Arlington , VA office.
Create a learning organization
We’ve heard it called “alphabet soup after your name.” While certifications are important, real-world experience and constant learning trumps a course any day of the week. And the best way for the SOC to learn? By doing first-hand and sharing those learnings with everyone. Here’s some of the lessons learned:
First, eliminate silos. Each of our MDR SOCs are composed of three tiers of analysts, working together on customer environments. There’s complete threat detection coverage, multiple layers of escalation and validation, and redundant knowledge. Additionally, the technology used by the SOC captures relevant details of the environment, detected threats, and analysis notes which are available to all analysts.
Second, train constantly. Rapid7 has a robust training program: a combination of external live training (SANS, Chris Sanders courses), self-paced learning (TCM malware analysis & forensics courses), as well as a robust internal security training program (modeled after specific incidents Rapid7 MDR has handled) to train our analysts quickly and effectively. All training is heavily focused on endpoint forensics, incident response, threat hunting, coding/scripting, and foundational security concepts. All analysts have the chance to attend external training every year. Internally, analysts learn from each other with weekly “lunch n’ learns” to level up their stills by learning from others around them and show off the latest threat they were able to thwart for our customers.
Third, we organized around learning in new ways. Over a year ago, Rapid7 merged our Incident Response Consulting Team with our MDR SOC to create an integrated team of Detection and Response experts. If an incident investigation appears to be major, analysts simply (and literally) swivel their chairs and tap Senior IR consultants and DFIR practitioners on the shoulder.
For major incidents, Rapid7’s TIDE Team (Threat Intelligence and Detections Engineering) is right there too. “We ride along with them and are watching what they’re discovering and we develop new detections,” says Eoin Miller, Manager of Detection and Response Services. “It helps not only that customer but any other customer that may be a current or future victim of that same attacker.”
Rapid7 MDR also created a “Tactical Operations” (TacOps) team, which is primarily used as a “farm system” for analyst development. Typically, Associate Analysts at other Security Operation Centers are relegated to Tier 1 roles, focusing on low severity alert triage with little exposure to actual malicious activity or complex investigations. Rapid7 takes a different approach by throwing these Associate Analysts into the deep end to deal with real, high priority threats (the things we know are evil), which accelerates their learning curve. They’re actually looking at malicious activity all day, not just hundreds of benign alerts.
Our Associate level analysts have even gone on to publish their work and were tapped to lead a technical malware deep dive on one of the most popular security webinars in the world (Ultimate IT Security). Not too shabby for “entry level” folks to be presenting to a broad audience after only a year working in our SOC. Not surprisingly, we focus on promoting from within, with many analysts taking on advanced roles in forensic analysis and IR.
Finally, we’ve reorganized our services organization to bring our penetration testing team SOC analysts under one roof. We feel the best way to learn (and improve our ability to detect and respond effectively) is to encourage collaboration and knowledge sharing between both our offensive-minded and defensive-minded security practitioners. Iron sharpens iron.
Never compromise your standards
MDR analyst candidates go through an initial technical assessment (live on phone responses) with our Talent Acquisition partners in order to pre-screen candidates before the live technical interview panel.
During the interview Technical Panel, our interviewers’ goal is to push the candidate to the edge of their knowledge. We ask a series of questions which are progressively more difficult using real-world scenarios: “If you see XYZ behavior, walk me through the process from start to finish:
What technology and methodology would you use?
What data are you looking for?
Deep dive into why and how you’re looking at it?
How do you come to the determination that the behavior is malicious or benign?
This allows us to question various tools and techniques used in the course of an investigation. We then hire based on the candidate’s knowledge, skill set, and culture fit.
Rapid7 has company core values. We’ve added to it with our “Culture Code for the MDR SOC.” Every organization and each SOC’s values are different. These are ours:
Ownership: Know what you’re responsible for and own it. We expect you to own your mission fully. Don’t make excuses, and don’t point fingers at others.
Customer-Centric: We are here for one reason—to deliver the managed security services our customers expect and deserve.
Passion and Purpose: Love what you do. While not everything you do every day is exciting, our team members genuinely enjoy their work and understand the importance of it.
Don’t Just ‘Turn The Wheel’: We’re not here just to handle alerts, run scans, perform hunts, or throw alerts over the fence for our customers to handle. We’re here to bring our security expertise to bear in the most effective way to better protect our customers at scale.
Risk Taking: Choosing not to take a risk is often the biggest risk. We will never fault someone for taking a well informed risk in order to better serve our customers.
Integrity: We never mislead customers or prospects or act against their best interests, and we are open and honest with our fellow Moose.
Never Done: This is not a clock-in / clock-out kind of job. While many days are predictable, others are not. Our North Star is customer outcomes, not time-based.
Glass Half Full: Security operations can be unforgiving—but we will remain positive and optimistic.
Have Fun: Get your job done, but have fun doing it.
We’re always looking for great security professionals to join our team. If the above piques your interest and you’re looking to join a part of something special, come check out our open Career opportunities.
When a balloon crossed through Canada and the United States, everyone lost their minds. The news was all-balloon, all-the-time. And the big, obvious, serious questions flew too: “why didn’t we see the balloon sooner? Have there been other balloons?”
That sounded pretty familiar to Rapid7 Detection and Response Practice Advisor Jeffrey Gardner. When the U.S. Military responded to the visibility problem in the airspace, it discussed “adjusting filters.” And that sounded familiar too. Because that’s what security practitioners are expected to do every day: find things they don’t even know exist.
While this Lost Bots episode is packed with practical guidance (you’ll likely watch parts of it more than once) it’s delivered by the “Team America” avatars of Gardner and co-host Stephen Davis, Lead Technical Customer Advisor for MDR.
Anyone in cybersecurity is in it for the humans, but we can still be fun.
Humans are great at adapting to change—but objectively the pace of technological change has been way, way too fast.
Security teams manage an average of 76 different tools. Breaches have gone from “s#&@!” to “inevitable.” That’s why we built Managed Threat Complete to address the reality of today’s threat environment. By 2025, Gartner says 50% of organizations will decide to partner with an MDR (Managed Detection and Response) service for 24×7 monitoring.
Now, one move can consolidate and rebalance your work
Managed Threat Complete: It’s always-on MDR plus unlimited vulnerability management with a single subscription.
Combine these two historically siloed pieces of a security program, and you have a complete picture of your risk profile and threat landscape. Since the service combines proactive, responsive, and strategic support of your program, it gets smarter and more resilient over time: a continuously-improving, virtuous cycle.
Most importantly, Managed Threat Complete lets you prove you’re building measurable capacity to be effective at detection and response—and improve the definitions of success that matter most to you. We call it the R-factor, and it measures:
How ready you are to react to your sprawling attack surface
How responsive you can be when something inevitably gets through
How effectively you’re able to remediate after the fact
How you measure your results and show provable outcomes
Forrester Consulting did the math on Rapid7 MDR, and you win
Forrester’s June 2022 Total Economic Impact™ study commissioned by Rapid7 found that Rapid7 MDR produced extraordinary results:
5.5x ROI over 3 years
<3 month payback
90% reduction in the likelihood of a breach
While your team methodically reduces your risks with unlimited VRM scanning, Managed Threat Complete gives you a full team of SOC experts dealing with threats in your environment using advanced XDR technology. And that means really responding, remediating, and making your organization safe and secure—no matter what.
It’s MDR so different, think of it as MDR 2.0.
Typical MDR vendors will simply alert a CISO to a problem. If you’re breached, they’ll tell you to hire an outside Incident Response firm to take it the rest of the way. Managed Threat Complete gives you unlimited Incident Response (the same level you’d get with an IR retainer) included, with DFIR professionals already embedded on your team.
Typical MDR vendors charge by data ingestion and retention. We prioritize visibility into your environment so our analysts can detect and respond without compromise.
Typical MDR vendors take a black box approach to their technology. But with Managed Threat Complete, we give customers unlimited access to our cloud-native XDR technology, sprawling detections library, all of it. See transparently into what your Rapid7 MDR partners are doing. Run your own investigations and threat hunting. Log in once a day or once a year, it’s at your fingertips.
Managed Threat Complete delivers a holistic approach to risk and threat management, so you can consolidate costs and be ready for whatever comes next.
Managed Threat Complete
Focus on proactive, strategic work, while our team delivers 24/7/365, end-to-end detection and response.
You’re in cybersecurity, so we’ll guess: 2022 crashed in with Log4Shell and, for the most part, got more challenging—never less. So, we kept making tangible improvements to InsightIDR, our cloud-native next-gen SIEM and XDR. We worked with some of our most forward-deployed practitioners: Rapid7 MDR, Threat Intelligence and Detections Engineering, our open source communities, and our customers. New features and functions address pain points and achieve specific goals.
Let’s review some of the highlights:
Accelerated response time with automated Quick Actions
Earlier in the year, InsightIDR launched the Quick Actions feature which provides teams with instant automation to reduce the time it takes to search, investigate, and respond with a simple click. Example use-cases include:
Threat hunting within log search. Using the “Look Up File Hash with Threat Crowd” quick action, teams can learn more about a hash within an endpoint log. If the output of the quick action finds the file hash is malicious, practitioners can choose to investigate further.
More context around alerts in investigations. Leveraging the “Look Up Domain with WHOIS” quick action enables teams to receive more context around an IP associated with an alert in an investigation
“InsightIDR is a real savior, we have reduced our time for log correlation, responding to incidents, not opening multiple tabs and logging into different platforms to understand what happened.”—Abhi Patel, Information Security Officer, Prime Bank. Source: TechValidate
Expanded visibility across cloud and external attack surface
With InsightIDR, teams have security that grows and scales alongside their business – both on-prem and in the cloud. This year we focused on empowering security teams with cloud incident response capabilities by providing robust integrations with AWS CloudTrail and Microsoft Azure, while also enabling cloud detections with our AWS Guard Duty Detections, AWS Cloud Trail Detections, and more.Customers have the full context of their cloud telemetry and detections alongside their wider environment to get a full, cohesive picture and investigate malicious activity and threats that may move across multiple devices and infrastructures.
Additionally, with Threat Command and InsightIDR together, customers can unlock a complete view of your external and internal attack surface. You can now view Threat Command alerts alongside their broader detection set in InsightIDR:
Prioritize and investigate Threat Command alerts:Use InsightIDR’s investigation management capabilities and seamlessly pivot back to Threat Command to remediate the threat or ask an analyst for help.
Tune Threat Command detection rules directly in InsightIDR: Adjust the rule action, set the rule priority, and add exceptions.
Lastly, Rapid7 provides all customers with 13 months of data retention by default—so they are always audit-ready. To support compliance regulations, we launched new dashboards for organizations to ensure they are meeting requirements. For example, we launched new dashboards for CIS, a common security framework, covering:
CIS Control 5 – Account Management
CIS Control 9 – Email and Web Browser Protections
CIS Control 10 – Malware Defense
“With Rapid7’s InsightIDR, we have a greater handle on threats. We are able to resolve issues quicker and reduce maximum tolerable downtime, our incident management procedures and real-time actions have improved immeasurably too, and we have better cyber hygiene as well.”—Security Officer, Medium Enterprise Chemicals Company. Source: TechValidate
Confidence with expertly curated and vetted detections
Rapid7 Threat Intelligence and Detection Engineering (TIDE) team has curated and is continuously updating our XDR detection library that is expertly vetted by the Rapid7 MDR SOC. The detection library is a result of meticulous research, our vast open source community, security forums, and industry expertise to provide your teams the data they need for sophisticated detection and response. Last year we launched a slew of new detections, a bulk being IDS rules, but worth highlighting is the expanded coverage of tracked threat actors with the Threat Command integration. By integrating our Attacker Behavior Analytics (ABA) detection engine with Threat Command’s threat library intelligence, customers can access broader detections, and new threat groups with around 400 new ABA detection rules powered by thousands of new IOCs.
We also added a new ABA detection rule – Anomalous Data Transfer (ADT) that uses the Insight Network Sensor to identify large transfers of data sent by assets on a network and outputs alerts for easier monitoring of unusual behavior and potential exfiltration.
“InsightIDR provided value to us on Day-1. We didn’t have to write long lists of rules or tweak hundreds of settings in order to get security alerts from our operating environment. Better still, the signal-to-noise ratio of the alerts is great; little-to-no false positives.”—Philip Daly, VP Infrastructure and Information Security, Carlton One Engagement. Source: TechValidate
Looking ahead
Watch this space! We’re always working on new product enhancements and functionality to ensure your team can stay ahead of potential threats and malicious activity. Keep an eye on the Rapid7 blog and the InsightIDR release notes to keep up to date with the latest detection and response releases at Rapid7.
The “right” criteria is whatever works to further your security organization’s specific needs in detection and response (D&R). There’s only so much budget to go around—and successfully obtaining a significant year-over-year increase can be rare. The last thing anyone wants to be known for is depleting that budget on a service provider that doesn’t deliver.
At Rapid7, we’ve spoken extensively about how a security operations center (SOC) can evaluate its current D&R proficiency to determine if it would be beneficial to extend those capabilities with a managed detection and response (MDR) provider. In an ongoing effort to help security organizations thoughtfully consider potential providers, we’re pleased to offer this complimentary Gartner® report, Quick Answer: What Key Questions Should I Ask When Selecting an MDR Provider?
This asset acts as a time-saving report for quick answers when vetting several potential providers. Key questions to ask yourself and your service providers include:
Yourself: Are we looking for providers that can improve our incident response capabilities?
Yourself: Do we have use cases specific to our environment that the MDR provider must accommodate?
Yourself: What functionality do we need from the provider’s portal?
Provider: How good are you at detecting threats that have bypassed existing, preventative controls?
Provider: How do you secure, and how long do you retain, the data you collect from customers?
Provider: What response types are provided as a component of the MDR service, and what is the limit of those response activities?
Before expecting any quick answers though, it’s crucial to consider…
Your criteria framework
Your organization might conduct a new audit of desired outcomes and team capabilities and discover it actually can handle the vast majority of D&R tasks. That’s why it’s crucial to go through that process of discovery of what you really need and determine if you can responsibly avoid spending money. Gartner says:
“Many buyers struggle to formulate effective RFPs that can solicit relevant information from providers to help in the initial evaluation and down-select process. Therefore, it is critical that buyers construct the must have, should have, could have and won’t have (MoSCoW) framework. Using these criteria will ensure they are able to effectively make selection choices based on genuine business needs.”
Also, what is the platform from which you are launching your evaluation process? Will this be the first engagement of an MDR service provider or are you changing providers for one reason or another? If the latter is true, then you’ll most likely have loads of existing data to inform your buying experience this time around. It’s also critical to get a strong sense of what the implementation process will look like after a service agreement has been signed. Gartner says:
“Selecting an MDR service provider to obtain modern SOC services can be a challenging process that requires the appropriate planning and evaluation processes before, during and after an agreement. Gartner clients face several unique challenges when evaluating and implementing MDR services.”
An urgent need
The need for additional or enhanced threat monitoring creeps ever upward, thus the need for regular re-evaluation of your D&R capabilities. Rather than ramping up the evaluation and MDR engagement process at a faster pace each time out, taking the time to think through and document desired outcomes with key stakeholders will ultimately save your security organization headaches…and money. Gartner says:
“The process for scoping use cases and requirements, and assessing MDR service offerings, often includes a negotiation and evaluation exercise where a “best match” and “ideal partner” is identified. Prior to starting any outsourcing initiative, requirements need to be documented and ratified (and continuously updated post onboarding), or else the old adage of “garbage in, garbage out” is likely to be realized.”
Take the time
It can be a rigorous evaluation process when determining your organization’s capacity for effective D&R. If your team is stretched too thin, a managed services provider could help. For a deeper dive into the MDR evaluation process, check out the complimentary Gartner report.
Gartner, “Quick Answer: What Key Questions Should I Ask When Selecting an MDR Provider?” John Collins, Andrew Davies, Craig Lawson, 10 November 2021.
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
Rapid7 MDR was excited to participate in this inaugural evaluation, along with 16 other Managed Service providers. We battle adversaries on behalf of our customers every single day, but most of this work goes largely unseen. This evaluation was an opportunity to show a wider audience the early detection, accelerated action, and deep partnership engagement that Rapid7 MDR delivers to customers across the globe every day.
Rapid7 reported malicious activity across all 10 ATT&CK Evaluation steps
Rapid7 MDR reported 63 of the 74 total attacker ‘techniques’ within these steps, accurately describing the full scope and impact of the breach while maintaining the strong signal-to-noise ratio that everyone expects of Rapid7.
This evaluation offers visibility into a real-world engagement with Rapid7. What our team delivered to MITRE Engenuity wasn’t ‘special’ treatment, but rather a demonstration of the resources, experience, and technology we bring to bear for all customers as part of the unlimited incident response service included with Rapid7 MDR.
Here are other highlights:
Reliable, early detection: we stopped OilRig (a.k.a. APT34) at the starting line
The attack began in a familiar way: a phishing email was used to drop a malicious payload and establish persistence on the workstation of an unsuspecting user. With a foothold in the environment, the attacker performed discovery actions and dumped user credentials, before moving laterally across the organization and eventually collecting and exfiltrating sensitive data.
Rapid7 MDR identified the very first step in the attack, notifying MITRE about the download and execution of the initial malicious payload and providing recommended actions to contain the threat. Had this been a ‘real world’ customer incident, the attack would have stopped here.
Comprehensive coverage across kill chain
As the attack was allowed to continue, our team went on to identify and report to MITRE Engenuity all major steps of the compromise – from discovery and credential dumping to Web shell installation, data staging, data exfiltration, and cleanup.
Robust, actionable reporting
The evaluation also highlights the comprehensive reporting, robust communications, detailed timelines, and deep forensic investigation that Rapid7 MDR customers receive. At the conclusion of the engagement, we delivered a comprehensive 40 page incident report describing in detail the full scope and impact of the breach and attributed the activity to APT group OilRig, an Iran-linked hacking group known to target critical infrastructure.
MDR left the environment better than we found it
While containment was out of scope for this evaluation, you’ll see that Rapid7 provided detailed response and mitigation recommendations along the way. While other Managed Services put work back on the customer to figure out how to resolve incidents and harden their security to prevent similar incidents in the future, Rapid7 provides this guidance and partners with customers to ensure these recommendations are implemented. We provide an end-to-end detection and response program.
Finally, what the MITRE ATT&CK Evaluation doesn’t show you
What’s reported out here is just a slice of what’s possible with Rapid7 MDR.
While this evaluation was largely endpoint-focused, our customers get complete coverage: endpoints, network, users, cloud, and more. As the attack surface grows in complexity, you need a real MDR partner, scaling with your business, driving the end-to-end results, staying ahead of the most advanced attacks, working as a seamless extension of your team.
Our many differences, including integrated DFIR, add up.
Anyone involved in hiring security analysts in the last few years is likely painfully aware of the cybersecurity skills shortage – but the talent hasn’t “gone anywhere” so much as it’s been bouncing around all over the place, looking for the highest bidder and most impactful work environment. Particularly since the advent of the pandemic, more highly skilled cybersecurity talent has been able to take advantage of work-from-anywhere opportunities, as well as other factors like work/life balance, the desire to avoid negative office politics – and, of course, potentially higher wages elsewhere.
Retain where it counts
Money isn’t everything, but it’s a lot. An awful lot. That’s what it may seem like to an experienced analyst who’s been working in the security operations center (SOC) for long hours over years, who doesn’t feel like they can really take time off, and who perhaps has been on LinkedIn of late just to “see what’s out there.” Having casual conversations with a recruiter can quickly turn into a conversation with you, their manager, that begins, “I need to put in my two-week notice.”
There are simply companies out there that will pay more and hire away your talent faster than you can say “onboarding.” You can attempt to shore up some budget to retain talent, but if money isn’t just one prong of a larger mix to keep your best and brightest, they’ll slowly start to join the quiet-quitting club and look elsewhere.
The balance shouldn’t be an act
It’s true that life – especially as we become adults – becomes a delicate balancing act. But for companies pitching a great work/life balance to prospective cybersecurity talent, that pitch needs to be genuine. A 2021 Gartner survey saw 43% of respondents say that flexibility in work hours helped them achieve greater productivity. And if the attempt is to woo talent with longer, more illustrious resumes, that attempt should highlight a meaningful work/life balance that’s able to coexist with the company’s values and mission – not to mention one that fits in well with the team dynamic that talent is entering or helping to build.
After all, you’re asking potential employees to sit in the trenches with their peers, fending off threats from some of the most ruthless attackers and organizations in the world. That can sometimes be a dark place to spend your days. Thus, the pervading environment around that function should be one of positivity, camaraderie, inclusivity, and celebration.
The pandemic took work/life balance to another level, one in which companies were forced to adopt work-from-home measures at least semi-permanently. In that scenario, the employee gained the ability to demand a better balance. And that’s something that can’t be taken away, even in part. Because talent loves a good party – and they can always leave yours.
Burn(out) ban in effect
One of the major reasons talent might decide that the party at your SOC has come to an end? Burnout. Currently, around 71% of SOC analysts say they feel burned out on the job. Reasons for this may have nothing to do with the environment in your SOC shop or greater organization. Burnout could be the result of a seasonal uptick in incident-response activities (end-of-year or holiday retail activities come to mind) or in response to the latest emergent threat. However, it’s good to be vigilant of how talent churn might become a common occurrence and how you can institute a ban on burnout.
It takes a team: To build out a fully operational SOC and achieve something close to 24×7 coverage, it takes several people. So, if you’re placing the hopes of round-the-clock coverage on the shoulders of, say, six analysts, they’re likely to burn bright for a short period of time and then leave the party.
The same thing, over and over: Your workday expectations may be music to the ears of prospective talent: 9 to 5, and then you log off and go home. That kind of schedule can be great for work/life balance. But is it pretty much the same thing, every day, year in and year out? Is there a heavy amount of alert fatigue that could be offset by a more efficient solution? Are you leveraging automation to its fullest, so that your SOC doesn’t become full of expert talent spending their days doing mundane tasks?
Burnout may come back to bite you: Glassdoor… it’s a thing. And people will talk. Your SOC may have developed a reputation for burnout without you even realizing it. It’s called social media, and you can sink or succeed by it – especially if it isn’t just one former analyst on Glassdoor talking about your security organization in relation to burnout. What if you find out it’s 50 people over the span of five years? Sure, it’s actionable data, but by then it may be too late.
The soul of your SOC
Think about it from their point of view. What do your employees consider a positive work environment? What would constitute a brain-drain culture? Taking proactive measures like sending out a survey and soliciting anonymous responses is an easy way of taking the temperature of the culture.
And if burnout is becoming a real thing, maybe it’s time to think about a managed services partner who can take on some of the more mundane security tasks and free up your in-house talent to innovate.
The goal of a detection and response (D&R) program is to act as quickly as possible to identify and remove threats while minimizing any fallout. Many organizations have identified the need for D&R as a critical piece of their security program, but it’s often the hardest — and most costly — piece to implement and run.
As a result, D&R programs tend to suffer from common mistakes, and security teams often run into obstacles that hamper the value a solid program can deliver.
Recognizing this fact, our team of security experts at Rapid7 has put together a list of the top mistakes companies make in their D&R programs as well as tips to overcome or avoid them entirely.
1. Trying to analyze too much data
To have a successful and truly comprehensive D&R program, you should have complete visibility across your modern environment – from endpoints to users, cloud, network, and all other avenues attackers may enter. With all this visibility, you may think you need all the data you can get your hands on. The reality? Data “analysis paralysis” is real.
While data fuels detection and response, too much of it will leave you wading through thousands of false positives and alert noise, making it hard to focus on the needle in a haystack full of other needles. The more data, the harder it is to understand which of those needles are sharp and which are dull.
So it ends up being about collecting the right data without turning your program into an alert machine. It’s key to understand which event sources to connect to your SIEM or XDR platform and what information is the most relevant. Typically, you’re on the right path if you’re aligning your event sources with use cases. The most impactful event sources we usually see ingested are:
Endpoint agents (including start/stop processes)
DHCP
LDAP
DNS
Cloud services (O365, IIS, load balancers)
VPN
Firewall
Web proxy
Active Directory for user attribution
For even greater detail, throw on network sensors, IDS, deception technology, and other log types
At the end of the day, gaining visibility into your assets, understanding user behaviors, collecting system logs, and piecing it all together will help you build a clearer picture of your environment. But analyzing all that data can prove challenging, especially for larger-scale environments.
That’s where Managed Security Service Providers (MSSP) and Managed Detection and Response (MDR) providers can come in to offload that element to a 24×7 team of experts.
2. Not prioritizing risks and outcomes
Not all D&R programs will focus on the same objectives. Different companies have different risks. For example, healthcare providers and retail chains will likely deal with threats unique to their respective industries. Hospitals, in particular, are prime targets for ransomware. Something as simple as not having two-factor authentication in place could leave a privileged account susceptible to a brute-force attack, creating wide-open access to medical records. It’s not overstating to say that could ultimately make it more difficult to save lives.
Taking this into account, your D&R program should identify the risks and outcomes that will directly impact your business. One of the big mistakes companies make is trying to cover all the bases while ignoring more targeted, industry-specific threats.
As mentioned above, healthcare is a heavily targeted industry. Phishing attacks like credential harvesting are extremely common. As we should all know by now, it can be disastrous for even one employee to click a suspicious link or open an attachment in an email. In the healthcare sector, customer and patient data were leaked about 58% of the time, or in about 25 out of 43 incidents. Adversaries can now move laterally with greater ease, quickly escalating privileges and getting what they want faster. And when extortion is the name of the game, the goal is often to disrupt mission-critical business operations. This can cripple a hospital’s ability to run, holding data for ransom and attempting to tarnish a company’s reputation in the process.
3. Finding help in the wrong place
Building a modern security operations center (SOC) today requires significant investments. An internal 24×7 SOC operation essentially needs around a dozen security personnel, a comprehensive security playbook with best practices clearly defined and outlined, and a suite of security tools that all go toward providing 24/7 monitoring. Compound these requirements with the cybersecurity skills shortage, and not many organizations will be able to set up or manage an internal SOC, let alone helm a fully operational D&R program. In a recent Forrester Consulting Total Economic Impact™ (TEI) study commissioned by Rapid7, it was identified that Rapid7’s MDR service was able to prevent security teams from hiring five full-time analysts – each at an annual salary of at least $135,000.
There are two critical mistakes organizations make that can send D&R programs down the wrong path:
Choosing to go it all alone and set up your own SOC without the right people and expertise
Partnering with a provider that doesn’t understand your needs or can’t deliver on what they promise
Partnering with an MDR provider is an effective way to ramp up security monitoring capabilities and fill this gap. But first, it’s important to evaluate an MDR partner across the following criteria:
Headcount and expertise: How experienced are the MDR analysts? Does the provider offer alert triage and investigation as well as digital forensics and incident response (DFIR) expertise?
Technology: What level of visibility will you have across the environment? And what detection methods will be used to find threats?
Collaboration and partnership: What do daily/monthly service interactions look like? Is the provider simply focused on security operations, or will they also help you advance your maturity?
Threat hunting: Will they go beyond real-time threat monitoring and offer targeted, human-driven threat hunting for unknown threats?
Process and service expectations: How will they help you achieve rapid time-to-value?
Managed response and incident response (IR) expertise: How will they respond on your behalf, and what will they do if an incident becomes a breach?
Security orchestration, automation, and response (SOAR): Will they leverage SOAR to automate processes?
Pricing: Will they price their solution to ensure transparency, predictability, and value?
An extension of your team
Services like MDR can enable you to obtain 24/7, remotely delivered SOC capabilities when you have limited or no existing internal detection and response expertise or need to augment your existing security operations team.
The key questions and critical areas of consideration discussed above can help you find the MDR partner who will best serve your needs — one who will provide the necessary MDR capabilities that can serve your short- and long-term needs. After all, the most important thing is that your organization comes out the other side better protected in the face of today’s threats.
Looking for more key considerations and questions to ask on your D&R journey to keeping your business secure? Check out our 2022 MDR Buyer’s Guide that details everything you need to know about evaluating MDR solutions.
Cyber threats have risen to the #1 concern of CEOs, which means security teams — in the hot seat for years — are really feeling it now. Files and data live in the cloud. Work is hybrid or remote. There’s turmoil around the world. Cyberattacks are not just a distant boogieman – they’re here and happening every day.
As companies try to make sure their existing security infrastructure can keep up, they confront the skills gap, a 0% industry unemployment rate, and no room for mistakes. Managed Detection and Response (MDR) is having a moment.
According to a recent ESG study, MDR is one of the fastest growing areas of cybersecurity today. A whopping 85% of surveyed organizations currently use or plan to use managed services for their security operations. And 88% say they will increase their use of managed services in the next 1-2 years.
What’s driving this move to MDR? Let’s take a look at six main factors.
1. Focus
Augmenting an internal security team means internal security personnel can focus on more strategic security initiatives rather than day-to-day operational tasks. In fact, 55% of surveyed organizations want to focus their internal security teams on more strategic initiatives rather than spend time on daily basics, the ESG study found.
By partnering with an MDR provider, alert triaging and investigations are generally taken care of by the external team. Of course, your organization still has some things you’ll need to do – partnership is the name of the game. But by working with a MDR service, security teams suddenly have more time and bandwidth to work strategically.
2. Services
ESG reports that 52% of companies surveyed believe managed service providers can do a better job with security operations than they can.
What you would once have to train your detection and response team to do, MDR providers take over. That means they’re able to detect active attackers within your environment and contain threats. Analyze incidents and provide recommendations for remediation, and apply learnings from other environments they manage to your environment to make sure you’re protected from the latest attacker behaviors. Finally, good MDR providers are able to pivot into breach response if an attacker is live within your network.
To learn more about how to evaluate MDR providers on eight core capabilities, read the MDR Buyers Guide here.
3. Augmentation
About half of organizations (49%) believe a service provider can augment their security operations center (SOC) team with additional support.
Most companies that are able to build internal SOCs are generally well-funded, can afford roughly 10-12 full-time personnel, have a large array of security tools at their disposal, and have extensive processes already outlined. Sound doable? Great! If not, augmentation by way of an MDR provider is your tall glass of water.
Sign on with an MDR provider, get deployed, and your team is instantly extended. Benefits include time savings, cost savings, and experience level that most companies can’t afford to hire at scale.
4. Skills
No surprise, 42% of surveyed organizations in the ESG study believe they don’t have adequate skills for security operations in-house.
MDR is more than outsourcing 24x7x365 monitoring. It’s a partnership that helps you move towards a more secure stature with guidance and expertise.
This type of partnership allows teams to contextualize metrics and reports, get a better understanding of investigations that take place within their environment, and have someone to walk through processes should an attack take place. You also have an expert in your corner during CISO, board, or executive meetings.
5. Price
40% of surveyed organizations did a cost analysis and found that it would cost less to use a service provider than to do it themselves.
We won’t sugar-coat it – partnering with an MDR service provider is expensive. But so is building out an internal team that can actually monitor and investigate within an organization’s environment round the clock.
The cost of partnering with an MDR provider pales in comparison to the cost of employing 10-12 security personnel that operate an around-the-clock SOC, and it can offer ROI much more quickly.
Check out this recent Forrester study to learn more about cost-saving outcomes of partnering with Rapid7’s MDR team.
6. Staff
Finally, ESG tells us that 35% of surveyed organizations don’t have an adequately sized staff for security operations.
Even with unlimited budget to hire a full team, it would be an incredibly labor-intensive and time-consuming process. It would be nearly impossible for most organizations to accomplish. Not only is finding qualified candidates and hiring a huge pain point, but the resources needed to onboard and train staff often aren’t there.
Of course, all MDR services are not the same
Keep these three things in mind:
Forrester found Rapid7 MDR reduced breaches by 90%
Forrester found Rapid7 MDR delivered 549% ROI
In the event of a breach, Rapid7 MDR pivots to full-on digital forensics and incident response, no delay, no limits
Check out our full MDR Buyer’s Guide for 2022 here.
From one person to the next, the word “impact” may have wildly different connotations. Is the word being used in a positive or negative sense? For an understaffed security organization attempting to fend off attacks and plug vulnerabilities, the impact of all of that work is most likely negative: more work, less success to show for it, and more stress to take home.
That’s why Rapid7 commissioned Forrester Consulting to conduct a June 2022 Total Economic Impact™ (TEI) study to learn how our real MDR customers are seeing tangible impacts to their bottom line by partnering with Rapid7.
The study found that Rapid7’s SOC expertise – with XDR technology that generated improved visibility – enabled a composite organization using Rapid7 Managed Detection and Response (MDR) to:
Quickly extend its coverage with skilled headcount
Put formal processes in place for cyberattack detection and response
The analysis was conducted using a hypothetical composite organization created for the purposes of the study, with insights gleaned from four real-life MDR customers. This composite reflects a profile we see often: a small team of two security analysts tasked with protecting 1,800 employees and 2,100 assets.
The study concluded that partnering with Rapid7 MDR services experts enabled the composite organization to achieve end-to-end coverage and cut down on detection and response times. Impact like that can open the door to true progress.
Any MDR financial justification like this will come down to four main factors: return on investment (ROI), savings from building out your SOC team, the reduction in risk to your organization, and the time to see value/impact. Let’s break down these four key statistics from the study in more detail.
1. ROI
In the Forrester study, the composite organization – once partnered with Rapid7 – saw productivity gains accelerate efficiencies across alert investigation, response actions, and report creation. They were also protected with 24/7 eyes-on-glass and expert security support. Savings from security-team productivity gains totaled over $930,000 and Rapid7 MDR services in total delivered an ROI of 549% for the composite organization over the course of the three-year analysis. That kind of money can be reinvested to strengthen other parts of a security program and act as a profit driver for the business.
This greater overall visibility is powered by XDR capabilities that can customize protection to assess and block specific threats. Continuously analyzing activity in this way enables more targeted and prioritized containment actions that lead to better curation.
2. Hiring savings
In any sort of managerial capacity, the word “headcount” can have an exhausting connotation. Having to hire a skilled professional, onboard that person to the point they’re contributing in a meaningful way, and then do it all again to fill out perhaps multiple vacancies in pursuit of a productive SOC team – it’s a lot. And it sucks up time and valuable resources, which is perhaps the biggest advantage attackers have over a security organization in need.
Partnering with Rapid7 MDR afforded the composite organization:
Time savings for existing security team members
Avoided headcount and onboarding for potential new team members
Security-breach cost avoidance by extending the team with a dedicated MDR services provider
This led to total quantified benefits with a present value of $4.03 million over three years.
3. Potential benefit
The above stat is great, but you may be asking what sort of start-up costs did the composite organization incur? According to the Forrester study, for the composite organization, partnering with Rapid7 MDR meant spending around $620,000 over the course of three years. Digging into that number a bit more, the organization spread the investment into smaller yearly increments.
Compared to the costs of hiring multiple full-time employees (FTEs) who can do exactly what one needs them to do (and hopefully more), $620,000 quickly begins to look more attractive than what one might pay those FTEs over, say, five years. For a deeper dive into the actual purchasing process of MDR services, check out this handy MDR buyer’s guide.
4. Payback period
For the total three-year investment of just over $620,000, the composite organization experienced payback in less than three months! At the time of the investment in Rapid7 MDR, the composite organization had key objectives like improved visibility across the entire security environment, a complete security solution backed by the right expertise, and 24/7/365 coverage.
The chief information security officer at a healthcare firm said it took two members of their security team, each working four hours a day over the course of two weeks, to complete implementation. In some instances, Rapid7 MDR was able to detect and respond to incidents the first day the service was live.
A complete economic picture
When it comes to under-resourced teams, the economics boil down to a simple comparison: The costs for an MDR provider like Rapid7 versus a potential multiyear attempt to stretch an already-overloaded staff to investigate every alert and mitigate every threat.
Impact aside, a year of MDR service can often equate to the cost of one or two open headcounts. At that point, the economic benefits are the cherry on top. After all, it’s always easier (and more impactful) to instantly extend your team with expert headcount, saving time and resources in onboarding and bringing in experts ready to make an impact from day one. Bundle it all together and you’re building a business case for the potential to bring your organization greater expertise, significant cost avoidance, and positive ROI.
At the end of the day, Rapid7 MDR can give existing security specialists some much-needed breathing room while helping the business into a better overall competitive position. Put another way: More coverage. More money. More time. Less stress. You can read the entire Forrester Consulting TEI study to get the deep-dive from interviewed customers – along with the numbers and stories they shared – on Rapid7 MDR.
When a security operations center (SOC) is operating at a deficit, they increase the possibility of beach reductions. That is, the likelihood they won’t be able to travel to any beaches – or any vacation destinations whatsoever – anytime in the near future. That can lead to burnout, which can lead to security talent loss, which can lead to the entire business being incredibly vulnerable.
So now let’s talk about breach reduction. As in, the charter of any security team.
No team can investigate every alert, but forging a valuable partnership with a Managed Detection and Response (MDR) provider can provide a turnkey solution and near-immediate headcount extension to your SOC.
A June 2022 Total Economic Impact™ study by Forrester Consulting commissioned by Rapid7 found that Rapid7’s SOC expertise – with XDR technology that generated improved visibility – enabled a composite organization using Rapid7 MDR to reduce the likelihood of a breach by 90% in the first year of partnership
The analysis was conducted using a hypothetical composite organization created for the purposes of the study, with insights gleaned from four real-life MDR customers. This composite reflects a security team profile we see often: a small team of two security analysts tasked with protecting 1,800 employees and 2,100 assets. We at Rapid7 see this as a tall order, but it’s one that (unfortunately) represents the state of security operations today.
The study concluded that partnering with Rapid7 MDR services experts enabled the composite organization to achieve end-to-end coverage and cut down on detection and response times. Let’s break down how Rapid7 MDR helped security teams reduce the likelihood of breaches by 90%.
1. Complete visibility into security environments
OK, so extended detection and response (XDR) isn’t exactly apples-to-apples with X-ray technology, but it’s an apt metaphor. Greater visibility, after all, helps to improve your overall security risk posture, and customers interviewed for the TEI study said their organizations were more secure thanks in part to this improved visibility. Rapid7’s InsightIDR uses its XDR superpowers to unify data from all over and beyond your modern environment, so it’s easier than ever to see and respond to a transgression.
The Rapid7 MDR team’s expertise in cloud-scalable XDR technology enables stronger signal-to-noise capabilities, so you only become aware of alerts that matter and get the peace of mind that comes from knowing we’ve got you covered. After all, being aware of a breach is better than not being aware of one – or having a customer alert you to the existence of a breach, which could lead to a different kind of breach: the relationship.
2. Detect and respond literally all day, every day
According to the Forrester TEI study, interviewed organizations had outdated technology that was used by staff to manually investigate each alert prior to partnering with Rapid7 MDR. These organizations’ security teams lacked expertise, were understaffed, and lacked visibility – the perfect storm to miss security incidents. Interviewees said there would be no way for them to implement a 24×7 detection and response program on their own without using Rapid7 MDR. As an interviewed director of information security for a financial services company said, “If we didn’t acquire Rapid7 MDR, I would have had to do a lot more manual work, and it would have kept me from other tasks.”
With the modern proliferation of threats, the only thing to do is to have 24x7x365 coverage of your entire network. As referenced above, that can be expensive and near-impossible to maintain, unless you’re gaining leverage with the right MDR partner.
For example, with Rapid7 MDR, customers can opt in to Active Response, which enables our expert SOC analysts to respond to a validated threat on your behalf. The service also removes quite a few headaches, providing the flexibility to configure or cancel responses so that unauthorized quarantines occur less frequently (as they may with automated containment actions).
A customer SOC team will also have their own access to InsightIDR, the underlying technology of Rapid7’s MDR services. With the ability to also run your own investigations, your team will be able to see what we see, and follow along with the process. No black boxes or Wizard of Oz reenactments here.
These days we say that round-the-clock monitoring isn’t just important – it’s a must. A good MDR provider will be able to take on those duties, raising any incidents discovered and validated, day and night. In particular, Rapid7 utilizes a follow-the-sun methodology. This purpose-built monitoring engine leverages incident-response (IR) teams all over the world – Australia, Ireland, the United States, and more – to ensure awake and active detection and response experts are investigating security alerts and only notifying you when there’s an actual incident. From the SOC or remote locations, these IR teams can perform real-time log analysis, threat hunting, and alert validation, for any customer.
Redundancy is key here. Attackers never take a day off, but security professionals working 9 to 5 do. Whether it’s national holidays or vacation season, the majority of attacks occur around these specific times security experts might set their status to “away.”
3. Gain more freedom to focus their energy elsewhere
In the TEI study, Forrester found that Rapid7 MDR was able to provide security teams with greater information and curated alert detections, with the ability to block specific threats. MDR also improved response times to detections by providing teams with a security resource dedicated to security incidents that require any response. This meant internal security teams could focus on other priorities and business objectives without dealing with:
Alert triage and investigations
An interviewed senior cybersecurity analyst at a technology solutions firm said analysts previously spent three to four hours a day on alert management. Now, with MDR, that same process only takes 10 minutes of their time! That means the small team can focus on other elements of their security program knowing there’s another team of experts monitoring their environment around the clock.
Threat response
An interviewed CISO at a healthcare firm reported that their response could take up to two weeks prior to MDR. That’s a long time! With Rapid7 MDR, the security team was able to detect and respond in three days instead. The interviewed senior cybersecurity analyst from the technology solutions firm said response may have taken days prior to Rapid7 MDR, but now the security team can respond in 30 minutes! Greater efficiency (and faster response) meant lower likelihood of future breaches and lower impact of any breaches.
Post-detection reporting
The interviewed cybersecurity analyst from the technology solutions firm said that before Rapid7 MDR, it took an entire day to compile a quarterly executive summary and two monthly reports because it meant parsing through log data and finding the right information. Now with MDR, the report is created for them and their ability to create and deliver this to their team is more efficient. That means they can spend more time protecting the organization, not reporting.
4. $1.6 million in savings over 3 years
When an organization can reduce the likelihood of attacks by 90%, that can result in some serious ROI. How serious? The composite organization profiled in the Forrester study was able to see a breach cost avoidance – or savings – of $1.6 million over three years when partnered with Rapid7 MDR.
The composite organization saw an average of 2.5 incidents per year, with an average cost per security breach $654,846. This average cost included damage to brand equity and customer loyalty. We at Rapid7 are also cognizant of the mental toll those incidents take on the entire business, as well as the loss of forward momentum on any current initiatives – it all comes to a stop when a breach occurs and disrupts. This is why it’s critical to have a team spot threats early and respond to them quickly.
For the more advanced, large-scale breaches, sometimes it requires backup. Luckily, Rapid7 MDR now includes Unlimited IR to ensure major incidents are handled by our Digital Forensics and Incident Response (DFIR) experts. The merger of the MDR and IR Consulting teams accelerates a breach investigation by instantly pulling in senior-level IR experts to an emergency situation and ensuring the response is as efficient as possible.
Rapid7 MDR teams use our open-source DFIR tool, Velociraptor, the same tools and experience you’d receive if you called the breach hotline. These experts leverage multiple types of forensics (file-system, memory, and network), as well as attack intelligence and enhanced endpoint visibility to quickly organize and interpret data. Then? Kick the threat out and slam the door behind them.
Defense in depth
Beyond the need for agile detection and response abilities, preventive solutions are also of critical importance. At a device level, it is of course always prudent to ensure things like multifactor authentication (MFA), antivirus or NGAV (NextGen Antivirus) software, and/or an endpoint protection platform (EPP) – designed to detect suspicious behavior and stop attacks – are part of your preventive behavior.
At a more macro level (i.e., a SOC in the security organization of a Fortune 500 company independent of the Forrester study), the following preventive solutions should always be part of the mix:
Vulnerability Risk Management: It’s easier to detect and respond to the bad guys in the environment when you limit the number of doors they can walk through. Vulnerabilities are always at risk of exploitation. Managing that risk is what InsightVM was made to do. It helps to secure your entire attack surface with visibility and behavioral assessment of your network-wide assets, as well as analyzing business context so it can prioritize the most critical issues.
Cloud Security: It takes cloud-native to protect cloud-based. InsightCloudSec provides visibility of all of your cloud assets in one, user-friendly place. Get immediate risk assessment with full context across infrastructure, orchestration, workload, and data tiers.
Application Security: More complex apps means more security required. With the ability to crawl and assess these modern web apps, InsightAppSec returns fewer false positives via features like the Universal Translator and its ability to bring flexibility to the security testing process. Finding threats with Dynamic Application Security Testing (DAST) – using the same exploits that an attacker would – is one of the keys to stopping web application-based attacks.
Security Orchestration Automation and Response (SOAR): The composite organization from the Forrester study took advantage of Rapid7 MDR’s utilization of Active Response, Rapid7’s Security Orchestration, Automation, and Response (SOAR) technology, as well as skilled SOC experts to quickly respond to and remediate threats.
By incorporating preventive and responsive solutions, you’ll work less by working smarter. Which, oftentimes, means letting someone else take on key aspects of your program. You can read the entire Forrester TEI study to get the deep-dive from interviewed customers – along with the numbers and stories they shared – on Rapid7 MDR.
But what the study does not quantify is Rapid7’s commitment to partnering with our customers to improve their security maturity, providing expertise that drives returns for your detection and response program where and when you need it. Considering MDR but don’t know where to start? We put together an MDR Buyer’s Guide that includes priority questions to ask when you’re seeking the right partner.
In-house security organizations these days are operating at an extreme deficit. Skeleton crews are running entire security operations centers (SOCs). A constant barrage of alerts is making it difficult for these teams to detect and investigate every alert and stay ahead of today’s evolving threats. The odds are heavily in favor of the attacker.
But there is hope. Managed security service providers (MSSPs) – and more specifically, managed detection and response (MDR) providers – enable access to specialized detection and response expertise and headcount, bypassing the talent- and skill-gap challenges that plague the industry.
MDR offers a way for internal security teams to extend their capabilities in threat detection, alert triage, malware analysis, incident investigation, and response capabilities quickly and at scale. For under-resourced teams, MDR is a turnkey solution for a fully operational SOC at a fraction of the cost to build one out internally. How much, exactly?
A June 2022 Total Economic Impact™ study by Forrester Consulting commissioned by Rapid7 found that Rapid7’s “secret sauce” – a blend of extended detection and response (XDR) technology, improved visibility, and SOC expertise – enabled a composite Rapid7 MDR customer to capture an estimated 549% return on their investment (ROI) over three years and to see a payback for that investment in less than 3 months! That’s almost a 5.5x ROI!
The analysis was conducted using a hypothetical composite organization created for the purposes of the study, using insights gleaned from four real-life MDR customers. This composite reflects a security team profile we see often: a small team of two security professionals tasked with protecting 1,800 employees and 2,100 assets. A tall order, and one that (unfortunately) represents the state of security operations today.
The study concluded that Rapid7 MDR services experts integrate with an existing security organization to quickly cut down on detection and response times. Subsequently, the interviewed customers saw substantial returns from working alongside the MDR team as a trusted partner to mature their program.
Here are four key takeaways from the Forrester Consulting study.
Rapid7 MDR offered improved visibility through XDR technology
Detection can only be as good as the visibility the technology provides and what’s being monitored. In the words of an interviewed director of information security for a financial services company, “I didn’t have full visibility into the security activity of all devices across my enterprise. It was a ‘fingers-crossed’ [hope] that there isn’t something going on within my network.”
Luckily, MDR as a partner can ensure complete monitoring and visibility across the entire environment – comprehensive coverage to detect across all endpoints, user accounts, network traffic, deception technologies, the cloud, and more – offering a winning strategy.
In the study, Forrester found that Rapid7 MDR utilizes XDR capabilities to help customers see beyond the confines of a traditional security information and event management (SIEM) and endpoint detection and response (EDR) tools, with coverage across the entire modern environment.
Combined with the latest threat intelligence and machine learning to continuously analyze attacker activity, the MDR provider can help you anticipate that threat and form a more proactive response. That’s a winning strategy.
Rapid7 MDR saved time for security teams
Alerts can fire constantly. Each of them needs triaging and investigation. Every confirmed incident then needs a response plan, remediation, mitigation actions, and a post-incident report. The challenge is, all of this takes time.
With MDR, those alerts are handled without spending countless cycles from the customer’s internal teams. Investigation, response, and reporting are, too. This frees up the security team to focus on other aspects of their program.
Going from understaffed to capably staffed can be an incredible time saver. As a director of information security in financial services said to Forrester, “If we didn’t acquire MDR, I would have had to do a lot more manual work and it would have kept me from other tasks.”
The Forrester study concluded that Rapid7 MDR – by providing improved focus and outsourcing of detection and response activities – reduced the amount of time spent by:
87.5% on alert investigation
97.5% on response, remediation, and recovery
83.3% on research and reporting
Rapid7 MDR helped avoid the hefty costs of hiring security talent
The Gartner® 2021 SOC Model Guide report suggests that “by 2025, 33% of organizations that currently have internal security functions will attempt and fail to build an effective internal SOC due to resource constraints, such as lack of budget, expertise, and staffing.” This is partially because of the difficulty to hire and retain top detection and response talent.
Hiring a full SOC team is incredibly expensive. For example, the Gartner SOC Model Guide suggested an industry benchmark closer to “at least 10-12 personnel for 24/7 coverage,” with the Forrester TEI study placing one full-time employee (FTE) at $135,000 annually.
Because of this, many teams are turning to MDR to implement a hybrid-SOC model that integrates an MDR SOC alongside an internal SOC team. Gartner suggests, “By 2025, 90% of SOCs in the G2000 will use a hybrid model by outsourcing at least 50% of the operational workload.” This approach has certainly become the most optimal and economic option.
Partnering with an MDR provider is certainly one way to avoid prohibitive time and hiring costs. According to the Forrester Consulting study, Rapid7 was able to save the composite organization $1.5 million over the course of three years by avoiding the need to hire five full-time security analysts in order to achieve 24×7 coverage (in year 1). And those numbers might be low compared to other industry SOC FTE benchmarks.
Rapid7 MDR greatly reduced the risk of a security breach
There will always be new zero-days, new TTPs, and emerging threats that make it impossible to prevent (and stop) every breach. The Forrester Consulting Cost Of A Cybersecurity Breach Survey from 2020 Q4 estimated that an organization will have an average of 2.5 significant security breaches each year with an average cost of $654,846 per breach.
That’s where partnering with an MDR provider can help reduce that number. In fact, the Forrester study notes that Rapid7 MDR reduced the likelihood of a major security breach by 90% for the composite organization!
At Rapid7, some of our MDR capabilities that help prevent breaches from occurring are:
XDR technology to see complete visibility across your attack surface (with an ability for customers to have full access to InsightIDR for log search, data storage, reporting, and more)
24x7x365 monitoring of the environment from a global, follow-the-sun SOC team of detection and response experts
Proactive, hypothesis-driven threat hunts from human MDR analysts
Active Response to contain assets and users instantly when there’s a validated incident
What about the 10% of incidents that get through? We at Rapid7 offer an industry-first, unlimited Incident/Breach Response baked into our MDR service, leveraging our integrated Digital Forensics and Incident Response (DFIR) team to ensure we’re able to assist customers with any security incident, no matter how minor or major.
All of this is why a director of information security in financial services who was interviewed for the Forrester study said, “I’d say we’re 100% more prepared to handle a security incident with Rapid7 MDR.”
MDROI
Ultimately, the goal of the security department is to invest in technology and services that help protect the organization. But when that investment is able to positively impact the company’s bottom line, it’s a win-win.
It’s not just about alleviating some of the stress on the security team. It’s also about having access to that MDR provider’s technology, their library of advanced detection methodologies and resources, and the collaboration that can lead to strengthening your security posture.
You can read the entire Forrester TEI study to get the full breakdown on Rapid7 MDR alongside the numbers and stories from customers.
But what the study does not quantify is our commitment to partnering with our customers to improve their security maturity, providing expertise that drives returns for your detection and response program where and when you need it.
Considering MDR but don’t know where to start? We put together an MDR Buyer’s Guide that includes the questions to ask and what to look for to help the decision-making process.
Forrester Consulting Study, “The Total Economic Impact™ Of Rapid7 Managed Detection And Response (MDR)” commissioned by Rapid7.
The Gartner® 2021 SOC Model Guide, 19 October 2021, John Collins, Mitchell Schneider, Pete Shoard
Gartner® is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. All rights reserved.
When that breach happens, you’ll most likely need help. For Rapid7 MDR customers, we’re there for you when you need us, period. Our belief is that, if a breach is inevitable, then a logical, transparent, collaborative, and effective approach to response should be, too.
I’m not just talking about the table-stakes “response” to everyday security threats. I’m talking about digital forensics and world-class incident response for any incident – no matter if it’s a minor breach like a phishing email with an attached maldoc or a major targeted breach involving multiple endpoints compromised by an advanced attacker.
Protecting your environment is our shared responsibility. As long as you are willing and able to partner with us during and after the Incident Response process, we are here for you. Rapid7 does the DFIR heavy lift. You cooperate to eradicate the threat and work to improve your security posture as a result.
Unfortunately, that’s not how all of the market sees it.
How vendors typically provide DFIR
Some managed detection and response (MDR) vendors or managed security services providers (MSSPs) do understand that there’s an R in MDR. Typically, they’ll do a cursory investigation, validation and – if you’re lucky – some form of basic or automated response.
For most, that’s where the R stops. If they can’t handle an emergency breach response situation (or if you’re on your own without any DFIR on staff), you’ll wind up hiring a third-party incident response (IR) consulting service. This will be a service you’ve found, or one that’s required by your cyber insurance provider. Perhaps you planned ahead and pre-purchased an hourly IR retainer.
Either way, how you pay for IR determines your customer experience during “response.” It’s a model designed to maximize provider profits, not your outcomes.
At a glance
IR Consulting Services
IR Included in Managed Services
Scope
Unbounded
Limited to managed services in-scope environments
Time Limit
Capped by number of hours or number of incidents
Capped by number of hours or number of incidents
Expertise
Senior IR Consultants
Capped by number of hours or number of incidents
24×7 IR
No
Yes
Tooling
Often will deploy a separate tooling stack, without easy access to historical data
Existing tooling, utilizing historical data but potentially lacking in forensic capability
Time to Respond
Slower (limited by legal documents, SLAs, lack of familiarity in the customer environment, time for tool deployment)
Proactively purchased as a retainer or reactively on an hourly basis
Included in purchase, up to an arbitrarily defined limit
There’s a good reason DFIR experts are reserved for expensive consulting services engagements. They’re a rare breed.
Most MDR teams can’t afford to staff the same DFIR experts that answer the Breach Response hotline. Security vendors price, package, and deliver these services in a way to reserve their more experienced (and expensive) experts for IR consulting.
Either you purchase Managed Services and expensive IR consulting hours (and play intermediary between these two separate teams), or you settle for “Incident Response lite” from your Managed Services SOC team.
If this seems like a “lesser of two evils” approach with two unappealing options, it is.
The future of incident response has arrived
Over a year ago, Rapid7 merged our Incident Response Consulting Team with our MDR SOC to ensure all MDR customers receive the same high-caliber DFIR expertise as a core capability of our service – no Breach Response hotlines or retainer hours needed.
This single, integrated team of Detection and Response experts started working together to execute on our response mission: early detection and rapid, highly effective investigation, containment, and eradication of threats.
Our SOC analysts are experts on alert triage, tuning, and threat hunting. They have the most up-to-date knowledge of attackers’ current tactics, techniques, and procedures and are extremely well-versed in attacker behavior, isolating malicious activity and stopping it in its tracks. When a minor incident is detected, our SOC analysts begin incident investigation – root cause analysis, malware reverse engineering, malicous code deobfuscation, and more – and response immediately. If the scope becomes large and complex, we (literally) swivel our chair to tap our IR reinforcements on the shoulder.
Senior IR consultants are seasoned DFIR practitioners. They’re also the experts leading the response to major breaches, directing investigation, containment, and eradication activities while clearly communicating with stakeholders on the status, scope, and impact of the incident.
Both teams benefit. The managed services SOC team has access to a world class Incident Response team. And the expert incident response consultants have a global team of (also world class) security analysts trained to assist with forensic investigation and response around the clock (including monitoring the compromised environment for new attacker activity).
Most importantly, our MDR customers benefit. This reimagining of how we work together delivers seamless, effective incident response for all. When every second counts, an organization cannot afford the limited response of most MDR providers, or the delay and confusion that comes with engaging a separate IR vendor.
Grab a coffee, it’s major breach story time
Here’s a real-life example of how our integrated approach works.
In early January, a new MDR client was finishing the onboarding process by installing the Insight Agent on their devices. Almost immediately upon agent installation, the MDR team noticed critical alerts flowing into InsightIDR (our unified SIEM and XDR solution).
Our SOC analysts dug in and realized this wasn’t a typical attack. The detections indicated a potential major incident, consistent with attacker behavior for ransomware. SOC analysts immediately used Active Response to quarantine the affected assets and initiated our incident response process.
The investigation transitioned to the IR team within minutes, and a senior IR consultant (from the same team responsible for leading breach response for Rapid7’s off-the-street or retainer customers) took ownership of the incident response engagement.
After assessing the early information provided by the SOC, the IR consultant identified the highest-priority investigation and response actions, taking on some of these tasks directly and assigning other tasks to additional IR consultants and SOC analysts. The objective: teamwork and speed.
The SOC worked around the clock together with the IR team to search these systems and identify traces of malicious activity. The team used already-deployed tools, such as InsightIDR and Velociraptor (Rapid7’s open-source DFIR tool).
This major incident was remediated and closed within three days of the initial alert, stopping the installation of ransomware within the customer’s environment and cutting out days and even weeks of back-and-forth between the customer, the MDR SOC team, and a third-party Breach Response team.
Now, no limits and a customer experience you’ll love
The results speak for themselves. Not only does the embedded IR model enable each team to reach beyond its traditional boundaries, it brings faster and smoother outcomes to our customers.
And now we’re taking this a step further.
Previously, our MDR services included up to two “uncapped” (no limit on IR team time and resources) Remote Incident Response engagements per year. While this was more than enough for most customers (and highly unusual for an MDR provider), we realized that imposing any arbitrary limits on DFIR put unnecessary constraints on delivering on our core mission.
For this reason, we have removed the Remote Incident Response limits from our MDR service across all tiers. Rapid7 will now respond to ALL incidents within our MDR customers’ in-scope environments, regardless of incident scope and complexity, and bring all the necessary resources to bear to effectively investigate, contain and eradicate these threats.
Making these DFIR engagements – often reserved for breach response retainer customers – part of the core MDR service (not just providing basic response or including hours for a retainer) just raised the “best practices” bar for the industry.
It’s not quite unlimited, but it’s close. The way we see it, we’ll assist with the hard parts of DFIR, while you partner with us to eradicate the threat and implement corrective actions. That partnership is key: Implementing required remediation, mitigation, and corrective actions will help to reduce the likelihood of incident recurrence and improve your overall security posture.
After all, that’s what MDR is all about.
P.S.: If you’re a security analyst or incident responder, we’re hiring!
In addition to providing world-class breach response services to our MDR customers, this new approach makes Rapid7 a great place to work and develop new skills.
Our SOC analysts develop their breach response expertise by working shoulder-to-shoulder with our Incident Response team. And our IR team focuses on doing what they love – not filling out time cards and stressing over their “utilization” as consultants, but leading the response to complex, high-impact breaches and being there for our customers when they need us the most. Plus, with the support and backing of a global SOC, our IR team can actually sleep at night!
Despite the worldwide cybersecurity skills crisis and The Great Resignation sweeping the industry, Rapid7’s MDR team grew by 30% last year with only 5% voluntary analyst turnover – in line with our last three years.
Part of this exceptionally low turnover is due to:
Investment in continuing education, diversity, and employee retention benefits
A robust training program, clear career progression, the opportunity to level up skills by teaming with IR mentors, and flexibility for extra-curricular “passion project” work (to automate processes and improve aspects of MDR services)
Competitive pay, and a focus on making sure analysts are doing work they enjoy day in and day out with a healthy work-life balance (there’s no such thing as a “night shift” since we use a follow-the-sun SOC model)
If you’re a Security Analyst or Incident Responder looking for a new challenge, come join our herd. I think Jeremiah Dewey, VP of Rapid7’s Managed Services, said it best:
“Work doesn’t have to be a soul-sucking, boring march to each Friday. You can follow your passion, have fun in what you’re doing, and be successful in growing your career and growing as a human being.”
Even if a security team was given a blank check to spend whatever they wanted and hire however they wanted, it would still be a massive effort to build a detection and response (D&R) program tailored to that organization’s specific needs. Thankfully, the plethora of managed services options available can help with that problem.
But with multiple types of managed services providers out there, how do you know which type of services are right for your organization? How can you effectively interview providers, attempt to then construct a D&R suite with the right vendor, and simultaneously continue to fortify your security program against threats?
For an organization beginning the search for a managed services partner that can actually add value, there is some starter legwork that can be done. There are many approaches to managed services providers along the D&R vein, such as:
Managed Detection and Response (MDR)
Managed Endpoint Detection and Response (MEDR)
Managed Security Service Provider (MSSP)
That last one, MSSP, is a blanket term for a provider that can assist with many specialized services like outsourced Security Operations Center-as-a-service (SOCaaS), MDR, or management of security tools such as a security information and event management (SIEM), firewalls, vulnerability risk management, and more. Knowing all this, while looking for the right managed service it’s simply a fact that you’re going to talk to a lot of vendors. Each one of them can say they’ll help you boost security defenses – they’ll say they have great people, they use the best technology, and they have a process to ensure your success.
The challenge? Every vendor’s marketing material will begin to sound the same. What it really comes down to is determining which provider’s strategy is best suited for your program’s needs. Let’s take a closer look at these three types of managed services to help you decide the best fit for your organization.
MDR
An MDR provider works with a customer to gain visibility and complete coverage across the customer’s entire environment. This helps a security practitioner better see when and where malicious-looking activity may be taking place.
MDR providers help solve operational challenges by instantly becoming an extension of their customers’ teams – providing headcount and extending coverage to 24x7x365. An MDR partner can also provide expertise and technologies to help find attacker behavior quickly and stop it before it becomes a wider issue.
More and more companies are becoming the focus of targeted attacks – specific aggressions designed to infiltrate an individual organization’s defenses. An MDR provider becomes a partner in helping to identify a targeted threat (read: reputational threat), repair affected systems, and focus efforts into both taking down the threat and providing recommendations for making the affected system more secure in the future.
There are a lot of MDR providers that go beyond “throwing alerts over the fence” to let clients parse and triage themselves. These days more MDR providers are finding it worth their while – and their bottom lines – to become a more strategic partner to security organizations. They help further security initiatives, build cyber resilience, and work with clients to get deeper visibility in their threat landscapes by:
Providing post-incident investigational insights
Weeding out benign events and only reporting true positive threats
Providing tailored remediation and mitigation recommendations
Recognizing there is no perimeter for data as it’s rushing back and forth from endpoints to clouds and beyond
Relieving security teams of steep analytical analysis so more of the focus is on threat hunting, as parsing alerts is automatically incorporated into threat intelligence
Curating high-fidelity detections and actionable telemetry to create efficient responses
These are all great benefits in extending what is possible with D&R and being proactive about extinguishing threats. However, MDR providers incorporating XDR into their approaches can’t simply add the letter “X” into the list of services and call it a day. XDR must help the organization actually gain control and visibility across its entire attack surface, from the nearest endpoint(s) to compromised user accounts, network traffic, cloud sources, and more.
When folded into a cohesive strategy that places emphasis on more proactive efforts, products like InsightIDR can be that solution that takes in telemetry from these disparate sources, correlates the data, and provides greater context to a potential threat.
MEDR
MEDR is a flavor of MDR that’s aligned more as an add-on management service that sits on top of endpoint-protection technology deployment. While MEDR does provide benefits like gaining visibility across wherever agents are set up, the EDR-centric approach won’t show the full story of a threat and its scope; an agent will simply tell the service provider what it gathers from the endpoint.
Many breaches, however, do begin at the endpoint. Why? Attackers can easily bypass firewalls and all sorts of implemented security controls by compromising just one endpoint, such as a user’s laptop. From there, they can move throughout a network, scooping up valuable internal/external data and quickly ruining a company’s reputation in the process. Even if they’re quickly found, what have they gotten away with?
Thus, focusing on endpoints is important. That’s simply an indisputable fact. EDR-based services are powerful tools within a managed services program. They provide advantages like:
Prevention aspects with integrated endpoint prevention platform (EPP) agent capabilities, such as Antivirus (NGAV) and stopping malicious file execution
Detecting compromised endpoints earlier in the attack chain
File integrity monitoring (FIM) capabilities so your team is alerted on changes to specific files on a given endpoint (if you’re monitoring for yourself)
Focusing only on endpoints, however, does miss key network- and cloud-spanning analysis that can deliver important telemetry in the fight against potential threats. MEDR typically lacks the ability to analyze network-spanning data, user analytics, and compliance behaviors, glean actionable insights, and use them to effectively respond to an incident. So the downside comes with the engagement model. Some MEDR players will rely on the tech to do most of the heavy lifting. Prevention is there to stop the threat early.
But if the attacker gets past this point, the managed services provider might take automated actions to handle alerts using the EDR tool or, worse, pass that alert on to their client for them to manage the investigation and response efforts. (And if you think that automated EDR actions are great, you’re encouraged to read about the risks associated with taking automated response actions without human intervention.)
SOCaaS
SOCaaS. That’s a heavy acronym. But the concept of “security operations center-as-a-service” is trying to fill a heavy need of any modern company: the implementation and management of a strong and sound cybersecurity program. Any MSSP who offers a holistic SOCaaS option should be able to provide the bottom-line benefit of enabling security practitioners to focus time and energy on innovations in other parts of the business.
A team of experts who can proactively defend, respond to threats, and provide (hopefully) round-the-clock support on behalf of a customer is probably the closest definition to SOCaaS that’s been bandied about in recent years. They can be a virtual SOC for a company, serving as a tactical console to enable team members to perform day-to-day tasks. They’ll also help teams strategize amidst bigger, longer-term security trends. So, in what ways can SOCaaS providers act as that strategic detection-and-response center for security teams?
Advanced SIEM functionality – In the midst of potentially billions of security events each day, a SIEM can help to prioritize the ones that truly deserve follow-up. A good SOCaaS provider will contextualize a proper response plan by taking into account user- and attacker-behavior analytics, performance metrics, incident response, and endpoint detection.
The human element – In the incredibly competitive marketplace for today’s security talent, it can be a daunting task for company leadership to source, develop, and retain an entire SOC of capable personnel. This is particularly true in efforts to maintain diversity in cybersecurity hiring. For example, Forrester says that women currently make up just 24% of security professionals worldwide.
Established processes – It typically takes nothing less than an extremely sophisticated process framework – established over a long period of time and testing – to be able to accurately identify, prioritize, and remediate a potential threat. It can be an incredible benefit to a business to forgo having to build out their own SOC with key personnel that – even when assembled – must take the necessary trial-and-error time to be able to work together efficiently and respond to threats effectively.
D&R expertise – If the goal of engaging SOCaaS is not to augment an existing D&R program, then vetting the provider for their expertise in that area is incredibly important. It really comes down to what you’re looking to achieve; as mentioned above, a modern MDR provider will leverage multiple sources of telemetry to detect and respond to threats. But when fully outsourcing a SOC, it’s incumbent upon security personnel representing the customer to figure out how D&R expertise figures into the larger picture of outsourced SOC operations at the vendor organization.
Communications – Beyond anything at all to do with technology and security, a SOCaaS provider must have great communication skills. How will the provider present information – especially about a potentially dire threat that could affect the company, its reputation, and its bottom line – to their client’s customer and executive team? Is there a dedicated point-of-contact (POC) or a team with whom you’ll be regularly working and interfacing?
If this is looking like a menu from which security teams looking for managed services can choose, that’s because it is. However, in this context we’re discussing SOCaaS as a fully outsourced arm of a business. For whatever reason – the need for speed/growth in other parts of the business, lack of recruitment power for talented security practitioners, etc. – a business may simply wish to staff a security “skeleton crew” who interfaces with the SOCaaS provider and relies on that provider to run, monitor, manage, and support all of the functionalities.
Bottom line: Choose the managed security services partner that best fits your needs
If your security organization is considering a managed services provider, that means your team is most likely looking to offload tedious and/or technical operational tasks that your existing security team simply doesn’t have the hours in a day to manage. Or you might need some augmentation and expertise to help with round-the-clock coverage. It also means you’re ready to find a partner to provide deep analysis and actionable insights so you can find out:
What is going on, and…
Is it something the company should worry about?
After that, your specialized provider should be able to make recommendations on how to respond – or, better yet, take those actions on your behalf. Because at the end of the day, it all depends on the outcome(s) you’re looking to achieve. Turnkey D&R services while your team focuses on other important things? Simple endpoint monitoring from a traditional MSSP? Or, are you looking to farm out your SOC operations and let someone else deal with all things security, not just some things security?
For those looking for that more comprehensive solution targeted at strictly strengthening the D&R muscle, leveraging an MDR provider with XDR capabilities is the way to go.
It’s going to take some budget, sure. But most of the time that same budget is earmarked for a similar cost as one of an open headcount (depending on the size of the environment). The capital expenditure (CapEx) cost is relative – and oftentimes far more affordable – when compared to the ongoing operating expenses (OpEx) outlay it takes to hire, train, and build an in-house SOC program. Whichever outcome your team is focused on, managed services as a whole is an affordable way to help build a D&R program at scale.
Looking for even more analysis to help you make an informed managed services decision? Check out the 2022 MDR Buyer’s Guide from Rapid7, or contact us for more info.
Last summer, Rapid7 acquired IntSights and its advanced external threat intelligence solution (now Threat Command by Rapid7). Threat Command monitors hundreds of thousands of sources across the clear, deep, and dark web, identifying malicious actors and notifying customers of potential attacks against their organizations.
The reason for the acquisition? With these external intelligence sources built into InsightIDR, its breadth of high-fidelity, low-noise detections would be unmatched.
Detections have been a Rapid7 thing since the start.
In an industry focused on ingesting data – and placing the burden on security teams to write their own detections – we went another way. We went detections first, delivering the most robust set of actionable detections out of the box.
Today, our detections library includes threat intelligence from our open-source communities, advanced attack surface mapping, proprietary machine learning, research projects, real-world follow-the-sun security operations center (SOC) experience, and 2.1+ trillion weekly security events observed across our detection and response (D&R) platform.
Now, Threat Command’s threat intelligence platform (TIP) content is integrated with our leading detection and response products and services. You get earlier threat identification and faster remediation.
MDR and InsightIDR customers have an even larger, expertly curated library
Right now, Rapid7 customers can find a lot more needles in haystacks. And we’ve made sure you can spot them quickly, easily, and reliably.
Our Threat Intelligence and Detection Engineering Team (TIDE) has done its work developing signatures and analytic detections for existing and emerging threats. TIDE analysts continuously provide InsightIDR users and managed detection and response (MDR) SOC analysts with the surrounding context needed to defend against threats with new detection mechanisms for vulnerability exploits and attack campaigns.
The detections are for newcomers as well as familiar names like the notorious Russian hacking group EvilCorp. As always, detections ensure coverage for various indicators of compromise (IOCs) that they and other attackers use in the wild.
Think of us as your research and execution team: As additional IOCs are added to the Rapid7 Threat Command Threat Library, they are automatically tested and applied to your logs to create alerts when identified.
What’s better and better, by the numbers
Now, InsightIDR has your back with:
138 threats powered by Threat Command’s Threat Library
414 detection rules powered by dynamic IOC feeds
Monitoring for all IOCs associated with each threat actor is automatic as they are added to the Threat Library
The mission is always to deliver more actionable alerts (with recommendations) and to reduce noise. So our TIDE Team tests IOCs and disables those we find to be unsuitable for alerting.
And this is just the beginning: All detections improve in fidelity over time as our MDR analysts inform the threat intelligence team of rule suppressions to provide a tailored approach for customers, add granularity, reduce noise, and avoid recurrency. And as Threat Command adds IOCs, they’ll turn into meticulous, out-of-the-box detections – whether you use InsightIDR, rely on our MDR SOC analysts, or collaborate with us to keep your environment secure.
If you’re an MDR customer or just considering it, here are other numbers to know:
With a 95% 4-year analyst retention rate, Rapid7 is an employer of choice during the cybersecurity staffing crisis and The Great Resignation
Our team of 24/7/365 global SOC analysts are proven threat hunters and DFIR experts
Together, the staff has a combined 500+ security certifications
Now, with even more detections, the strongest back-end system capturing threats as they evolve, and unmatched knowledge in the field, you can level up your D&R program with Rapid7 InsightIDR — or a partnership with the best-in-breed MDR analyst teams out there.
You’re tasked with protecting your environment, and you’ve invested significant time and resources into deploying and configuring your tools — but how do you know if the security controls you’ve put into place are effective? The challenge continues to grow as attacker tactics, techniques, and procedures (TTPs) constantly evolve. In today’s landscape, a security breach is nearly inevitable.
Amid an ever-changing threat landscape, do you have confidence your tools are able to immediately detect threats when they occur? And more importantly, does your team know how to effectively respond to stop the attack, and do it fast?
While we don’t have a crystal ball to offer, we can help make sure your detection and response plan holds up against a breach.
Say hello to Rapid7’s newest incident response service: the Detection and Response Workshop.
Put your safeguards to the test with a guided attack simulation
The Detection and Response Workshop is a guided exercise led by Rapid7’s digital forensics and incident response (DFIR) experts to confirm that your team can quickly detect threats and evaluate your response procedures against a simulated attack within your environment.
This workshop isn’t a Tabletop Exercise (TTX), an IR Planning engagement, or a Purple Team exercise. We’ll pit your organization’s defenders against the latest attack campaigns, within the tools they use on a daily basis, to test your ability to respond when an incident happens under live conditions, without your company’s reputation at stake.
Each Workshop simulation is tailored to your specific needs and mapped to the MITRE ATT&CK Framework. Throughout the Workshop, our experts make recommendations to help strengthen your program – from existing configurations of tools, products, and devices to analysis processes and documentation.
The workshop itself is hands-on and doesn’t require current use of a Rapid7 product. Any security team can utilize this new service to understand what TTPs an adversary may use against them and make sure their program detects and responds accordingly.
Your team will leave the multi-day workshop feeling confident that you have an understanding of where and how to strengthen your existing IR process and detection and response program. You’ll receive a detailed report of the workshop, including our written assessment and recommendations to build resilience into your response program.
Rapid7 Incident Response consulting services
Security is the core of our business, and IR plays a huge role in the security landscape. Our team of DFIR experts — the same experts that respond to incidents for all 1,200+ of our MDR customers — have decades of experience under their belt that they utilize to analyze your security fit-up from all angles. Our team is complete with experts in threat analysis, forensics, and malware analysis, as well as a deep understanding of industry-leading technologies.
Knowing where your program stands is a crucial part of enhancing it, and our IR team has built specialized services to help your team build resiliency at each stage in the process. We now offer a full Incident Response Service Curriculum, allowing teams to engage in a single course for their IR goals or register for the entire curriculum.
From planning to full attack simulations, your team can level up its skills with tailored guidance and coaching through each course:
Course 101: Incident Response Program Development
Course 201: Tabletop Exercise (TTX)
Course 301: Detection & Response Workshop
Course 401: Purple Team Exercise
No matter what stage your team is in building your incident response program, our experts are able to help analyze and provide recommendations for improvement.
The Detection & Response Workshop is available now for all security teams. To learn more, talk to a Rapid7 sales representative by filling out this form today.
Cyberthreats are now the No. 1 source of stress among CEOs, with 71% of respondents to PwC’s 2021 CEO Study reporting they are “extremely concerned” about the issue. At the same time, the cybersecurity skills gap continues to grow, with 95% of security pros saying the shortage of talent in their field hasn’t improved. So while the seriousness of the problem has increased, the availability of in-house resources to adequately address it has not — particularly when it comes to finding talent with the specialized skills in detection and response.
These trends have led many organizations to partner with managed detection and response (MDR) service providers to address resource and skills gap challenges and build a strong competency to find and stop attackers in their environment.
By instantly extending your internal team’s capabilities with detection and response experts, MDR services can provide you the confidence that your environment is protected at all times.
And for those that struggle to build a fully staffed security operations center (SOC) with the right headcount, technology, and process to be effective — all while staying under a tight budget — MDR may provide a cost-effective method to quickly stand up a complete detection and response program.
In our 2022 MDR Buyer’s Guide, we outline the core capabilities that provide the foundation for evaluating MDR vendors. They include:
Digital forensics and incident/breach response (DFIR)
Automation
A simple, predictable pricing
SLA delivery standards
If you’re looking for a deep dive into each of these criteria, download the full guide!
In this post, we’ll streamline the discussion into 4 big-picture questions, providing you a quick-reference guide to use in the early stages of your MDR vendor selection journey, as you begin to identify your needs and narrow down your options.
1. Is this partner simply an outsourced SOC, or can they help us advance our overall security program?
An MDR provider is not just a vendor but a partner — and people are the foundation of any great partnership. You’ll want to ensure you ask the right questions regarding who will be servicing your organization and how, including:
How many MDR SOC analysts will be monitoring my environment 24×7?
What’s the experience level of the MDR SOC team we’ll be working with?
What is the average tenure and attrition rate of the team?
Will your partner suggest operational and strategic guidance to improve your program based on real-time threat monitoring and proactive threat hunting?
Is there someone who will be our Security Advisor that we meet with regularly?
What is the customer experience like when I need to connect with the MDR team?
2. Do they have the right tools at their disposal?
MDR combines real-time threat monitoring across the most critical elements of your IT environment — endpoints, network, users, and cloud sources. And in case you haven’t noticed, those environments are becoming increasingly complex. The cloud is enabling rapid scaling, and threats can come from virtually anywhere.
To carry out their duties well in this context, MDR providers need to be using the right XDR technology for complete visibility and coverage. Here are some questions to ask that can help you get a better sense of how the MDR vendors you’re considering approach their technology implementation — and how that affects you as the customer.
Is the MDR SOC team using multiple third-party solutions, or a technology built by an embedded engineering team?
How do you detect threats that bypass preventative controls?
Will I have full access to your back-end technology? If not, will you provide self-service log search and dashboards?
Does the SOC perform proactive threat hunts on top of the real-time detections?
Will we have the ability to add SOAR automation capabilities to expedite the remediation process?
3. Can they pair insight with action?
The last thing you want to hear from an MDR provider is, “Hey, we found this threat — now you have to go fix it.” The vendors you’re considering should have a managed response approach to effectively curb attacks after detection.
To understand when and how vendors will respond to threats they detect, start with these key questions:
What types of managed response actions will the MDR SOC advisors take?
In what instances will the MDR service take response action on our behalf?
Will I have the opportunity to deny the containment response if I don’t want the SOC team to take action?
4. Does the service scale to our needs and budget?
Even if an MDR vendor sounds great on paper across all of these points, that doesn’t necessarily mean they’re right for you. After all, you wouldn’t buy a two-seater car as your primary vehicle for a family of four. It’s critical to evaluate your MDR provider on the axes of your program maturity and desired security outcomes — both as it is now and for your goals for the future. Here are a few questions that will help you get a sense of whether an MDR vendor’s service and pricing structure fits your organization’s requirements.
How is the MDR service priced?
In the event of a breach, does MDR include DFIR as you’d get if you had an incident response retainer?
Are there data allotment or retention limitations?
What is your mean time to detect (MTTD) and mean time to respond (MTTR)?
These kinds of questions should help point you in the right direction in your initial conversations with potential MDR vendors. As you begin to make more fine-tuned decisions, you’ll want to have a few more detailed questions to ask — which means understanding the ins and outs of the MDR landscape a little more fully.
Check out our full MDR Buyer’s Guide for 2022 to help you navigate your choices with confidence and clarity.
MITRE ATT&CK is considered by practitioners and the analyst community to be the most comprehensive framework of cybersecurity attacks and mitigation techniques available today. MITRE helps the security industry speak the same language and stick to a well-known, common framework.
To get more details on MITRE’s ATT&CK Matrix for Enterprise and its impact, I spoke with 3 members of Rapid7’s Managed Detection and Response team who have firsthand experience working with this framework every day — read our conversation below!
Laying some groundwork here, what are your thoughts on the MITRE ATT&CK framework?
John Fenninger, Manager of Rapid7’s Detection and Response Services, kicked us off by sharing his perspective:
“MITRE ATT&CK is an incredibly valuable framework for both vendors and customers. From things like compliance to more immediate needs like investigating an ongoing attack, MITRE makes it easy to see specific techniques that customers may not have heard of and helps think of tactical moves customers can protect against. With InsightIDR specifically, we align our detections to MITRE to give both our MDR SOC analysts and customers visibility into how far along a threat is on the ATT&CK chain.”
Rapid7 is not only a consumer of the MITRE ATT&CK Framework but an active contributor as well — in 2020, Rapid7 Incident Response Consultant Ted Samuels made a contribution to MITRE around a discovery for group policy objects that is now in the latest version of the ATT&CK framework.
Can you share your perspective on how the MITRE framework is used, and by who?
When it comes to leveraging the MITRE ATT&CK framework, there are 2 key audiences to consider, says Rapid7’s Senior Detection & Response Analyst, Vidya Tambe:
“There are 2 main categories of users — people who write detections and people who do the analysis of the detections, and the MITRE framework is important for both. From the analyst side, we want to know what stage of attack each alert is at, and based on where the alert falls, we know how critical an incident is. With MITRE, we can track how an attacker got to where they are and what kind of escalations they did — overall, it helps us back-track to see what they were able to compromise.
“From the detection writing standpoint, we want to stop attacks before they get too far into someone’s environment. Attacker techniques are always evolving, and while we aim to write detections for all the phases, a primary focus is to try and write detections early on to stop attackers as early in the ATT&CK chain as possible.”
What advice do you have for security teams when it comes to leveraging the MITRE framework to drive successful detection and response?
Rapid7 Detection and Response Analyst Carlo Anez Mazurco shared some advice for teams when it comes to using the MITRE framework at their organization:
“The MITRE Framework allows us to build a threat-informed defense. It shows us the 3 main areas that we need to focus on for data collection, data analysis, and expansion of detections. For teams to successfully utilize the MITRE framework, they need visibility into the following data sources at a minimum:
Process and process command line monitoring can be collected via Sysmon, Windows Event Logs, and many EDR platforms
File and registry monitoring is also often collected by Sysmon, Windows Event Logs, and many EDR platforms
Authentication logs collected from the domain controller
Packet capture, especially east/west capture, such as those collected between hosts and enclaves in your network
“Teams need a platform like InsightIDR, Rapid7’s extended detection and response solution, where the data from all of these sources can be ingested. Whatever platform or tool teams choose to use for this data ingestion should include MITRE mappings to attacker behaviors to understand what attackers are trying to do inside our environment at each stage, the TTPs (Tactics, Techniques, Procedures) of each threat actor should be documented in each alert — InsightIDR maps its detections to the MITRE framework to do just this for users.”
You mentioned InsightIDR has MITRE mapping — can you dig a little more into how this impacts customers?
“Our InsightIDR platform helps our customers collect all the necessary data sources,” Carlo continued. “That includes process and process command line monitoring via our endpoint Insight Agent, as well as file monitoring. Plus, authentication logs are collected from domain controllers and also via the Insight Agent, and network flow inside the environment can be gathered through our Insight Network Sensor.
“Our ABA and UBA detections are mapped to the MITRE framework to show our customers which TTPs are the most commonly used by threat actors in their environment, and it gives an insight into the attack patterns in real time. You can see an example of this in one of our past Rapid7 Threat Reports here.
“Additionally, our Rapid7 Threat Intelligence team is always developing new threat detections based on the threat intelligence feeds and public repositories of attacker behaviors. These new detections are mapped to the TTPs inside the MITRE framework and pushed out to all Rapid7 customers.”
We also recently released a new view of Detection Rules in InsightIDR where all detections are mapped to the MITRE ATT&CK Framework, and users can see associated MITRE tactics, techniques, and sub-techniques for detections while performing an investigation.
Interested in learning more?
As you can see, we really value the MITRE ATT&CK framework here at Rapid7. With InsightIDR your detections are vetted by a team of professional SOC analysts and mapped to MITRE to take the guessing game of what an attacker might do next.
If you’re looking to hear more from us on MITRE, watch a quick 3-minute rundown on the framework here.
This post also includes contributions from Reese Lewis, Andrew Christian, and Seth Lazarus.
Rapid7’s Managed Detection and Response (MDR) team leverages specialized toolsets, malware analysis, tradecraft, and collaboration with our colleagues on the Threat Intelligence and Detection Engineering (TIDE) team to detect and remediate threats.
Recently, we identified a malware campaign whose payload installs itself as a Windows application after delivery via a browser ad service and bypasses User Account Control (UAC) by abusing a Windows environment variable and a native scheduled task to ensure it persistently executes with elevated privileges. The malware is classified as a stealer, which intends to steal sensitive data from an infected asset (such as browser credentials and cryptocurrency), prevent browser updates, and allow for arbitrary command execution.
Detection
The MDR SOC first became aware of this malware campaign upon analysis of “UAC Bypass – Disk Cleanup Utility” and “Suspicious Process – TaskKill Multiple Times” alerts (authored by Rapid7’s TIDE team) within Rapid7’s InsightIDR platform.
As the “UAC Bypass – Disk Cleanup Utility” name implies, the alert identified a possible UAC bypass using the Disk Cleanup utility due to a vulnerability in some versions of Windows 10 that allows a native scheduled task to execute arbitrary code by modifying the content of an environment variable. Specifically, the alert detected a PowerShell command spawned by a suspicious executable named HoxLuSfo.exe. We determined that HoxLuSfo.exe was spawned by sihost.exe, a background process that launches and maintains the Windows action and notification centers.
Figure 1: PowerShell command identified by Rapid7’s MDR on infected assets
We determined the purpose of the PowerShell command was, after sleeping, to attempt to perform a Disk Cleanup Utility UAC bypass. The command works because, on some Windows systems, it is possible for the Disk Cleanup Utility to run via the native scheduled task “SilentCleanup” that, when triggered, executes the following command with elevated privileges:
The PowerShell command exploited the use of the environment variable %windir% in the path specified in the “SilentCleanup” scheduled task by altering the value set for the environment variable %windir%. Specifically, the PowerShell command deleted the existing %windir% environment variable and replaced it with a new %windir% environment variable set to:
%LOCALAPPDATA%\Microsoft\OneDrive\setup\st.exe REM
The environment variable replacement therefore configured the scheduled task “SilentCleanup” to execute the following command whenever the task “SilentCleanup” was triggered:
The binary st.exe was a copied version of HoxLuSfo.exe from the file path C:\Program Files\WindowsApps\3b76099d-e6e0-4e86-bed1-100cc5fa699f_113.0.2.0_neutral__7afzw0tp1da5e\HoxLuSfo\.
The trailing “REM” at the end of the Registry entry commented out the rest of the native command for the “SilentCleanup” scheduled task, effectively configuring the task to execute:
%LOCALAPPDATA%\Microsoft\OneDrive\setup\st.exe
After making the changes to the %windir% environment variable, the PowerShell command ran the “SilentCleanup” scheduled task, thereby hijacking the “SilentCleanup” scheduled task to run st.exe with elevated privileges.
The alert for “Suspicious Process – TaskKill Multiple Times” later detected st.exe spawning multiple commands attempting to kill any process named Google*, MicrosoftEdge*, or setu*.
Analysis of HoxLuSfo.exe
Rapid7’s MDR could not remotely acquire the files HoxLuSfo.exe and st.exe from the infected assets because they were no longer present at the time of the investigation. However, we obtained a copy of the executable from VirusTotal based on its MD5 hash, 1cc0536ae396eba7fbde9f35dc2fc8e3.
Figure 2: Overview of HoxLuSfo.exe (originally named TorE.exe) within dnSpy and its partially obfuscated contents.
Rapid7’s MDR concluded that HoxLuSfo.exe had the following characteristics and behaviors:
32-bit Microsoft Visual Studio .NET executable containing obfuscated code
Originally named TorE.exe
At the time of writing, only 10 antivirus solutions detected HoxLuSfo.exe as malicious
Figure 3: Low detection rate for HoxLuSfo.exe on VirusTotal
Fingerprints the infected asset
Drops and leverages a 32-bit Microsoft Visual Studio .NET DLL, JiLuT64.dll (MD5: 14ff402962ad21b78ae0b4c43cd1f194), which is an Agile .NET obfuscator signed by SecureTeam Software Ltd, likely to (de)obfuscate contents
Modifies the hosts file on the infected asset to prevent correct resolution of common browser update URLs to prevent browser updates
Figure 4: Modifications made to the hosts file on infected assets
Enumerates installed browsers and steals credentials from installed browsers
Kills processes named Google*, MicrosoftEdge*, setu*
Contains functionality to steal cryptocurrency
Contains functionality for the execution of arbitrary commands on the infected asset
Figure 5: Sample of the functionality within HoxLuSfo.exe to execute arbitrary commands
Communicates with s1.cleancrack[.]tech and s4.cleancrack[.]tech (both of which resolve to 172.67.187[.]162 and 104.21.92[.]68 at the time of analysis) via AES-encrypted messages with a key of e84ad660c4721ae0e84ad660c4721ae0. The encryption scheme employed appears to be reused code from here.
Has a PDB path of E:\msix\ChromeRceADMIN4CB\TorE\obj\Release\TorE.pdb.
Rapid7’s MDR interacted with s4.cleancrack[.]tech and discovered what appears to be a login portal for the attacker to access stolen data.
Figure 6: Login page hosted at hXXps://s4.cleancrack[.]tech/login
Source of infection
Rapid7’s MDR observed the execution of chrome.exe just prior to HoxLuSfo.exe spawning the PowerShell command we detected with our alert.
In one of our investigations, our analysis of the user’s Chrome browser history file showed redirects to suspicious domains before initial infection: hXXps://getredd[.]biz/ → hXXps://eu.postsupport[.]net → hXXp://updateslives[.]com/
In another investigation, DNS logs showed a redirect chain that followed a similar pattern: hXXps://getblackk[.]biz/ → hXXps://eu.postsupport[.]net → hXXp://updateslives[.]com/ → hXXps://chromesupdate[.]com
In the first investigation, the user’s Chrome profile revealed that the site permission settings for a suspicious domain, birchlerarroyo[.]com, were altered just prior to the redirects. Specifically, the user granted permission to the site hosted at birchlerarroyo[.]com to send notifications to the user.
Figure 7: Notifications enabled for birchlerarroyo[.]com within the user’s site settings of Chrome
Rapid7’s MDR visited the website hosted at birchlerarroyo[.]com and found that the website presented a browser notification requesting permission to show notifications to the user.
Figure 8: Website hosted at birchlerarroyo[.]com requesting permission to show notifications to the user
We suspect that the website hosted at birchlerarroyo[.]com was compromised, as its source code contained a reference to a suspicious JavaScript file hosted at fastred[.]biz:
Figure 9: Suspicious JavaScript file found within the source code of the website hosted at birchlerarroyo[.]com
We determined that the JavaScript file hosted at fastred[.]biz was responsible for the notification observed at birchlerarroyo[.]com via the code in Figure 10.
Figure 10: Partial contents of the JavaScript file hosted at fastred[.]biz
Pivoting off of the string “Код RedPush” within the source code of birchlerarroyo[.]com (see highlighted lines in Figure 9), as well as the workerName and applicationServerKey settings within the JavaScript file in Figure 10, Rapid7’s MDR discovered additional websites containing similar source code: ostoday[.]com and magnetline[.]ru.
Rapid7’s MDR analyzed the websites hosted at each of birchlerarroyo[.]com, ostoday[.]com, and magnetline[.]ru and found that each:
Displayed the same type of browser notification shown in Figure 8
Was built using WordPress and employed the same WordPress plugin, “WP Rocket”
Had source code that referred to similar Javascript files hosted at either fastred[.]biz or clickmatters[.]biz and the JavaScript files had the same applicationServerKey: BIbjCoVklTIiXYjv3Z5WS9oemREJPCOFVHwpAxQphYoA5FOTzG-xOq6GiK31R-NF--qzgT3_C2jurmRX_N6nY4g
Figure 11: Partial contents of the JavaScript file hosted at clickmatters[.]biz. The unicode in the “text” key decodes to “Нажмите \”Разрешить\”, чтобы получать уведомления”, which translates to “Click \ “Allow \” to receive notifications”.
Had source code that contained a similar rbConfig parameter referencing takiparkrb[.]site and a varying rotator value
Figure 12: Example rbConfig parameter found in website source code
Had source code that contained references to either “Код RedPush” (translates to “Redpush code”), “Код РБ” (translates to “CodeRB”), or “Код нативного ПУШа RB” (translates to “Native PUSH code RB”)
Pivoting off of the similar strings of “CodeRB” and “Redpush” within source code led to other findings.
First, Rapid7’s MDR discovered an advertising business, RedPush (see redpush[.]biz). RedPush provides its customers with advertisement code to host on customers’ websites. The code produces pop-up notifications to allow for advertisements to be pushed to users browsing the customers’ websites. RedPush’s customers make a profit based on the number of advertisement clicks generated from their websites that contain RedPush’s code.
Figure 13: Summary of RedPush’s ad delivery model via push notifications
Second, Rapid7’s MDR discovered a publication by Malwaretips describing a browser pop-up malware family known as Redpush. Upon visiting a website compromised with Redpush code, the code presents a browser notification requesting permission to send notifications to the user. After the user grants permission, the compromised site appears to gain the ability to push toast notifications, which could range from spam advertisements to notifications for malicious fake software updates. Similar publications by McAfee here and here describe that threat actors have recently been employing toast notifications that advertise fake software updates to trick users into installing malicious Windows applications.
Rapid7’s MDR could not reproduce a push of a malicious software after visiting the compromised website at birchlerarroyo[.]com, possibly for several reasons:
Notification-enabled sites may send notifications at varying frequencies, as explained here, and varying times of day.
Malicious packages are known to be selectively pushed to users based on geolocation, as explained here. (Note: Rapid7’s MDR interacted with the website using IP addresses having varying geolocations in North America and Europe.)
The malware was no longer being served at the time of investigation.
However, the malware delivery techniques described by Malwaretips and McAfee were likely employed to trick the users in our investigations into installing the malware while they were browsing the Internet. As explained in the “Forensic analysis” section, in one of our investigations, there was evidence of an initial toast notification, a fake update masquerade, and installation of a malicious Windows application. Additionally, the grandparent process of the PowerShell command we detected, sihost.exe, indicated to us that the malware may have leveraged the Windows Notification Center during the infection chain.
Forensic analysis
Analysis of the User’s Chrome profile and Microsoft-Windows-PushNotifications-Platform Windows Event Logs suggests that upon the user enabling notifications to be sent from the compromised site at birchlerarroyo[.]com, the user was presented with and cleared a toast notification. We could not determine what the contents of the toast notification were based on available evidence.
Figure 14: Windows Event Log for the user clearing a toast notification to proceed with the malware’s infection chain
Based on our analysis of timestamp evidence, the user was likely directed to each of getredd[.]biz, postsupport[.]net, and updateslives[.]com after clicking the toast notification, and presented a fake update webpage.
Similar to the infection mechanism described by McAfee, the installation path of the malware on disk within C:\Program Files\WindowsApps\ suggests that the users were tricked into installing a malicious Windows application. The Microsoft-Windows-AppXDeploymentServerOperational and Microsoft-Windows-AppxPackagingOperational Windows Event logs contained suspicious entries confirming installation of the malware as a Windows application, as shown in Figures 15-19.
Figure 15: Windows Event Log displaying the reading of the contents of a suspicious application package “3b76099d-e6e0-4e86-bed1-100cc5fa699f_113.0.2.0_neutral__7afzw0tp1da5e”Figure 16: Windows Event Log displaying the deployment of the application package and the passing of suspicious installation parameters to the application via App Installer, as explained here. (Note: Rapid7’s MDR noticed the value of the ran parameter changed across separate and distinct interactions with the threat actor’s infrastructure, suggesting the ran parameter may be employed for tracking purposes.)Figure 17: Windows Event Log that appears to show the URI installation parameter being processed by the application via App InstallerFigure 18: Windows Event Log showing validation of the application package’s digital signature. See here for more information about signatures for Windows application packagesFigure 19: Windows Event Log showing successful deployment of the application package
Rapid7’s MDR visited chromesupdate[.]com in a controlled environment and discovered that it was hosting a convincing Chrome-update-themed webpage.
Figure 20: Lure hosted at chromesupdate[.]com
The website title, “Google Chrome – Download the Fast, Secure Browser from Google,” was consistent with those we observed of the redirect URLs getredd[.]biz, postsupport[.]net, and updateslives[.]com. The users in our investigations likely arrived at the website in Figure 20 after clicking a malicious toast notification, and proceeded to click the “Install” link presented on the website to initiate the Windows application installation.
The “Install” link presented at the website led to a Windows application installer URL (similar to that seen in Figure 17), which is consistent with MSIX distribution via the web.
Figure 21: Portion of the source code of the webpage hosted at chromesupdate[.]com showing a Windows application installer URL for a malicious MSIX package
Rapid7’s MDR obtained the MSIX file, oelgfertgokejrgre.msix, hosted at chromesupdate[.]com, and confirmed that it was a Windows application package.
Figure 22: Extracted contents of oelgfertgokejrgre.msix
Analysis of the contents extracted from oelgfertgokejrgre.msix revealed the following notable characteristics and features:
Two files, HoxLuSfo.exe and JiLutime.dll, were contained within the HoxLuSfo subdirectory. JiLutime.dll (MD5: 60bb67ebcffed2f406ac741b1083dc80) was a 32-bit Agile .NET obfuscator DLL signed by SecureTeam Software Ltd, likely to (de)obfuscate contents.
The AppxManifest.xml file contained more references to the Windows application’s masquerade as a Google Chrome update, as well as details related to its package identity and signature.
Figure 24: Partial contents of AppxManifest.xml
The DeroKuilSza.build.appxrecipe file contained strings that referenced a project “DeroKuilSza,” which is likely associated with the malware author.
Figure 25: References to a “DeroKuilSza” project found within DeroKuilSza.build.appxrecipe
Our dynamic analysis of oelgfertgokejrgre.msix provided clarity around the malware’s installation process. Detonation of oelgfertgokejrgre.msix caused a Windows App Installer window to appear, which displayed information about a fake Google Chrome update.
Figure 26: Windows App Installer window showing a fake Google Chrome update installation prompt
The information displayed to the user in Figure 26 is spoofed to masquerade as a legitimate Google Chrome update. The information correlates to the AppxManifest.xml configuration shown in Figure 24.
Once we proceeded with the installation, the MSIX package registered a notification sender via App Installer and immediately presented a notification to launch the fake Google Chrome update.
Figure 27: Registration of App Installer as a notification sender and notification to launch the fake Google Chrome update
Since the malicious Windows application package installed by the MSIX file was not hosted on the Microsoft Store, a prompt is presented to enable installation of sideload applications, if not already enabled, to allow for installation of applications from unofficial sources.
Figure 28: Requirement for sideload apps mode to be enabled to proceed with installationFigure 29: Menu presented to the user to enable sideload apps mode to complete the installation of the malware
The malware needs the enablement of “Sideload apps” to complete its installation.
Pulling off the mask
The malware we summarized in this blog post has several tricks up its sleeve. Its delivery mechanism via an ad service as a Windows application (which does not leave typical web-based download forensic artifacts behind), Windows application installation path, and UAC bypass technique by manipulation of an environment variable and native scheduled task can go undetected by various security solutions or even by a seasoned SOC analyst. Rapid7’s MDR customers can rest assured that, by leveraging our attacker behavior analytics detection methodology, our analysts will detect and respond to this infection chain before the malware can steal valuable data.
The following blog post was co-authored by Andrew Christian and Brendan Watters.
Beginning Feb. 27, 2021, Rapid7’s Managed Detection and Response (MDR) team has observed a notable increase in the automated exploitation of vulnerable Microsoft Exchange servers to upload a webshell granting attackers remote access. The suspected vulnerability being exploited is a cross-site request forgery (CSRF) vulnerability: The likeliest culprit is CVE-2021-24085, an Exchange Server spoofing vulnerability released as part of Microsoft’s February 2021 Patch Tuesday advisory, though other CVEs may also be at play (e.g., CVE-2021-26855, CVE-2021-26865, CVE-2021-26857).
The following China Chopper command was observed multiple times beginning Feb. 27 using the same DigitalOcean source IP (165.232.154.116):
cmd /c cd /d C:\inetpub\wwwroot\aspnet_client\system_web&net group "Exchange Organization administrators" administrator /del /domain&echo [S]&cd&echo [E]
Exchange or other systems administrators who see this command—or any other China Chopper command in the near future—should look for the following in IIS logs:
165.232.154.116 (the source IP of the requests)
/ecp/y.js
/ecp/DDI/DDIService.svc/GetList
Indicators of compromise (IOCs) from the attacks we have observed are consistent with IOCs for publicly available exploit code targeting CVE-2021-24085 released by security researcher Steven Seeley last week, shortly before indiscriminate exploitation began. After initial exploitation, attackers drop an ASP eval webshell before (usually) executing procdump against lsass.exe in order to grab all the credentials from the box. It would also be possible to then clean some indicators of compromise from the affected machine[s]. We have included a section on CVE-2021-24085 exploitation at the end of this document.
Rapid7 recommends that Exchange customers apply Microsoft’s February 2021 updates immediately. InsightVM and Nexpose customers can assess their exposure to CVE-2021-24085 and other February Patch Tuesday CVEs with vulnerability checks. InsightIDR provides existing coverage for this vulnerability via our out-of-the-box China Chopper Webshell Executing Commands detection, and will alert you about any suspicious activity. View this detection in the Attacker Tool section of the InsightIDR Detection Library.
CVE-2021-24085 exploit chain
As part of the PoC for CVE-2021-24085, the attacker will search for a specific token using a request to /ecp/DDI/DDIService.svc/GetList. If that request is successful, the PoC moves on to writing the desired token to the server’s filesystem with the request /ecp/DDI/DDIService.svc/SetObject. At that point, the token is available for downloading directly. The PoC uses a download request to /ecp/poc.png (though the name could be anything) and may be recorded in the IIS logs themselves attached to the IP of the initial attack.
Indicators of compromise would include the requests to both /ecp/DDI/DDIService.svc/GetList and /ecp/DDI/DDIService.svc/SetObject, especially if those requests were associated with an odd user agent string like python. Because the PoC utilizes aSetObject to write the token o the server’s filesystem in a world-readable location, it would be beneficial for incident responders to examine any files that were created around the time of the requests, as one of those files could be the access token and should be removed or placed in a secure location. It is also possible that responders could discover the file name in question by checking to see if the original attacker’s IP downloaded any files.
NEVER MISS A BLOG
Get the latest stories, expertise, and news about security today.
By continuing to use the site, you agree to the use of cookies. more information
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.
![[The Lost Bots] S03E02: Finding unknowns, even spy balloons](https://blog.rapid7.com/content/images/2023/04/The-Lost-Bots-logo-large.png){kind=logo}
