We are excited to announce that as of today, network security teams can procure and use Magic Transit, Cloudflare’s industry-leading DDoS mitigation solution, and Kentik’s network observability as an integrated solution. We are excited to help our customers not just with technical simplicity, but business process simplicity as well.
Why monitoring and mitigation?
Distributed Denial of Service (DDoS) attacks are highly disruptive to businesses everywhere. According to the Cloudflare DDoS Attack Trends report, in the first half of 2021 the world witnessed massive ransomware and ransom DDoS attack campaigns that interrupted critical infrastructure, including oil pipelines, healthcare, and financial services. In the second half, we saw a growing swarm of attacks, including one of the most powerful botnets deployed (Meris), with record-breaking network-layer attacks observed on the Cloudflare network.
Along with an increase in severity, there is a proliferation of automated toolkits that make it simple and cheap for anyone to launch these attacks. Detecting and stopping these attacks manually is not effective, and network security engineers are increasingly turning to automated tools to help ensure network and application availability.
DDoS protection has evolved over the years from appliances to hybrid models to fully Internet-native solutions, like Cloudflare’s Magic Transit. Cloudflare has been protecting millions of Internet properties against DDoS attacks, ensuring they are available at all times. Magic Transit extends Cloudflare’s industry-leading DDoS protection to shield entire IP subnets from DDoS attacks, while also accelerating network traffic, ensuring your data centers, cloud services and corporate networks are always reachable from the Internet. Our powerful global network spanning 250+ cities and 121 Tbps of capacity ensures that customers can have always-on DDoS protection without impacting network latency and application performance. Magic Transit also supports on-demand mode, which allows customers to activate DDoS protection when they need it most.
Network observability becomes critical to understand what normal looks like for your environment so that DDoS attacks are readily detected. Flow-based monitoring helps you understand not only how much traffic is flowing over your network, but also where it came from, where it’s going, and what applications are consuming bandwidth.
Magic Transit protection for every network configuration
Magic Transit is one of the most powerful DDoS mitigation platforms available today. We have worked hard to ensure Magic Transit is flexible enough for the most demanding network architectures. We need to fit into your world, not the other way around. And that involves partnering with leading network observability vendors to give you multiple options for how you choose to protect your network.
With this new partnership, customers can now consume Cloudflare’s Magic Transit service in one of three modes:
Always On — Customers looking for fast mitigation and traffic acceleration can deploy Magic Transit in Always On mode.
On Demand — Customers can choose to turn on Magic Transit response to a DDoS attack via Cloudflare’s UI or Cloudflare’s Magic Transit API.
On Demand + Flow-based Monitoring — Customers can now purchase and deploy an integrated network observability and DDoS protection solution consisting of Cloudflare Magic Transit On Demand and Kentik Protect from a single vendor.
In each configuration, Magic Transit is seamlessly paired with Magic Firewall — our cloud-native firewall-as-a-service.
Why Kentik’s flow-based monitoring?
At Cloudflare, we continuously take feedback from our customers on both our product and on what other tools they use. Customer feedback helps us build our products and how we grow Cloudflare’s Technology Partner Program.
For our Magic Transit customers, we found that many of our customers who chose Magic Transit On Demand have adopted solutions from Kentik, the network observability company with one of the leading flow-based monitoring tools in the ecosystem. Kentik empowers network professionals to plan, run, and fix any network with observability into all their traffic.
Simplifying network security
Cloudflare strives to simplify how customers can shield their network from cybersecurity threats like DDoS attacks. Magic Transit gives network security professionals the confidence that their network resources are immune from DDoS-related outages. We have now extended that same simplicity to this joint solution, making it simple for our customers to procure, provision, and integrate Magic Transit and Kentik. Our end goal is always creating the best experience possible for our customers, with Cloudflare’s services fitting seamlessly into their existing technology stack.
Kentik’s powerful network observability cloud collects flow logs from your network components and continuously learns network behavior, detecting anomalies such as DDoS attacks. Using our native API integration, the Kentik platform can trigger Magic Transit to start attracting network traffic when there’s an attack underway. Magic Transit’s autonomous DDoS mitigation automatically analyzes incoming traffic and filters out DDoS traffic across the entire Cloudflare network, protecting your network from unwanted traffic and avoiding service availability issues and outages.
Together, Kentik and Cloudflare have created a well-supported integration and a more streamlined procurement process to combine Kentik’s best-of-breed network observability and Cloudflare’s industry-leading DDoS protection in Magic Transit. Customers can now receive the best DDoS protection and network observability in a completely SaaS-based offering.
“We are excited to partner with Cloudflare to make it easier for our mutual customers to integrate our leading technology solutions and deploy industry-leading DDoS protection in a fully SaaS-based environment”, said Mike Mooney, CRO at Kentik.
Conclusion
Now, customers seeking to combine purpose-built, best-of-breed network observability and visualization from Kentik with Cloudflare’s Magic Transit On Demand can do so through a single vendor agreement and an integrated solution.
Welcome to our first DDoS report of 2022, and the ninth in total so far. This report includes new data points and insights both in the application-layer and network-layer sections — as observed across the global Cloudflare network between January and March 2022.
The first quarter of 2022 saw a massive spike in application-layer DDoS attacks, but a decrease in the total number of network-layer DDoS attacks. Despite the decrease, we’ve seen volumetric DDoS attacks surge by up to 645% QoQ, and we mitigated a new zero-day reflection attack with an amplification factor of 220 billion percent.
In the Russian and Ukrainian cyberspace, the most targeted industries were Online Media and Broadcast Media. In our Azerbaijan and Palestinian Cloudflare data centers, we’ve seen enormous spikes in DDoS activity — indicating the presence of botnets operating from within.
The Highlights
The Russian and Ukrainian cyberspace
Russian Online Media companies were the most targeted industries within Russia in Q1. The next most targeted was the Internet industry, then Cryptocurrency, and then Retail. While many attacks that targeted Russian Cryptocurrency companies originated in Ukraine or the US, another major source of attacks was from within Russia itself.
The majority of HTTP DDoS attacks that targeted Russian companies originated from Germany, the US, Singapore, Finland, India, the Netherlands, and Ukraine. It’s important to note that being able to identify where cyber attack traffic originates is not the same as being able to attribute where the attacker is located.
Attacks on Ukraine targeted Broadcast Media and Publishing websites and seem to have been more distributed, originating from more countries — which may indicate the use of global botnets. Still, most of the attack traffic originated from the US, Russia, Germany, China, the UK, and Thailand.
In January 2022, over 17% of under-attack respondents reported being targeted by ransom DDoS attacks or receiving a threat in advance.
That figure drastically dropped to 6% in February, and then to 3% in March.
When compared to previous quarters, we can see that in total, in Q1, only 10% of respondents reported a ransom DDoS attack; a 28% decrease YoY and 52% decrease QoQ.
Application-layer DDoS attacks
2022 Q1 was the busiest quarter in the past 12 months for application-layer attacks. HTTP-layer DDoS attacks increased by 164% YoY and 135% QoQ.
Diving deeper into the quarter, in March 2022 there were more HTTP DDoS attacks than in all of Q4 combined (and Q3, and Q1).
After four consecutive quarters in a row with China as the top source of HTTP DDoS attacks, the US stepped into the lead this quarter. HTTP DDoS attacks originating from the US increased by a staggering 6,777% QoQ and 2,225% YoY.
Network-layer DDoS attacks
Network-layer attacks in Q1 increased by 71% YoY but decreased 58% QoQ.
The Telecommunications industry was the most targeted by network-layer DDoS attacks, followed by Gaming and Gambling companies, and the Information Technology and Services industry.
Volumetric attacks increased in Q1. Attacks above 10 Mpps (million packets per second) grew by over 300% QoQ, and attacks over 100 Gbps grew by 645% QoQ.
This report is based on DDoS attacks that were automatically detected and mitigated by Cloudflare’s DDoS Protection systems. To learn more about how it works, check out this deep-dive blog post.
A note on how we measure DDoS attacks observed over our network To analyze attack trends, we calculate the “DDoS activity” rate, which is either the percentage of attack traffic out of the total traffic (attack + clean) observed over our global network, or in a specific location, or in a specific category (e.g., industry or billing country). Measuring the percentages allows us to normalize data points and avoid biases reflected in absolute numbers towards, for example, a Cloudflare data center that receives more total traffic and likely, also more attacks.
To view an interactive version of this report view it on Cloudflare Radar.
Ransom Attacks
Our systems constantly analyze traffic and automatically apply mitigation when DDoS attacks are detected. Each DDoS’d customer is prompted with an automated survey to help us better understand the nature of the attack and the success of the mitigation.
For over two years now, Cloudflare has been surveying attacked customers — one question on the survey being if they received a threat or a ransom note demanding payment in exchange to stop the DDoS attack. In the last quarter, 2021 Q4, we observed a record-breaking level of reported ransom DDoS attacks (one out of every five customers). This quarter, we’ve witnessed a drop in ransom DDoS attacks with only one out of 10 respondents reporting a ransom DDoS attack; a 28% decrease YoY and 52% decrease QoQ.
When we break it down by month, we can see that January 2022 saw the largest number of respondents reporting receiving a ransom letter in Q1. Almost one out of every five customers (17%).
Application-layer DDoS attacks
Application-layer DDoS attacks, specifically HTTP DDoS attacks, are attacks that usually aim to disrupt a web server by making it unable to process legitimate user requests. If a server is bombarded with more requests than it can process, the server will drop legitimate requests and — in some cases — crash, resulting in degraded performance or an outage for legitimate users.
Application-layer DDoS attacks by month
In Q1, application-layer DDoS attacks soared by 164% YoY and 135% QoQ – the busiest quarter within the past year.
Application-layer DDoS attacks increased to new heights in the first quarter of 2022. In March alone, there were more HTTP DDoS attacks than in all of 2021 Q4 combined (and Q3, and Q1).
Application-layer DDoS attacks by industry
Consumer Electronics was the most targeted industry in Q1.
Globally, the Consumer Electronics industry was the most attacked with an increase of 5,086% QoQ. Second was the Online Media industry with a 2,131% increase in attacks QoQ. Third were Computer Software companies, with an increase of 76% QoQ and 1,472 YoY.
To understand the origin of the HTTP attacks, we look at the geolocation of the source IP address belonging to the client that generated the attack HTTP requests. Unlike network-layer attacks, source IP addresses cannot be spoofed in HTTP attacks. A high percentage of DDoS activity in a given country usually indicates the presence of botnets operating from within the country’s borders.
After four consecutive quarters in a row with China as the top source of HTTP DDoS attacks, the US stepped into the lead this quarter. HTTP DDoS attacks originating from the US increased by a staggering 6,777% QoQ and 2,225% YoY. Following China in second place are India, Germany, Brazil, and Ukraine.
Application-layer DDoS attacks by target country
In order to identify which countries are targeted by the most HTTP DDoS attacks, we bucket the DDoS attacks by our customers’ billing countries and represent it as a percentage out of all DDoS attacks.
The US drops to second place, after being first for three consecutive quarters. Organizations in China were targeted the most by HTTP DDoS attacks, followed by the US, Russia, and Cyprus.
Network-layer DDoS attacks
While application-layer attacks target the application (Layer 7 of the OSI model) running the service that end users are trying to access (HTTP/S in our case), network-layer attacks aim to overwhelm network infrastructure (such as in-line routers and servers) and the Internet link itself.
Network-layer DDoS attacks by month
While HTTP DDoS attacks soared in Q1, network-layer DDoS attacks actually decreased by 58% QoQ, but still increased by 71% YoY.
Diving deeper into Q1, we can see that the amount of network-layer DDoS attacks remained mostly consistent throughout the quarter with about a third of attacks occurring every month.
Amongst these network-layer DDoS attacks are also zero-day DDoS attacks that Cloudflare automatically detected and mitigated.
In the beginning of March, Cloudflare researchers helped investigate and expose a zero-day vulnerability in Mitel business phone systems that amongst other possible exploitations, also enables attackers to launch an amplification DDoS attack. This type of attack reflects traffic off vulnerable Mitel servers to victims, amplifying the amount of traffic sent in the process by an amplification factor of 220 billion percent in this specific case. You can read more about it in our recent blog post.
We observed several of these attacks across our network. One of them targeted a North American cloud provider using the Cloudflare Magic Transit service. The attack originated from 100 source IPs mainly from the US, UK, Canada, Netherlands, Australia, and approximately 20 other countries. It peaked above 50 Mpps (~22 Gbps) and was automatically detected and mitigated by Cloudflare systems.
In this report, for the first time, we’ve begun classifying network-layer DDoS attacks according to the industries of our customers using the Spectrum and Magic products. This classification allows us to understand which industries are targeted the most by network-layer DDoS attacks.
When we look at Q1 statistics, we can see that in terms of attack packets and attack bytes launched towards Cloudflare customers, the Telecommunications industry was targeted the most. More than 8% of all attack bytes and 10% of all attack packets that Cloudflare mitigated targeted Telecommunications companies.
Following not too far behind, in second and third place were the Gaming / Gambling and Information Technology and Services industries.
Network-layer DDoS attacks by target country
Similarly to the classification by our customers’ industry, we can also bucket attacks by our customers’ billing country as we do for application-layer DDoS attacks, to identify the top attacked countries.
Looking at Q1 numbers, we can see that the US was targeted by the highest percentage of DDoS attacks traffic — over 10% of all attack packets and almost 8% of all attack bytes. Following the US is China, Canada, and Singapore.
Network-layer DDoS attacks by ingress country
When trying to understand where network-layer DDoS attacks originate, we cannot use the same method as we use for the application-layer attack analysis. To launch an application-layer DDoS attack, successful handshakes must occur between the client and the server in order to establish an HTTP/S connection. For a successful handshake to occur, the attacker cannot spoof their source IP address. While the attacker may use botnets, proxies, and other methods to obfuscate their identity, the attacking client’s source IP location does sufficiently represent the attack source of application-layer DDoS attacks.
On the other hand, to launch network-layer DDoS attacks, in most cases, no handshake is needed. Attackers can spoof the source IP address in order to obfuscate the attack source and introduce randomness into the attack properties, which can make it harder for simple DDoS protection systems to block the attack. So if we were to derive the source country based on a spoofed source IP, we would get a ‘spoofed country’.
For this reason, when analyzing network-layer DDoS attack sources, we bucket the traffic by the Cloudflare edge data center locations where the traffic was ingested, and not by the (potentially) spoofed source IP to get an understanding of where the attacks originate from. We are able to achieve geographical accuracy in our report because we have data centers in over 270 cities around the world. However, even this method is not 100% accurate, as traffic may be back hauled and routed via various Internet Service Providers and countries for reasons that vary from cost reduction to congestion and failure management.
In Q1, the percentage of attacks detected in Cloudflare’s data centers in Azerbaijan increased by 16,624% QoQ and 96,900% YoY, making it the country with the highest percentage of network-layer DDoS activity (48.5%).
Following our Azerbaijanian data center is our Palestinian data center where a staggering 41.9% of all traffic was DDoS traffic. This represents a 10,120% increase QoQ and 46,456% YoY.
To view all regions and countries, check out the interactive map.
Attack vectors
SYN Floods remain the most popular DDoS attack vector, while use of generic UDP floods drops significantly in Q1.
An attack vector is a term used to describe the method that the attacker uses to launch their DDoS attack, i.e., the IP protocol, packet attributes such as TCP flags, flooding method, and other criteria.
In Q1, SYN floods accounted for 57% of all network-layer DDoS attacks, representing a 69% increase QoQ and a 13% increase YoY. In second place, attacks over SSDP surged by over 1,100% QoQ. Following were RST floods and attacks over UDP. Last quarter, generic UDP floods took the second place, but this time, generic UDP DDoS attacks plummeted by 87% QoQ from 32% to a mere 3.9%.
Emerging threats
Identifying the top attack vectors helps organizations understand the threat landscape. In turn, this may help them improve their security posture to protect against those threats. Similarly, learning about new emerging threats that may not yet account for a significant portion of attacks, can help mitigate them before they become a significant force.
When we look at new emerging attack vectors in Q1, we can see increases in DDoS attacks reflecting off of Lantronix services (+971% QoQ) and SSDP reflection attacks (+724% QoQ). Additionally, SYN-ACK attacks increased by 437% and attacks by Mirai botnets by 321% QoQ.
Attacker reflecting traffic off of Lantronix Discovery Service
Lantronix is a US-based software and hardware company that provides solutions for Internet of Things (IoT) management amongst their vast offering. One of the tools that they provide to manage their IoT components is the Lantronix Discovery Protocol. It is a command-line tool that helps to search and find Lantronix devices. The discovery tool is UDP-based, meaning that no handshake is required. The source IP can be spoofed. So an attacker can use the tool to search for publicly exposed Lantronix devices using a 4 byte request, which will then in turn respond with a 30 byte response from port 30718. By spoofing the source IP of the victim, all Lantronix devices will target their responses to the victim — resulting in a reflection/amplification attack.
Simple Service Discovery Protocol used for reflection DDoS attacks
The Simple Service Discovery Protocol (SSDP) protocol works similarly to the Lantronix Discovery protocol, but for Universal Plug and Play (UPnP) devices such as network-connected printers. By abusing the SSDP protocol, attackers can generate a reflection-based DDoS attack overwhelming the target’s infrastructure and taking their Internet properties offline. You can read more about SSDP-based DDoS attacks here.
Network-layer DDoS attacks by attack rate
In Q1, we observed a massive uptick in volumetric DDoS attacks — both from the packet rate and bitrate perspective. Attacks over 10 Mpps grew by over 300% QoQ, and attacks over 100 Gbps grew by 645% QoQ.
There are different ways of measuring the size of an L3/4 DDoS attack. One is the volume of traffic it delivers, measured as the bit rate (specifically, terabits per second or gigabits per second). Another is the number of packets it delivers, measured as the packet rate (specifically, millions of packets per second).
Attacks with high bit rates attempt to cause a denial-of-service event by clogging the Internet link, while attacks with high packet rates attempt to overwhelm the servers, routers, or other in-line hardware appliances. These devices dedicate a certain amount of memory and computation power to process each packet. Therefore, by bombarding it with many packets, the appliance can be left with no further processing resources. In such a case, packets are “dropped,” i.e., the appliance is unable to process them. For users, this results in service disruptions and denial of service.
Distribution by packet rate
The majority of network-layer DDoS attacks remain below 50,000 packets per second. While 50 kpps is on the lower side of the spectrum at Cloudflare scale, it can still easily take down unprotected Internet properties and congest even a standard Gigabit Ethernet connection.
When we look at the changes in the attack sizes, we can see that attacks of over 10 Mpps grew by over 300% QoQ. Similarly, attacks of 1-10 Mpps grew by almost 40% QoQ.
Distribution by bitrate
In Q1, most of the network-layer DDoS attacks remain below 500 Mbps. This too is a tiny drop in the water at Cloudflare scale, but can very quickly shut down unprotected Internet properties with less capacity or at the very least congest, even a standard Gigabit Ethernet connection.
Graph of the distribution of network-layer DDoS attacks by bit rate in 2022 Q1
Similarly to the trends observed in the packet-per-second realm, here we can also see large increases. The amount of DDoS attacks that peaked over 100 Gbps increased by 645% QoQ; attacks peaking between 10 Gbps to 100 Gbps increased by 407%; attacks peaking between 1 Gbps to 10 Gbps increased by 88%; and even attacks peaking between 500 Mbps to 1 Gbps increased by almost 20% QoQ.
Network-layer DDoS attacks by duration
Most attacks remain under one hour in duration, reiterating the need for automated always-on DDoS mitigation solutions.
We measure the duration of an attack by recording the difference between when it is first detected by our systems as an attack and the last packet we see with that attack signature towards that specific target.
In previous reports, we provided a breakdown of ‘attacks under an hour’, and larger time ranges. However, in most cases over 90 percent of attacks last less than an hour. So starting from this report, we broke down the short attacks and grouped them by shorter time ranges to provide better granularity.
One important thing to keep in mind is that even if an attack lasts only a few minutes, if it is successful, the repercussions could last well beyond the initial attack duration. IT personnel responding to a successful attack may spend hours and even days restoring their services.
In the first quarter of 2022, more than half of the attacks lasted 10-20 minutes, approximately 40% ended within 10 minutes, another ~5% lasted 20-40 minutes, and the remaining lasted longer than 40 minutes.
Short attacks can easily go undetected, especially burst attacks that, within seconds, bombard a target with a significant number of packets, bytes, or requests. In this case, DDoS protection services that rely on manual mitigation by security analysis have no chance in mitigating the attack in time. They can only learn from it in their post-attack analysis, then deploy a new rule that filters the attack fingerprint and hope to catch it next time. Similarly, using an “on-demand” service, where the security team will redirect traffic to a DDoS provider during the attack, is also inefficient because the attack will already be over before the traffic routes to the on-demand DDoS provider.
It’s recommended that companies use automated, always-on DDoS protection services that analyze traffic and apply real-time fingerprinting fast enough to block short-lived attacks.
Summary
Cloudflare’s mission is to help build a better Internet. A better Internet is one that is more secure, faster, and reliable for everyone — even in the face of DDoS attacks. As part of our mission, since 2017, we’ve been providing unmetered and unlimited DDoS protection for free to all of our customers. Over the years, it has become increasingly easier for attackers to launch DDoS attacks. But as easy as it has become, we want to make sure that it is even easier — and free — for organizations of all sizes to protect themselves against DDoS attacks of all types.
Not using Cloudflare yet? Start now with our Free and Pro plans to protect your websites, or contact us for comprehensive DDoS protection for your entire network using Magic Transit.
A zero-day vulnerability in the Mitel MiCollab business phone system has recently been discovered (CVE-2022-26143). This vulnerability, called TP240PhoneHome, which Cloudflare customers are already protected against, can be used to launch UDP amplification attacks. This type of attack reflects traffic off vulnerable servers to victims, amplifying the amount of traffic sent in the process by an amplification factor of 220 billion percent in this specific case.
Cloudflare has been actively involved in investigating the TP240PhoneHome exploit, along with other members of the InfoSec community. Read our joint disclosure here for more details. As far as we can tell, the vulnerability has been exploited as early as February 18, 2022. We have deployed emergency mitigation rules to protect Cloudflare customers against the amplification DDoS attacks.
Mitel has been informed of the vulnerability. As of February 22, they have issued a high severity security advisory advising their customers to block exploitation attempts using a firewall, until a software patch is made available. Cloudflare Magic Transit customers can use the Magic Firewall to block external traffic to the exposed Mitel UDP port 10074 by following the example in the screenshot below, or by pasting the following expression into their Magic Firewall rule editor and selecting the Block action:
(udp.dstport eq 10074).
Creating a Magic Firewall rule to block traffic to port 10074
To learn more, register for our webinar on March 23rd, 2022.
Exploiting the vulnerability to launch DDoS attacks
Mitel Networks is based in Canada and provides business communications and collaboration products to over 70 million business users around the world. Amongst their enterprise collaboration products is the aforementioned Mitel MiCollab platform, known to be used in critical infrastructure such as municipal governments, schools, and emergency services. The vulnerability was discovered in the Mitel MiCollab platform.
The vulnerability manifests as an unauthenticated UDP port that is incorrectly exposed to the public Internet. The call control protocol running on this port can be used to, amongst other things, issue the debugging command startblast. This command does not place real telephone calls; rather, it simulates a “blast” of calls in order to test the system. For each test call that is made, two UDP packets are emitted in response to the issuer of the command.
According to the security advisory, the exploit can “allow a malicious actor to gain unauthorized access to sensitive information and services, cause performance degradations or a denial of service condition on the affected system. If exploited with a denial of service attack, the impacted system may cause significant outbound traffic impacting availability of other services.”
Since this is an unauthenticated and connectionless UDP-based protocol, you can use spoofing to direct the response traffic toward any IP and port number — and by doing so, reflect and amplify a DDoS attack to the victim.
We’ve mainly focused on the amplification vector because it can be used to hurt the whole Internet, but the phone systems themselves can likely be hurt in other ways with this vulnerability. This UDP call control port offers many other commands. With some work, it’s likely that you could use this UDP port to commit toll fraud, or to simply render the phone system inoperable. We haven’t assessed these other possibilities, because we do not have access to a device that we can safely test with.
The good news
Fortunately, only a few thousand of these devices are improperly exposed to the public Internet, meaning that this vector can “only” achieve several hundred million packets per second total. This volume of traffic can cause major outages if you’re not protected by an always-on automated DDoS protection service, but it’s nothing to be concerned with if you are.
Furthermore, an attacker can’t run multiple commands at the same time. Instead, the server queues up commands and executes them serially. The fact that you can only launch one attack at a time from these devices, mixed with the fact that you can make that attack for many hours, has fascinating implications. If an attacker chooses to start an attack by specifying a very large number of packets, then that box is “burned” – it can’t be used to attack anyone else until the attack completes.
How Cloudflare detects and mitigates DDoS attacks
To defend organizations against DDoS attacks, we built and operate software-defined systems that run autonomously. They automatically detect and mitigate DDoS attacks across our entire network.
Initially, traffic is routed through the Internet via BGP Anycast to the nearest Cloudflare edge data center. Once the traffic reaches our data center, our DDoS systems sample it asynchronously allowing for out-of-path analysis of traffic without introducing latency penalties.
The analysis is done using data streaming algorithms. Packet samples are compared to the fingerprints and multiple real-time signatures are created based on the dynamic masking of various fingerprint attributes. Each time another packet matches one of the signatures, a counter is increased. When the system qualifies an attack, i.e., the activation threshold is reached for a given signature, a mitigation rule is compiled and pushed inline. The mitigation rule includes the real-time signature and the mitigation action, e.g., drop.
Cloudflare’s mission is to help build a better Internet. A better Internet is one that is more secure, faster, and reliable for everyone — even in the face of DDoS attacks and emerging zero-day threats. As part of our mission, since 2017, we’ve been providing unmetered and unlimited DDoS protection for free to all of our customers. Over the years, it has become increasingly easier for attackers to launch DDoS attacks. To counter the attacker’s advantage, we want to make sure that it is also easy and free for organizations of all sizes to protect themselves against DDoS attacks of all types.
Beginning in mid-February 2022, security researchers, network operators, and security vendors observed a spike in DDoS attacks sourced from UDP port 10074 targeting broadband access ISPs, financial institutions, logistics companies, and organizations in other vertical markets.
Upon further investigation, it was determined that the devices abused to launch these attacks are MiCollab and MiVoice Business Express collaboration systems produced by Mitel, which incorporate TP-240 VoIP- processing interface cards and supporting software; their primary function is to provide Internet-based site-to-site voice connectivity for PBX systems.
Approximately 2600 of these systems have been incorrectly provisioned so that an unauthenticated system test facility has been inadvertently exposed to the public Internet, allowing attackers to leverage these PBX VoIP gateways as DDoS reflectors/amplifiers.
Mitel is aware that these systems are being abused to facilitate high-pps (packets-per-second) DDoS attacks, and have been actively working with customers to remediate abusable devices with patched software that disables public access to the system test facility.
In this blog, we will further explore the observed activity, explain how the driver has been abused, and share recommended mitigation steps. This research was created cooperatively among a team of researchers from Akamai SIRT, Cloudflare, Lumen Black Lotus Labs, NETSCOUT ASERT, TELUS, Team Cymru, and The Shadowserver Foundation.
DDoS attacks in the wild
While spikes of network traffic associated with the vulnerable service were observed on January 8th and February 7,th 2022, we believe the first actual attacks leveraging the exploit began on February 18th.
Observed attacks were primarily predicated on packets-per-second, or throughput, and appeared to be UDP reflection/amplification attacks sourced from UDP/10074 that were mainly directed towards destination ports UDP/80 and UDP/443. The single largest observed attack of this type preceding this one was approximately 53 Mpps and 23 Gbps. The average packet size for that attack was approximately 60 bytes, with an attack duration of approximately 5 minutes. The amplified attack packets are not fragmented.
This particular attack vector differs from most UDP reflection/amplification attack methodologies in that the exposed system test facility can be abused to launch a sustained DDoS attack of up to 14 hours in duration by means of a single spoofed attack initiation packet, resulting in a record-setting packet amplification ratio of 4,294,967,296:1. A controlled test of this DDoS attack vector yielded more than 400 Mmpps of sustained DDoS attack traffic.
It should be noted that this single-packet attack initiation capability has the effect of precluding network operator traceback of the spoofed attack initiator traffic. This helps mask the attack traffic generation infrastructure, making it less likely that the attack origin can be traced compared with other UDP reflection/amplification DDoS attack vectors.
Abusing the tp240dvr driver
The abused service on affected Mitel systems is called tp240dvr (“TP-240 driver”) and appears to run as a software bridge to facilitate interactions with TDM/VoIP PCI interface cards. The service listens for commands on UDP/10074 and is not meant to be exposed to the Internet, as confirmed by the manufacturer of these devices. It is this exposure to the Internet that ultimately allows it to be abused.
The tp240dvr service exposes an unusual command that is designed to stress-test its clients in order to facilitate debugging and performance testing. This command can be abused to cause the tp240dvr service to send this stress-test to attack victims. The traffic consists of a high rate of short informative status update packets that can potentially overwhelm victims and cause the DDoS scenario.
This command can also be abused by attackers to launch very high-throughput attacks. Attackers can use specially-crafted commands to cause the tp240dvr service to send larger informative status update packets, significantly increasing the amplification ratio.
By extensively testing isolated virtual TP-240-based systems in a lab setting, researchers were able to cause these devices to generate massive amounts of traffic in response to comparatively small request payloads. We will cover this attack scenario in greater technical depth in the following sections.
Calculating the potential attack impact
As previously mentioned, amplification via this abusable test facility differs substantially from how it is accomplished with most other UDP reflection/amplification DDoS vectors. Typically, reflection/amplification attacks require the attacker to continuously transmit malicious payloads to abusable nodes for as long as they wish to attack the victim. In the case of TP-240 reflection/amplification, this continuous transmission is not necessary to launch high-impact DDoS attacks.
Instead, an attacker leveraging TP-240 reflection/amplification can launch a high-impact DDoS attack using a single packet. Examination of the tp240dvr binary reveals that, due to its design, an attacker can theoretically cause the service to emit 2,147,483,647 responses to a single malicious command. Each response generates two packets on the wire, leading to approximately 4,294,967,294 amplified attack packets being directed toward the attack victim.
For each response to a command, the first packet contains a counter that increments with each sent response. As the counter value increments, the size of this first packet will grow from 36 bytes to 45 bytes. The second packet contains diagnostic output from the function, which can be influenced by the attacker. By optimizing each initiator packet to maximize the size of the second packet, every command will result in amplified packets that are up to 1,184 bytes in length.
In theory, a single abusable node generating the upper limit of 4,294,967,294 packets at a rate of 80kpps would result in an attack duration of roughly 14 hours. Over the course of the attack, the “counter” packets alone would generate roughly 95.5GB of amplified attack traffic destined for the targeted network. The maximally-padded “diagnostic output” packets would account for an additional 2.5TB of attack traffic directed towards the target.
This would yield a sustained flood of just under 393mb/sec of attack traffic from a single reflector/amplifier, all resulting from a single spoofed attack initiator packet of only 1,119 bytes in length. This results in a nearly unimaginable amplification ratio of 2,200,288,816:1 — a multiplier of 220 billion percent, triggered by a single packet.
Upper boundaries of attack volume and simultaneity
The tp240dvr service processes commands using a single thread. This means they can only process a single command at a time, and thus can only be used to launch one attack at a time. In the example scenario presented above, during the 14 hours that the abused device would be attacking the target, it cannot be leveraged to attack any other target. This is somewhat unique in the context of DDoS reflection/amplification vectors.
Although this characteristic also causes the tp240dvr service to be unavailable to legitimate users, it is much preferable to having these devices be leveraged in parallel by multiple attackers — and leaving legitimate operators of these systems to wonder why their outbound Internet data capacity is being consumed at much higher rates.
Additionally, it appears these devices are on relatively low-powered hardware, in terms of their traffic-generation capabilities. On an Internet where 100/Gbps links, dozens of CPU cores, and multi-threading capabilities have become commonplace, we can all be thankful this abusable service is not found on top-of-the-line hardware platforms capable of individually generating millions of packets per second, and running with thousands of parallelized threads.
Lastly, it is also good news that of the tens of thousands of these devices, which have been purchased and deployed historically by governments, commercial enterprises, and other organizations worldwide, a relatively small number of them have been configured in a manner that leaves them in this abusable state, and of those, many have been properly secured and taken offline from an attacker’s perspective.
Collateral impact
The collateral impact of TP-240 reflection/amplification attacks is potentially significant for organizations with Internet-exposed Mitel MiCollab and MiVoice Business Express collaboration systems that are abused as DDoS reflectors/amplifiers.
This may include partial or full interruption of voice communications through these systems, as well as additional service disruption due to transit capacity consumption, state-table exhaustion of NATs, and stateful firewalls, etc.
Wholesale filtering of all UDP/10074-sourced traffic by network operators may potentially overblock legitimate Internet traffic, and is therefore contraindicated.
Recommended actions
TP-240 reflection/amplification DDoS attacks are sourced from UDP/10074 and destined for the UDP port of the attacker’s choice. This amplified attack traffic can be detected, classified, traced back, and safely mitigated using standard DDoS defense tools and techniques.
Flow telemetry and packet capture via open-source and commercial analysis systems can alert network operators and end customers of TP-240 reflection/amplification attacks.
Network access control lists (ACLs), flowspec, destination-based remotely triggered blackhole (D/RTBH), source-based remotely triggered blackhole (S/RTBH), and intelligent DDoS mitigation systems can be used to mitigate these attacks.
Network operators should perform reconnaissance to identify and facilitate remediation of abusable TP-240 reflectors/amplifiers on their networks and/or the networks of their customers. Operators of Mitel MiCollab and MiVoice Business Express collaboration systems should proactively contact Mitel in order to receive specific remediation instructions from the vendor.
Organizations with business-critical public-facing Internet properties should ensure that all relevant network infrastructure, architectural, and operational Best Current Practices (BCPs) have been implemented, including situationally specific network access policies that only permit Internet traffic via required IP protocols and ports. Internet access network traffic to/from internal organizational personnel should be isolated from Internet traffic to/from public-facing Internet properties, and served via separate upstream Internet transit links.
DDoS defenses for all public-facing Internet properties and supporting infrastructure should be implemented in a situationally appropriate manner, including periodic testing to ensure that any changes to the organization’s servers/services/applications are incorporated into its DDoS defense plan.
It is imperative that organizations operating mission-critical public-facing Internet properties and/or infrastructure ensure that all servers/services/application/datastores/infrastructure elements are protected against DDoS attack, and are included in periodic, realistic tests of the organization’s DDoS mitigation plan. Critical ancillary supporting services such as authoritative and recursive DNS servers must be included in this plan.
Network operators should implement ingress and egress source address validation in order to prevent attackers from initiating reflection/amplification DDoS attacks.
All potential DDoS attack mitigation measures described in this document MUST be tested and customized in a situationally appropriate manner prior to deployment on production networks.
Mitigating factors
Operators of Internet-exposed TP-240-based Mitel MiCollab and MiVoice Business Express collaboration systems can prevent abuse of their systems to launch DDoS attacks by blocking incoming Internet traffic destined for UDP/10074 via access control lists (ACLs), firewall rules, and other standard network access control policy enforcement mechanisms.
Mitel have provided patched software versions that prevent TP-240-equipped MiCollab and MiVoice Business Express collaboration systems from being abused as DDoS reflectors/amplifiers by preventing exposure of the service to the Internet. Mitel customers should contact the vendor for remediation instructions.
Collateral impact to abusable TP-240 reflectors/amplifiers can alert network operators and/or end-customers to remove affected systems from “demilitarized zone” (DMZ) networks or Internet Data Centers (IDCs), or to disable relevant UDP port-forwarding rules that allow specific UDP/10074 traffic sourced from the public Internet to reach these devices, thereby preventing them from being abused to launch reflection/amplification DDoS attacks.
The amplified attack traffic is not fragmented, so there is no additional attack component consisting of non-initial fragments, as is the case with many other UDP reflection/amplification DDoS vectors.
Implementation of ingress and egress source-address validation (SAV; also known as anti-spoofing) can prevent attackers from launching reflection/amplification DDoS attacks.
Conclusion
Unfortunately, many abusable services that should not be exposed to the public Internet are nevertheless left open for attackers to exploit. This scenario is yet another example of real-world deployments not adhering to vendor guidance. Vendors can prevent this situation by adopting “safe by default” postures on devices before shipping.
Reflection/amplification DDoS attacks would be impossible to launch if all network operators implemented ingress and egress source-address validation (SAV, also known as anti-spoofing). The ability to spoof the IP address(es) of the intended attack target(s) is required to launch such attacks. Service providers must continue to implement SAV in their own networks, and require that their downstream customers do so.
As is routinely the case with newer DDoS attack vectors, it appears that after an initial period of employment by advanced attackers with access to bespoke DDoS attack infrastructure, TP-240 reflection/amplification has been weaponized and added to the arsenals of so-called “booter/stresser” DDoS-for-hire services, placing it within the reach of the general attacker population.
Collaboration across the operational, research, and vendor communities is central to the continued viability of the Internet. The quick response to and ongoing remediation of this high-impact DDoS attack vector has only been possible as a result of such collaboration. Organizations with a vested interest in the stability and resiliency of the Internet should embrace and support cross-industry cooperative efforts as a core principle.
The combined efforts of the research and mitigation task force demonstrates that successful collaboration across industry peers to quickly remediate threats to availability and resiliency is not only possible, but is also increasingly critical for the continued viability of the global Internet.
In particular, the Mitigation Task Force would like to cite Mitel for their exemplary cooperation, rapid response, and ongoing participation in remediation efforts. Mitel quickly created and disseminated patched software, worked with their customers and partners to update affected systems, and supplied valuable expertise as the Task Force worked to formulate this document.
We’re excited to announce that customers using our Free plan can now get real-time alerts about HTTP DDoS attacks that were automatically detected and mitigated by Cloudflare. The real-time DDoS alerts were originally announced over a year ago but were made available to customers on the Pro plan or higher. This announcement extends the DDoS alerts feature to Free plan users. You can read the original announcement blog post here.
What is a DDoS attack?
A Distributed Denial of Service (DDoS) attack is a cyber-attack that attempts to disrupt your online business. Whether your business relies on VoIP servers, UDP-based gaming servers, or HTTP servers, DDoS attacks can be used to disrupt any type of Internet property, server, or network.
In this blog post, we’ll focus on DDoS attacks that target HTTP servers. Whether your HTTP server is powering a mobile app, an eCommerce website, an API gateway, or any other HTTP application, if an attacker sends you more requests than it can handle, your server won’t be able to serve your real users. A flood of requests can cause service disruptions or even take your entire server offline. DDoS attacks can have real-world consequences such as a blow to your revenue and reputation.
How Cloudflare detects and mitigates DDoS attacks
Protecting your server against DDoS attacks requires two main capabilities:
The bandwidth to absorb both your users’ requests and the attack requests
The ability to differentiate between your users’ requests and the attack requests
Using our home-grown systems, we do just that, regardless of the size, frequency and duration of the attacks. All Cloudflare customers, including those using the Free plan, are protected by our unmetered DDoS mitigation commitment.
To protect against DDoS attacks, first, we route your traffic to our network of data centers. Our network spans more than 250 cities in over 100 countries around the world. Its capacity is over 100 Tbps — fifty times larger than the largest attack we’ve ever seen. Our bandwidth is more than enough to absorb both your users’ traffic and attack traffic.
Cloudflare’s global network
Cloudflare’s global network
Second, once your traffic reaches our data centers, it goes through state-of-the-art analysis mechanisms that constantly scan for DDoS attacks. Once an attack is detected, a real-time mitigation rule is automatically generated to surgically mitigate the attack requests based on the attack pattern, whilst leaving your users’ requests untouched. Using the HTTP DDoS Managed Ruleset you can customize the settings of the mitigation system to tailor it to your needs and specific traffic patterns.
Not sure what to do? That’s ok. For the most part, you won’t need to do anything and our system will automatically keep your servers protected. You can read more about it in our Get Started guide or in the original blog post. If you’re interested, you can also read more about how our mitigation system works in this technical blog post: A deep-dive into Cloudflare’s autonomous edge DDoS protection
Configuring a DDoS alert
Once our system detects and mitigates a DDoS attack, you’ll receive a real-time alert. To receive an alert, make sure you, first, configure a notification policy by following these steps:
Cloudflare’s mission is to help build a better Internet, and it guides everything we do. As part of this mission, we believe that a better Internet is one where enterprise-grade DDoS protection is available for everyone, not just bigger organizations.
Furthermore, we’ve also made our DDoS Managed Ruleset available for everyone to make sure that even non-paying customers can tailor and optimize their DDoS protection settings. Taking a step further, we want all of our users to be able to react as fast as possible when needed. This is why we’re providing real-time alerts for free. Knowledge is power, and notifying our users of attacks in real-time empowers them to ensure their website is safe, available, and performant.
The first half of 2021 witnessed massive ransomware and ransom DDoS attack campaigns that interrupted aspects of critical infrastructure around the world (including one of the largest petroleum pipeline system operators in the US) and a vulnerability in IT management software that targeted schools, public sector, travel organizations, and credit unions, to name a few.
The second half of the year recorded a growing swarm of one of the most powerful botnets deployed (Meris) and record-breaking HTTP DDoS attacks and network-layer attacks observed over the Cloudflare network. This besides the Log4j2 vulnerability (CVE-2021-44228) discovered in December that allows an attacker to execute code on a remote server — arguably one of the most severe vulnerabilities on the Internet since both Heartbleed and Shellshock.
Prominent attacks such as the ones listed above are but a few examples that demonstrate a trend of intensifying cyber-insecurity that affected everyone, from tech firms and government organizations to wineries and meat processing plants.
Here are some DDoS attack trends and highlights from 2021 and Q4 ‘21 specifically:
In December alone, one out of every three survey respondents reported being targeted by a ransom DDoS attack or threatened by the attacker.
Application-layer DDoS attacks
The Manufacturing industry was the most attacked in Q4 ’21, recording a whopping 641% increase QoQ in the number of attacks. The Business Services and Gaming/Gambling industries were the second and third most targeted industries by application-layer DDoS attacks.
For the fourth time in a row this year, China topped the charts with the highest percentage of attack traffic originating from its networks.
Q4 ’21 was the busiest quarter for attackers in 2021. In December 2021 alone, there were more than all the attacks observed in Q1 and Q2 ’21 separately.
While the majority of attacks were small, terabit-strong attacks became the new norm in the second half of 2021. Cloudflare automatically mitigated dozens of attacks peaking over 1 Tbps, with the largest one peaking just under 2 Tbps — the largest we’ve ever seen.
Attacks originating from Moldova quadrupled in Q4 ’21 QoQ, making it the country with the highest percentage of network-layer DDoS activity.
SYN floods and UDP floods were the most frequent attack vectors while emerging threats such as SNMP attacks increased by nearly 5,800% QoQ.
This report is based on DDoS attacks that were automatically detected and mitigated by Cloudflare’s DDoS Protection systems. To learn more about how it works, check out this deep-dive blog post.
A note on how we measure DDoS attacks observed over our network
To analyze attack trends, we calculate the “DDoS activity” rate, which is the percentage of attack traffic out of the total traffic (attack + clean) observed over our global network. Measuring attack numbers as a percentage of the total traffic observed allows us to normalize data points and avoid biases reflected in absolute numbers towards, for example, a Cloudflare data center that receives more total traffic and likely, also more attacks.
An interactive version of this report is available on Cloudflare Radar.
Ransom Attacks
Our systems constantly analyze traffic and automatically apply mitigation when DDoS attacks are detected. Each DDoS’d customer is prompted with an automated survey to help us better understand the nature of the attack and the success of the mitigation.
For over two years now, Cloudflare has been surveying attacked customers — one question on the survey being if they received a ransom note demanding payment in exchange to stop the DDoS attack. Q4 ’21 recorded the highest survey responses ever that indicated ransom threats — ransom attacks increased by 29% YoY and 175% QoQ. More specifically, one out of every 4.5 respondents (22%) reported receiving a ransom letter demanding payment by the attacker.
The percentage of respondents reported being targeted by a ransom DDoS attack or that have received threats in advance of the attack.
When we break it down by month, we can see that December 2021 topped the charts with 32% of respondents reporting receiving a ransom letter — that’s nearly one out of every three surveyed respondents.
Application-layer DDoS attacks
Application-layer DDoS attacks, specifically HTTP DDoS attacks, are attacks that usually aim to disrupt a web server by making it unable to process legitimate user requests. If a server is bombarded with more requests than it can process, the server will drop legitimate requests and — in some cases — crash, resulting in degraded performance or an outage for legitimate users.
Application-layer DDoS attacks by industry
In Q4, DDoS attacks on Manufacturing companies increased by 641% QoQ, and DDoS attacks on the Business Services industry increased by 97%.
When we break down the application-layer attacks targeted by industry, the Manufacturing, Business Services, and Gaming/Gambling industries were the most targeted industries in Q4 ’21.
Application-layer DDoS attacks by source country
To understand the origin of the HTTP attacks, we look at the geolocation of the source IP address belonging to the client that generated the attack HTTP requests. Unlike network-layer attacks, source IP addresses cannot be spoofed in HTTP attacks. A high percentage of DDoS activity in a given country usually indicates the presence of botnets operating from within the country’s borders.
For the fourth quarter in a row, China remains the country with the highest percentage of DDoS attacks originating from within its borders. More than three out of every thousand HTTP requests that originated from Chinese IP addresses were part of an HTTP DDoS attack. The US remained in second place, followed by Brazil and India.
Application-layer DDoS attacks by target country
In order to identify which countries are targeted by the most HTTP DDoS attacks, we bucket the DDoS attacks by our customers’ billing countries and represent it as a percentage out of all DDoS attacks.
For the third consecutive time this year, organizations in the United States were targeted by the most HTTP DDoS attacks, followed by Canada and Germany.
Network-layer DDoS attacks
While application-layer attacks target the application (Layer 7 of the OSI model) running the service that end users are trying to access, network-layer attacks aim to overwhelm network infrastructure (such as in-line routers and servers) and the Internet link itself.
Cloudflare thwarts an almost 2 Tbps attack
In November, our systems automatically detected and mitigated an almost 2 Tbps DDoS attack. This was a multi-vector attack combining DNS amplification attacks and UDP floods. The entire attack lasted just one minute. The attack was launched from approximately 15,000 bots running a variant of the original Mirai code on IoT devices and unpatched GitLab instances.
Network-layer DDoS attacks by month
December was the busiest month for attackers in 2021.
Q4 ‘21 was the busiest quarter in 2021 for attackers. Over 43% of all network-layer DDoS attacks took place in the fourth quarter of 2021. While October was a relatively calmer month, in November, the month of the Chinese Singles’ Day, the American Thanksgiving holiday, Black Friday, and Cyber Monday, the number of network-layer DDoS attacks nearly doubled. The number of observed attacks increased towards the final days of December ’21 as the world prepared to close out the year. In fact, the total number of attacks in December alone was higher than all the attacks in Q2 ’21 and almost equivalent to all attacks in Q1 ’21.
Network-layer DDoS attacks by attack rate
While most attacks are still relatively ‘small’ in size, terabit-strong attacks are becoming the norm.
There are different ways of measuring the size of an L3/4 DDoS attack. One is the volume of traffic it delivers, measured as the bit rate (specifically, terabits per second or gigabits per second). Another is the number of packets it delivers, measured as the packet rate (specifically, millions of packets per second).
Attacks with high bit rates attempt to cause a denial-of-service event by clogging the Internet link, while attacks with high packet rates attempt to overwhelm the servers, routers, or other in-line hardware appliances. These devices dedicate a certain amount of memory and computation power to process each packet. Therefore, by bombarding it with many packets, the appliance can be left with no further processing resources. In such a case, packets are “dropped,” i.e., the appliance is unable to process them. For users, this results in service disruptions and denial of service.
The distribution of attacks by their size (in bit rate) and month is shown below. As seen in the graph above, the majority of attacks took place in December. However, the graph below illustrates that larger attacks, over 300 Gbps in size, took place in November. Most of the attacks between 5-20 Gbps took place in December.
Distribution by packet rate
An interesting correlation Cloudflare has observed is that when the number of attacks increases, their size and duration decrease. In the first two-thirds of 2021, the number of attacks was relatively small, and correspondingly, their rates increased, e.g., in Q3 ’21, attacks ranging from 1-10 million packets per second (mpps) increased by 196%. In Q4 ’21, the number of attacks increased and Cloudflare observed a decrease in the size of attacks. 91% of all attacks peaked below 50,000 packets per second (pps) — easily sufficient to take down unprotected Internet properties.
Larger attacks of over 1 mpps decreased by 48% to 28% QoQ, while attacks peaking below 50K pps increased by 2.36% QoQ.
Distribution by bit rate
Similar to the trend observed in packet-intensive attacks, the amount of bit-intensive attacks shrunk as well. While attacks over 1 Tbps are becoming the norm, with the largest one we’ve ever seen peak just below 2 Tbps, the majority of attacks are still small and peaked below 500 Mbps (97.2%).
In Q4 ’21, larger attacks of all ranges above 500 Mbps saw massive decreases ranging from 35% to 57% for the larger 100+ Gbps attacks.
Network-layer DDoS attacks by duration
Most attacks remain under one hour in duration, reiterating the need for automated always-on DDoS mitigation solutions.
We measure the duration of an attack by recording the difference between when it is first detected by our systems as an attack and the last packet we see with that attack signature towards that specific target. In the last quarter of 2021, 98% of all network-layer attacks lasted less than one hour. This is very common as most of the attacks are short-lived. Even more so, a trend we’ve seen is that when the number of attacks increases, as in this quarter, their rate and duration decreases.
Short attacks can easily go undetected, especially burst attacks that, within seconds, bombard a target with a significant number of packets, bytes, or requests. In this case, DDoS protection services that rely on manual mitigation by security analysis have no chance in mitigating the attack in time. They can only learn from it in their post-attack analysis, then deploy a new rule that filters the attack fingerprint and hope to catch it next time. Similarly, using an “on-demand” service, where the security team will redirect traffic to a DDoS provider during the attack, is also inefficient because the attack will already be over before the traffic routes to the on-demand DDoS provider.
It’s recommended that companies use automated, always-on DDoS protection services that analyze traffic and apply real-time fingerprinting fast enough to block short-lived attacks.
Attack vectors
SYN floods remain attackers’ favorite method of attack, while attacks over SNMP saw a massive surge of almost 5,800% QoQ.
An attack vector is a term used to describe the method that the attacker uses to launch their DDoS attack, i.e., the IP protocol, packet attributes such as TCP flags, flooding method, and other criteria.
For the first time in 2021, the percentage of SYN flood attacks significantly decreased. Throughout 2021, SYN floods accounted for 54% of all network-layer attacks on average. While still grabbing first place as the most frequent vector, its share dropped by 38% QoQ to 34%.
However, it was a close-run for SYN attacks and UDP attacks. A UDP flood is a type of denial-of-service attack in which a large number of User Datagram Protocol (UDP) packets are sent to a targeted server with the aim of overwhelming that device’s ability to process and respond. Oftentimes, the firewall protecting the targeted server can also become exhausted as a result of UDP flooding, resulting in a denial-of-service to legitimate traffic. Attacks over UDP jumped from fourth place in Q3 ’21 to second place in Q4 ’21, with a share of 32% of all network-layer attacks — a 1,198% increase in QoQ.
In third place came the SNMP underdog that made a massive leap with its first time 2021 appearance in the top attack vectors.
Emerging threats
When we look at emerging attack vectors — which helps us understand what new vectors attackers are deploying to launch attacks — we observe a massive spike in SNMP, MSSQL, and generic UDP-based DDoS attacks.
Both SNMP and MSSQL attacks are used to reflect and amplify traffic on the target by spoofing the target’s IP address as the source IP in the packets used to trigger the attack.
Simple Network Management Protocol (SNMP) is a UDP-based protocol that is often used to discover and manage network devices such as printers, switches, routers, and firewalls of a home or enterprise network on UDP well-known port 161. In an SNMP reflection attack, the attacker sends out a large number of SNMP queries while spoofing the source IP address in the packet as the targets to devices on the network that, in turn, reply to that target’s address. Numerous responses from the devices on the network results in the target network being DDoSed.
Similar to the SNMP amplification attack, the Microsoft SQL (MSSQL) attack is based on a technique that abuses the Microsoft SQL Server Resolution Protocol for the purpose of launching a reflection-based DDoS attack. The attack occurs when a Microsoft SQL Server responds to a client query or request, attempting to exploit the Microsoft SQL Server Resolution Protocol (MC-SQLR), listening on UDP port 1434.
Network-layer DDoS attacks by country
Attacks originating from Moldova quadrupled, making it the country with the highest percentage of network-layer DDoS activity.
When analyzing network-layer DDoS attacks, we bucket the traffic by the Cloudflare edge data center locations where the traffic was ingested, and not by the source IP. The reason for this is that, when attackers launch network-layer attacks, they can spoof the source IP address in order to obfuscate the attack source and introduce randomness into the attack properties, which can make it harder for simple DDoS protection systems to block the attack. Hence, if we were to derive the source country based on a spoofed source IP, we would get a spoofed country.
Cloudflare is able to overcome the challenges of spoofed IPs by displaying the attack data by the location of the Cloudflare data center in which the attack was observed. We are able to achieve geographical accuracy in our report because we have data centers in over 250 cities around the world.
To view all regions and countries, check out the interactive map.
Summary
Cloudflare’s mission is to help build a better Internet. A better Internet is one that is more secure, faster, and reliable for everyone — even in the face of DDoS attacks. As part of our mission, since 2017, we’ve been providing unmetered and unlimited DDoS protection for free to all of our customers. Over the years, it has become increasingly easier for attackers to launch DDoS attacks. To counter the attacker’s advantage, we want to make sure that it is also easy and free for organizations of all sizes to protect themselves against DDoS attacks of all types.
After initially providing our customers control over the HTTP-layer DDoS protection settings earlier this year, we’re now excited to extend the control our customers have to the packet layer. Using these new controls, Cloudflare Enterprise customers using the Magic Transit and Spectrum services can now tune and tweak their L3/4 DDoS protection settings directly from the Cloudflare dashboard or via the Cloudflare API.
The new functionality provides customers control over two main DDoS rulesets:
Network-layer DDoS Protectionruleset — This ruleset includes rules to detect and mitigate DDoS attacks on layer 3/4 of the OSI model such as UDP floods, SYN-ACK reflection attacks, SYN Floods, and DNS floods. This ruleset is available for Spectrum and Magic Transit customers on the Enterprise plan.
Advanced TCP Protectionruleset — This ruleset includes rules to detect and mitigate sophisticated out-of-state TCP attacks such as spoofed ACK Floods, Randomized SYN Floods, and distributed SYN-ACK Reflection attacks. This ruleset is available for Magic Transit customers only.
A Distributed Denial of Service (DDoS) attack is a type of cyberattack that aims to disrupt the victim’s Internet services. There are many types of DDoS attacks, and they can be generated by attackers at different layers of the Internet. One example is the HTTP flood. It aims to disrupt HTTP application servers such as those that power mobile apps and websites. Another example is the UDP flood. While this type of attack can be used to disrupt HTTP servers, it can also be used in an attempt to disrupt non-HTTP applications. These include TCP-based and UDP-based applications, networking services such as VoIP services, gaming servers, cryptocurrency, and more.
To defend organizations against DDoS attacks, we built and operate software-defined systems that run autonomously. They automatically detect and mitigate DDoS attacks across our entire network. You can read more about our autonomous DDoS protection systems and how they work in our deep-dive technical blog post.
Unmetered and unlimited DDoS Protection
The level of protection that we offer is unmetered and unlimited — It is not bounded by the size of the attack, the number of the attacks, or the duration of the attacks. This is especially important these days because as we’ve recently seen, attacks are getting larger and more frequent. Consequently, in Q3, network-layer attacks increased by 44% compared to the previous quarter. Furthermore, just recently, our systems automatically detected and mitigated a DDoS attack that peaked just below 2 Tbps — the largest we’ve seen to date.
Mirai botnet launched an almost 2 Tbps DDoS attack
You can think of our autonomous DDoS protection systems as groups (rulesets) of intelligent rules. There are rulesets of HTTP DDoS Protection rules, Network-layer DDoS Protection rules and Advanced TCP Protection rules. In this blog post, we will cover the latter two rulesets. We’ve already covered the former in the blog post How to customize your HTTP DDoS protection settings.
Cloudflare L3/4 DDoS Managed Rules
In the Network-layer DDoS Protection rulesets, each rule has a unique set of conditional fingerprints, dynamic field masking, activation thresholds, and mitigation actions. These rules are managed (by Cloudflare), meaning that the specifics of each rule is curated in-house by our DDoS experts. Before deploying a new rule, it is first rigorously tested and optimized for mitigation accuracy and efficiency across our entire global network.
In the Advanced TCP Protection ruleset, we use a novel TCP state classification engine to identify the state of TCP flows. The engine powering this ruleset is flowtrackd — you can read more about it in our announcement blog post. One of the unique features of this system is that it is able to operate using only the ingress (inbound) packet flows. The system sees only the ingress traffic and is able to drop, challenge, or allow packets based on their legitimacy. For example, a flood of ACK packets that don’t correspond to open TCP connections will be dropped.
How attacks are detected and mitigated
Sampling
Initially, traffic is routed through the Internet via BGP Anycast to the nearest Cloudflare edge data center. Once the traffic reaches our data center, our DDoS systems sample it asynchronously allowing for out-of-path analysis of traffic without introducing latency penalties. The Advanced TCP Protection ruleset needs to view the entire packet flow and so it sits inline for Magic Transit customers only. It, too, does not introduce any latency penalties.
Analysis & mitigation
The analysis for the Advanced TCP Protection ruleset is straightforward and efficient. The system qualifies TCP flows and tracks their state. In this way, packets that don’t correspond to a legitimate connection and its state are dropped or challenged. The mitigation is activated only above certain thresholds that customers can define.
The analysis for the Network-layer DDoS Protection ruleset is done using data streaming algorithms. Packet samples are compared to the conditional fingerprints and multiple real-time signatures are created based on the dynamic masking. Each time another packet matches one of the signatures, a counter is increased. When the activation threshold is reached for a given signature, a mitigation rule is compiled and pushed inline. The mitigation rule includes the real-time signature and the mitigation action, e.g., drop.
Example
As a simple example, one fingerprint could include the following fields: source IP, source port, destination IP, and the TCP sequence number. A packet flood attack with a fixed sequence number would match the fingerprint and the counter would increase for every packet match until the activation threshold is exceeded. Then a mitigation action would be applied.
However, in the case of a spoofed attack where the source IP addresses and ports are randomized, we would end up with multiple signatures for each combination of source IP and port. Assuming a sufficiently randomized/distributed attack, the activation thresholds would not be met and mitigation would not occur. For this reason, we use dynamic masking, i.e. ignoring fields that may not be a strong indicator of the signature. By masking (ignoring) the source IP and port, we would be able to match all the attack packets based on the unique TCP sequence number regardless of how randomized/distributed the attack is.
Configuring the DDoS Protection Settings
For now, we’ve only exposed a handful of the Network-layer DDoS protection rules that we’ve identified as the ones most prone to customizations. We will be exposing more and more rules on a regular basis. This shouldn’t affect any of your traffic.
Overriding the sensitivity level and mitigation action
For the Network-layer DDoS Protection ruleset, for each of the available rules, you can override the sensitivity level (activation threshold), customize the mitigation action, and apply expression filters to exclude/include traffic from the DDoS protection system based on various packet fields. You can create multiple overrides to customize the protection for your network and your various applications.
Configuring expression fields for the DDoS Managed Rules to match on
In the past, you’d have to go through our support channels to customize the rules. In some cases, this may have taken longer to resolve than desired. With today’s announcement, you can tailor and fine-tune the settings of our autonomous edge system by yourself to quickly improve the accuracy of the protection for your specific network needs.
For the Advanced TCP Protection ruleset, for now, we’ve only exposed the ability to enable or disable it as a whole in the dashboard. To enable or disable the ruleset per IP prefix, you must use the API. At this time, when initially onboarding to Cloudflare, the Cloudflare team must first create a policy for you. After onboarding, if you need to change the sensitivity thresholds, use Monitor mode, or add filter expressions you must contact Cloudflare Support. In upcoming releases, this too will be available via the dashboard and API without requiring help from our Support team.
Pre-existing customizations
If you previously contacted Cloudflare Support to apply customizations, your customizations have been preserved, and you can visit the dashboard to view the settings of the Network-layer DDoS Protection ruleset and change them if you need. If you require any changes to your Advanced TCP Protection customizations, please reach out to Cloudflare Support.
If so far you didn’t have the need to customize this protection, there is no action required on your end. However, if you would like to view and customize your DDoS protection settings, follow this dashboard guide or review the API documentation to programmatically configure the DDoS protection settings.
Helping Build a Better Internet
At Cloudflare, everything we do is guided by our mission to help build a better Internet. The DDoS team’s vision is derived from this mission: our goal is to make the impact of DDoS attacks a thing of the past. Our first step was to build the autonomous systems that detect and mitigate attacks independently. Done. The second step was to expose the control plane over these systems to our customers (announced today). Done. The next step will be to fully automate the configuration with an auto-pilot feature — training the systems to learn your specific traffic patterns to automatically optimize your DDoS protection settings. You can expect many more improvements, automations, and new capabilities to keep your Internet properties safe, available, and performant.
Earlier this week, Cloudflare automatically detected and mitigated a DDoS attack that peaked just below 2 Tbps — the largest we’ve seen to date. This was a multi-vector attack combining DNS amplification attacks and UDP floods. The entire attack lasted just one minute. The attack was launched from approximately 15,000 bots running a variant of the original Mirai code on IoT devices and unpatched GitLab instances.
DDoS attack peaking just below 2 Tbps
Network-layer DDoS attacks increased by 44%
Last quarter, we saw multiple terabit-strong DDoS attacks and this attack continues this trend of increased attack intensity. Another key finding from our Q3 DDoS Trends report was that network-layer DDoS attacks actually increased by 44% quarter-over-quarter. While the fourth quarter is not over yet, we have, again, seen multiple terabit-strong attacks that targeted Cloudflare customers.
DDoS attacks peaking at 1-1.4 Tbps
How did Cloudflare mitigate this attack?
To begin with, our systems constantly analyze traffic samples “out-of-path” which allows us to asynchronously detect DDoS attacks without causing latency or impacting performance. Once the attack traffic was detected (within sub-seconds), our systems generated a real-time signature that surgically matched against the attack patterns to mitigate the attack without impacting legitimate traffic.
Once generated, the fingerprint is propagated as an ephemeral mitigation rule to the most optimal location in the Cloudflare edge for cost-efficient mitigation. In this specific case, as with most L3/4 DDoS attacks, the rule was pushed in-line into the Linux kernel eXpress Data Path (XDP) to drop the attack packet at wirespeed.
A conceptual diagram of Cloudflare’s DDoS protection systems
Cloudflare’s mission is to help build a better Internet — one that is secure, faster, and more reliable for everyone. The DDoS team’s vision is derived from this mission: our goal is to make the impact of DDoS attacks a thing of the past. Whether it’s the Meris botnet that launched some of the largest HTTP DDoS attacks on record, the recent attacks on VoIP providers or this Mirai-variant that’s DDoSing Internet properties, Cloudflare’s network automatically detects and mitigates DDoS attacks. Cloudflare provides a secure, reliable, performant, and customizable platform for Internet properties of all types.
For more information about Cloudflare’s DDoS protection, reach out to us or have a go with a hands-on evaluation of Cloudflare’s Free plan here.
Meris first got our attention due to an exceptionally large 17.2 million requests per second (rps) DDoS attack that it launched against one of our customers. This attack, along with subsequent attacks originated by the Meris botnet, was automatically detected and mitigated by our DDoS protection systems. Cloudflare customers, even ones on the free plan, are protected against Meris attacks.
Over the past months, we’ve been tracking and analyzing the activity of the Meris botnet. Some main highlights include:
Meris targets approximately 50 different websites every single day with a daily average of 104 unique DDoS attacks.
More than 33% of all Meris DDoS attack traffic targeted China-based websites.
More than 12% of all websites that were attacked by Meris are operated by US-based companies.
View more Meris attack insights and trends in the interactive Radar dashboard.
So what is Meris?
Meris (Latvian for plague) is the name of an active botnet behind a series of recent DDoS attacks that have targeted thousands of websites around the world. It was originally detected in late June 2021 by QRator in joint research they conducted with Yandex. Their initial research identified 30,000 to 56,000 bots, but they estimated that the numbers are actually much higher, in the ballpark of 250,000 bots.
The Meris botnet is formed of infected routers and networking hardware manufactured by the Latvian company MikroTik. According to MikroTik’s blog, the attackers exploited a vulnerability in the router’s operating system (RouterOS) which enabled attackers to gain unauthenticated remote access to read and write arbitrary files (CVE-2018-14847).
RouterOS is the router operating system that’s used by MikroTik’s routers and the RouterBOARD hardware product family, which can also be used to turn any PC into a router. Administration of RouterOS can be done either via direct SSH connection or by using a configuration utility called WinBox. The vulnerability itself was possible due to a directory traversal vulnerability in the WinBox interface with RouterOS.
Directory traversal is a type of exploit that allows attackers to travel to the parent directories to gain access to the operating system’s file system, a method and structure of how data is stored and retrieved in the operating system. Once they gain access to the file system, attackers can then read the existing files that administer the router and write files directly into the file system to administer the routers to their botnet needs.
While the vulnerability was patched after its detection back in 2018, it’s still being exploited in compromised devices that do not use the patched RouterOS versions, or that use the default usernames and passwords. MicroTik has advised its customers to upgrade their devices’ OS version, to only allow access to the devices via secure IPsec, and to inspect for any abnormalities such as unknown SOCKS proxy settings and scripts.
To launch volumetric attacks, the botnet uses HTTP pipelining which allows it to send multiple requests over a single connection, thus increasing its total attack throughput. Furthermore, in an attempt to obfuscate the attack source, the botnet uses open SOCKS proxies to proxy their attack traffic to the target.
Cloudflare’s DDoS protection systems automatically detect and mitigate Meris attacks. One of the mitigation actions that the system can choose to use is the ‘Connection Close’ action which eliminates the risk of HTTP pipelining and helps slow down attackers. Additionally, as part of Cloudflare’s threat intelligence suite, we provide a Managed IP List of Open SOCKS Proxies that customers can use as part of their firewall rules — to block, challenge or rate-limit traffic that arrives via SOCKS proxies.
How does Meris compare to Mirai?
About five years ago, Mirai (Japanese for future) — the infamous botnet that infected hundreds of thousands of IoT devices — launched record-breaking DDoS attacks against websites.
There have been many variants of the Mirai botnet since its source code was leaked. One version of Mirai, called Moobot, was detected last year when it attacked a Cloudflare customer with a 654 Gbps DDoS attack. Another variant recently made a resurgence when it targeted Cloudflare customers with over a dozen UDP and TCP based DDoS attacks that peaked multiple times above 1 Tbps, with a max peak of approximately 1.2 Tbps.
While Mirai infected IoT devices with low computational power, Meris is a swarm of routers that have significantly higher processing power and data transfer capabilities than IoT devices, making them much more potent in causing harm at a larger scale to web properties that are not protected by sophisticated cloud-based DDoS mitigation.
Tracking the Meris botnet attacks
Since the appearance of Meris, Cloudflare’s systems automatically detected and mitigated Meris attacks using the existing mitigation rules. During our analysis of the Meris botnet attacks, our security experts noticed the attack vectors adapt to try and bypass Cloudflare’s defenses. Needless to say, they were not successful. But we wanted to stay many steps ahead of attackers — and so our engineers deployed additional rules that mitigate Meris attacks even more comprehensively. A side effect of these mitigation rules is that it also provides us with more granular threat intelligence on the Meris attacks.
Since we deployed the new rules in early August, we’ve seen Meris launch an average of 104 DDoS attacks on Cloudflare customers every day. The highest figure we’ve seen was on September 6, when Meris was used to launch 261 unique attacks against Cloudflare customers.
Overall, Meris targets about 50 different websites and applications every single day. Although the average attack peaked at 106K rps, the median attack size was actually smaller at 17.6K rps. The largest attack we’ve seen was 17.2M rps and that occurred in July. In the graph below, you can see the daily highest requests per second rate after we deployed the new rules. Since then, the largest attack we’ve seen was 16.7M rps, which took place on August 19.
Meris used to target Banks, Financial Services, and Insurance companies
Over the past few months, the industry that received the most attack traffic from the Meris botnet was the Banking, Financial Services, and Insurance (BFSI) industry
Following the BFSI industry, the most attacked industries were the Publishing, Gaming/Gambling, and IT Services industries. And while BFSI was the number one most attacked industry when considering the Meris DDoS activity rate, it only came in fourth place when considering the percentage of targeted websites.
In terms of the percentage of targeted websites, the Computer Software industry came in first place. Almost 4% of all impacted websites were of Computer Software companies protected by Cloudflare, followed by Gaming/Gambling and IT Services with 3% and 2%, respectively.
Besides the total breakdowns shown above, we can also view the top industries the botnet attacked over time to understand the changing trends. These trends may be tied to political events, new video game releases, sporting events, or any other global or local public interest events.
Off the top, we can already see the two largest peaks on August 9 and August 29 — mainly on the Computer Software, Gaming/Gambling, and IT industries. Another interesting peak occurred on August 14 against Cryptocurrency providers.
In late August, the botnet was pointed against gambling and casino websites, generating attacks at rates of hundreds of thousands to millions of requests per second. A second significant wave against the same industry was launched in early September.
Meris targets websites in China, Australia, and US
Similarly to the analysis of the top industries, we can calculate the Meris DDoS activity rate per target country to identify which countries came under the most attacks. In total, China-based companies saw the largest amount of DDoS attacks. More than 33% of all requests generated by Meris were destined for China-based companies that are protected by Cloudflare. Australia came in second place, and the US in third.
On the other hand, when we look at the number of websites that were targeted by Meris, the US came in first place. More than 12% of all websites that were targeted by Meris are operated by US-based companies. China came in second place with 5.6% and Russia in third with 4.4%.
Over time, we can see how the attacks on the top countries change. Similarly to the per-industry breakdown, we can also see two large peaks. The first one occurred on the same spike as the per-industry breakdown on August 9. However, the second one here occurred on September 1.
Although only tens of thousands of bots have been detected per attack, it is estimated that there are roughly 250,000 bots worldwide. As indicated above, the botnet is formed of MikroTik routers. Using the source IP address of the routers, we’re able to identify the origin country of the bots to paint a geographical representation of the bots’ presence and growth over time.
The change in the location of the bots doesn’t necessarily indicate that the botnet is growing or shrinking. It could also be that different bot groups are activated from time to time to spread the load of the attacks while attempting not to get caught.
At the beginning of August, the majority of the bots were located in Brazil. But by the end of August, that number plummeted to a single digit percentage close to zero. Meanwhile, the number of infected devices grew in the United States. From the beginning of September, the number of bots was significantly higher in the US, Russia, India, Indonesia, and China.
Cloudflare operates autonomous DDoS protection systems that automatically detect and mitigate DDoS attacks of all types, including attacks launched by Meris and Mirai. These systems are also customizable, and Cloudflare customers can tweak and tune their DDoS protection settings as needed with the HTTP DDoS Managed Rulesetand the L3/4 DDoS Managed Ruleset.
Here’s a summary of the trends observed in Q3 ‘21:
Application-layer (L7) DDoS attack trends:
For the second consecutive quarter in 2021, US-based companies were the most targeted in the world.
For the first time in 2021, attacks on UK-based and Canada-based companies skyrocketed, making them the second and third most targeted countries, respectively.
Attacks on Computer Software, Gaming/ Gambling, IT, and Internet companies increased by an average of 573% compared to the previous quarter.
Meris, one of the most powerful botnets in history, aided in launching DDoS campaigns across various industries and countries. You can read more on that here.
Network-layer (L3/4) DDoS attack trends:
DDoS attacks increased by 44% worldwide compared to the previous quarter.
The Middle East and Africa recorded the largest average attack increase of approximately 80%.
Morocco recorded the highest DDoS activity in the third quarter globally — three out of every 100 packets were part of a DDoS attack.
While SYN and RST attacks remain the dominant attack method used by attackers, Cloudflare observed a surge in DTLS amplification attacks — recording a 3,549% increase QoQ.
Attackers targeted (and continue to target going into the fourth quarter this year) VoIP service providers with massive DDoS attack campaigns in attempts to bring SIP infrastructure down.
Note on avoiding data biases: When we analyze attack trends, we calculate the “DDoS activity” rate, which is the percentage of attack traffic of the total traffic (attack + clean). When reporting application- and network-layer DDoS attack trends, we use this metric, which allows us to normalize the data points and avoid biases toward, for example, a larger Cloudflare data center that naturally handles more traffic and therefore also, possibly, more attacks compared to a smaller Cloudflare data center located elsewhere.
Application-layer DDoS attacks
Application-layer DDoS attacks, specifically HTTP DDoS attacks, are attacks that usually aim to disrupt a web server by making it unable to process legitimate user requests. If a server is bombarded with more requests than it can process, the server will drop legitimate requests and — in some cases — crash, resulting in degraded performance or an outage for legitimate users.
Q3 ‘21 was the quarter of Meris — one of the most powerful botnets deployed to launch some of the largest HTTP DDoS attacks in history.
This past quarter, we observed one of the largest recorded HTTP attacks — 17.2M rps (requests per second) — targeting a customer in the financial services industry. One of the most powerful botnets ever observed, called Meris, is known to be deployed in launching these attacks.
Meris (Latvian for plague) is a botnet behind recent DDoS attacks that have targeted networks or organizations around the world. The Meris botnet infected routers and other networking equipment manufactured by the Latvian company MikroTik. According to MikroTik’s blog, a vulnerability in the MikroTik RouterOS (that was patched after its detection back in 2018) was exploited in still unpatched devices to build a botnet and launch coordinated DDoS attacks by bad actors.
Similar to the Mirai botnet of 2016, Meris is one of the most powerful botnets recorded. While Mirai infected IoT devices with low computational power such as smart cameras, Meris is a growing swarm of networking infrastructure (such as routers and switches) with significantly higher processing power and data transfer capabilities than IoT devices — making them much more potent in causing harm at a larger scale. Be that as it may, Meris is an example of how the attack volume doesn’t necessarily guarantee damage to the target. As far as we know, Meris, despite its strength, was not able to cause significant impact or Internet outages. On the other hand, by tactically targeting the DYN DNS service in 2016, Mirai succeeded in causing significant Internet disruptions.
Application-layer DDoS attacks by industry
The tech and gaming industries were the most targeted industries in Q3 ‘21.
When we break down the application-layer attacks targeted by industry, Computer Software companies topped the charts. The Gaming/Gambling industry, also known to be regular targets of online attacks, was a close second, followed by the Internet and IT industries.
Application-layer DDoS attacks by source country
To understand the origin of the HTTP attacks, we look at the geolocation of the source IP address belonging to the client that generated the attack HTTP requests. Unlike network-layer attacks, source IPs cannot be spoofed in HTTP attacks. A high DDoS activity rate in a given country usually indicates the presence of botnets operating from within.
In the third quarter of 2021, most attacks originated from devices/servers in China, the United States, and India. While China remains in first place, the number of attacks originating from Chinese IPs actually decreased by 30% compared to the previous quarter. Almost one out of every 200 HTTP requests that originated from China was part of an HTTP DDoS attack.
Additionally, attacks from Brazil and Germany shrank by 38% compared to the previous quarter. Attacks originating from the US and Malaysia reduced by 40% and 45%, respectively.
Application-layer DDoS attacks by target country
In order to identify which countries are targeted the most by L7 attacks, we break down the DDoS activity by our customers’ billing countries.
For the second consecutive time this year, organizations in the United States were targeted the most by L7 DDoS attacks in the world, followed by those in the UK and Canada.
Network-layer DDoS attacks
While application-layer attacks target the application (Layer 7 of the OSI model) running the service that end users are trying to access, network-layer attacks aim to overwhelm network infrastructure (such as in-line routers and servers) and the Internet link itself.
Mirai-variant botnet strikes with a force of 1.2 Tbps.
Q3 ‘21 was also the quarter when the infamous Mirai made a resurgence. A Mirai-variant botnet launched over a dozen UDP- and TCP-based DDoS attacks that peaked multiple times above 1 Tbps, with a max peak of approximately 1.2 Tbps. These network-layer attacks targeted Cloudflare customers on the Magic Transit and Spectrum services. One of these targets was a major APAC-based Internet services, telecommunications, and hosting provider and the other was a gaming company. In all cases, the attacks were automatically detected and mitigated without human intervention.
Network-layer DDoS attacks by month
September was, by far, the busiest month for attackers this year.
Q3 ‘21 accounted for more than 38% of all attacks this year. September was the busiest month for attackers so far in 2021 — accounting for over 16% of all attacks this year.
Network-layer DDoS attacks by attack rate
Most attacks are ‘small’ in size, but the number of larger attacks continues to rise.
There are different ways of measuring the size of a L3/4 DDoS attack. One is the volume of traffic it delivers, measured as the bit rate (specifically, terabits per second or gigabits per second). Another is the number of packets it delivers, measured as the packet rate (specifically, millions of packets per second).
Attacks with high bit rates attempt to cause a denial-of-service event by clogging the Internet link, while attacks with high packet rates attempt to overwhelm the servers, routers, or other in-line hardware appliances. Appliances dedicate a certain amount of memory and computation power to process each packet. Therefore, by bombarding it with many packets, the appliance can be left with no further processing resources. In such a case, packets are “dropped,” i.e., the appliance is unable to process them. For users, this results in service disruptions and denial of service.
The distribution of attacks by their size (in bit rate) and month is shown below. Interestingly enough, all attacks over 400 Gbps took place in August, including some of the largest attacks we have seen; multiple attacks peaked above 1 Tbps and reached as high as 1.2 Tbps.
Packet rate As seen in previous quarters, the majority of attacks observed in Q3 ‘21 were relatively small in size — nearly 89% of all attacks peaked below 50K packets per second (pps). While a majority of attacks are smaller in size, we observed that the number of larger attacks is increasing QoQ — attacks that peaked above 10M pps increased by 142% QoQ.
Attacks of packet rates ranging from 1-10 million packets per second increased by 196% compared to the previous quarter. This trend is similar to what we observed the last quarter as well, suggesting that larger attacks are increasing.
Bit rate From the bit rate perspective, a similar trend was observed — a total of 95.4% of all attacks peaked below 500 Mbps.
QoQ data shows that the number of attacks of sizes ranging from 500 Mbps to 10 Gbps saw massive increases of 126% to 289% compared to the previous quarter. Attacks over 100 Gbps decreased by nearly 14%.
The number of larger bitrate attacks increased QoQ (with the one exception being attacks over 100 Gbps, which decreased by nearly 14% QoQ). In particular, attacks ranging from 500 Mbps to 1 Gbps saw a surge of 289% QoQ and those ranging from 1 Gbps to 100 Gbps surged by 126%.
This trend once again illustrates that, while (in general) a majority of the attacks are indeed smaller, the number of “larger” attacks is increasing. This suggests that more attackers are garnering more resources to launch larger attacks.
Network-layer DDoS attacks by duration
Most attacks remain under one hour in duration, reiterating the need for automated always-on DDoS mitigation solutions.
We measure the duration of an attack by recording the difference between when it is first detected by our systems as an attack and the last packet we see with that attack signature. As in previous quarters, most of the attacks are short-lived. To be specific, 94.4% of all DDoS attacks lasted less than an hour. On the other end of the axis, attacks over 6 hours accounted for less than 0.4% in Q3 ‘21, and we did see a QoQ increase of 165% in attacks ranging 1-2 hours. Be that as it may, a longer attack does not necessarily mean a more dangerous one.
Short attacks can easily go undetected, especially burst attacks that, within seconds, bombard a target with a significant number of packets, bytes, or requests. In this case, DDoS protection services that rely on manual mitigation by security analysis have no chance in mitigating the attack in time. They can only learn from it in their post-attack analysis, then deploy a new rule that filters the attack fingerprint and hope to catch it next time. Similarly, using an “on-demand” service, where the security team will redirect traffic to a DDoS provider during the attack, is also inefficient because the attack will already be over before the traffic routes to the on-demand DDoS provider.
Cloudflare recommends that companies use automated, always-on DDoS protection services that analyze traffic and apply real-time fingerprinting fast enough to block the short-lived attacks. Cloudflare analyzes traffic out-of-path, ensuring that DDoS mitigation does not add any latency to legitimate traffic, even in always-on deployments. Once an attack is identified, our autonomous edge DDoS protection system (dosd) generates and applies a dynamically crafted rule with a real-time signature. Pre-configured firewall rules comprising allow/deny lists for known traffic patterns take effect immediately.
Attack vectors
SYN floods remain attackers’ favorite method of attack, while attacks over DTLS saw a massive surge — 3,549% QoQ.
An attack vector is the term used to describe the method that the attacker utilizes in their attempt to cause a denial-of-service event.
As observed in previous quarters, attacks utilizing SYN floods remain the most popular method used by attackers.
A SYN flood attack is a DDoS attack that works by exploiting the very foundation of the TCP protocol — the stateful TCP connection between a client and a server as a part of the 3-way TCP handshake. As a part of the TCP handshake, the client sends an initial connection request packet with a synchronize flag (SYN). The server responds with a packet that contains a synchronized acknowledgment flag (SYN-ACK). Finally, the client responds with an acknowledgment (ACK) packet. At this point, a connection is established and data can be exchanged until the connection is closed. This stateful process can be abused by attackers to cause denial-of-service events.
By repeatedly sending SYN packets, the attacker attempts to overwhelm a server or the router’s connection table that tracks the state of TCP connections. The server replies with a SYN-ACK packet, allocates a certain amount of memory for each given connection, and falsely waits for the client to respond with the final ACK. Given a sufficient number of connections occupying the server’s memory, the server is unable to allocate further memory for legitimate clients, causing the server to crash or preventing it from handling legitimate client connections, i.e., a denial-of-service event.
More than half of all attacks observed over our network were SYN floods. This was followed by RST, ACK, and UDP floods.
Emerging threats
While SYN and RST floods remain popular overall, when we look at emerging attack vectors — which helps us understand what new vectors attackers are deploying to launch attacks — we observed a massive spike in DTLS amplification attacks. DTLS floods increased by 3,549% QoQ.
Datagram Transport Layer Security (DTLS) is a protocol similar to Transport Layer Security (TLS) designed to provide similar security guarantees to connectionless datagram-based applications to prevent message forgery, eavesdropping, or tampering. DTLS, being connectionless, is specifically useful for establishing VPN connections, without the TCP meltdown problem. The application is responsible for reordering and other connection properties.
Just as with most UDP-based protocols, DTLS is spoofable and being used by attackers to generate reflection amplification attacks to overwhelm network gateways.
Network-layer DDoS attacks by country
While Morocco topped the charts in terms of the highest network attack rate observed, Asian countries closely followed.
When analyzing network-layer DDoS attacks, we bucket the traffic by the Cloudflare edge data center locations where the traffic was ingested, and not by the source IP. The reason for this is that, when attackers launch network-layer attacks, they can spoof the source IP address in order to obfuscate the attack source and introduce randomness into the attack properties, which may make it harder for simple DDoS protection systems to block the attack. Hence, if we were to derive the source country based on a spoofed source IP, we would get a spoofed country.
Cloudflare is able to overcome the challenges of spoofed IPs by displaying the attack data by the location of the Cloudflare data center in which the attack was observed. We are able to achieve geographical accuracy in our report because we have data centers in over 250 cities around the world.
A note on recent attacks on voice over-IP service providers — and ransom DDoS attacks
We recently reported and provided an update on the surge in DDoS attacks on VoIP service providers — some of who have also received ransom threats. As of early Q4 ‘21, this attack campaign is still ongoing and current. At Cloudflare, we continue to onboard VoIP service providers and shield their applications and networks against attacks.
HTTP attacks against API gateways and the corporate websites of the providers have been combined with network-layer and transport-layer attacks against VoIP infrastructures.
Examples include:
TCP floods targeting stateful firewalls: These are being used in “trial-and-error” type attacks. They are not very effective against telephony infrastructure specifically (because it is mostly UDP) but very effective at overwhelming stateful firewalls.
UDP floods targeting SIP infrastructure: Floods of UDP traffic that have no well-known fingerprint, aimed at critical VoIP services. Generic floods like this may look like legitimate traffic to unsophisticated filtering systems.
UDP reflection targeting SIP infrastructure: These methods, when targeted at SIP or RTP services, can easily overwhelm Session Border Controllers (SBCs) and other telephony infrastructure. The attacker seems to learn enough about the target’s infrastructure to target such services with high precision.
SIP protocol-specific attacks: Attacks at the application layer are of particular concern because of the higher resource cost of generating application errors versus filtering on network devices.
Organizations also continue to receive ransom notes that threaten attacks in exchange for bitcoin. Ransomware and ransom DDoS attacks, for the fourth consecutive quarter, continue to be a germane threat to organizations all over the world.
Cloudflare products close off several threat vectors that can lead to a ransomware infection and ransom DDoS attacks:
Cloudflare Browser Isolation prevents drive-by downloads and other browser-based attacks.
A Zero Trust architecture can help prevent ransomware from spreading within a network.
Magic Transit protects organizations’ networks against DDoS attacks using BGP route redistribution — without impacting latency.
Helping build a better Internet
Cloudflare was founded on the mission to help build a better Internet. And part of that mission is to build an Internet where the impact of DDoS attacks is a thing of the past. Over the last 10 years, we have been unwavering in our efforts to protect our customers’ Internet properties from DDoS attacks of any size or kind. In 2017, we announced unmetered DDoS protection for free — as part of every Cloudflare service and plan, including the Free plan — to make sure every organization can stay protected and available. Organizations big and small have joined Cloudflare over the past several years to ensure their websites, applications, and networks are secure from DDoS attacks, and remain fast and reliable.
But cyberattacks come in various forms, not just DDoS attacks. Malicious bots, ransomware attacks, email phishing, and VPN / remote access hacks are some many attacks that continue to plague organizations of all sizes globally. These attacks target websites, APIs, applications, and entire networks — which form the lifeblood of any online business. That is why the Cloudflare security portfolio accounts for everything and everyone connected to the Internet.
Attackers continue targeting VoIP infrastructure around the world. In our blog from last week, May I ask who’s calling, please? A recent rise in VoIP DDoS attacks, we reviewed how the SIP protocol works, ways it can be abused, and how Cloudflare can help protect against attacks on VoIP infrastructure without impacting performance.
Cloudflare’s network stands in front of some of the largest, most performance-sensitive voice and video providers in the world, and is uniquely well suited to mitigating attacks on VoIP providers.
Because of the sustained attacks we are observing, we are sharing details on recent attack patterns, what steps they should take before an attack, and what to do after an attack has taken place.
Below are three of the most common questions we’ve received from companies concerned about attacks on their VoIP systems, and Cloudflare’s answers.
Question #1: How is VoIP infrastructure being attacked?
The attackers primarily use off-the-shelf booter services to launch attacks against VoIP infrastructure. The attack methods being used are not novel, but the persistence of the attacker and their attempts to understand the target’s infrastructure are.
Attackers have used various attack vectors to probe the existing defenses of targets and try to infiltrate any existing defenses to disrupt VoIP services offered by certain providers. In some cases, they have been successful. HTTP attacks against API gateways and the corporate websites of the providers have been combined with network-layer and transport-layer attack against VoIP infrastructures. Examples:
TCP floods targeting stateful firewalls These are being used in “trial-and-error” type attacks. They are not very effective against telephony infrastructure specifically (because it’s mostly UDP) but very effective at overwhelming stateful firewalls.
UDP floods targeting SIP infrastructure Floods of UDP traffic that have no well-known fingerprint, aimed at critical VoIP services. Generic floods like this may look like legitimate traffic to unsophisticated filtering systems.
UDP reflection targeting SIP infrastructure These methods, when targeted at SIP or RTP services, can easily overwhelm Session Border Controllers (SBCs) and other telephony infrastructure. The attacker seems to learn enough about the target’s infrastructure to target such services with high precision.
SIP protocol-specific attacks Attacks at the application layer are of particular concern because of the higher resource cost of generating application errors vs filtering on network devices.
Question #2: How should I prepare my organization in case our VoIP infrastructure is targeted?
Deploy an always-on DDoS mitigation service Cloudflare recommends the deployment of always-on network level protection, like Cloudflare Magic Transit, prior to your organization being attacked.
Do not rely on reactive on-demand SOC-based DDoS Protection services that require humans to analyze attack traffic — they take too long to respond. Instead, onboard to a cloud service that has sufficient network capacity and automated DDoS mitigation systems.
Cloudflare has effective mitigations in place for the attacks seen against VoIP infrastructure, including for sophisticated TCP floods and SIP specific attacks.
Enforce a positive security model Block TCP on IP/port ranges that are not expected to receive TCP, instead of relying on on-premise firewalls that can be overwhelmed. Block network probing attempts (e.g. ICMP) and other packets that you don’t normally expect to see.
Build custom mitigation strategies Work together with your DDoS protection vendor to tailor mitigation strategies to your workload. Every network is different, and each poses unique challenges when integrating with DDoS mitigation systems.
Educate your employees Train all of your employees to be on the lookout for ransom demands. Check email, support tickets, form submissions, and even server access logs. Ensure employees know to immediately report ransom demands to your Security Incident Response team.
Question #3: What should I do if I receive a ransom/threat?
Do not to pay the ransom Paying the ransom only encourages bad actors—and there’s no guarantee that they won’t attack your network now or later.
Notify Cloudflare We can help ensure your website and network infrastructure are safeguarded against these attacks.
Notify local law enforcement They will also likely request a copy of the ransom letter that you received.
Cloudflare is here to help
With over 100 Tbps of network capacity, a network architecture that efficiently filters traffic close to the source, and a physical presence in over 250 cities, Cloudflare can help protect critical VoIP infrastructure without impacting latency, jitter, and call quality. Test results demonstrate a performance improvement of 36% on average across the globe for a real customer network using Cloudflare Magic Transit.
Some of the largest voice and video providers in the world rely on Cloudflare to protect their networks and ensure their services remain online and fast. We stand ready to help.
Talk to a Cloudflare specialist to learn more. Under attack? Contact our hotline to speak with someone immediately.
We’re excited to announce the availability of the HTTP DDoS Managed Ruleset. This new feature allows Cloudflare customers to independently tailor their HTTP DDoS protection settings. Whether you’re on the Free plan or the Enterprise plan, you can now tweak and optimize the settings directly from within the Cloudflare dashboard or via API.
We expect that in most cases, Cloudflare customers won’t need to customize any settings. Our mission is to make DDoS disruptions a thing of the past, with no customer overhead. To achieve this mission we’re constantly investing in our automated detection and mitigation systems. In some rare cases, there is a need to make some configuration changes, and so now, Cloudflare customers can customize those protection mechanisms independently. The next evolutionary step is to make those settings learn and auto-tune themselves for our customers, based on their unique traffic patterns. Zero-touch DDoS protection at scale.
Unmetered DDoS Protection
Back in 2017, we announced that we will never kick a customer off of our network because they face large attacks, even if they are not paying us at all (i.e., using the Free plan). Furthermore, we committed to never charge a customer for DDoS attack traffic — no matter the size and duration of the attack. Just this summer, our systems automatically detected and mitigated one of the largest DDoS attacks of all time. As opposed to other vendors, Cloudflare customers don’t need to request a service credit for the attack traffic: we simply exclude DDoS traffic from our billing systems. This is done automatically, just like our attack detection and mitigation mechanisms.
Autonomous DDoS Protection
Our unmetered DDoS protection commitment is possible due to our ongoing investment in our network and technology stack. The global coverage and capacity of our network allows us to absorb the largest attacks ever recorded, without manual intervention. Using BGP Anycast, traffic is routed to the closest Cloudflare edge data center as a form of global inter-data center load balancing. Traffic is then load balanced efficiently inside the data center between servers with the help of Unimog, our home-grown L4 load balancer, to ensure that traffic arrives at the least loaded server. Then, each server scans for malicious traffic and, if detected, applies mitigations in the most optimal location in the tech stack. Each server detects and mitigates attacks completely autonomously without requiring any centralized consensus, and shares details with each other using multicast. This is done using our proprietary autonomous edge detection and mitigation system, and this is how we’re able to continue offering unmetered DDoS protection for free at the scale we operate at.
Configurable DDoS Protection
Our autonomous systems use a set of dynamic rules that scan for attack patterns, known attack tools, suspicious patterns, protocol violations, requests causing large amounts of origin errors, excessive traffic hitting the origin/cache, and additional attack vectors. Each rule has a predefined sensitivity level and default action that varies based on the rule’s confidence that the traffic is indeed part of an attack.
But how do we determine those confidence levels? The answer to that depends on each specific rule and what that rule is looking for. Some rules look for the patterns in HTTP attributes that are generated by known attack tools and botnets, known protocol violations and other general suspicious patterns andtraffic abnormalities. If a given rule is searching for the HTTP patterns of known attack tools, then once found, the likelihood (i.e., confidence) that this traffic is part of an attack is high, and we can therefore safely block all the traffic that matches that rule. However, in other cases, the detected patterns or abnormal activity might resemble an attack but can actually be caused by faulty applications that generate abnormal HTTP calls, misbehaving API clients that flood their origin server, and even legitimate traffic that naively violates protocol standards. In those cases, we might want to rate-limit the traffic that matches the rule or serve a challenge action to verify and allow legitimate users in while blocking bad bots and attackers.
Configuring the DDoS Protection Settings
In the past, you’d have to go through our support channels to customize any of the default actions and sensitivity levels. In some cases, this may have taken longer to resolve than desired. With today’s release, you can tailor and fine-tune the settings of our autonomous edge system by yourself to quickly improve the accuracy of the protection for your specific application needs.
If you previously contacted Cloudflare Support to apply customizations, the DDoS Ruleset has been set to Essentially off or Low for your zone, based on your existing customization. You can visit the dashboard to view the settings and change them if you need.
If you’ve requested to exclude or bypass the mitigations for specific HTTP attributes or IPs, or if you’ve requested a significantly high threshold that requires Cloudflare approval, then those customizations are still active but may not yet be visible in the dashboard.
If you haven’t experienced this issue previously, there is no action required on your end. However, if you would like to customize your DDoS protection settings, go directly to the DDoS tab or follow these steps:
Next to HTTP DDoS attack protection, click Configure.
In Ruleset configuration, select the action and sensitivity values for all the rules in the HTTP DDoS Managed Ruleset.
Alternatively, follow the API documentation to programmatically configure the DDoS protection settings.
In the configuration page, you can select a different Action and Sensitivity Level to override all the DDoS protection rules as a group of rules (i.e., the “ruleset”).
Alternatively, you can click Browse Rules to override specific rules, rather than all of them as a set of rules.
Mitigation Action
The mitigation action defines what action to take when the mitigation rule is applied. Our systems constantly analyze traffic and track potentially malicious activity. When certain request-per-second thresholds exceed the configured sensitivity level, a mitigation rule with a dynamically generated signature will be applied to mitigate the attack. The default mitigation action can vary according to the specific rule. A rule with less confidence may apply a Challenge action as a form of soft mitigation, and a rule with a Block action is applied when there is higher confidence that the traffic is part of an attack — as a form of a stricter mitigation action.
The available values for the action are:
Block
Challenge (CAPTCHA)
Log
Use Rule Defaults
Some examples of when you may want to change the mitigation action include:
Safer onboarding: You’re onboarding a new HTTP application which has odd traffic patterns, naively violates protocol violations or causes spiky behavior. In this case, you can set the action to Log and see what traffic our systems flag. Afterwards, you can make the necessary changes to the sensitivity levels as required and switch the mitigation action back to the default.
Stricter mitigation: A DDoS attack has been detected but a Rate-limit or Challenge action have been applied due to the rule’s default logic. However, in this specific case, you’re sure that this is malicious traffic, so you can change the action to Block for a more complete mitigation.
Mitigation Sensitivity
The sensitivity level defines when the mitigation rule is applied. Our systems constantly analyze traffic and track potentially malicious activity. When certain request-per-second thresholds are crossed, a mitigation rule with a dynamically generated signature will be applied to mitigate the attack. Toggling the sensitivity levels allows you to define when the mitigation is applied. The higher the sensitivity, the faster the mitigation is applied. The available values for sensitivity are:
High (default)
Medium
Low
Essentially Off
Essentially Off means that we’ve set an exceptionally low sensitivity level so in most cases traffic won’t be mitigated for you. However, attack traffic will be mitigated at exceptional levels to ensure the safety and stability of the Cloudflare network.
Some examples of when you may want to change the sensitivity action include:
Avoid impact to legitimate traffic: One of the rules has applied mitigation to your legitimate traffic due to a suspicious pattern. In this case, you may want to reduce the rule sensitivity to avoid recurrence of the issue and negative impact to your traffic.
Legacy applications: One of your legacy HTTP applications is violating protocol standards, or you may have mistakenly introduced a bug into your mobile application/API client. These cases may result in abnormal traffic activity that may be flagged by our systems. In such a case, you can select the Essentially Off sensitivity level until you’ve resolved the issue on your end, to avoid false positives.
Overriding Specific Rules
As mentioned above, you can also select a specific rule to override its action and sensitivity levels. The per-rule override takes priority over the ruleset override.
When configuring per-rule overrides, you’ll see that some rules have a DDoS Dynamic action. This means that the mitigation is multi-staged and will apply different mitigation actions depending on various factors including attack type, request characteristics, and various other factors. This dynamic action can also be overridden if you choose to do so.
DDoS Attack Analytics
When a DDoS attack is detected and mitigated, you’ll receive a real-time DDoS alert (if you’ve configured one) and you’ll be able to view the attack in the Firewall analytics dashboard. The attack details and the rule ID that was triggered will also be displayed in the Activity log as part of each HTTP request log that was mitigated.
Helping Build a Better Internet
At Cloudflare, everything we do is guided by our mission to help build a better Internet. A significant part of that mission is to make DDoS downtime and service disruptions a thing of the past. By giving our users the visibility and tools they need in order to understand and improve their DDoS protection, we’re hoping to make another step towards a better Internet.
Do you have feedback about the user interface or anything else? In the new DDoS tab, we’ve added a link to provide feedback, so you too can help shape the future of Cloudflare’s DDoS protection Managed Rules.
Earlier this summer, Cloudflare’s autonomous edge DDoS protection systems automatically detected and mitigated a 17.2 million request-per-second (rps) DDoS attack, an attack almost three times larger than any previous one that we’re aware of. For perspective on how large this attack was: Cloudflare serves over 25 million HTTP requests per second on average. This refers to the average rate of legitimate traffic in 2021 Q2. So peaking at 17.2 million rps, this attack reached 68% of our Q2 average rps rate of legitimate HTTP traffic.
Comparison graph of Cloudflare’s average request per second rate versus the DDoS attack
Automated DDoS mitigation with Cloudflare’s autonomous edge
This attack, along with the additional attacks provided in the next sections, were automatically detected and mitigated by our autonomous edge DDoS protection systems. The system is powered by our very own denial of service daemon (dosd). Dosd is a home-grown software-defined daemon. A unique dosd instance runs in every server in each one of our data centers around the world. Each dosd instance independently analyzes traffic samples out-of-path. Analyzing traffic out-of-path allows us to scan asynchronously for DDoS attacks without causing latency and impacting performance. DDoS findings are also shared between the various dosd instances within a data center, as a form of proactive threat intelligence sharing.
Once an attack is detected, our systems generate a mitigation rule with a real-time signature that matches the attack patterns. The rule is propagated to the most optimal location in the tech stack. As an example, a volumetric HTTP DDoS attack may be blocked at L4 inside the Linux iptables firewall instead of at L7 inside the L7 reverse proxy which runs in the user space. Mitigating lower in the stack, e.g. dropping the packets at L4 instead of responding with a 403 error page in L7, is more cost-efficient. It reduces our edge CPU consumption and intra-data center bandwidth utilization — thus helping us mitigate large attacks at scale without impacting performance.
This autonomous approach, along with our network’s global scale and reliability, allow us to mitigate attacks that reach 68% of our average per-second-rate, and higher, without requiring any manual mitigation by Cloudflare personnel, nor causing any performance degradation.
The resurgence of Mirai and new powerful botnets
This attack was launched by a powerful botnet, targeting a Cloudflare customer in the financial industry. Within seconds, the botnet bombarded the Cloudflare edge with over 330 million attack requests.
Graph of 17.2M rps attack
The attack traffic originated from more than 20,000 bots in 125 countries around the world. Based on the bots’ source IP addresses, almost 15% of the attack originated from Indonesia and another 17% from India and Brazil combined. Indicating that there may be many malware infected devices in those countries.
Distribution of the attack sources by top countries
Volumetric attacks increase
This 17.2 million rps attack is the largest HTTP DDoS attack that Cloudflare has ever seen to date and almost three times the size of any other reported HTTP DDoS attack. This specific botnet, however, has been seen at least twice over the past few weeks. Just last week it also targeted a different Cloudflare customer, a hosting provider, with an HTTP DDoS attack that peaked just below 8 million rps.
Graph of 8M rps attack
Two weeks before, a Mirai-variant botnet launched over a dozen UDP and TCP based DDoS attacks that peaked multiple times above 1 Tbps, with a max peak of approximately 1.2 Tbps. And while the first HTTP attacks targeted Cloudflare customers on the WAF/CDN service, the 1+ Tbps network-layer attacks targeted Cloudflare customers on the Magic Transit and Spectrum services. One of these targets was a major APAC-based Internet services, telecommunications and hosting provider. The other was a gaming company. In all cases, the attacks were automatically detected and mitigated without human intervention.
Graph of Mirai botnet attack peaking at 1.2 Tbps
The Mirai botnet started with roughly 30K bots and slowly shrinked to approximately 28K. However, despite losing bots from its fleet, the botnet was still able to generate impressive volumes of attack traffic for short periods. In some cases, each burst lasted only a few seconds.
These attacks join the increase in Mirari-based DDoS attacks that we’ve observed on our network over the past weeks. In July alone, L3/4 Mirai attacks increased by 88% and L7 attacks by 9%. Additionally, based on the current August per-day average of the Mirai attacks, we can expect L7 Mirai DDoS attacks and other similar botnet attacks to increase by 185% and L3/4 attacks by 71% by the end of the month.
Graph of change in Mirai based DDoS attacks by month
Back to the Mirai
Mirai, which means ‘future’ in Japanese, is a codename for malware that was first discovered in 2016 by MalwareMustDie, a non-profit security research workgroup. The malware spreads by infecting Linux-operated devices such as security cameras and routers. It then self-propagates by searching for open Telnet ports 23 and 2323. Once found, it then attempts to gain access to vulnerable devices by brute forcing known credentials such as factory default usernames and passwords. Later variants of Mirai also took advantage of zero-day exploits in routers and other devices. Once infected, the devices will monitor a Command & Control (C2) server for instructions on which target to attack.
Diagram of Botnet operator controlling the botnet to attack websites
How to protect your home and business
While the majority of attacks are small and short, we continue to see these types of volumetric attacks emerging more often. It’s important to note that these volumetric short burst attacks can be especially dangerous for legacy DDoS protection systems or organizations without active, always-on cloud-based protection.
Furthermore, while the short duration may say something about the botnet’s capability to deliver sustained levels of traffic over time, it can be challenging or impossible for humans to react to it in time. In such cases, the attack is over before a security engineer even has time to analyze the traffic or activate their stand-by DDoS protection system. These types of attacks highlight the need for automated, always-on protection.
How to protect your business and Internet properties
Follow our preventive best practices, to ensure that both your Cloudflare settings and your origin server settings are optimized. As an example, make sure that you allow only traffic from Cloudflare’s IP range. Ideally, ask your upstream Internet Service Provider (ISP) to apply an access control list (ACL), otherwise, attackers may target your servers’ IP addresses directly and bypass your protection.
Recommendations on how to protect your home and IoT appliances
Change the default username and password of any device that is connected to the Internet such as smart cameras and routers. This will reduce the risk that malware such as Mirai can gain access to your router and IoT devices.
Protect your home against malware with Cloudflare for Families. Cloudflare for Families is a free service that automatically blocks traffic from your home to malicious websites and malware communication.
Recent weeks have witnessed massive ransomware and ransom DDoS (Distributed Denial of Service) attack campaigns that interrupted aspects of critical infrastructure around the world, including one of the largest petroleum pipeline system operators, and one of the world’s biggest meat processing companies. Earlier this quarter, more than 200 organizations across Belgium, including the government and parliament websites and other services, were also DDoS’d.
And when most of the United States were celebrating Independence Day on July 4, hundreds of US companies were hit by a ransomware attack demanding 70 million USD in Bitcoin. Attackers known to be affiliated with REvil, a Russian ransomware group, exploited multiple previously unknown vulnerabilities in IT management software. The targets included schools, small public-sector bodies, travel and leisure organizations, and credit unions, to name a few. While the threat of ransomware and ransom DDoS is not new (read our posts on ransomware and ransom DDoS from 2021 Q1), the latest attacks on Internet properties ranging from wineries, professional sports teams, ferry services and hospitals has brought them from just being background noise to front page headlines affecting our day-to-day lives. In fact, recent attacks have propelled ransomware and DDoS to the top of US President Biden’s national security agenda.
The DDoS attack trends observed over Cloudflare’s network in 2021 Q2 paint a picture that reflects the overall global cyber threat landscape. Here are some highlights.
Over 11% of our surveyed customers who were targeted by a DDoS attack reported receiving a threat or ransom letter threatening in advance, in the first six months of this year. Emergency onboarding of customers under an active DDoS attack increased by 41.8% in 2021 H1 compared to 2020 H2.
HTTP DDoS attacks targeting government administration/public sector websites increased by 491%, making it the second most targeted industry after Consumer Services whose DDoS activity increased by 684% QoQ.
China remains the country with the most DDoS activity originating from within their borders — 7 out of every 1,000 HTTP requests originating from China were part of an HTTP DDoS attack targeting websites, and more than 3 out of every 100 bytes that were ingested in our data centers in China were part of a network-layer DDoS attack.
Emerging threats included amplification DDoS attacks that abused the Quote of the Day (QOTD) protocol which increased by 123% QoQ. Additionally, as the adoption of QUIC protocol continues to increase, so do attacks over QUIC — registering a whopping 109% QoQ surge in 2021 Q2. The number of network-layer DDoS attacks in the range of 10-100 Gbps increased by 21.4% QoQ. One customer that was attacked is Hypixel, an American gaming company. Hypixel remained online with no downtime and no performance penalties to their gamer users, even when under an active DDoS attack campaign larger than 620 Gbps. Read their story here.
To view all DDoS attack insights across all regions and industries worldwide, visit Cloudflare’s interactive Radar DDoS dashboard.
Application-layer DDoS attacks
Application-layer DDoS attacks, specifically HTTP DDoS attacks, are attacks that usually aim to disrupt an HTTP server by making it unable to process legitimate user requests. If a server is bombarded with more requests than it can process, the server will drop legitimate requests or even crash resulting in performance penalties or a denial of service event for legitimate users.
DDoS activity per market industry
When we analyze attacks, we calculate the ‘DDoS activity’ rate, which is the percentage of attack traffic out of the total traffic (attack + clean). This allows us to normalize the data points and avoid biases towards, for example, a larger data center that naturally handles more traffic and therefore also more attacks.
In 2021 Q2, Consumer Services was the most targeted industry followed by Government Administration and Marketing & Advertising.
DDoS activity per source country
To understand the origin of the HTTP attacks we observed over Cloudflare’s network, we look at the source IP address of the client generating the attack HTTP requests. Unlike network-layer attacks, source IPs cannot be spoofed in HTTP attacks. A high DDoS activity rate in a given country indicates large botnets operating from within.
China and the US remain in the first and second places, respectively, regarding the percentage of DDoS activity originating from within their territories. In China, more than 7 out of every 1,000 HTTP requests were part of an HTTP DDoS attack, while in the US almost 5 out of 1,000 HTTP requests were part of an attack.
DDoS activity per target country
In order to identify which countries the targets of the DDoS attacks resided in, we break down the DDoS activity by our customers’ billing countries. Note that Cloudflare does not charge for attack traffic and has pioneered providing unmetered and unlimited DDoS protection since 2017. By cross-referencing the attack data with our customers’ billing country, we can identify which countries were attacked the most.
Data observed in 2021 Q2 suggest that organizations in the US and China were the most targeted by HTTP DDoS attacks. In fact, one out of every 200 HTTP requests destined to US-based organizations was part of a DDoS attack.
Network-layer DDoS attacks
While application-layer attacks strike the application (Layer 7 of the OSI model) running the service end users are trying to access, network-layer attacks target network infrastructure (such as in-line routers and other network servers) and the Internet link itself.
The chart above shows the distribution of network-layer DDoS attacks in 2021 Q2.
Distribution of attacks by size (packet rate and bit rate)
There are different ways of measuring the size of a L3/4 DDoS attack. One is the volume of traffic it delivers, measured as the bit rate (specifically, gigabits-per-second). Another is the number of packets it delivers, measured as the packet rate (specifically, packets-per-second). Attacks with high bit rates attempt to saturate the Internet link, while attacks with high packet rates attempt to overwhelm the servers, routers or other in-line hardware appliances.
The distribution of attacks by their size (in bit rate) and month is shown below. As observed in the chart, all attacks over 300 Gbps were observed in the month of June.
In terms of bit rate, attacks under 500 Mbps constituted a majority of all DDoS attacks observed in 2021 Q2.
Similarly, looking from the lens of packet rate, nearly 94% of attacks were under 50K pps. Even though attacks from 1-10M pps constituted only 1% of all DDoS attacks observed, this number is 27.5% higher than that observed in the previous quarter, suggesting that larger attacks are not diminishing either — but rather increasing.
Note that while attacks under 500 Mbps and 50K pps might seem ‘small’ compared to other headline-making large attacks, they are often sufficient to create major disruptions for Internet properties that are not protected by an always-on, automated cloud-based DDoS protection service. Moreso, many organisations have uplinks provided by their service providers with a bandwidth capacity smaller than 1 Gbps. Assuming their public-facing network interface also serves legitimate traffic, DDoS attacks smaller than 500 Mbps are often capable of taking down exposed Internet properties.
Distribution by attack duration
Cloudflare continues to see a large percentage of DDoS attacks that last under an hour. In Q2, over 97% of all DDoS attacks lasted less than an hour.
Short burst attacks may attempt to cause damage without being detected by DDoS detection systems. DDoS services that rely on manual analysis and mitigation may prove to be useless against these types of attacks because they are over before the analyst even identifies the attack traffic.
Alternatively, the use of short attacks may be used to probe the cyber defenses of the target. Load-testing tools and automated DDoS tools, that are widely available on the dark web, can generate short bursts of a SYN flood, for example, and then follow up with another short attack using a different attack vector. This allows attackers to understand the security posture of their targets before they decide to launch larger attacks at larger rates and longer durations — which come at a cost.
In other cases, attackers generate small DDoS attacks as proof and warning to the target organization of the attacker’s ability to cause real damage later on. It’s often followed by a ransom email to the target organization, demanding payment to avoid suffering an attack that could more thoroughly cripple network infrastructure.
This highlights the need for an always on, automated DDoS protection approach. DDoS protection services that rely on manual re-routing, analysis and mitigation may prove to be useless against these types of attacks because they are over before the analyst can even identify the attack traffic.
Distribution of attacks by attack vectors
An attack vector is the term used to describe the method that the attacker utilizes in their attempt to cause a denial of service event.
As observed in previous quarters, attacks utilizing SYN floods and UDP-based protocols remain the most popular methods by attackers.
What is a SYN flood attack? It’s a DDoS attack that exploits the very foundation of the TCP protocol. A stateful TCP connection between a client and a server begins with a 3-way TCP handshake. The client sends an initial connection request packet with a synchronize flag (SYN). The server responds with a packet that contains a synchronized acknowledgment flag (SYN-ACK). Finally, the client responds with an acknowledgment (ACK) packet. At this point, a connection is established and data can be exchanged until the connection is closed. This stateful process can be abused by attackers to cause denial of service events.
By repeatedly sending SYN packets, the attacker attempts to overwhelm a server or the router’s connection table that tracks the state of TCP connections. The router replies with a SYN-ACK packet, allocates a certain amount of memory for each given connection, and falsely waits for the client to respond with the final ACK. Given a sufficient number of connections occupying the router’s memory, the router is unable to allocate further memory for legitimate clients, causing the router to crash or preventing it from handling legitimate client connections, i.e., a denial of service event.
Emerging threats
Emerging threats included amplification DDoS attacks that abuse the Quote of the Day (QOTD) service which increased by 123% QoQ. QOTD was defined in RFC-865 (1983) and can be sent over either the UDP or TCP protocols. It was originally designed for debugging and as a measurement tool, with no specific syntax for the quote. The RFC does however recommend the use of ASCII characters and to limit the length to 512 characters.
Furthermore, we’ve seen a 107% increase QoQ in UDP Portmap and Echo attacks — all of which are really old attack vectors. This may indicate attackers digging up old methods and attack tools to try and overcome protection systems. As we’ve seen in previous quarters, the adoption of the QUIC protocol continues to increase. Consequently, so do attacks over QUIC, or more specifically floods and amplification attacks of non-QUIC traffic in places where we’d expect to see QUIC traffic. In 2021 Q2, these types of attacks increased by 109% QoQ. This continued trend may indicate that attackers are attempting to abuse the QUIC-designated ports and gateways into organizations’ networks — searching for vulnerabilities and security holes.
DDoS activity by Cloudflare data center country
In 2021 Q2, our data center in Haiti observed the largest percentage of network-layer DDoS attack traffic, followed by Brunei (almost 3 out of every 100 packets were part of an attack) and China.
Note that when analyzing network-layer DDoS attacks, we bucket the traffic by the Cloudflare edge data center locations where the traffic was ingested, and not by the source IP. The reason for this is that, when attackers launch network-layer attacks, they can spoof the source IP address in order to obfuscate the attack source and introduce randomness into the attack properties, which may make it harder for simple DDoS protection systems to block the attack. Hence, if we were to derive the source country based on a spoofed source IP, we would get a spoofed country. Cloudflare is able to overcome the challenges of spoofed IPs by displaying the attack data by the location of Cloudflare’s data center in which the attack was observed. We’re able to achieve geographical accuracy in our report because we have data centers in over 200 cities around the world.
A note on ransomware and ransom DDoS — a growing global threat
The last few weeks have seen a resurgence of ransom-driven cyber threats: ransomware and ransom DDoS (RDDoS).
So what is ransomware and ransom DDoS, and how are they different?
Ransomware is malicious software that encrypts an organization’s systems and databases, rendering them inaccessible and unusable. Malware is usually introduced into an organization’s systems via phishing emails — tricking employees to click on a link or download a file. Once the malware is installed on the employee’s device, it encrypts the device and can propagate to the entire network of the organization’s servers and employee devices. The attacker will demand money, usually in the form of Bitcoin, in exchange for decrypting the organization’s systems and granting them access back to their systems.
Unlike a ransomware attack, a ransom DDoS attack does not encrypt a company’s systems; it aims to knock them offline if the ransom is not paid. What makes ransom DDoS attacks even more dangerous is that they do not require the attacker to gain access to a business’s internal systems to execute the attack. However, with a strong DDoS protection strategy in place, a ransom DDoS attack has little to no effect on businesses.
Ransomware and ransom DDoS threats are impacting most industries across the globe — the financial industry, transportation, oil and gas, consumer goods, and even education and healthcare.
Entities claiming to be ‘Fancy Lazarus’, ‘Fancy Bear’, ‘Lazarus Group’, and ‘REvil’ are once again launching ransomware and ransom-DDoS attacks against organizations’ websites and network infrastructure unless a ransom is paid before a given deadline. In the case of DDoS threats, prior to the ransom note, a small DDoS attack is usually launched as a form of demonstration. The demonstration attack is typically over UDP, lasting roughly 30-120 minutes.
The ransom note is typically sent to the common group email aliases of the company that are publicly available online such as noc@, support@, help@, legal@, abuse@, etc. In several cases, it has ended up in spam. In other cases, we’ve seen employees disregard the ransom note as spam, increasing the organization’s response time which resulted in further damage to their online properties.
Cloudflare’s recommendation for organizations that receive a threat or ransom note:
Do not panic, and we recommend you do not pay the ransom: Paying ransom only encourages and funds bad actors. There’s also no guarantee that you won’t be attacked again anyway.
Contact local law enforcement: Be ready to provide a copy of the ransom letter you received and any other logs or packet captures.
Activate an effective DDoS protection strategy: Cloud-based DDoS protection can be quickly onboarded in the event of an active threat, and with a team of security experts on your side, risks can be mitigated quickly and effectively.
Here’s a short video by Cloudflare CTO, John Graham-Cumming addressing the threat of ransom DDoS attacks.
Cloudflare protects Hypixel against a massive DDoS attack campaign
At Cloudflare, our teams have been exceptionally busy this past quarter rapidly onboarding (onto our Magic Transit service) a multitude of new and existing customers that have either received a ransom letter or were under an active DDoS attack.
One such customer is Hypixel Inc, the development studio behind the world’s largest Minecraft minigame server. With over 24M total unique logins to date and a world record 216,000+ concurrent players on PC, the Hypixel team works hard to add value to the experience of millions of players across the globe.
The gaming industry is often subject to some of the largest volumetric DDoS attacks — and as a marquee brand, Hypixel attracts more than its fair share. Uptime and high performance are fundamental to the functioning of Hypixel’s servers. Any perceived downtime or noticeable lag could result in an exodus of gamers.
When Hypixel was under a massive DDoS attack campaign, they turned to Cloudflare to extend their services with Cloudflare to include Magic Transit, Cloudflare’s BGP-based DDoS protection service for network infrastructure. After rapidly onboarding them overnight, Cloudflare was automatically able to detect and mitigate DDoS attacks targeting their network — several of which were well over 620 Gbps. The DDoS attack comprised mostly TCP floods and UDP amplification attacks. In the graph, the various colors represent the multiple Cloudflare systems that contribute to detecting and mitigating the multi-vector attack — emphasising the value of our multi-layered DDoS approach.
Even as attack patterns changed in real-time, Magic Transit shielded Hypixel’s network. In fact, because all their clean traffic routed over Cloudflare’s high performing low-latency network, Hypixel’s users noticed no change in gamer experience — even during an active volumetric DDoS attack.
During the attack campaign, Cloudflare automatically detected and mitigated over 5,000 DDoS attacks: 53% were ACK floods, 39% were UDP-based attacks and 8% SYN floods.
“We had several attacks of well over 620 Gbps with no impact at all on our players. Their gaming experience remained uninterrupted and fast, thanks to Cloudflare Magic Transit.” – Simon Collins-Laflamme, CEO, Hypixel Inc.
Hypixel’s journey with Cloudflare began with them employing Cloudflare Spectrum to help protect their gaming infrastructure against DDoS attacks. As their user base grew, they adopted additional Cloudflare products to bolster the robustness and resilience of all of their critical infrastructure. Today, they use multiple Cloudflare products including CDN, Rate Limiting, Spectrum, Argo Smart Routing, and Load Balancing to build and secure infrastructure that provides gamers around the world the real-time gaming experiences they need.
Get holistic protection against cyber attacks of any kind
DDoS attacks constitute just one facet of the many cyber threats organizations are facing today. As businesses shift to a Zero Trust approach, network and security buyers will face larger threats related to network access, and a continued surge in the frequency and sophistication of bot-related and ransomware attacks.
A key design tenet while building products at Cloudflare is integration. Cloudflare One is a solution that uses a Zero Trust security model to provide companies a better way to protect devices, data, and applications — and is deeply integrated with our existing platform of security and DDoS solutions.
In fact, Cloudflare offers an integrated solution that comprises an all-star cast featuring the following to name a few:
DDoS: LEADER in Forrester Wave™ for DDoS Mitigation Solutions, Q1 20211
WAF: Cloudflare is a CHALLENGER in the 2020 Gartner Magic Quadrant for Web Application Firewall (receiving the highest placement in the ‘Ability to Execute’)2
Zero Trust: Cloudflare is a LEADER in the Omdia Market Radar: Zero-Trust Access Report, 20203
Web protection: Innovation leader in the Global Holistic Web Protection Market for 2020 by Frost & Sullivan4
Cloudflare’s global (and growing) network is uniquely positioned to deliver DDoS protection and other security, performance, and reliability services with unparalleled scale, speed, and smarts.
AWS Shield is a managed service that protects applications that are running on Amazon Web Services (AWS) against external threats, such as bots and distributed denial of service (DDoS) attacks. Shield detects network and web application-layer volumetric events that may indicate a DDoS attack, web content scraping, or other unauthorized non-human traffic that is interacting with AWS resources.
In this blog post, I’ll show you some of the volumetric event trends from network traffic and web request patterns that we observed in 2020 as more workloads moved to the cloud. It includes insights that are broadly applicable to cloud applications and insights that are specific to gaming applications. I will also share tips and best practices that you can follow to protect the availability of the applications that you run on AWS.
DDoS trends as more developers rely on the cloud
In 2020, we saw an increase in developers building applications on AWS and protecting their availability with AWS Shield Advanced, which includes AWS WAF at no additional cost. The DDoS threat vectors we observed were similar to the ones that were observed in 2019, but they occurred with greater frequency. Between February 2020 and April 2020, we observed a 72% increase in the monthly number of events that were detected by Shield.
TCP SYN floods and UDP reflection attacks, which attempt to reflect and amplify packets off legitimate services running on the internet, were among the most common infrastructure-layer events detected by AWS Shield in 2020. (In this blog post, we’ll use the term infrastructure layer to refer to Layers 3 and 4 of the OSI model.) These tactics attempt to affect the availability of an application by overwhelming its ability to process packets or establish new connections on behalf of legitimate users. One of the oldest UDP reflection vectors, DNS reflection, remains the most common, at 15.5% of all infrastructure-layer events detected by Shield. TCP SYN floods were the second most common at 13.8%. This is unsurprising, because web applications commonly rely upon both DNS and TCP traffic. Bad actors can find a consistent supply of systems on the internet that can be used as reflectors, due to the properties of these protocols, or system misconfiguration.
Bad actors may use application-layer requests, in isolation or together with infrastructure-layer attacks, in their attempt to affect the availability of an application. The most common application-layer attack observed by Shield in 2020 was the web request flood, an observation that is consistent with prior years. This vector gives a bad actor more leverage, meaning that they can have a greater effect with less traffic and effort. Instead of having to exhaust the capacity of a network path, device, or other lower-level component, they only need to send more web requests than the application is able to handle. This attack vector was a significant cause of increased volumetric events detected by Shield in the first half of 2020. For more information about events detected by Shield during 2020, see Figure 1.
Figure 1: Monthly number of volumetric events detected by AWS Shield in 2020
A closer look at web application-layer attacks
The request volume of web application-layer events that are detected by AWS Shield has increased, an indication that bad actors are making greater investments in tactics that are more challenging to detect and mitigate than infrastructure-layer events. Shield continuously monitors DDoS activity and alerts customers if there is an elevated threat at any point in time. In 2020, Shield reported elevated threats on 53 days, 33 of which were caused by high-volume web request floods. There were 55 events with a volume of greater than 500,000 requests per second (RPS), some of which reached millions of RPS. The RPS of the 99th percentile (P99) of the volume of web request floods detected by Shield nearly doubled between the first and second halves of the year. (The 99th percentile is the request volume in RPS, below which 99% of request floods were observed.). For more information about the volume of web request floods detected by Shield in 2020, see Figure 2.
Figure 2: Quarterly P90 and P99 volume of web request floods detected by AWS Shield in 2020
It’s important to protect web applications against DDoS attacks of any size. The more common request floods are relatively small, but smaller attacks can affect an application if it isn’t architected for DDoS resiliency. You can follow these best practices to help protect your web application against request floods and other DDoS attacks:
Protect internet-facing resources with AWS Shield Advanced. You can use AWS Shield Advanced to protect your applications that are running on AWS against most common, frequently occurring network and transport layer DDoS attacks. When you add protected resources in AWS Shield Advanced, network volumetric attacks against those resources are detected and mitigated more quickly. You also receive visibility into security events by using the AWS Shield console, API, or Amazon CloudWatch metrics. If you need assistance during an active event, you can quickly engage with AWS Shield experts or escalate to the AWS Shield Response Team (SRT).
Access greater network and request capacity with Amazon CloudFront and Amazon Route 53. You can use these services to serve static and dynamic web content, as well as DNS answers, by using the global network of AWS edge locations. This provides you with greater capacity to help mitigate large volumetric attacks. Applications that are fronted by Amazon CloudFront and Amazon Route 53 also benefit from inline mitigation that continually inspects all traffic and mitigates most infrastructure-layer DDoS attempts in less than one second. CloudFront and the AWS Shield DDoS mitigation systems use SYN cookies to verify new connections, which protects against SYN floods and other traffic floods that aren’t valid for the application. (A SYN cookie is a technique by which the Shield infrastructure encodes connection setup information into the SYN response (SYN-ACK packet) in such a way that the TCP connection resources are only consumed for legitimate clients who complete the TCP handshake.)
Use AWS WAF and rate-based rules to mitigate application-layer attacks. AWS Shield Advanced provides you with protection against infrastructure-layer attacks that can be mitigated with network-based DDoS mitigation systems. When you add Shield Advanced protection to CloudFront or Application Load Balancer (ALB) for serving web content, you receive AWS WAF at no additional cost. AWS Managed Rules for AWS WAF makes it easy to select and apply pre-configured rules, depending on your specific requirements. You also receive web request flood detection and can mitigate security events by configuring rate-based rules to match and temporarily block IP addresses that are sending traffic above a rate that you define. For larger applications, or applications that span multiple AWS accounts, you can use AWS Firewall Manager to deploy and manage rules across all of your resources.
Considerations unique to gaming use cases
On AWS, you can build and protect any kind of application. Internet-facing applications are more likely to receive DDoS attacks, particularly if a bad actor is motivated to disrupt the normal function of the application. We looked across AWS Shield data and found that one type of application stood out as the most likely to be targeted by DDoS attacks: gaming servers. Gaming servers host matches between players on their personal computers or gaming consoles. 16% of infrastructure-layer events detected by Shield in 2020 targeted gaming applications. The application might be targeted simply out of malice, or to gain an advantage in the game. Between Q1 2020 and Q2 2020, we observed a 46% increase in the frequency of events that were detected on behalf of gaming applications. This increase aligns with the increased use of residential internet networks during the same time.
There are unique considerations for protecting a gaming application against DDoS attacks. Many gaming applications rely upon UDP traffic, which makes it infeasible to block UDP as a countermeasure against the most common DDoS attacks, like UDP reflection attacks or UDP floods. You can nevertheless protect your gaming application and the experience of your players by using Elastic IP addresses and protecting these resources with AWS Shield Advanced. Shield Advanced has the ability to perform deep packet inspection of all traffic, even at extremely high PPS rates. Using that powerful tool, the AWS Shield Response Team (SRT) can work with you to understand your application and build a custom mitigation that allows only valid player traffic.
Reacting to extortion attempts
From August 2020 through November 2020, we saw a revival of DDoS extortion attempts, a tactic that is now more than six years old. Each extortion attempt reported by customers to the AWS SRT had familiar characteristics. A malicious actor would target an application that wasn’t running on AWS as a proof of concept and then threaten a larger, follow-on attack if a ransom wasn’t paid. Although it’s very uncommon for the follow-on attack to actually occur, application owners take these threats seriously and use the opportunity to assess their own protection and operational readiness. In approximately 90% of AWS support cases related to these attempts, the SRT assisted the application owners directly with their preparation. We also assisted Shield Advanced customers who weren’t directly targeted by extortion attempts but were aware of other extortion campaigns.
One question that we frequently hear is how AWS can help developers monitor their applications and take quick action if a possible DDoS attack is detected. When you protect your resources with AWS Shield Advanced, you have the option to associate an Amazon Route 53 health check. The status of the health check is used to improve the decisions that are made by the Shield detection system. If you have Shield Advanced proactive engagement enabled, the SRT is automatically engaged any time a Shield event corresponds to an unhealthy Route 53 health check that is associated to your protected resource. Based on the contact information provided in the Shield console, an SRT engineer will contact you to coordinate a response to the detected event. If you’re running a web application, you can choose to delegate access to your Shield Advanced and AWS WAF APIs to the SRT and provide the team with copies of your AWS WAF logs. During an escalation, an SRT engineer will evaluate your logs for DDoS signatures and robotic patterns and assist in building effective mitigations.
Summary
In this blog post, I shared some of the trends that were observed by AWS Shield in 2020, as well as steps that you can take to protect the availability of your applications against DDoS attacks. If you’d like to learn more about DDoS protection on AWS and configuring AWS Shield Advanced, check out the following resources:
If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on the AWS Shield forum or contact AWS Support.
Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.
We’re thrilled to announce that Cloudflare has been named a leader in The Forrester WaveTM: DDoS Mitigation Solutions, Q1 2021. You can download a complimentary copy of the report here.
According to the report, written by, Forrester Senior Analyst for Security and Risk, David Holmes, “Cloudflare protects against DDoS from the edge, and fast… customer references view Cloudflare’s edge network as a compelling way to protect and deliver applications.”
Unmetered and unlimited DDoS protection for all
Cloudflare was founded with the mission to help build a better Internet — one where the impact of DDoS attacks is a thing of the past. Over the last 10 years, we have been unwavering in our efforts to protect our customers’ Internet properties from DDoS attacks of any size or kind. In 2017, we announced unmetered DDoS protection for free — as part of every Cloudflare service and plan including the Free plan — to make sure every organization can stay protected and available.
Thanks to our home-grown automated DDoS protection systems, we’re able to provide unmetered and unlimited DDoS protection for free. Our automated systems constantly analyze traffic samples asynchronously as to avoid impact to performance. They scan for DDoS attacks across layers 3-7 of the OSI model. They look for patterns in IP packets, HTTP requests and HTTP responses. When an attack is identified, a real-time signature is generated in the form of an ephemeral mitigation rule. The rule is propagated to the most optimal location in our edge for the most cost-efficient mitigation: either in the Linux kernel’s eXpress Data Path (XDP), Linux userspace iptables or in the HTTP reverse-proxy. A cost-efficient mitigation strategy means that we can mitigate the most volumetric, distributed attacks without impacting performance.
Read more about how Cloudflare’s DDoS protection systems work here.
DDoS attacks increasing
We’d like to say DDoS attacks are a thing of the past. But unfortunately, they are not.
On the contrary, we continue to see the frequency, sophistication, and geographical distribution of DDoS attacks rise every quarter – in quantity or size. See our reports from last year (Q1 ‘20, Q2 ‘20, Q3 ‘20, and Q4 ‘20) and view overall Internet traffic trends here on Cloudflare Radar.
Over the past year, Cloudflare has seen and automatically mitigated some of the largest and arguably the most creative cyber attacks. As attackers are getting bolder and smarter in their ways, organizations are looking for ways to battle these kinds of attacks with no disruption to the services they provide.
DDoS attacks in 2020
Organizations are being extorted under threat of DDoS
In January this year, we shared the story of how we helped a Fortune Global 500 company stay online and protected whilst they were targeted by a ransom DDoS attack. They weren’t the only one. In fact, in the fourth quarter of 2020, 17% of surveyed Cloudflare customers reported receiving a ransom or a threat of DDoS attack. In Q1 2021, this increased to 26% — roughly 1 out of every 4 respondents reported a ransom threat and a subsequent DDoS attack on their network infrastructure.
Whether organizations are targeted with ransom attacks or amateur ‘cyber vandalism’, it’s important for organizations to utilize an always-on, automated DDoS protection service that doesn’t require manual human intervention in the hour of need. We take great pride in being able to provide this level of protection to our customers.
Continuous improvement
As attacks have continued to evolve, and the number of customers using our services has increased, Cloudflare has continually invested in our technology to stay several steps ahead of attackers. We’ve made significant investments in bolstering our mitigation capacity, honing our detection algorithms, and providing better analytics capabilities to our customers. Our aim is to make impact from DDoS attacks a thing of the past, for all customers, just like spam in the 90s.
In 2019, we rolled out our autonomous DDoS detection and mitigation system, dosd. This component of our mitigation stack is fully software-defined, leverages Linux’s eXpress Data Path (XDP), and allows us to quickly and automatically deploy eBPF rules that run on each packet received for inspection — mitigating the most sophisticated attacks within less than 3 seconds on average at the edge and other common attacks instantly. It works by detecting patterns in the attack traffic and then quickly deploying rules autonomously to drop the offenders at wire speed. Additionally, because dosd operates independently within each data center, with no reliance on a centralized data center, it greatly increases the resilience of our network.
While dosd is great at mitigating attacks by detecting patterns in the traffic, what about patternless attacks? That’s where flowtrackd comes in, our novel TCP state classification engine, built in 2020, to defend against disruptive L3/L4 attacks targeting our Magic Transit customers. It’s able to detect and mitigate the most randomized, sophisticated attacks. Additionally, at L7, we also learn our customer’s traffic baselines and identify when their origin is in distress. When an origin server shows signs of deterioration, our systems begin soft mitigation in order to reduce the impact on the server and allow it to recuperate.
Building advanced DDoS protection systems is not only about the detection, but also about cost efficient mitigation. We aim to mitigate attacks without impacting performance that can be caused due to excessive computational consumption. This requirement is why we introduced IP Jails to the world: IP Jails is a gatebot capability that mitigates the most volumetric and distributed attacks without impacting performance. Gatebot activates IP Jails when attacks become significantly volumetric, and then instead of blocking at L7, IP Jails temporarily drops the connection of the offending IP address that generated the request which matched the attack signature that gatebot created. IP Jails leverages the Linux iptables mechanism to drop packets at wirespeed. Dropping L7 attacks at L4, is significantly more cost-efficient, and benefits both our customers and our Site Reliability Engineering team.
Lastly, to provide our customers better visibility and insight into the increasingly sophisticated attacks we’re seeing and mitigating, we released the Firewall Analytics dashboard in 2019. This dashboard provides insights into both HTTP application security and DDoS activity at L7, allowing customers to configure rules directly from within analytics dashboards thus tightening the feedback loop for responding to events. Later in 2020, we released an equivalent dashboard for L3/4 activity for our enterprise Magic Transit and Spectrum customers, in the form of the Network Analytics dashboard. Network Analytics provides insight into packet-level traffic and DDoS attack activity, along with periodical Insights and Trends. To complement the dashboards and provide our users the right information as they need it, we rolled out real-time DDoS alerts and also periodical DDoS reports — right into your inboxes. Or if you prefer, directly into your SIEM dashboards.
Cloudflare received the top score in the strategy category
This year, due to our advanced DDoS protection capabilities, Cloudflare received the top score in the strategy category and among the top three in the current offering category. Additionally, we were given the highest possible scores in 15 criteria in the report, including:
Threat detection
Burst attacks
Response automation
Speed of implementation
Product vision
Performance
Security operation center (SOC) service
We believe that our standing stems from the sustained investments we’ve made over the last few years in our global Anycast network — which serves as the foundation of all services we provide to our customers.
Our network is architected for scale — every service runs on every server in every Cloudflare data center that spans over 200 cities globally. And as opposed to some of the other vendors in the report, every Cloudflare service is delivered from every one of our edge data centers.
Integrated security and performance
A leading application performance monitoring company that uses Cloudflare’s services for serverless compute and content delivery recently told us that they wanted to consolidate their performance and security services under one provider. They got rid of their incumbent L3 services provider and onboarded Cloudflare for their application and network services (with Magic Transit) for easier management and better support.
We see this more and more. The benefits of using a single cloud provider for bundled security and performance services are plentiful:
Deep service integration – all of our services are deeply integrated which allows our users to truly leverage the power of Cloudflare. As an example, Bot Management rules are implemented with our Application Firewall.
Easier troubleshooting — instead of having to reach out to multiple providers, our customers have a single point of contact when troubleshooting. Additionally, we provide immediate human response in our under attack hotline.
Lower latency — because every one of our services are delivered from all of our data centers, there are no performance penalties. As an example, there are no additional routing hops between the DDoS service to Bot Management service to CDN service.
However, not all cloud services are built the same, i.e. most vendors today do not have a comprehensive and robust solution to offer. Cloudflare’s unique architecture enables it to offer an integrated solution that comprises an all-star cast featuring the following to name a few:
CDN: Customer’s Choice LEADER in 2020 Gartner Peer Insights ‘Voice of the Customer’: Global CDN1
DDoS: Received the highest number of high scores in the 2020 Gartner report for Solution Comparison for DDoS Cloud Scrubbing Centers2
WAF: Cloudflare is a CHALLENGER in the 2020 Gartner Magic Quadrant for Web Application Firewall (receiving the highest placement in the ‘Ability to Execute’)3
Zero Trust: Cloudflare is a LEADER in the Omdia Market Radar: Zero-Trust Access Report, 20204
Bot Management: Leader in the 2020 SPARK Matrix of Bot Management Market5
Integrated solution: Innovation leader in the Global Holistic Web Protection Market for 2020 by Frost & Sullivan6
We are pleased to be named a LEADER in The Forrester Wave™: for DDoS Mitigation Solutions, Q1 2021 report, and will continue to work tirelessly to remain, as the report puts it, a “compelling way to protect and deliver applications” for our customers.
For more information about Cloudflare’s DDoS protection, reach out to us here or hands-on evaluation of Cloudflare, sign up here.
Network-layer DDoS attacks are on the rise, prompting security teams to rethink their L3 DDoS mitigation strategies to prevent business impact. Magic Transit protects customers’ entire networks from DDoS attacks by placing our network in front of theirs, either always on or on demand. Today, we’re announcing new functionality to improve the experience for on-demand Magic Transit customers: flow-based monitoring. Flow-based monitoring allows us to detect threats and notify customers when they’re under attack so they can activate Magic Transit for protection.
Magic Transit is Cloudflare’s solution to secure and accelerate your network at the IP layer. With Magic Transit, you get DDoS protection, traffic acceleration, and other network functions delivered as a service from every Cloudflare data center. With Cloudflare’s global network (59 Tbps capacity across 200+ cities) and <3sec time to mitigate at the edge, you’re covered from even the largest and most sophisticated attacks without compromising performance. Learn more about Magic Transit here.
Using Magic Transit on demand
With Magic Transit, Cloudflare advertises customers’ IP prefixes to the Internet with BGP in order to attract traffic to our network for DDoS protection. Customers can choose to use Magic Transit always on or on demand. With always on, we advertise their IPs and mitigate attacks all the time; for on demand, customers activate advertisement only when their networks are under active attack. But there’s a problem with on demand: if your traffic isn’t routed through Cloudflare’s network, by the time you notice you’re being targeted by an attack and activate Magic Transit to mitigate it, the attack may have already caused impact to your business.
On demand with flow-based monitoring
Flow-based monitoring solves the problem with on-demand by enabling Cloudflare to detect and notify you about attacks based on traffic flows from your data centers. You can configure your routers to continuously send NetFlow or sFlow (coming soon) to Cloudflare. We’ll ingest your flow data and analyze it for volumetric DDoS attacks.
Send flow data from your network to Cloudflare for analysis
When an attack is detected, we’ll notify you automatically (by email, webhook, and/or PagerDuty) with information about the attack.
Cloudflare detects attacks based on your flow data
You can choose whether you’d like to activate IP advertisement with Magic Transit manually – we support activation via the Cloudflare dashboard or API – or automatically, to minimize the time to mitigation. Once Magic Transit is activated and your traffic is flowing through Cloudflare, you’ll receive only the clean traffic back to your network over your GRE tunnels.
Activate Magic Transit for DDoS protection
Using flow-based monitoring with Magic Transit on demand will provide your team peace of mind. Rather than acting in response to an attack after it impacts your business, you can complete a simple one-time setup and rest assured that Cloudflare will notify you (and/or start protecting your network automatically) when you’re under attack. And once Magic Transit is activated, Cloudflare’s global network and industry-leading DDoS mitigation has you covered: your users can continue business as usual with no impact to performance.
Example flow-based monitoring workflow: faster time to mitigate for Acme Corp
Let’s walk through an example customer deployment and workflow with Magic Transit on demand and flow-based monitoring. Acme Corp’s network was hit by a large ransom DDoS attack recently, which caused downtime for both external-facing and internal applications. To make sure they’re not impacted again, the Acme network team chose to set up on-demand Magic Transit. They authorize Cloudflare to advertise their IP space to the Internet in case of an attack, and set up Anycast GRE tunnels to receive clean traffic from Cloudflare back to their network. Finally, they configure their routers at each data center to send NetFlow data to a Cloudflare Anycast IP.
Cloudflare receives Acme’s NetFlow data at a location close to the data center sending it (thanks, Anycast!) and analyzes it for DDoS attacks. When traffic exceeds attack thresholds, Cloudflare triggers an automatic PagerDuty incident for Acme’s NOC team and starts advertising Acme’s IP prefixes to the Internet with BGP. Acme’s traffic, including the attack, starts flowing through Cloudflare within minutes, and the attack is blocked at the edge. Clean traffic is routed back to Acme through their GRE tunnels, causing no disruption to end users – they’ll never even know Acme was attacked. When the attack has subsided, Acme’s team can withdraw their prefixes from Cloudflare with one click, returning their traffic to its normal path.
When the attack subsides, withdraw your prefixes from Cloudflare to return to normal
Get started
To learn more about Magic Transit and flow-based monitoring, contact us today.
DDoS attack trends in the final quarter of 2020 defied norms in many ways. For the first time in 2020, Cloudflare observed an increase in the number of large DDoS attacks. Specifically, the number of attacks over 500Mbps and 50K pps saw a massive uptick.
In addition, attack vectors continued to evolve, with protocol-based attacks seeing a 3-10x increase compared to the prior quarter. Attackers were also more persistent than ever — nearly 9% of all attacks observed between October and December lasted more than 24 hours.
Below are additional noteworthy observations from the fourth quarter of 2020, which the rest of this blog explores in greater detail.
Number of attacks: For the first time in 2020, the total number of attacks observed in Q4 decreased compared to the prior quarter.
Attack duration: 73% of all attacks observed lasted under an hour, a decrease from 88% in Q3.
Attack vectors: While SYN, ACK, and RST floods continued to be the dominant attack vectors deployed, attacks over NetBIOS saw a whopping 5400% increase, followed by those over ISAKMP and SPSS.
Global DDoS activity: Our data centers in Mauritius, Romania, and Brunei recorded the highest percentages of DDoS activity relative to non-attack traffic.
Additional attack tactics:Ransom DDoS (RDDoS) attacks continue to target organizations around the world as criminal groups attempt to extort a ransom in the form of Bitcoin under a threat of a DDoS attack.
Number of attacks
For the first time in 2020, the total number of network layer DDoS attacks we observed decreased compared to the previous quarter. Q4 constituted 15% of all attacks observed in 2020, compared to Q3’s 48%. In fact, the total number of attacks in Q4 was less than that seen in the month of September alone by a whopping 60%. On a monthly basis, December was Q4’s busiest month for attackers.
Attack rates
There are different ways of measuring an L3/4 DDoS attack’s size. One is the volume of traffic it delivers, or its ‘bit rate’ (measured in gigabits-per-second). Another is the number of packets it delivers, or its ‘packet rate’ (measured in packets-per-second). Attacks with high bit rates attempt to saturate last-mile network links of the target, and attacks with high packet rates attempt to overwhelm routers or other in-line hardware devices.
In Q4, as in previous quarters, the majority of attacks were quite small — under 1 Gbps and 1M pps, specifically. This trend is not surprising, since most attacks are launched by amateur attackers using tools that are easy to use and cost a few dollars at most. Small attacks may also serve as a smokescreen to distract security teams from other kinds of cyberattacks, or to test a network’s existing defense mechanisms.
However, the overall popularity of small attacks didn’t tell the whole story in Q4. Attacks over 500Mbps and 50K pps constituted a larger percentage of total attacks than they did in previous quarters. In fact, the number of attacks over 100 Gbps increased by 10x from Q3, and those over 10M pps increased by 3.6x.
One unique large attack Cloudflare observed was an ACK flood DoS attack that was automatically detected and mitigated by Cloudflare’s systems. What was unique about this attack was not the max packet rate, but the attack method that appears to have been borrowed from the world of acoustics.
As can be seen in the graph above, the attack’s packet rate followed a wave-shaped pattern for over 19 hours. It seems as though the attacker was inspired by an acoustics concept called beat. For this reason, we codenamed this attack “Beat”. In acoustics, a beat is a term that is used to describe an interference of two different wave frequencies. You can read more about the Beat attack in our blog post: Beat – An Acoustics Inspired DDoS Attack
Whether packet intensive or bit intensive, the increase in large DDoS attacks is a disturbing trend. It indicates that attackers are getting more brazen, and are using tools that allow them to launch larger attacks. What’s worse, often larger attacks have implications to not just target the network, but also intermediary service providers that serve the target network downstream.
Attack Duration
73% of attacks in Q4 ‘20 lasted for under an hour. On the other end of the spectrum, nearly 9% of attacks lasted over 24 hrs (compared to a mere 1.5% in Q3 ’20). This increase reinforces the need for a real-time, always-on defense system to protect against attacks of every size and duration.
Attack vectors
An ‘attack vector’ is a term used to describe the attack method. The most popular method, SYN floods, constituted nearly 42% of all attacks observed in Q3, followed by ACK, RST, and UDP-based DDoS attacks. This is relatively consistent with observations from previous quarters. However, ACK attacks jumped from ninth place in Q3 to second place — a 13x increase quarter-over-quarter— dethroning RST attacks from second place.
Top emerging threats
While TCP based attacks like SYN and RST floods remain popular, UDP-protocol specific attacks such as NetBIOS and ISAKMP-based DDoS attacks are seeing an explosion compared to the prior quarter.
NetBIOS is a protocol that allows applications on separate machines to communicate and access shared resources over a local area network, and ISAKMP is a protocol used to establish Security Associations (SAs) and cryptographic keys when setting up an IPsec VPN connection (IPsec uses the Internet Key Exchange (IKE) protocol to ensure secure connections and will authenticate and encrypt packets of data sent over an Internet Protocol (IP) network.)
Cloudflare continues to see protocol based attacks — and indeed, multi-vector attacks — deployed to attempt to bring networks down. As the complexity of attacks elevates, adequate DDoS protection needs to be put in place to keep organizations secure and online at all times.
Global DDoS activity
To understand where these attacks come from, we look at the Cloudflare edge network data centers where the traffic was ingested, rather than the location of the source IP. The reason? When attackers launch L3/4 attacks, they can spoof the source IP address in order to obfuscate their attack’s source.
In this report, we also measure the attack traffic observed at a Cloudflare data center relative to the non-attack traffic observed at the same data center for geo-based distribution. This gives us more accuracy in our endeavor to pinpoint geographic locations that are observing more threats than others. We’re able to achieve geographical accuracy in our report because we have data centers in over 200 cities, in more than 100 countries around the world.
Looking at Q4 metrics, we observed interesting insights — our data centers in Mauritius, Romania, and Brunei recorded the highest percentages of attack traffic relative to non-attack traffic. Specifically, between 4.4% and 4.9% of all traffic in those countries came from DDoS attacks. Another way of saying this is that almost 5 out of every 100 bytes was part of attack traffic. These observations indicate increased botnet activities in those countries.
What might explain the comparatively high incidence of DDoS attacks in these countries? While it’s impossible to say for sure, here are some possibilities for the top two countries on the list:
Mauritius – In August 2020, a state of environmental emergency was declared in Mauritius after a ship carrying nearly 4,000 tons of fuel cracked its hull. The oil spill ignited anti-government protests calling for the resignation of the prime minister. Since then, the government has suspended the parliament twice, and has also been accused of suppressing local media and independent reporting covering the incident. Even five months after, following a series of human-rights scandals, the protests continue. The events in Mauritius may be linked to the increased DDoS activity.
Source: wikipedia
Romania – Two events may be behind the increased DDoS activity in Romania. Romania recently held parliamentary elections which ended on December 6, 2020. In addition, the EU announced on December 9th that Romania will host their new cyber security research hub, the European Cybersecurity Industrial, Technology and Research Competence Centre (ECCC). Another possible explanation is that Romania is the country with the cheapest super-fast broadband Internet in the world — making it easier for anyone to launch volumetric attacks from within Romania.
🎉#Bucharest to be the host of the future EU #Cyber Centre🎉Hub for high tech and innovation, featuring a thriving digital ecosystem, dynamic and young, Romania’s capital will take this task in a responsible and dedicated manner, to the benefit of the entire European Union👏👏👏
Ransom-based attacks continue to plague organizations
In our previous quarterly DDoS report, we noted a rise in extortion and ransom-based DDoS (RDDoS) attacks around the world. In a RDDoS attack, a malicious party threatens a person or organization with a cyberattack that could knock their networks, websites, or applications offline for a period of time, unless the person or organization pays a ransom. You can read more about RDDoS attacks here.
In Q4 ‘20, this disturbing trend continued. Organizations large and small came to Cloudflare asking for help in keeping their network infrastructure online while they figured out how to respond to ransom notes. Read this story of what a Fortune Global 500 company did when they received a ransom note, and about their recommendations for organizations.
Cloudflare continues to closely monitor this trend. If you receive a threat:
Do not panic — we recommend you to not pay the ransom: Paying the ransom only encourages bad actors and finances illegal activities — and there’s no guarantee attackers won’t attack your network anyway.
Notify local law enforcement: They will also likely request a copy of the ransom letter that you received.
Contact Cloudflare: We can help ensure your website and network infrastructure are safeguarded from these ransom attacks.
Cloudflare DDoS Protection
Cloudflare provides comprehensive L3-L7 DDoS protection. In 2017, we pioneered the elimination of the industry standard surge pricing for DDoS attacks, providing customers with unmetered and unlimited DDoS protection. Since then, we’ve onboarded thousands of customers of all sizes — including Wikimedia, Panasonic, and Discord — that use Cloudflare to protect and accelerate their Internet properties. Why do they choose Cloudflare? Three main reasons:
1. No scrubs Cloudflare doesn’t operate scrubbing centers as we believe that the scrubbing center model is a flawed approach to DDoS protection. Scrubbing centers cause delays and cost too much to build and run. What’s more, DDoS attacks are asymmetric — attackers have more available bandwidth than a single scrubbing center will ever be able to handle.
Cloudflare’s network is architected so that every machine in every data center performs DDoS mitigation. Doing this at the edge is the only way to mitigate at scale without impacting performance. Our Anycast-based architecture makes our capacity equivalent to our DDoS scrubbing capacity, the largest in the market at 51 Tbps. This means Cloudflare detects and mitigates DDoS attacks close to the source of attack. Better yet, Cloudflare’s global threat intelligence acts like an immune system for the Internet — employing our machine learning models to learn from and mitigate attacks against any customer to protect them all.
2. It’s about time Most organizations are in some stage of their journey from on-prem to the cloud. The threat landscape, functional requirements, and scale of business applications are evolving faster than ever before, and the volume and sophistication of network attacks are already straining the defensive capabilities of even the most advanced enterprises. One concern many enterprises have when adopting the cloud is added latency for applications. Most cloud-based DDoS protection services rely on specialized data centers aka “scrubbing centers” for DDoS mitigation. Backhauling traffic to those data centers can add significant latency depending on its location relative to the destination server.
This problem compounds when an organization uses different providers for different networking functions. When traffic must hop from provider to provider, latency can be measured in hundreds of milliseconds.
Cloudflare’s distributed geographical presence ensures that attacks are globally detected and mitigated in under 3 seconds on average — making it one of the fastest in the industry.
3. It’s not just about DDoS DDoS attacks constitute just one facet of the many cyber threats organizations are facing today. As businesses shift to a Zero Trust approach, network and security buyers will face larger threats related to network access, and a continued surge in the frequency and sophistication of bot-related attacks.
A key design tenet while building products at Cloudflare is integration. Cloudflare One is a solution that uses a Zero Trust security model to provide companies a better way to protect devices, data, and applications — and is deeply integrated with our existing platform of security and DDoS solutions.
To learn more about Cloudflare’s DDoS solution contact us or get started today by signing up on our dashboard.
I love building products that solve real problems for our customers. These days I don’t get to do so as much directly with our Engineering teams. Instead, about half my time is spent with customers listening to and learning from their security challenges, while the other half of my time is spent with other Cloudflare Product Managers (PMs) helping them solve these customer challenges as simply and elegantly as possible. While I miss the deeply technical engineering discussions, I am proud to have the opportunity to look back every year on all that we’ve shipped across our application security teams.
Taking the time to reflect on what we’ve delivered also helps to reinforce my belief in the Cloudflare approach to shipping product: release early, stay close to customers for feedback, and iterate quickly to deliver incremental value. To borrow a term from the investment world, this approach brings the benefits of compounded returns to our customers: we put new products that solve real-world problems into their hands as quickly as possible, and then reinvest the proceeds of our shared learnings immediately back into the product.
It is these sustained investments that allow us to release a flurry of small improvements over the course of a year, and be recognized by leading industry analyst firms for the capabilities we’ve accumulated and distributed to our customers. Today we’re excited to announce that Frost & Sullivan has named Cloudflare the Innovation Leader in their Frost Radar™: Global Holistic Web Protection Market Report. Frost & Sullivan’s view that this market “will gradually absorb the markets formed around legacy and point solutions” is consistent with our view of the world, and we’re leading the way in “the consolidation of standalone WAF, DDoS mitigation, and Bot Risk Management solutions” they believe is “poised to happen before 2025”.
We are honored to receive this recognition, based on the analysis of 10 providers’ competitive strengths and opportunities as assessed by Frost & Sullivan. The rest of this post explains some of the capabilities that we shipped in 2020 across our Web Application Firewall (WAF), Bot Management, and Distributed Denial-of-Service product lines—the scope of Frost & Sullivan’s report. Get a copy of the Frost & Sullivan Frost Radar report to see why Cloudflare was named the Innovation Leader here.
2020 Web Security Themes and Roundup
Before jumping into specific product and feature launches, I want to briefly explain how we think about building and delivering our web security capabilities. The most important “product” by far that’s been built at Cloudflare over the past 10 years is the massive global network that moves bits securely around the world, as close to the speed of light as possible. Building our features atop this network allows us to reject the legacy tradeoff of performance or security. And equipping customers with the ability to program and extend the network with Cloudflare Workers and Firewall Rules allows us to focus on quickly delivering useful security primitives such as functions, operators, and ML-trained data—then later packaging them up in streamlined user interfaces.
We talk internally about building up the “toolbox” of security controls so customers can express their desired security posture, and that’s how we think about many of the releases over the past year that are discussed below. We begin by providing the saw, hammer, and nails, and let expert builders construct whatever defenses they see fit. By watching how these tools are put to use and observing the results of billions of attempts to evade the erected defenses, we learn how to improve and package them together as a whole for those less inclined to build from components. Most recently we did this with API Shield, providing a guided template to create “positive security” models within Firewall Rules using existing primitives plus new data structures for strong authentication such as Cloudflare-managed client SSL/TLS certificates. Each new tool added to the toolbox increases the value of the existing tools. Each new web request—good or bad—improves the models that our threat intelligence and Bot Management capabilities depend upon.
Web application firewall (WAF) usability at scale
Last year we spoke with many customers about our plan to decouple configuration from the zone/domain model and allow rules to be set for arbitrary paths and groups of services across an account. In 4Q2020 we put this granular control in the hands of a few developers and some of our most sophisticated enterprise customers, and we’re currently collecting and incorporating feedback before defaulting the capabilities on for new customers.
Rules are great, especially with increased flexibility, but without data structures and request enrichment at the edge (such as the Bot Management techniques described below) they cannot act on anything beyond static properties of the request. In 3Q2020 we released our IP Lists capabilities and customers have been steadily uploading their home-grown and third-party subscription lists. These lists can be referenced anywhere in a customer’s account as named variables and then combined with all other attributes of the request, even Bot Management scores, e.g., http.request.uri.path contains “/login” and (not ip.src in $pingdom_probes and cf.bot_management.score < 30) is a Firewall Rule filter that blocks all bots except Pingdom from accessing the login endpoint.
Built predominantly in 4Q2020 and currently being tested in the Firewall Rules engine is a brand new implementation of our Rate Limiting engine. By moving this matching and enforcement logic from a standalone tool to a component within a performant, memory-safe, expressive engine built in Rust, we have increased the utility of existing functions. Additional examples of improving this library of capabilities include the work completed in 1Q2020 to add HMAC functions and regex-based HTTP header and body inspection to the engine.
Bots and machine learning (ML)
In addition to making edge data sets accessible for request evaluation, we continued to invest heavily within our Bot Management team to provide actionable data so that our customers could decide what (if any) automated traffic they wanted to allow to interact with their applications. Our highest priority for Bot research and development has always been efficacy, and last year was no different. A significant portion of our engineering effort was dedicated to our detection engines — both updating and iterating on existing systems or creating entirely new detection engines from scratch.
In 1Q2020 we completed a total rewrite of our Machine Learning engine, and are continually focused on improving the efficacy of our ML engines. To do this, we draw on one of our major competitive advantages: the massive amount of data flowing through Cloudflare’s network. The early 2020 upgrade to our ML model nearly doubled the number of features we use to evaluate and score requests. And to help customers better understand why requests are flagged as bots, we have recently complemented the bot likelihood score in our logs with attribution to the specific engine that generated the score.
Also in 1Q2020, we upgraded our behavioral analysis engine to incorporate more features and increase overall accuracy. This engine conducts histogram-based outlier scoring and is now fully deployed to nearly all Bot Management zones.
In 2Q2020, we developed a lightweight JavaScript element that further advanced our browser fingerprinting capabilities and aids in detection. Specifically, we now silently challenge browsers and detect if a browser is misrepresenting its User Agent. This technique will be incorporated into our ML models and combined with our heuristics engine for more accurate browser fingerprinting. This feature is entirely optional and can be enabled or disabled by customers through our UI and API. Customers with extremely performance sensitive zones or traffic types that are unsuitable for JavaScript (such as API or some mobile app traffic) can still be accurately scored by our Bot Management engine.
In addition to detection, we also spent (and will continue to spend) engineering effort on mitigation. Our entire JavaScript and CAPTCHA challenge platform was rewritten in the last year and deployed to our customer zones in a staged fashion in the second half of 2020. Our new platform is faster and more robust at detecting automated systems attempting to solve the challenges. More importantly, this platform allows us to further invest in new challenge types and modes as we enter 2021.
The biggest and most well received feature released in 2020 was our dedicated Bot Management analytics, released in 3Q2020. We now present informative graphs that double as diagnostic tools. Customers have found that analytics are far more than interesting charts and statistics: in the case of Bot Management, analytics are essential to spotting and subsequently eliminating false positives.
Last but definitely not least, we announced the deprecation of the __cfduid cookie in 4Q2020 which was used primarily to detect bots but caused confusion for some customers including questions about whether they needed to display a cookie banner because of what we do.
To get a sense of the Bot Attack trends we saw in the first half of 2020, take a read through this blog post. And if you’re curious about how our ML models and heuristic engines work to keep your properties safe, this deep dive by Alex Bocharov, Machine Learning Tech Lead on the Bots team, is an excellent guide.
API and IoT security and protection
At the beginning of 4Q2020, we released a product called API Shield that was purpose built to secure, protect, and accelerate API traffic — and will eventually provide much of the common functionality expected in traditional API Gateways. The UI for API Shield was built on top of Firewall Rules for maximum flexibility, and will serve as the jump-off point for configuring additional API security features we have planned this year.
As part of API Shield, every customer now gets a fully managed, domain-scoped private CA generated for each of their zones, and we plan to continue working closely with the SSL/TLS team to expand CA management options based on feedback. Since the release, we’ve seen great adoption from in particular IoT companies focused on locking down their APIs using short-lived client certificates distributed out to devices. Customers can also now upload OpenAPI schemas to be matched against incoming requests from these devices, with bad requests being dropped at the edge rather than passed on to origin infrastructure.
Another capability we released in 4Q2020 was support for gRPC-based API traffic. Since that release, customers have expressed significant interest in using Cloudflare as a secure API gateway between easy-to-use customer-facing JSON endpoints and internal-facing gRPC or GraphQL endpoints. Like most customer challenges at Cloudflare, early adopters are looking to solve these use cases initially with Cloudflare Workers, but we’re keeping an eye on whether there are aspects for which we’ll want to provide first-class feature support.
Distributed Denial-of-Service (DDoS) protections for web applications and APIs
The application-layer security of a web application or API is of minimal importance if the service itself is not available due to a persistent DDoS attack at L3-L7. While mitigating such attacks has long been one of Cloudflare’s strengths, attack methodologies evolve and we continued to invest heavily in 2020 to drop attacks more quickly, more efficiently, and more precisely; as a result, automatic mitigation techniques are applied immediately and most malicious traffic is blocked in less than 3 seconds.
Early in 2020 we responded to a persistent increase in smaller, more localized attacks by fine-tuning a system that can autonomously detect attacks on any server in any datacenter. In the month prior to us first posting about this tool, it mitigated almost 300,000 network-layer attacks, roughly 55 times greater than the tool we previously relied upon. This new tool, dubbed “dosd”, leverages Linux’s eXpress Data Path (XDP) and allows our system to quickly — and automatically — deploy rules eBPF rules that run on each packet received. We further enhanced our edge mitigation capabilities in 3Q2020 by developing and releasing a protection layer that can operate even in environments where we only see one side of the TCP flow. These network layer protections help protect our customers who leverage both Magic Transit to protect their IP ranges and our WAF to protect their applications and APIs.
To document and provide visibility into these attacks, we released a GraphQL-backed interface in 1Q2020 called Network Analytics. Network Analytics extends the visibility of attacks against our customers’ services from L7 to L3, and includes detailed attack logs containing data such as top source and destination IPs and ports, ASNs, data centers, countries, bit rates, protocol and TCP flag distributions. A litany of improvements made to this graphical rendering engine over the course of 2020 have benefitted all analytics tools using the same front-end. In 4Q2020, Network Analytics was extended to provide traffic and attack insights into Cloudflare Spectrum-protected applications, which are terminated at L4 (TCP/UDP).
2020 was a tough year around the world. Throughout what has also been, and continues to be, a period of heightened cyberattacks and breaches, we feel proud that our teams were able to release a steady flow of new and improved capabilities across several critical security product areas reviewed by Frost & Sullivan. These releases culminated in far greater protections for customers at the end of the year than the beginning, and a recognition for our sustained efforts.
We are pleased to have been named the Innovation Leader in their Frost Radar™: Global Holistic Web Protection Market Report, which “addresses organizations’ demand for consolidated, single pane of glass solutions, which not only reduce the security gaps of legacy products but also provide simplified management capabilities”.
As we look towards 2021 we plan to continue releasing early and often, listening to feedback from our customers, and delivering incremental value along the way. If you have ideas on what additional capabilities you’d like to use to protect your applications and networks, we’d love to hear them below in the comments.
In late 2020, a major Fortune Global 500 company was targeted by a Ransom DDoS (RDDoS) attack by a group claiming to be the Lazarus Group. Cloudflare quickly onboarded them to the Magic Transit service and protected them against the lingering threat. This extortion attempt was part of wider ransom campaigns that have been unfolding throughout the year, targeting thousands of organizations around the world. Extortionists are threatening organizations with crippling DDoS attacks if they do not pay a ransom.
Throughout 2020, Cloudflare onboarded and protected many organizations with Magic Transit, Cloudflare’s DDoS protection service for critical network infrastructure, the WAF service for HTTP applications, and the Spectrum service for TCP/UDP based applications — ensuring their business’s availability and continuity.
Unwinding the attack timeline
I spoke with Daniel (a pseudonym) and his team, who work at the Incident Response and Forensics team at the aforementioned company. I wanted to learn about their experience, and share it with our readers so they could learn how to better prepare for such an event. The company has requested to stay anonymous and so some details have been omitted to ensure that. In this blog post, I will refer to them as X.
Initially, the attacker sent ransom emails to a handful of X’s publicly listed email aliases such as press@, shareholder@, and hostmaster@. We’ve heard from other customers that in some cases, non-technical employees received the email and ignored it as being spam which delayed the incident response team’s time to react by hours. However, luckily for X, a network engineer that was on the email list of the hostmaster@ alias saw it and immediately forwarded it to Daniel’s incident response team.
In the ransom email, the attackers demanded 20 bitcoin and gave them a week to pay up, or else a second larger attack would strike, and the ransom would increase to 30 bitcoin. Daniel says that they had a contingency plan ready for this situation and that they did not intend to pay. Paying the ransom funds illegitimate activities, motivates the attackers, and does not guarantee that they won’t attack anyway.
…Please perform a google search of “Lazarus Group” to have a look at some of our previous work. Also, perform a search for “NZX” or “New Zealand Stock Exchange” in the news. You don’t want to be like them, do you?…
The current fee is 20 Bitcoin (BTC). It’s a small price to pay for what will happen if your whole network goes down. Is it worth it? You decide!…
If you decide not to pay, we will start the attack on the indicated date and uphold it until you do. We will completely destroy your reputation and make sure your services will remain offline until you pay…
–An excerpt of the ransom note
The contingency plan
Upon receiving the email from the network engineer, Daniel called him and they started combing through the network data — they noticed a significant increase in traffic towards one of their global data centers. This attacker was not playing around, firing gigabits per second towards a single server. The attack, despite just being a proof of intention, saturated the Internet uplink to that specific data center, causing a denial of service event and generating a series of failure events.
This first “teaser” attack came on a work day, towards the end of business hours as people were already wrapping up their day. At the time, X was not protected by Cloudflare and relied on an on-demand DDoS protection service. Daniel activated the contingency plan which relied on the on-demand scrubbing center service.
Daniel contacted their DDoS protection service. It took them over 30 minutes to activate the service and redirect X’s traffic to the scrubbing center. Activating the on-demand service caused networking failures and resulted in multiple incidents for X on various services — even ones that were not under attack. Daniel says hindsight is 2020 and he realized that an always-on service would have been much more effective than on-demand, reactionary control that takes time to implement, after the impact is felt. The networking failures amplified the one-hour attack resulting in incidents lasting much longer than expected.
Onboarding to Cloudflare
Following the initial attack, Daniel’s team reached out to Cloudflare and wanted to onboard to our automated always-on DDoS protection service, Magic Transit. The goal was to onboard to it before the second attack would strike. Cloudflare explained the pre-onboarding steps, provided details on the process, and helped onboard X’s network in a process Daniel defined as “quite painless and very professional. The speed and responsiveness were impressive. One of the key differentiation is the attack and traffic analytics that we see that our incumbent provider couldn’t provide us. We’re seeing attacks we never knew about being mitigated automatically.”
The attackers promised a second, huge attack which never happened. Perhaps it was just an empty threat, or it could be that the attackers detected that X is protected by Cloudflare which deterred them and they, therefore, decided to move on to their next target?
Recommendations for organizations
I asked Daniel if he has any recommendations for businesses so they can learn from his experience and be better prepared, should they be targeted by ransom attacks:
1. Utilize an automated always-on DDoS protection service
Do not rely on reactive on-demand SOC-based DDoS Protection services that require humans to analyze attack traffic. It just takes too long. Don’t be tempted to use an on-demand service: “you get all of the pain and none of the benefits”. Instead, onboard to a cloud service that has sufficient network capacity and automated DDoS mitigation systems.
2. Work with your vendor to build and understand your threat model
Work together with your DDoS protection vendor to tailor mitigation strategies to your workload. Every network is different, and each poses unique challenges when integrating with DDoS mitigation systems.
3. Create a contingency plan and educate your employees
Be prepared. Have plans ready and train your teams on them. Educate all of your employees, even the non-techies, on what to do if they receive a ransom email. They should report it immediately to your Security Incident Response team.
Cloudflare customers need not worry as they are protected. Enterprise customers can reach out to their account team if they are being extorted in order to review and optimize their security posture if needed. Customers on all other plans can reach out to our support teams and learn more about how to optimize your Cloudflare security configuration.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
