On Monday, September 30, customers on Verizon’s mobile network in multiple cities across the United States reported experiencing a loss of connectivity. Impacted phones showed “SOS” instead of the usual bar-based signal strength indicator, and customers complained of an inability to make or receive calls on their mobile devices.
AS6167 (CELLCO) is the autonomous system used by Verizon for its mobile network. To better understand how the outage impacted Internet traffic on Verizon’s network, we took a look at HTTP request volume from AS6167 independent of geography, as well as traffic from AS6167 in various cities that were reported to be the most significantly impacted.
Although initial reports of connectivity problems started around 09:00 ET (13:00 UTC), we didn’t see a noticeable change in request volume at an ASN level until about two hours later. Just before 12:00 ET (16:00 UTC), Verizon published a social media post acknowledging the problem, stating “We are aware of an issue impacting service for some customers. Our engineers are engaged and we are working quickly to identify and solve the issue.”
As the Cloudflare Radar graph below shows, a slight decline (-5%) in HTTP traffic as compared to traffic at the same time a week prior is first visible around 11:00 ET (15:00 UTC). Request volume fell as much as 9% below expected levels at 13:45 ET (17:45 UTC).
Just after 17:00 ET (21:00 UTC), Verizon published a second social media post noting, in part, “Verizon engineers are making progress on our network issue and service has started to be restored.” Request volumes returned to expected levels around the same time, surpassing the previous week’s levels at 17:15 ET (21:15 UTC). At 19:18 ET (23:18 UTC), a social media post from Verizon noted “Verizon engineers have fully restored today’s network disruption that impacted some customers. Service has returned to normal levels.”
Media reports listed cities including Chicago, Indianapolis, New York City, Atlanta, Cincinnati, Omaha, Phoenix, Denver, Minneapolis, Seattle, Los Angeles, and Las Vegas as being most impacted. In addition to looking at comparative traffic trends across the whole Verizon Wireless network, we also compared request volumes in the listed cities to the same time a week prior (September 23).
Declines in request traffic starting around 11:00 ET (15:00 UTC) are clearly visible in cities including Los Angeles, Seattle, Omaha, Denver, Phoenix, Minneapolis, Indianapolis, and Chicago. In contrast to other cities, Omaha’s request volume was already trending lower than last week heading into today’s outage, but its graph clearly shows the impact of today’s disruption as well. Omaha’s difference in traffic was the most significant, down approximately 30%, while other cities saw declines in the 10-20% range.
Request traffic from Las Vegas initially appeared to exhibit a bit of volatility around 11:00 ET (15:00 UTC), but continues to track fairly closely to last week’s levels before exceeding them starting at 16:00 ET (20:00 UTC). Cincinnati was tracking slightly above last week’s request volume before the outage began, and tracked closely to the prior week during the outage period.
We observed week-over-week traffic increases during the outage period in New York and Atlanta. However, in both cities, traffic was already slightly above last week’s levels, and that trend continued throughout the day.
Based on our observations, it appears that voice services on Verizon’s network may have been more significantly impacted than data services, as we saw some declines in request traffic across impacted cities, but none experienced full outages.
As of this writing (19:15 ET, 23:15 UTC), no specific information has been made available by Verizon regarding the root cause of the network problems.
Cloudflare Radar showcases global Internet traffic patterns, attack activity, and technology trends and insights. It is powered by data from Cloudflare’s global network, as well as aggregated and anonymized data from Cloudflare’s 1.1.1.1 public DNS Resolver, and is built on top of a rich, publicly accessible API. This API allows users to explore Radar data beyond the default set of visualizations, for example filtering by protocol, comparing metrics across multiple locations or autonomous systems, or examining trends over two different periods of time. However, not every user has the technical know-how to make a raw API query or process the JSON-formatted response.
Today, we are launching the Cloudflare Radar Data Explorer, which provides a simple Web-based interface to enable users to easily build more complex API queries, including comparisons and filters, and visualize the results. And as a complement to the Data Explorer, we are also launching an AI Assistant, which uses Cloudflare Workers AI to translate a user’s natural language statements or questions into the appropriate Radar API calls, the results of which are visualized in the Data Explorer. Below, we introduce the AI Assistant and Data Explorer, and also dig into how we used Cloudflare Developer Platform tools to build the AI Assistant.
Ask the AI Assistant
Sometimes, a user may know what they are looking for, but aren’t quite sure how to build the relevant API query by selecting from the available options and filters. (The sheer number may appear overwhelming.) In those cases, they can simply pose a question to the AI Assistant, like “Has there been an uptick in malicious email over the last week?” The AI Assistant makes a series of Workers AI and Radar API calls to retrieve the relevant data, which is visualized within seconds:
The AI Assistant pane is found on the right side of the page in desktop browsers, and appears when the user taps the “AI Assistant” button on a mobile browser. To use the AI Assistant, users just need to type their question into the “Ask me something” area at the bottom of the pane and submit it. A few sample queries are also displayed by default to provide examples of how and what to ask, and clicking on one submits it.
The submitted question is evaluated by the AI Assistant (more below on how that happens), and the resulting visualization is displayed in the Results section of the Data Explorer. In addition to the visualization of the results, the appropriate Data, Filter, and Compare options are selected in the Query section above the visualization, allowing the user to further tune or refine the results if necessary. The Keep current filters toggle within the AI Assistant pane allows users to build on the previous question. For example, with that toggle active, a user could ask “Traffic in the United States”, see the resultant graph, and then ask “Compare it with traffic in Mexico” to add Mexico’s data to the graph.
Building a query directly
For users that prefer a more hands-on approach, a wide variety of Radar datasets are available to explore, including traffic metrics, attacks, Internet quality, email security, and more. Once the user selects a dataset, the Breakdown By: dropdown is automatically populated with relevant options (if any), and Filter options are also dynamically populated. As the user selects additional options, the visualization in the Result section is automatically updated.
In addition to building the query of interest, Data Explorer also enables the user to compare the results, both against a specific date range and/or another location or autonomous system (AS). To compare results with the immediately previous period (the last seven days with the seven days before that, for instance), just toggle on the Previous period switch. Otherwise, clicking on the Date Range field brings up a calendar that enables the user to select a starting date — the corresponding date range is intelligently selected, based on the date range selected in the Filter section. To compare results across locations or ASNs, clicking on the “Location or ASN” field brings up a search box in which the user can enter a location (country/region) name, AS name, or AS number, with search results updating as the user types. Note that locations can be compared with other locations or ASes, and ASes can be compared with other ASes or locations. This enables a user, for example, to compare trends for their ISP with trends for their country.
Visualizing the results
Much of the value of Cloudflare Radar comes from its visualizations – the graphs, maps, and tables that illustrate the underlying data, and Data Explorer does not disappoint here. Depending on the dataset and filters selected, and the volume of data returned, results may be visualized in a time series graph, bar chart, treemap, or global choropleth map. The visualization type is determined automatically based on the contents of the API response. For example, the presence of countryalpha2 keys in the response means a choropleth map will be used, the presence of timestamps in the response means a line graph (“xychart”) should be shown, and more than 40 items in the response selects a treemap as the visualization type.
To illustrate the extended visualizations available in Data Explorer, the figure below is an expanded version of one that would normally be found on Radar’s Adoption & Usage page. The “standard” version of the graph plots the shares of the HTTP versions over the last seven days for the United States, as well as the summary share values. In this extended version of the graph generated in the Data Explorer, we compare data for the United States with HTTP version share data for AS701 (Verizon), for both the past seven days and the previous seven-day period. In addition to the comparisons plotted on the time series graph, the associated summary values are also compared in an accompanying bar chart. This comprehensive visualization makes comparisons easy.
For some combinations of datasets/filters/comparisons, time series graphs can get quite busy, with a significant number of lines being plotted. To isolate just a single line on the graph, double-click on the item in the legend. To add/remove additional lines back to/from the graph, single-click on the relevant legend item.
Similar to other visualizations on Radar, the resulting graphs or maps can be downloaded, copied, or embedded into another website or application. Simply click on the “Share” button above the visualization card to bring up the Share modal dialog. We hope to see these graphs shared in articles, blog posts, and presentations, and to see embedded visualizations with real-time data in your portals and operations centers!
Still want to use the API? No problem.
Although Data Explorer was designed to simplify the process of building, and viewing the results of, more complex API queries, we recognize that some users may still want to retrieve data directly from the API. To enable that, Data Explorer’s API section provides copyable API calls as a direct request URL and a cURL command. The raw data returned by the query is also available to copy or download as a JSON blob, for those users that want to save it locally, or paste it into another application for additional manipulation or analysis.
How we built the AI Assistant
Knowing all that AI is capable of these days, we thought that creating a system for an LLM to answer questions didn’t seem like an overly complex task. While there were some challenges, Cloudflare’s developer platform tools thankfully made it fairly straightforward.
LLM-assisted API querying
The main challenge we encountered in building the API Assistant was the large number of combinations of datasets and parameters that can potentially be visualized in the Data Explorer. There are around 100 API endpoints from which the data can be fetched, with most able to take multiple parameters.
There were a few potential approaches to getting started. One was to take a previously trained LLM and further train it with the API endpoint descriptions in order to have it return the output in the required structured format which would then be used to execute the API query. However, for the first version, we decided against this approach of fine-tuning, as we wanted to quickly test a few different models supported by Workers AI, and we wanted the flexibility to easily add or remove parameter combinations, as Data Explorer development was still under way. As such, we decided to start with prompt engineering, where all the endpoint-specific information is placed in the instructions sent to the LLM.
Putting the full detailed description of the API endpoints supported by the Data Explorer into the system prompt would be possible for an LLM with a larger context window (the number of tokens the model takes as input before generating output). Newer models are getting better with the needle in a haystack problem, which refers to the issue whereby LLMs do not retrieve information (the needle) equally well if it is placed in different positions within the long textual input (the haystack). However, it has been empirically shown that the position of information within the large context still matters. Additionally, many of the Radar API endpoints have quite similar descriptions, and putting all the descriptions in a single instruction could be more confusing for the model, and the processing time also increases with larger contexts. Based on this, we adopted the approach of having multiple inference calls to an LLM.
First, when the user enters a question, a Worker sends this question and a short general description of the available datasets to the LLM, asking it to determine the topic of the question. Then, based on the topic returned by the model, a system prompt is generated with the endpoint descriptions, including only those related to the topic. This prompt, along with the original question, is sent to the LLM asking it to select the appropriate endpoint and its specific parameters. At the same time, two parallel inference calls to the model are also made, one with the question and the system prompt related to the description of location parameters, and another with the description of time range parameters. Then, all three model outputs are put together and validated.
If the final output is a valid dataset and parameter combination, it is sent back to the Data Explorer, which executes the API query and displays the resulting visualization for the user. Different LLMs were tested for this task, and at the end, openhermes-2.5-mistral-7b, trained on code datasets, was selected as the best option. To give the model more context, not only is the user’s current question sent to the model, but the previous one and its response are as well, in case the next question asked by the user is related to the previous one. In addition, calls to the model are sent through Cloudflare’s AI Gateway, to allow for caching, rate limiting, and logging.
After the user is shown the result, they can indicate whether what was shown to them was useful or not by clicking the “thumbs up” or “thumbs down” icons in the response. This rating information is saved with the original question in D1, our serverless SQL database, so the results can be analyzed and applied to future AI Assistant improvements.
The full end-to-end data flow for the Cloudflare Radar AI Assistant is illustrated in the diagram below.
When the LLM doesn’t know the answer
In some cases, however, the LLM may not “know” the answer to the question posed by the user. If the model does not generate a valid final response, then the user is shown three alternative questions. The intent here is to guide the user into asking an answerable question — that is, a question that is answerable with data from Radar.
This is achieved using a previously compiled (static) list of various questions related to different Radar datasets. For each of these questions, their embedding is calculated using an embeddings model, and stored in our Vectorizevector database. “Embeddings” are numerical representations of textual data (vectors) capturing their semantic meaning and relationships, with similar text having vectors that are closer. When a user’s question does not generate a valid model response, the embedding of that question is calculated, and its vector is compared against all the stored vectors from the vector database, and the three most similar ones are selected. These three questions, determined to be similar to the user’s original question, are then shown to the user.
There are also cases when the LLM gives answers which do not correspond to what the user asked, as hallucinations are currently inevitable in LLMs, or when time durations are calculated inaccurately, as LLMs sometimes struggle with mathematical calculations. To help guard against this, AI Assistant responses are first validated against the API schema to confirm that the dataset and the parameter combination is valid. Additionally, Data Explorer dropdown options are automatically populated based on the AI Assistant’s response, and the chart titles are also automatically generated, so the user always knows exactly what data is shown in the visualization, even if it might not answer their actual question.
Looking ahead
We’re excited to enable more granular access to the rich datasets that currently power Cloudflare Radar. As we add new datasets in the future, such as DNS metrics, these will be available through Data Explorer and AI Assistant as well.
As noted above, Radar offers a predefined set of visualizations, and these serve as an excellent starting point for further exploration. We are adding links from each Radar visualization into Data Explorer, enabling users to further analyze the associated data to answer more specific questions. Clicking the “pie chart” icon next to a graph’s description brings up a Data Explorer page with the relevant metrics, options, and filters selected.
Correlating observations across two different metrics is another capability that we are also working on adding to Data Explorer. For example, if you are investigating an Internet disruption, you will be able to plot traffic trends and announced IP address space for a given country or autonomous system on the same graph to determine if both dropped concurrently.
But for now, use the Data Explorer and AI Assistant to go beyond what Cloudflare Radar offers, finding answers to your questions about what’s happening on the Internet. If you share Data Explorer visualizations on social media, be sure to tag us: @CloudflareRadar (X), noc.social/@cloudflareradar (Mastodon), and radar.cloudflare.com (Bluesky). You can also reach out on social media, or contact us via email, with suggestions for future Data Explorer and AI Assistant functionality.
Historically, traffic graphs on Cloudflare Radar have displayed two metrics: total traffic and HTTP traffic. These graphs show normalized traffic volumes measured in bytes, derived from aggregated NetFlow data. (NetFlow is a protocol used to collect metadata about IP traffic flows traversing network devices.) Today, we’re adding an additional metric that reflects the number of HTTP requests, normalized over the same time period. By comparing bytes with requests, readers can gain additional insights into traffic patterns and user behavior. Below, we review how this new data has been incorporated into Radar, and explore HTTP request traffic in more detail.
Note that while we refer to “HTTP request traffic” in this post and on Radar, the term encompasses requests made in the clear over HTTP and over encrypted connections using HTTPS – the latter accounts for ~95% of all requests to Cloudflare during July 2024.
New and updated graphs
Graphs including HTTP request-based traffic data have been added to the Overview and Traffic sections on Cloudflare Radar. On the Overview page, the “Traffic trends” graph now includes a drop-down selector at the upper right, where you can choose between “Total & HTTP bytes” and “HTTP requests & bytes”. We explore the distinction between these further in the following sections.
The default “Total & HTTP bytes” selection displays a time series graph, showing total bytes and HTTP bytes traffic over time, as Radar has done for several years now.
Selecting “HTTP requests & bytes” from the dropdown switches the view to a time series graph that HTTP requests traffic and HTTP bytes traffic over time. In both graphs, users can click on a metric in the legend to deselect it and remove it from the graph. These (de)selections are maintained when a user chooses to download or save a graph.
In addition, we’ve added a “Protocols” summary next to the graph that shows the share of bytes over the selected time period that HTTP accounts for, and the remaining aggregate share associated with the protocols used by other non-HTTP Cloudflare services (such as DNS, WARP, etc.). For most locations or ASNs, HTTP traffic will comprise the majority share of bytes-based traffic.
On Radar’s Traffic page, we have added the HTTP requests metric to the “Traffic volume” graph at the top of the page, allowing you to see how request volume has changed during the selected time period as compared to the previous period, in addition to the changes in the bytes-based metrics.
A new standalone request-based “HTTP traffic” graph was also added to the Traffic page, just below the bytes-based “Traffic trends” graph. This new graph shows normalized HTTP request traffic volume across the selected time period, and by default, also compares it with the previous time period.
Similar to other Radar graphs, these new HTTP request-based graphs can also be downloaded, copied to the clipboard, or embedded in other websites – just click on the share icon.
As always, the underlying data is also available through the Radar API. The “HTTP requests Time Series” API endpoint returns normalized HTTP request time series data across the specified time period for the requested location or autonomous system (ASN).
What is HTTP request traffic?
An HTTP GET request is a message sent from a client (such as your web browser) to a web server (such as one operated by Cloudflare), asking for a particular resource (file). In addition to returning the requested resource, which could range from a single-pixel GIF accounting for just a few bytes, to an API call that returns a few kilobytes of data, to a multi-gigabyte software package, the Web server also returns a set of headers, which can include information about the content type, the last time the resource was modified, cookie information, cacheability, and more. While GET requests account for the overwhelming majority of HTTP request traffic, such traffic also includes other HTTP request methods including HEAD, POST, PUT, and more.
Cloudflare temporarily logs HTTP requests received by our network, including associated header information and “metadata” about the request, such as the bot score computed for the request and the associated cache status. Request logs for a customer’s web properties are available for them to download, and after processing and analysis, this data is also presented in the Analytics section of the Cloudflare dashboard. The HTTP request data now available on Radar is based on a sample of this log data, aggregated across Cloudflare’s global customer base.
The value of request-based traffic insights
Cloudflare Radar already has HTTP data, so why add more? One key reason for analyzing and including HTTP request traffic is resilience. Having multiple sources of truth with respect to HTTP traffic allows us to better and more quickly distinguish between real events (such as an Internet disruption in a given country or network) and data pipeline issues.
While bytes-based metrics provide a reasonable proxy into human (user) behavior, especially with respect to activity surrounding Internet disruptions, request-based metrics provide an even better perspective. A lot of HTTP traffic involves relatively small responses – especially API traffic, which now accounts for 60% of all traffic. Furthermore, response sizes can vary widely, ranging from a single-pixel GIF accounting for just a few bytes, to an API call that returns a few kilobytes of data, to a multi-gigabyte software package
To that end, the scope of user activity may be insufficiently reflected by a bytes-based metric, or buried in the noise, whereas request activity provides a cleaner signal and a more direct proxy for user activity. This is especially important as we examine the restoration of connectivity after an Internet disruption, attempting to ascertain when activity has returned to “expected” pre-disruption levels.
Finally, incorporating request-based traffic insights into Radar is simply extending the way that the data is already being used on the site. All of the graphs, maps, and tables presented on Radar’s Adoption & Usage page, are based on analysis of HTTP request traffic, making use of information contained within request headers (such as HTTP version or user agent) or characteristics of the underlying connection (such as IP version).
Bytes vs requests – what’s the difference?
The current “HTTP traffic” view aggregates the bytes associated with HTTP requests to Cloudflare’s content delivery (CDN) services from the selected location or autonomous system (ASN). “Total traffic” aggregates this HTTP traffic along with the traffic associated with other Cloudflare services, including our 1.1.1.1 DNS resolver, authoritative DNS, WARP, and Spectrum, among others. (While Spectrum, WARP, and 1.1.1.1 also carry HTTP traffic, the share of HTTP traffic carried by these services is opaque to Radar, and isn’t accounted for as part of the HTTP traffic calculations.)
The bytes associated with a given request include the size of the request, the size of the headers associated with the response, and the size of the response itself. As noted above, the size of a file returned in response to a request can vary widely, depending on what was requested. The shape of the HTTP requests and HTTP bytes lines may be quite similar, but the potential variability in response sizes (in aggregate) can cause the lines to diverge, sometimes significantly so. For example, if an application regularly makes background requests to check for updates, the availability and subsequent download of a large file containing a software update would cause a spike in the HTTP bytes line, while the HTTP requests pattern remained consistent.
As another example, consider the graph below, capturing HTTP requests and bytes traffic trends for Portugal during the first week of August. HTTP bytes traffic initially grows each day between 06:00 and 09:00 UTC (07:00 – 10:00 local summer time), increases much more slowly until around 19:00 UTC (20:00 local summer time), and then increases rapidly before peaking around 21:00 UTC (22:00 local time). This suggests that content consumed during the workday is lighter in terms of bytes (such as API traffic, as discussed above), while evening traffic is more byte-heavy (possibly due to increased consumption of media content). In contrast, after starting to increase around 06:00 UTC (07:00 local summer time), request traffic generally sees three successively higher peaks each day – occurring around 10:00, 14:00, and 21:00 UTC respectively (11:00, 15:00, and 22:00 local summer time). These peaks are most pronounced on weekdays, but are still apparent on weekend days as well, suggesting regular patterns of user activity at those times.
It is important to remember that in looking at the “HTTP requests & bytes” graphs on Radar that they are showing two different metrics, and as such, only their shape over time is comparable, not their relative sizes. (As both metrics are normalized on a 0 to 1 (Max) scale, the lines on the graph are scaled relative to the maximum normalized value of each metric, including the previous period.)
Conclusion
The addition of HTTP request metrics to Cloudflare Radar brings additional visibility to traffic trends at a global, location, and network level, complementing the existing bytes-based HTTP traffic metrics. Derived from traffic to customer web properties, these new metrics can be found on Radar’s Overview and Traffic pages.
Historically, traffic graphs on Cloudflare Radar have displayed two metrics: total traffic and HTTP traffic. These graphs show normalized traffic volumes measured in bytes, derived from aggregated NetFlow data. (NetFlow is a protocol used to collect metadata about IP traffic flows traversing network devices.) Today, we’re adding another metric that reflects the number of HTTP requests, normalized over the same time period. By comparing bytes with requests, readers can gain additional insights into traffic patterns and user behavior. Below, we review how this new data has been incorporated into Radar, and explore HTTP request traffic in more detail.
Note that while we refer to “HTTP request traffic” in this post and on Radar, the term encompasses requests made in the clear over HTTP and over encrypted connections using HTTPS – the latter accounts for ~95% of all requests to Cloudflare during July 2024.
New and updated graphs
Graphs including HTTP request-based traffic data have been added to the Overview and Traffic sections on Cloudflare Radar. On the Overview page, the “Traffic trends” graph now includes a drop-down selector at the upper right, where you can choose between “Total & HTTP bytes” and “HTTP requests & bytes”. We explore the distinction between these further in the following sections.
The default “Total & HTTP bytes” selection displays a time series graph, showing total bytes and HTTP bytes traffic over time, as Radar has done for several years now.
Selecting “HTTP requests & bytes” from the dropdown switches the view to a time series graph that HTTP requests traffic and HTTP bytes traffic over time. In both graphs, users can click on a metric in the legend to deselect it and remove it from the graph. These (de)selections are maintained when a user chooses to download or save a graph.
In addition, we’ve added a “Protocols” summary next to the graph that shows the share of bytes over the selected time period that HTTP accounts for, and the remaining aggregate share associated with the protocols used by other non-HTTP Cloudflare services (such as DNS, WARP, etc.). For most locations or ASNs, HTTP traffic will comprise the majority share of bytes-based traffic.
On Radar’s Traffic page, we have added the HTTP requests metric to the “Traffic volume” graph at the top of the page, allowing you to see how request volume has changed during the selected time period as compared to the previous period, in addition to the changes in the bytes-based metrics.
A new standalone request-based “HTTP traffic” graph was also added to the Traffic page, just below the bytes-based “Traffic trends” graph. This new graph shows normalized HTTP request traffic volume across the selected time period, and by default, also compares it with the previous time period.
Similar to other Radar graphs, these new HTTP request-based graphs can also be downloaded, copied to the clipboard, or embedded in other websites – just click on the share icon.
As always, the underlying data is also available through the Radar API. The “HTTP requests Time Series” API endpoint returns normalized HTTP request time series data across the specified time period for the requested location or autonomous system (ASN).
What is HTTP request traffic?
An HTTP GET request is a message sent from a client (such as your web browser) to a web server (such as one operated by Cloudflare), asking for a particular resource (file). In addition to returning the requested resource, which could range from a single-pixel GIF accounting for just a few bytes, to an API call that returns a few kilobytes of data, to a multi-gigabyte software package, the Web server also returns a set of headers, which can include information about the content type, the last time the resource was modified, cookie information, cacheability, and more. While GET requests account for the overwhelming majority of HTTP request traffic, such traffic also includes other HTTP request methods including HEAD, POST, PUT, and more.
Cloudflare temporarily logs HTTP requests received by our network, including associated header information and “metadata” about the request, such as the bot score computed for the request and the associated cache status. Request logs for a customer’s web properties are available for them to download, and after processing and analysis, this data is also presented in the Analytics section of the Cloudflare dashboard. The HTTP request data now available on Radar is based on a sample of this log data, aggregated across Cloudflare’s global customer base.
The value of request-based traffic insights
Cloudflare Radar already has HTTP data, so why add more? One key reason for analyzing and including HTTP request traffic is resilience. Having multiple sources of truth with respect to HTTP traffic allows us to better and more quickly distinguish between real events (such as an Internet disruption in a given country or network) and data pipeline issues.
While bytes-based metrics provide a reasonable proxy into human (user) behavior, especially with respect to activity surrounding Internet disruptions, request-based metrics provide an even better perspective. A lot of HTTP traffic involves relatively small responses – especially API traffic, which now accounts for 60% of all traffic. Furthermore, response sizes can vary widely, ranging from a single-pixel GIF accounting for just a few bytes, to an API call that returns a few kilobytes of data, to a multi-gigabyte software package
To that end, the scope of user activity may be insufficiently reflected by a bytes-based metric, or buried in the noise, whereas request activity provides a cleaner signal and a more direct proxy for user activity. This is especially important as we examine the restoration of connectivity after an Internet disruption, attempting to ascertain when activity has returned to “expected” pre-disruption levels.
Finally, incorporating request-based traffic insights into Radar is simply extending the way that the data is already being used on the site. All the graphs, maps, and tables presented on Radar’s Adoption & Usage page, are based on analysis of HTTP request traffic, making use of information contained within request headers (such as HTTP version or user agent) or characteristics of the underlying connection (such as IP version).
Bytes vs requests – what’s the difference?
The current “HTTP traffic” view aggregates the bytes associated with HTTP requests to Cloudflare’s content delivery (CDN) services from the selected location or autonomous system (ASN). “Total traffic” aggregates this HTTP traffic along with the traffic associated with other Cloudflare services, including our 1.1.1.1 DNS resolver, authoritative DNS, WARP, and Spectrum, among others. (While Spectrum, WARP, and 1.1.1.1 also carry HTTP traffic, the share of HTTP traffic carried by these services is opaque to Radar, and isn’t accounted for as part of the HTTP traffic calculations.)
The bytes associated with a given request include the size of the request, the size of the headers associated with the response, and the size of the response itself. As noted above, the size of a file returned in response to a request can vary widely, depending on what was requested. The shape of the HTTP requests and HTTP bytes lines may be quite similar, but the potential variability in response sizes (in aggregate) can cause the lines to diverge, sometimes significantly so. For example, if an application regularly makes background requests to check for updates, the availability and subsequent download of a large file containing a software update would cause a spike in the HTTP bytes line, while the HTTP requests pattern remained consistent.
As another example, consider the graph below, capturing HTTP requests and bytes traffic trends for Portugal during the first week of August. HTTP bytes traffic initially grows each day between 06:00 and 09:00 UTC (07:00 – 10:00 local summer time), increases much more slowly until around 19:00 UTC (20:00 local summer time), and then increases rapidly before peaking around 21:00 UTC (22:00 local time). This suggests that content consumed during the workday is lighter in terms of bytes (such as API traffic, as discussed above), while evening traffic is more byte-heavy (possibly due to increased consumption of media content). In contrast, after starting to increase around 06:00 UTC (07:00 local summer time), request traffic generally sees three successively higher peaks each day – occurring around 10:00, 14:00, and 21:00 UTC respectively (11:00, 15:00, and 22:00 local summer time). These peaks are most pronounced on weekdays, but are still apparent on weekend days as well, suggesting regular patterns of user activity at those times.
It is important to remember that in looking at the “HTTP requests & bytes” graphs on Radar that they are showing two different metrics, and as such, only their shape over time is comparable, not their relative sizes. (As both metrics are normalized on a 0 to 1 (Max) scale, the lines on the graph are scaled relative to the maximum normalized value of each metric, including the previous period.)
Conclusion
The addition of HTTP request metrics to Cloudflare Radar brings additional visibility to traffic trends at a global, location, and network level, complementing the existing bytes-based HTTP traffic metrics. Derived from traffic to customer web properties, these new metrics can be found on Radar’s Overview and Traffic pages. In addition to HTTP traffic trends, visit Cloudflare Radar for additional insights around Internet disruptions, routing issues, attacks, domain popularity, and Internet quality. Follow us on social media at @CloudflareRadar (X), noc.social/@cloudflareradar (Mastodon), and radar.cloudflare.com (Bluesky), or contact us via email.
Cloudflare Radar is constantly monitoring the Internet for widespread disruptions. In mid-July, we published our Q2 2024 Internet Disruption Summary, and here we examine several recent noteworthy disruptions detected in the first month of Q3, including traffic anomalies observed in Bangladesh, Syria, Pakistan, and Venezuela.
Bangladesh
Violent student protests in Bangladesh against quotas in government jobs and rising unemployment rates led the government to order the nationwide shutdown of mobile Internet connectivity on July 18, reportedly to “ensure the security of citizens.” This government-directed shutdown ultimately became a near-complete Internet outage for the country, as broadband networks were taken offline as well. At a country level, Internet traffic in Bangladesh dropped to near zero just before 21:00 local time (15:00 UTC). Announced IP address space from the country dropped to near zero at that time as well, meaning that nearly every network in the country was disconnected from the Internet.
However, ahead of this nationwide shutdown, we observed outages across several Bangladeshi network providers, perhaps foreshadowing what was to come. At AS24389 (Grameenphone), a complete Internet outage started at 01:30 local time on July 18 (19:30 UTC on July 17), with a total loss of both Internet traffic and announced IP address space.
In the days before the shutdown, both median bandwidth and latency at a country level for Bangladesh were fairly stable. However, Cloudflare Radar’s Internet Quality measurements at a country level show a clear increase in median bandwidth and a concurrent drop in median latency, both likely due to the loss of measurements from mobile network providers as they disconnected from the Internet.
Five days after the full Internet shutdown started, broadband Internet services providers in Bangladesh began to restore connectivity on July 23. The initial restoration was characterized as a “trial run”, prioritizing banking, commercial sectors, technology firms, exporters, outsourcing providers and media outlets, according to the state minister for post, telecommunication and information technology. Announced IP address space began to increase around 19:00 local time (13:00 UTC), with traffic volumes beginning to trend upwards at that same time, as selected networks reconnected to the Internet.
Unfortunately, Syria is no stranger to Internet shutdowns, as they occur yearly during nationwide exams, implemented with the intent of preventing cheating on those exams. Our recent blog post titled Exam-ining recent Internet shutdowns in Syria, Iraq, and Algeria examined the first round of 2024 exams, which took place between May 26 and June 13. A second round of exams, and with them, multi-hour Internet shutdowns, began on July 25, and seen in the schedules below, published by Syrian Telecom on its Facebook page (English translation via Google Lens).
The Internet shutdowns implemented for the first four days of tests are clearly visible in the graph below, occurring on July 25, 28, 29, and 30.
However, you will also note another disruption is visible in both Syria’s Internet traffic and announced IP address space shortly after the planned shutdown on July 30. According to a (translated) Facebook post from Syrian Telecom, “while performing regular maintenance on one of the air conditioners located in one of the technical halls [data centers], an explosion occurred, causing the Internet circuits to temporarily go out of service.” This issue resulted in a disruption lasting approximately eight hours, between 11:00 – 19:00 local time (08:00 – 16:00 UTC) seen in both traffic and announced IP address space graphs for AS29256 (Syrian Telecom).
Pakistan
Closing out the month, on July 31, Pakistan experienced a wide-scale Internet disruption that lasted approximately two hours, between 13:30 – 15:30 local time (08:30 – 10:30 UTC). Traffic only dropped ~45% at a country level, but AS17557 (PTCL) experienced a near complete loss of traffic, while traffic at AS24499 (Telenor Pakistan) dropped nearly 90%. Together, the two network providers serve an estimated nine million users, and are among the top five Internet service providers in the country.
It was reported that the Pakistan Telecommunication Authority (PTA) attributed the disruptions to a technical glitch in the international submarine cable affecting the Pakistan Telecommunication Company Limited (PTCL) network. However, another published report noted “According to our sources, the government’s latest firewall edition to block the content was misconfigured, resulting in Internet connectivity disruption.” (Some additional information about the firewall can be found in this article.) The graphs below are from forthcoming TCP reset/timeout data on Cloudflare Radar, and show increased numbers of connections terminating immediately after the initial synchronization (SYN) packet used to establish new TCP connections (“Post SYN”) between 13:30 – 15:30 local time (08:30 – 10:30 UTC) on PTCL and Telenor Pakistan, coincident with the observed disruption. In other words, the rate of SYN packets arriving at Cloudflare was mostly consistent during the disruption, but there was a drop in other TCP packets, suggesting that the firewall explanation may be plausible. A Facebook post from the Pakistan Telecommunication Authority (PTA) simply highlighted that the issue had been resolved, and that “The exact issue is being investigated by PTA to avoid such instances in future.”
Regardless of the actual cause, the disruption had a clear impact on the country’s financial markets, with a published report stating “The KSE-100 index suffered a sharp decline on Wednesday, plummeting over 740 points in the final hour of trading amid a nationwide internet outage. Analysts attributed the sudden drop to panic selling as investors struggled with limited market data.”
Venezuela
In the past, some countries have implemented government-directed Internet shutdowns as a means of limiting communication about or organizing of protests and demonstrations associated with contested elections. Although such protests and demonstrations sprang up in the wake of a contested presidential election in Venezuela that took place on July 28, Internet shutdowns did not follow. However, in monitoring Internet traffic in Venezuela during the days around the election, the Cloudflare Radar team did observe several notable drops in traffic, as compared to the same times the week prior. After surging 35% at 05:00 local time (09:00 UTC) on Sunday, July 28 (election day), traffic dropped after the polls opened, down by as much as 23% at 09:00 local time (13:00 UTC). On July 29, the day following the election, traffic was as much as 28% lower than the same time the previous week at 06:15 local time (10:15 UTC) and 18:45 local time (22:45 UTC).
And while the observed drops in traffic appeared to be organic, and not caused by an Internet shutdown, it is worth noting that multiple websites are being blocked in Venezuela. An Internet Society Pulse blog post, published two days ahead of the election, reports that “Around 60 websites are currently blocked in Venezuela, including eight media sites and three that fact-check news and misinformation.”, citing data from the Open Observatory of Network Interference (OONI).
Cloudflare Radar is constantly monitoring the Internet for widespread disruptions. In mid-July, we published our Q2 2024 Internet Disruption Summary, and here we examine recent several noteworthy disruptions detected in the first month of Q3, including traffic anomalies observed in Bangladesh, Syria, Pakistan, and Venezuela.
Bangladesh
Violent student protests in Bangladesh against quotas in government jobs and rising unemployment rates led the government to order the nationwide shutdown of mobile Internet connectivity on July 18, reportedly to “ensure the security of citizens.” This government-directed shutdown ultimately became a near-complete Internet outage for the country, as broadband networks were taken offline as well. At a country level, Internet traffic in Bangladesh dropped to near zero just before 21:00 local time (15:00 UTC). Announced IP address space from the country dropped to near zero at that time as well, meaning that nearly every network in the country was disconnected from the Internet.
However, ahead of this nationwide shutdown, we observed outages across several Bangladeshi network providers, perhaps foreshadowing what was to come. At AS24389 (Grameenphone), a complete Internet outage started at 01:30 local time on July 18 (19:30 UTC on July 17), with a total loss of both Internet traffic and announced IP address space.
In the days before the shutdown, both median bandwidth and latency at a country level for Bangladesh were fairly stable. However, Cloudflare Radar’s Internet Quality measurements at a country level show a clear increase in median bandwidth and a concurrent drop in median latency, both likely due to the loss of measurements from mobile network providers as they disconnected from the Internet.
Five days after the full Internet shutdown started, broadband Internet services providers in Bangladesh began to restore connectivity on July 23. The initial restoration was characterized as a “trial run”, prioritizing banking, commercial sectors, technology firms, exporters, outsourcing providers and media outlets, according to the state minister for post, telecommunication and information technology. Announced IP address space began to increase around 19:00 local time (13:00 UTC), with traffic volumes beginning to trend upwards at that same time, as selected networks reconnected to the Internet.
Unfortunately, Syria is no stranger to Internet shutdowns, as they occur yearly during nationwide exams, implemented with the intent of preventing cheating on those exams. Our recent blog post titled Exam-ining recent Internet shutdowns in Syria, Iraq, and Algeria examined the first round of 2024 exams, which took place between May 26 and June 13.
A second round of exams, and with them, multi-hour Internet shutdowns, began on July 25, and seen in the schedules below, published by Syrian Telecom on its Facebook page (English translation via Google Lens).
The Internet shutdowns implemented for the first four days of tests are clearly visible in the graph below, occurring on July 25, 28, 29, and 30.
However, you will also note another disruption is visible in both Syria’s Internet traffic and announced IP address space shortly after the planned shutdown on July 30. According to a (translated) Facebook post from Syrian Telecom, “while performing regular maintenance on one of the air conditioners located in one of the technical halls [data centers], an explosion occurred, causing the Internet circuits to temporarily go out of service.” This issue resulted in a disruption lasting approximately eight hours, between 11:00 – 19:00 local time (08:00 – 16:00 UTC) seen in both traffic and announced IP address space graphs for AS29256 (Syrian Telecom).
Pakistan
Closing out the month, on July 31, Pakistan experienced a wide-scale Internet disruption that lasted approximately two hours, between 13:30 – 15:30 local time (08:30 – 10:30 UTC). Traffic only dropped ~45% at a country level, but AS17557 (PTCL) experienced a near complete loss of traffic, while traffic at AS24499 (Telenor Pakistan) dropped nearly 90%. Together, the two network providers serve an estimated nine million users, and are among the top five Internet service providers in the country.
It was reported that the Pakistan Telecommunication Authority (PTA) attributed the disruptions to a technical glitch in the international submarine cable affecting the Pakistan Telecommunication Company Limited (PTCL) network. However, another published report noted “According to our sources, the government’s latest firewall edition to block the content was misconfigured, resulting in Internet connectivity disruption.” (Some additional information about the firewall can be found in this article.) The graphs below are from forthcoming TCP reset/timeout data on Cloudflare Radar, and show increased numbers of connections terminating immediately after the initial synchronization (SYN) packet used to establish new TCP connections (“Post SYN”) between 13:30 – 15:30 local time (08:30 – 10:30 UTC) on PTCL and Telenor Pakistan, coincident with the observed disruption. In other words, the rate of SYN packets arriving at Cloudflare was mostly consistent during the disruption, but there was a drop in other TCP packets, suggesting that the firewall explanation may be plausible.
A Facebook post from the Pakistan Telecommunication Authority (PTA) simply highlighted that the issue had been resolved, and that “The exact issue is being investigated by PTA to avoid such instances in future.”
Regardless of the actual cause, the disruption had a clear impact on the country’s financial markets, with a published report stating “The KSE-100 index suffered a sharp decline on Wednesday, plummeting over 740 points in the final hour of trading amid a nationwide internet outage. Analysts attributed the sudden drop to panic selling as investors struggled with limited market data.”
Venezuela
In the past, some countries have implemented government-directed Internet shutdowns as a means of limiting communication about or organizing of protests and demonstrations associated with contested elections. Although such protests and demonstrations sprang up in the wake of a contested presidential election in Venezuela that took place on July 28, Internet shutdowns did not follow. However, in monitoring Internet traffic in Venezuela during the days around the election, the Cloudflare Radar team did observe several notable drops in traffic, as compared to the same times the week prior.
After surging 35% at 05:00 local time (09:00 UTC) on Sunday, July 28 (election day), traffic dropped after the polls opened, down by as much as 23% at 09:00 local time (13:00 UTC). On July 29, the day following the election, traffic was as much as 28% lower than the same time the previous week at 06:15 local time (10:15 UTC) and 18:45 local time (22:45 UTC).
And while the observed drops in traffic appeared to be organic, and not caused by an Internet shutdown, it is worth noting that multiple websites are being blocked in Venezuela. An Internet Society Pulse blog post, published two days ahead of the election, reports that “Around 60 websites are currently blocked in Venezuela, including eight media sites and three that fact-check news and misinformation.”, citing data from the Open Observatory of Network Interference (OONI).
Cloudflare’s network spans more than 320 cities in over 120 countries, where we interconnect with over 13,000 network providers in order to provide a broad range of services to millions of customers. The breadth of both our network and our customer base provides us with a unique perspective on Internet resilience, enabling us to observe the impact of Internet disruptions. Thanks to Cloudflare Radar functionality released earlier this year, we can explore the impact from a routing perspective, as well as a traffic perspective, at both a network and location level.
As we have seen in previous years, nationwide exams take place across several MENA countries in the second quarter, and with them come government directed Internet shutdowns. Cable cuts, both terrestrial and submarine, caused Internet outages across a number of countries, with the ACE submarine cable being a particular source of problems. Maintenance, power outages, and technical problems also disrupted Internet connectivity, as did unknown issues. And as we have frequently seen in the two-plus years since the conflict began, Internet connectivity in Ukraine suffers as a result of Russian attacks.
As we have noted in the past, this post is intended as a summary overview of observed disruptions, and is not an exhaustive or complete list of issues that have occurred during the quarter.
Government directed
Syria, Algeria, Iraq
Each spring, governments in several countries in the Middle East and North Africa (MENA) region order local telecommunications providers to shut down or disrupt Internet connectivity across the country in an effort to prevent students from cheating on national secondary and high school exams. These shutdowns/disruptions generally occur for several hours per day over a multi-week period. We covered such events in 2023, 2022, and 2021, as they occurred in locations including Syria, Sudan, Algeria, and Iraq.
In June, we published Exam-ining recent Internet shutdowns in Syria, Iraq, and Algeria, which examined the daily Internet shutdowns that took place in Iraq and Syria, as well as the two multi-hour daily disruptions in Algeria, which appeared to be pursuing a content blocking strategy, rather than a full nationwide shutdown. The post examined the impact that these shutdowns have on Internet traffic, and also analyzed routing information and traffic from other Cloudflare services in an effort to better understand how these shutdowns are being implemented.
In addition to the shutdowns covered in the previously referenced blog post, Iraq implemented a second round of shutdowns that started on June 23, and ran through at least July 14. Some of these shutdowns impacted the same set of networks seen in the first round, and some impacted networks in the autonomous Kurdistan region in the north.
Both sets of shutdowns reviewed above appeared to have followed the same approach as the first round covered in the earlier blog post.
Kenya, Burundi, Uganda, Rwanda, Tanzania
Concerns over a potential Internet shutdown during planned protests against tax increases proposed in “Finance Bill 2024” by the Kenyan government led to the publication of a joint statement signed by multiple organizations. The statement strongly urged the Kenyan government to refrain from enforcing any
Internet shutdowns or information controls, and highlighted the “disastrous economic effects” such a move could have. In response, the Communications Authority of Kenya issued a press release stating that “For the avoidance of doubt, the Authority has no intention whatsoever to shut down Internet traffic or interfere with the quality of connectivity. Such actions would be a betrayal of the Constitution as a whole, the freedom of expression in particular and our own ethos.”
As protests escalated on June 25, Internet traffic in Kenya dropped at 16:30 local time (13:30 UTC). Initially, this outage was thought to be due to issues with one or more undersea cables that provide international connectivity to the country, with the potential cause supported by social media posts from Safaricom and Airtel.
Similar concurrent drops in Internet traffic were observed in Burundi, Uganda, Rwanda, and Tanzania, as shown below. Issues with submarine cables connected to one country can impact Internet connectivity in other countries if there is a dependency on that country/cable for upstream Internet connectivity. As such, the observed disruptions in those four countries were not that unusual. To that end, a (subsequently deleted) post on X from MTN Uganda noted: “Our esteemed customers, We are experiencing a degraded service on all our internet services due to an outage caused by our connectivity supply through Kenya. Our technical teams and partners are working jointly to resolve the issue in the shortest time possible. In the interim, we kindly advise our customers to use *165# to access Mobile Money and other app based services. Thank you.“
However, other participants in the Internet infrastructure community in Africa called the undersea cable outage explanation into question. Kyle Spencer, Executive Director of the Uganda Internet eXchange Point, posted on X that “I am told the Kenyan government ordered sea cable landing stations to disconnect circuits.” Ben Roberts, Group CTIO at Liquid Intelligent Technologies (a pan-African network infrastructure provider), posted “No cables are damaged this week.” In addition, outages on undersea cables are rarely, if ever, resolved in a matter of hours, as this disruption was – they frequently last for days or weeks.
On June 26, Safaricom’s CEO claimed “This outage was occasioned by reduced bandwidth on some cables that carry Internet traffic”, contradicting the company’s original claim. No additional information was forthcoming from Airtel or the Communications Authority of Kenya, but as noted above, some within the industry believe that the disruption that impacted connectivity in Kenya, Burundi, Uganda, Rwanda, and Tanzania was directed by the government of Kenya, and was not caused by submarine cable outages.
Cable cuts
Haiti
At 17:36 local time (21:36 UTC) on April 28, Digicel Haiti posted an “important note” on X that stated in part (translated) “On April 27, 2024, the company suffered several attacks on its international optical infrastructure in the Drouya area on National Road #1. The optical fiber was damaged by the impact of cartridges after the armed clashes in the area for a few days. It affected several services such as internet (data), SMS, MonCash and international calling. For now, we are happy to inform the population that all services are restored to 100%.” The graph below shows the impact of the fiber damage, with AS27653 (Digicel Haiti) suffering an Internet outage lasting nearly 24 hours, from around 17:30 local time (21:30 UTC) on April 27 through approximately 16:00 local time (20:00 UTC) on April 28, after which traffic quickly recovered.
Then on May 3, The Director General of Digicel Haiti posted on X that (translated) “Digicel is informing the general public that it suffered two more damages to its international fiber infrastructure at 2am this morning. We have restored Moncash services, SMS, and Fiber Optic connections. Our crews are already on their way to address the apparent landslide in the Canaan area.” The disruption caused by this fiber damage lasted for approximately eight hours, between 02:15 – 10:30 local time (06:15 – 14:30 UTC), and as seen in the graph below, appeared to have a nominal impact on traffic.
On Sunday, May 12, issues with the EASSy and Seacom submarine cables again disrupted connectivity to East Africa, impacting a number of countries previously affected by a set of cable cuts that occurred nearly three months earlier. Insight into these earlier cable cuts and the initial impact of May’s cable damage was covered in our East African Internet connectivity again impacted by submarine cable cuts blog post.
Traffic levels across a number of the impacted countries dropped just before 11:00 local time (08:00 UTC). The magnitude of the initial impact varied by country, with traffic initially dropping by 10-25% in Kenya, Uganda, Madagascar, and Mozambique, while traffic in Rwanda, Malawi, and Tanzania dropped by one-third or more than compared to the previous week. The overall impact was most significant in Tanzania, Madagascar, and Rwanda, as seen in the graphs below. Traffic returned to expected levels at various times over the following week, ranging from a day and a half later (May 13) in Kenya to a week later (May 19) in Rwanda.
Repairs to the EASSy and Seacom cables were completed on May 31. Repairs to the cables damaged in February were ongoing as of July 9, as their location in a war zone complicates repair efforts.
Chad
A reported fiber optic cable cut in Cameroon disrupted Internet connectivity for customers of Moov Africa TChad on May 25. The outage lasted three hours, between 15:15 -18:15 local time (14:15 – 17:15 UTC), with the impact visible at a country level as well. Routing was disrupted too, as the number of IPv4 /24 prefixes (256 IPv4 addresses) announced by Moov Africa Tchad fell from eight to three during the disruption.
The event was similar to one that occurred on January 10, when Moov Africa Tchad and country-level traffic was disrupted for over 12 hours “due to a cut in the optical fiber coming from Cameroon through which Chad has access to the Internet”. During that event, significant volatility was also observed from a routing perspective, as the volume of announced IPv4 address space shifted frequently at a network and country level during the disruption. As we noted last quarter, as a landlocked country, Chad is dependent on terrestrial Internet connections to/through neighboring countries, and the AfTerFibre cable map illustrates Chad’s reliance on limited cable paths through Cameroon and Sudan.
Gambia, Mauritania, Senegal
A reported “network interruption” on the Africa Coast to Europe (ACE) submarine cable disrupted traffic across networks in the Gambia, Mauritania, and Senegal on June 5. AS25250 (Gamtel), AS29544 (Mauritel), and AS37649 (Free/Tigo) all saw traffic drop around 23:00 local time (23:00 UTC). As seen in the graphs below, the outage lasted for nearly 11 hours, with traffic recovering just 10:00 local time on June 6 (10:00 UTC). Mauritel saw a near complete outage, while Gamtel and Free/Tigo saw less severe impacts, possibly because they were able to shift traffic to back up links.
Maintenance
Guinea, Gambia, Sierra Leone, Liberia
Above, we discussed an unexpected network interruption on the ACE submarine cable that caused outages across multiple countries on June 5. However, two months earlier, a planned outage for repair work on the cable also disrupted connectivity across multiple African countries. A communiqúe issued by the Ministry of Posts, Telecommunications and the Digital Economy in Guinea noted in part (translated) “…the ACE (Africa Coast to Europe) network will undergo a planned outage on April 8, 2024, between midnight and 2:00 a.m. morning in the following countries: Guinea, Senegal, Gambia, Sierra Leone and Liberia. This total outage of approximately 2 hours will affect Internet traffic and international calls.”
The graphs below show the impact to traffic in the listed countries for the planned two-hour repair window, though it appears that traffic did not return fully to expected levels after the repair window concluded – it is unclear why it remained slightly depressed. In addition, despite being listed as one of the impacted countries, no impact to traffic was observed in Senegal.
Guinea
Rounding out a trifecta of entries about the ACE submarine cable, planned maintenance work on the cable by GUILAB reportedly caused a multi-hour outage at AS37461 (Orange Guinea) and at a country level as well, lasting from 12:15 – 15:45 local time (12:15 – 15:45 UTC). (GUILAB is the company in charge of managing the capacity allocated to Guinea on the ACE submarine cable.) The maintenance work was reported by Orange Guinea in two X posts (1, 2), although these posts were subsequently deleted.
Power outage
Kenya
At 18:30 local time (15:30 UTC) on May 2, Kenya Power posted a “Power Outage Alert” on X that stated “At 5:40 PM (EAT) today, Thursday, 2nd May 2024, we experienced a system disturbance on the grid, resulting in power supply disruption in most parts of the country.” The graph below shows the resultant impact on Internet connectivity in the country, with traffic dropping sharply between 17:30 – 17:45 local time (14:30 – 14:45 UTC). The drop in traffic lasted until approximately 21:30 local time (18:30 UTC), the same time that Kenya Power posted a “Power Supply Restoration” notice on X, highlighting the restoration of power to parts of the country. Although the post-outage spike seen in the graph would suggest pent-up demand for online content, a longer-term view of Kenya’s Internet traffic shows traffic peaks at the same time (22:00 local time, 19:00 UTC) during the preceding two days as well.
Ecuador
A nationwide power outage in Ecuador on June 19 impacted hospitals, homes, and the subway, in addition to causing a major disruption to Internet connectivity. The graph below shows Ecuador’s Internet traffic dropping sharply just after 15:00 local time (20:00 UTC). A post on X from Public Works Minister Roberto Luque explained (translated) “The immediate report that we received from CENACE is that there is a failure in the transmission line that caused a cascade disconnection, so there is no energy service on a national scale.” A subsequent post pointed at a lack of investment in the underlying systems, and noted that as of 18:41 pm local time (23:41 UTC), “95% of the energy has already been restored”. After the initial sharp drop, traffic began to recover fairly quickly, and was effectively back to expected levels by the stated time.
Albania, Bosnia, Montenegro
A sudden increase in power consumption related to increased usage due to high temperatures, as well electrical systems being impacted by the heat, caused a widespread power outage across Montenegro, Bosnia, and Montenegro on June 21. The outage reportedly originated in Montenegro after a 400-kilowatt transmission line exploded. While power outages are generally more localized to a single country, or region within a country, power distribution systems are linked across Balkan countries as part of the Trans-Balkan Electricity Corridor.
Published reports (MSN, Reuters) noted that electrical networks went down 12:00 – 13:00 local time (10:00 – 11:00 UTC), and that electricity suppliers in the impacted countries started restoring power by mid-afternoon, and had it largely restored by the evening. The graphs below show traffic from Albania, Bosnia, and Montenegro starting to drop around 12:00 local time (10:00 UTC), reaching its nadir in Albania and Bosnia at 12:30 local time (10:30 UTC) and at 13:00 local time (11:00 UTC) in Montenegro. Traffic recovered gradually over the next several hours as power was restored, returning to expected levels by 15:30 local time (13:30 UTC).
Croatia was reportedly impacted by the power outage as well, but no adverse impact to traffic at a country level is visible during the timeframe that connectivity in the other countries was disrupted.
Military action
Ukraine
During the two-plus years of the Russia-Ukraine conflict, Ukraine’s power grid has been a frequent target for Russian air attacks. When damage to Ukraine’s electrical power infrastructure occurs as a result of these attacks, Internet connectivity is also disrupted. Attacks on May 21 caused power outages across a number of areas in Ukraine. The most significant impact was in Sumy, where traffic dropped as low as 82% below the previous week at 00:00 on May 22 local time (21:00 UTC). As the graphs below illustrate, traffic was also lower than the previous week for several hours in Kyiv, Kharkiv, and Vinnytsia, with traffic returning to expected levels by around 08:00 local time (05:00 UTC) on May 22.
\
Technical problems
Malaysia
As we’ve covered in previous quarterly posts, Internet outages and disruptions aren’t always due to significant wide-scale events like severe weather, power outages, or cable cuts. Sometimes more mundane technical issues can cause problems when users try to access the Internet. One example of this occurred on April 15 in Malaysia, when customers of Time Internet experienced a network outage for nearly two hours. The company explained the reason for the outage in a contrite post on their Facebook page, stating in part “This Internet service outage was by far the worst in our history – affecting approximately 40% of our customers. … At 5.38pm today, both our primary and secondary Secure DNS servers became unreachable. This means that any browser or service requiring a DNS address resolution was not able to reach its intended site.” Because subscribers could not reach Time Internet’s DNS resolvers, they were unable to resolve hostnames for Internet services, sites, and applications, including those delivered by Cloudflare. This resulted in the drop in traffic seen in the graph below, which started just after 17:00 local time (05:00 UTC), and began to recover approximately an hour later. The company did not provide any additional information on what caused the DNS servers to fail.
Nepal
In Nepal, a number of local Internet service providers including AS45650 (Vianet) and AS139922 (Dishhome) rely on Indian provider Bharti Airtel for upstream connectivity, enabling them to reach the rest of the Internet. A published report underscores the reliance, noting “Nepali ISPs buy 70 percent of their internet from Airtel.”
On April 25, these ISPs warned that their services could be interrupted because the Nepali government had not provided them with foreign exchange services that would enable them to pay bandwidth vendors such as Airtel, whom they reportedly owed USD $30 million to. On May 1, Airtel informed the delinquent Nepali providers that Internet services may be interrupted at any time due to the overdue payment, and on May 2, Airtel took that step. The graphs below show Vianet’s traffic dropping to near zero at 16:15 local time (10:30 UTC), recovering to expected levels six hours later. An hour later, at 17:15 local time (11:30 UTC), Dishhome’s traffic dropped significantly, though not as severely as Vianet’s. Dishhome’s traffic also recovered approximately six hours later.
A month later, on June 3, AS45650 (Vianet) and AS17501 (Worldlink) in Nepal experienced Internet disruptions that were reportedly caused by routing issues on Bharti Airtel’s network. On Worldlink, a drop in traffic occurred between 12:15 – 14:00 local time (06:30 – 08:15 UTC), while on Vianet, the loss of traffic took place between 12:15 – 13:15 local time (06:30 – 07:30 local time).
Unknown
Most of the Internet disruptions covered in this blog post series have a known root cause, whether admitted/stated by the impacted provider(s) or closely associated with a real world event (severe weather, power outage, etc.) However, other disruptions are observed and even publicized by the impacted provider, but no underlying reason for the outage is ever made public.
Malaysia
On May 21, CelcomDigi (AS10030)posted on X that it was experiencing an outage on its network, and that it was working to resolve the issue as soon as possible. However. just 12 minutes later, it published a second post stating that it had fully restored Celcom Internet service. These posts were made at 21:35 and 21:47 local time (13:35 and 13:47) respectively. However, as the graph below shows, traffic volumes had returned to expected levels over an hour earlier, as the observed Internet disruption on Celcom’s network lasted between 18:00 – 20:15 local time (10:00 – 12:15 UTC). (Note that the second disruption shown in the graph below was due to an internal Cloudflare data pipeline issue, and not any sort of problem with Celcom’s network.)
Starlink
SpaceX Starlink’s satellite Internet service is unique in that it has an international subscriber base, so outages on its network have a more wide-reaching impact than issues with an ISP that covers a single country. At 01:59 UTC on May 29, Starlinkshared on X that it was currently experiencing a network outage, and that it was actively implementing a solution. Twenty-eight minutes later, it posted “The network issue has been fully resolved.” This brief outage is visible in the graph below as a slight dip in traffic. However, what is particularly interesting is the spike in traffic to Cloudflare from Starlink’s network following the resolution of the outage. The sharp increase and rapid decline of the traffic curve after service was restored suggests that it may be related to an automated connectivity check of some kind, rather than pent-up user demand for content.
Chad
A near-complete Internet outage was observed in Chad on June 5 between 08:15 – 12:00 local time (07:15 – 11:00 UTC), as seen in the graph below. Routing was also impacted, as the number of IPv4 /24 address blocks (256 IPv4 addresses) announced by network providers in the country dropped by as much as 75% during the outage.
A news item covering the outage noted that only Starlink subscribers retained Internet access during the outage. It also noted that Chad has faced recurring Internet disruptions since 2016, either because of problems with fiber-optic cables, or due to government directed shutdowns in the name of national security. It is unclear what ultimately caused this particular outage.
India
With an estimated subscriber base in excess of over 460 million, any Internet disruption affecting Reliance Jio’s network (AS55836) is going to have a widespread impact across India. On June 18, Reliance Jio experienced two disruptions that occurred between 13:15 – 17:15 local time (07:45 – 11:45 UTC). Each disruption lasted less than an hour, and dropped traffic levels to approximately half of those seen at the same time a week prior. Both mobile and fiber connectivity was affected, and no additional information has been provided by Reliance Jio regarding the root cause of the connectivity issues.
Conclusion
As we become increasingly dependent on reliable Internet connectivity, we must recognize that that connectivity is itself reliant on a complex and interconnected foundation of physical, technical, and political factors. A failure in any one of these foundational components, whether due to a cable cut, power outage, misconfiguration, or government action, can have a significant impact, disrupting Internet connectivity for millions of users, potentially across multiple countries. While the resilience and reliability of the physical and technical components can be improved through redundancy and best practices, political factors have arguably proven to be the hardest to address. However, organizations like AccessNow, through their #KeepItOn campaign, mobilize people, communities, and civil society actors globally to fight against government-directed Internet shutdowns, which can have significant financial consequences.
The practice of cheating on exams (or at least attempting to) is presumably as old as the concept of exams itself, especially when the results of the exam can have significant consequences for one’s academic future or career. As access to the Internet became more ubiquitous with the growth of mobile connectivity, and communication easier with an assortment of social media and messaging apps, a new avenue for cheating on exams emerged, potentially facilitating the sharing of test materials or answers. Over the last decade, some governments have reacted to this perceived risk by taking aggressive action to prevent cheating, ranging from targeted DNS-based blocking/filtering to multi-hour nationwide shutdowns across multi-week exam periods.
Syria and Iraq are well-known practitioners of the latter approach, and we have covered past exam-related Internet shutdowns in Syria (2021, 2022, 2023) and Iraq (2022, 2023) here on the Cloudflare blog. It is now mid-June 2024, and exams in both countries took place over the last several weeks, and with those exams, regular nationwide Internet shutdowns. In addition, Baccalaureate exams also took place in Algeria, and we have written about related Internet disruptions there in the past (2022, 2023). However, in contrast to the single daily shutdowns in Syria and Iraq, the Algerian government opted instead for two multi-hour disruptions each day – one in the morning, one in the afternoon – and appears to be pursuing a content blocking strategy, rather than a full nationwide shutdown.
As we have done in past year’s posts, we will examine the impact that these shutdowns have on Internet traffic, but also analyze routing information and traffic from other Cloudflare services in an effort to better understand how these shutdowns are being implemented.
Syria
The Syrian Telecom Company, to their credit, publishes an exam schedule on social media, with the image below published to their Facebook page. The English version was created by applying Google Translate to the image. The schedule shows the date & time of each Internet shutdown (“disconnection”), in addition to the subject(s) of that day’s exam(s). In 2024, exams started on May 26, and went through June 13.
In Syria, AS29256 (Syrian Telecom) is effectively the Internet, as shown in the table below. While there are a few other autonomous systems (ASNs/ASes) registered in Syria, there are only two that currently announce IP address space to the public Internet. As such, the trends seen at a country level for Syria reflect those seen for AS29256, and this is clearly evident in the traffic graphs below.
Nationwide Internet shutdowns in Syria began on May 26, taking place for varying multi-hour periods from Sunday to Thursday for three consecutive weeks. The graphs below show Internet traffic from the country, as well as AS29256, dropping to zero during the scheduled shutdowns.
In addition, graphs from the Cloudflare Radar Routing pages for Syria and AS29256 show the number of IPv4 and IPv6 prefixes being announced country-wide and by AS29256 dropping to at or near zero during each shutdown. This ultimately means that there is no Internet path back to systems (IP addresses) connected to Syrian Telecom. Below, we explore why this is important and problematic.
As has been observed in the past, the shutdowns in Syria are asymmetrical. That is, traffic can exit the country (via AS29256), but there are no paths for responses to return. The impact of this approach is clearly evident in traffic to Cloudflare’s 1.1.1.1 DNS Resolver. We continue to see traffic to the resolver when the shutdowns take place, and in fact, we see the traffic spike during the shutdowns, as the graph below shows.
If we dig into traffic to 1.1.1.1 by protocol, we can see that it is driven by requests over UDP port 53, the standard port used for DNS requests over UDP and TCP. (Given the request pattern, that also appears to be the primary way that we see traffic to the resolver from Syria.)
If we remove the UDP line from the graph, we see that request volume for DNS over TCP port 53, as well as DNS over HTTPS (DoH) and DNS over TLS (DoT), all drops to zero during the shutdowns.
Similarly, we can clearly see the shutdowns in HTTP(S) request-based traffic graphs as well, since HTTP(S) is also a TCP-based protocol.
Why do we see this impact? With DNS over UDP, the client simply makes a request to the resolver – no multi-step handshake is involved, as with TCP. So in this case, 1.1.1.1 is receiving these requests, but as shown above, there’s no path for the response to reach the client. Because it hasn’t received a response, the client retries the request, and this flood of retries is manifested as the spike seen in the graphs above.
However, as we see above, request volume for DNS over TCP, as well as DoH, DoT, and HTTP(S) (which all use TCP), falls to zero during the shutdowns. The lack of a path back to the client means that the TCP 3-way handshake can’t complete, and thus we don’t see DNS requests over these protocols.
In looking at 1.1.1.1 Resolver request volume from Syria for popular social media and messaging applications, we can see traffic for facebook.com most closely matches the spikes shown above. Removing facebook.com from the graph, we can also see similar, though more limited, increases for domains used by popular messaging applications WhatsApp, Signal, and Telegram. Facebook and WhatsApp are reportedly the most popular social media and messaging applications in Syria.
Although we have focused on the analysis of traffic to Cloudflare’s DNS resolver, and the patterns seen within that traffic, it is also worth highlighting an interesting pattern observed in traffic to Cloudflare’s Authoritative DNS platform. (DNS resolvers act as a middleman between clients, such as a laptop or phone, and an authoritative DNS server. Authoritative DNS servers contain information specific to the domain names they serve, including IP addresses and other types of records.)
The graph below shows bits/second traffic from Syria for Cloudflare’s authoritative DNS service on June 13. (Similar patterns were observed during the other days when shutdowns occurred, but data volume limits the ability to create a graph showing an extended period of time.) In this graph, we can see that at the start of the shutdown (03:00 UTC), traffic rises sharply, effectively plateaus for the duration of the shutdown, and then returns to normal levels. We believe that the traffic pattern illustrated here could be the result of some local resolvers in Syria having the IP addresses for our authoritative DNS servers cached, and are making requests to them. The increased traffic level could be because they are retrying their queries after not receiving responses, but in a less aggressive fashion than the client applications driving the resolver traffic spikes shown above.
In summary, Syria appears to be implementing their Internet shutdowns not through filtering, but rather by simply not announcing their IP address space for the duration of the shutdown, thereby preventing any responses from returning to the originating requestor, whether client application, web browser, or local DNS resolver.
Iraq
On May 19, the Iraqi Ministry of Communication posted an update that stated (translated) “The Ministry of Communications would like to note that the Internet service will be cut off for two hours during the general exams for intermediate studies, from six in the morning until eight in the morning, based on higher directives and at the request of the Ministry of Education.” The post came nearly a year after the Iraqi Ministry of Communication refused a request from the Ministry of Education to shut down the Internet during the baccalaureate exams as part of efforts to prevent cheating. On May 20, the Iraqi Ministry of Education posted the schedule for the upcoming set of exams to its Facebook page.
Iraq has a much richer network service provider environment than Syria does, with over 150autonomous systems (ASNs) registered in the country and announcing IP address space, compared to just two ASNs (both Syrian Telecom) in Syria announcing IP address space. Although traffic in Iraq is generally concentrated among the larger providers, shutdowns are rarely “complete” at a country level because not every autonomous system (network provider) in the country implements a shutdown. (This is due in part to the autonomous Kurdistan region in the north, which often implements similar shutdowns on their own schedule. Network providers in this region are included in Iraq’s country-level graphs.)
We can see this in a Cloudflare Radar traffic graph that shows the shutdowns at a country level, where traffic is dropping by around 87% during each multi-hour shutdown. In addition to the five networks also shown here (AS203214 (HulumTele), AS199739 (Earthlink), AS58322 (Halasat), AS51684 (Asiacell), and AS59588 (Zainas)), further analysis finds more than 30 where we observed a complete loss of traffic during the shutdowns, with a number of them downstream of these providers.
In contrast to Syria, the changes to announced IP address space during the shutdowns are much less severe in Iraq. Several of the shutdowns are correlated with a drop of ~20-25% in announced IPv4 address space, while a few others saw a drop closer to just 2%.
Similar to Syria, we can also look at 1.1.1.1 resolver traffic data to better understand how the shutdowns are being implemented. The country-level graphs below suggest that UDP traffic patterns are not visibly changing, suggesting that responses from the resolver are, in fact, getting back to the clients. However, this likely isn’t the case, and such a conclusion is at least in part an artifact of the graph’s time frame and hourly granularity, as well as the inclusion of resolver traffic from Kurdish network providers (ASNs). The shutdowns are more clearly evident in the DNS-over-TCP and DNS-over-HTTPS graphs below, as well as in the graph for HTTP(S) request traffic (both mobile & desktop), which is also TCP-based. In these graphs, the troughs on days that shutdowns occurred generally dip lower than those on the days that the Internet remained available.
In looking at authoritative DNS traffic from Iraq during a shutdown (for June 13 as an example day, as above), we see evidence of a decline in traffic during the time the shutdown occurs.
The decline in authoritative DNS traffic is more evident at an ASN level, such as in the graph below for AS203214 (Hulum), effectively confirming that UDP traffic is not getting through here either.
Considering the traffic, 1.1.1.1 Resolver, and authoritative DNS observations reviewed here, it suggests that the Internet shutdowns taking place in Iraq are more complex than Syria’s, as it appears that both UDP and TCP traffic are unable to egress from impacted network providers. As not all impacted network providers are showing a complete loss of announced IP address space during the shutdowns, Iraq is taking a different approach to disrupting Internet connectivity. Although analysis of our data doesn’t provide a definitive conclusion, there are several likely options, and network providers in the country may be combining several. These options revolve around:
IP: Block packets from reaching IP addresses. This may be done by withdrawing prefix announcements from the routing table (a brute force approach) or by blocking access to specific IP addresses, such as those associated with a specific application or service (a more surgical approach).
Connection: Block connections based on SNI/HTTP headers, or other application data. If a network or on-path device is able to observe the server name (or other relevant headers/data), then the connection can be terminated.
DNS: Operators of private or ‘internal’ DNS resolvers, offered by ISPs and enterprise environments for use by their own users, can apply content restrictions, blocking the resolution of hostnames associated with websites and other applications.
The consequences of these options are covered in more detail in a blog post. In addition, applying them at common network chokepoints, such as AS212330 (IRAQIXP) or AS208293 (AlSalam State Company, associated with the Iraqi Ministry of Communications), can disrupt connectivity at multiple downstream ISPs, without those providers necessarily having to take action themselves.
Algeria
As we noted in blog posts in 2022 and 2023, Algeria has a history of disrupting Internet connectivity during Baccalaureate exams. This has been taking place since 2018, following widespread cheating in 2016 that saw questions leaked online both before and during tests. On March 13, the Algerian Ministry of Education announced that the Baccalaureate exams would be held June 9-13. As expected, Internet disruptions were observed both country-wide and at a network level. Similar to previous years, two disruptions were observed each day. The first one began at 08:00 local time (07:00 UTC), and except for June 9, lasted three hours, ending at 11:00 local time (10:00 UTC). (On June 9, it lasted until 13:00 local time (12:00 UTC).) The second one began between 14:00-14:30 local time (13:00-13:30 UTC), and lasted until 16:00-17:00 local time (15:00-16:00 UTC) – the end time varied by day.
As seen in the graphs below, the impact to traffic was fairly nominal, suggesting that wide scale Internet shutdowns similar to those seen in Syria were not being implemented. While this is in line with 2023’s pronouncement by the Minister of Education that there would be no Internet shutdown on exam days, a number of posts on X complained of broader cuts to Internet connectivity.
Similar to the analysis above of the shutdowns in Syria and Iraq, we can also review changes to announced IP address space to better understand how connectivity was being disrupted. In this case, as the graphs below show, no meaningful changes to announced IPv4 address space were observed during the days the Baccalaureate exams were given. As such, the observed drops in traffic were not caused by routing changes.
In the HTTP(S) request traffic graph below, the twice-daily disruptions are highlighted, with the morning one appearing as a nominal drop in traffic, and the afternoon one causing a more severe decline. (The graph shows request traffic aggregated at a country level, but the graphs for the ASNs listed above also show similar patterns.)
In addition, similar patterns are observed in 1.1.1.1 resolver traffic at a country and ASN level, but only for DNS over TCP, DNS over TLS, and DNS over HTTPS, all of which leverage TCP. In the graph below showing only resolver traffic over UDP, there’s no clear evidence of disruptions. However, in the graph that shows resolver traffic over HTTPS, TCP, and TLS, a slight perturbation is visible in the morning, as traffic begins to rise for the day, and a sharper decrease is visible in the afternoon, with both disruptions aligning with the twice daily drops in traffic discussed above.
These observations support the conjecture that the Algerian government is likely taking a more nuanced approach to restricting access to content, interfering in some fashion with TCP-based traffic. The conjecture is also supported by an internal tool that helps to understand connection tampering that is based on research co-designed and developed by members of the Cloudflare Research team. We will be launching insights into TCP connection tampering on Cloudflare Radar later in 2024 and, in the meantime, technical details can be found in the peer-reviewed paper titled Global, Passive Detection of Connection Tampering.
The graph below, taken from the internal tool, highlights observed TCP connection tampering in connections from Algeria during the week that the Baccalaureate exams took place. While some baseline level of post-ACK and post-PSH tampering is consistently visible, we see significant increases in post-ACK twice a day during the exam period, at the times that align with the shifts in traffic discussed above. Technical descriptions of post-ACK and post-PSH tampering can be found in the Cloudflare Radar glossary, but in short, tampering post-ACK means an established TCP connection to Cloudflare’s server has been abruptly ended by one or more RST packets before the server sees data packets. Although clients do use RSTs, clients are more likely to close connections with a FIN (as specified by the RFC). The RST method can also be used by middleboxes that (i) sees the data packet, then (ii) drops the data packet, then (iii) sends an RST to the server to force the server to close the connection (and very likely another RST to the client too for the same reason). Tampering post-PSH means that something on the path, like a middlebox, (i) saw something it didn’t like on an established connection, then (ii) permitted the data to pass but then, (iii) it sends the RST to force endpoints to close the connection.
Looking beyond Cloudflare-sourced data, aggregated test results from the Open Observatory of Network Interference (OONI) also show evidence of anomalous behavior. Using OONI Probe, a mobile and desktop app, can probe for potential blocking of websites, instant messaging apps, and censorship circumvention tools. Examining test results from users in Algeria for popular messaging platforms WhatsApp, Telegram, Signal, and Facebook Messenger for the first two weeks of June, we clearly see the appearance of test results marked as “Anomaly” starting on June 9. (OONI defines “Anomaly” results as “Measurements that provided signs of potential blocking”.) OONI Tor testresults also show a similar “Anomaly” pattern. Anomalous traffic patterns are also visible for Google Web Search, YouTube, and GMail.
Although the analysis of these observations and data sets doesn’t provide us with specific details around exactly how the observed Internet disruptions are being implemented, it strongly supports the supposition that network providers in Algeria are, in some fashion, interfering with TCP connections, but not blocking them outright nor shutting down their networks completely. Given that popular messaging platforms, Google properties, Cloudflare’s 1.1.1.1 DNS resolver, and some number of Cloudflare customer sites all appear to be impacted, it suggests that a list of hostnames are being targeted for disruption/interference, either by the SNI or the destination IP address.
Conclusion
Perhaps recognizing the broad negative impact that brute-force nationwide Internet shutdowns have as a response to cheating on exams, some governments appear to be turning to more nuanced techniques, such as content blocking or connection tampering. However, because these are widely applied as well, they are arguably just as disruptive as a full nationwide Internet shutdown. The cause of full shutdowns, such as those seen in Syria, are arguably easier to diagnose than the disruptions to connectivity seen in Iraq and Algeria, which appear to use approaches that are hard to specifically identify from the outside.
On Sunday, May 12, issues with the ESSAy and Seacom submarine cables again disrupted connectivity to East Africa, impacting a number of countries previously affected by a set of cable cuts that occurred nearly three months earlier.
Already suffering from reduced capacity due to the February cable cuts, these countries were impacted by a second set of cable cuts that occurred on Sunday, May 12. According to a social media post from Ben Roberts, Group CTIO at Liquid Intelligent Technologies in Kenya, faults on the EASSy and Seacom cables again disrupted connectivity to East Africa, as he noted “All sub sea capacity between East Africa and South Africa is down.” A BBC article citing Roberts stated that the EASSy cable had been cut approximately 45km (28 miles) north of the South African port city of Durban. A subsequent press release issued by the Communications Authority of Kenya stated that the cut had occurred at the Mtunzini teleport station (in South Africa). As seen in the map below, both the EASSy and Seacom cables land in Mtunzini.
Map of African undersea cables, April 2024. Source: https://manypossibilities.net/african-undersea-cables/
Impacts to country-level Internet traffic
Cloudflare Radar saw traffic levels across a number of the impacted countries drop just before 11:00 local time (08:00 UTC). As seen in the graphs below, the magnitude of impact varied by country, with traffic initially dropping by 10-25% in Kenya, Uganda, Madagascar, and Mozambique, while traffic in Rwanda, Malawi, and Tanzania dropped by one-third or more as compared to the previous week.
In Kenya and Uganda, the overall impact appeared to be low, with traffic generally remaining just below expected levels in the day and a half following the cable faults. In the other countries, the overnight trough of the diurnal traffic patterns remained consistent with the previous week’s traffic levels, but otherwise traffic remains significantly lower than expected.
The importance of redundancy
In Kenya, the impact may have been nominal due to steps taken by providers like Safaricom and Airtel Kenya. In a May 12 social media post, Safaricom noted “…We have since activated redundancy measures to minimise service interruption and keep you connected as we await the full restoration of the cable.” In a subsequent social media post on May 13, Safaricom noted “Thanks to our redundancy plans and capacity investment across multiple undersea cables our services continue to be available, however some customers may experience slow connectivity and speeds.” Similarly, a social media post from Airtel Kenya noted “Following yesterday’s undersea fiber cut that has impacted internet connectivity, we would like to update you that we have taken measures to improve your browsing experience through additional capacity enhancement.”
Similarly, the previously referenced press release from the Communications Authority of Kenya talked about actions being taken, stating “Meanwhile, the Authority has directed service providers to take proactive steps to secure alternative routes for their traffic and is monitoring the situation closely to ensure that incoming and outbound internet connectivity is available. The East Africa Marine System (TEAMS) cable, which has not been affected by the cut, is currently being utilised for local traffic flow while redundancy on the South Africa route has been activated to minimize the impact.”
What’s next?
Once the necessary permits are secured and the cable faults are located, repairs can often be completed in several days. However, because cable repair ships are something of a scarce resource, there is often a delay to both engage a vessel and for it to travel to the area where the cable damage occurred, whether from its baseport or the location of a previous repair. However, in this case that delay may be comparatively short, as submarine cable industry observer @philBE2 predicts “Expecting the usual suspect, CS Leon Thevenin, now moored in Cape Town, to be swiftly mobilized for an expeditious repair mission…”
Cloudflare’s network spans more than 310 cities in over 120 countries, where we interconnect with over 13,000 network providers in order to provide a broad range of services to millions of customers. The breadth of both our network and our customer base provides us with a unique perspective on Internet resilience, enabling us to observe the impact of Internet disruptions. Thanks to recently released Cloudflare Radar functionality, this quarter we have started to explore the impact from a routing perspective, as well as a traffic perspective, at both a network and location level.
The first quarter of 2024 kicked off with quite a few Internet disruptions. Damage to both terrestrial and submarine cables caused problems in a number of locations, while military action related to ongoing geopolitical conflicts impacted connectivity in other areas. Governments in several African countries, as well as Pakistan, ordered Internet shutdowns, focusing heavily on mobile connectivity. Malicious actors known as Anonymous Sudan claimed responsibility for cyberattacks that disrupted Internet connectivity in Israel and Bahrain. Maintenance and power outages forced users offline, resulting in observed drops in traffic. And in a more unusual turn, RPKI, DNS, and DNSSEC issues were among the technical problems that disrupted connectivity for subscribers across multiple network providers.
As we have noted in the past, this post is intended as a summary overview of observed disruptions, and is not an exhaustive or complete list of issues that have occurred during the quarter.
Cable cuts
Moov Africa Tchad
Reported fiber optic cable damage that occurred in Cameroon on January 10 further disrupted connectivity for customers of AS327802 (Moov Africa Tchad / Millicom) a telecommunications provider in Chad. According to a (translated) Facebook post from Moov Africa Tchad, “On the afternoon of January 10, 2024, there was a breakdown of the internet due to a cut in the optical fiber coming from Cameroon through which Chad has access to the internet, the one coming from Sudan being unavailable for a while.” It is unclear whether the referenced cable cut occurred in Cameroon or Chad, and the mentioned Sudan cable issue may be the one covered in our Q4 2023 summary post. As a landlocked country, Chad is dependent on terrestrial Internet connections to/through neighboring countries, and the AfTerFibre cable map illustrates Chad’s reliance on limited cable paths through Cameroon and Sudan.
The graphs below show that Moov Africa Tchad traffic was disrupted for over 12 hours starting midday (UTC) on January 10, and the disruption was visible at a country level as well. The fiber cut also resulted in significant volatility from a routing perspective, as the volume of announced IPv4 address space shifted frequently at a network and country level during the disruption.
On February 15, a brief (~30 minute) but complete significant Internet disruption was observed at AS37577 (Orange Burkina Faso). According to the translation of a communiqué posted by the provider on social media, “The incident is due to a fiber cut, which causes a disruption of Internet services for certain customers.” Orange did not specify whether it was a more localized fiber cut, or damage to one of the terrestrial fibers that cross the country. The incident took the network completely offline, as the ASN’s amount of announced IPv4 address space dropped to zero for the duration.
MTN Nigeria
MTN Nigeria turned to social media on February 28 to let customers know that “You have been experiencing challenges connecting to the network due to a major service outage caused by multiple fibre cuts, affecting voice and data services.” A published report described the impact, noting “Millions of customers nationwide were impacted by the hours-long outage, especially in Lagos.” Connectivity was disrupted for approximately seven hours between 13:30 – 20:30 local time (12:30 – 19:30 UTC), and the provider posted a followup note just before midnight local time stating that service had been fully restored.
Digicel Haiti
A 16-hour Internet disruption on March 2/3 at AS27653 (Digicel Haiti) was due to a double fiber cut as a result of violence related to attempts to oust Prime Minister Ariel Henry. Starting around 22:00 local time on March 2 (03:00 on March 3), a complete outage was observed for approximately nine hours. Some recovery in traffic occurred for approximately two-and-a-half hours, followed by a three hour near-complete disruption. Digicel Haiti effectively disappeared from the Internet during the nine-hour outage, as no IPv4 or IPv6 address space was announced by the network during that time.
SKY (Philippines)
A brief traffic disruption observed on AS23944 (SKY) in the Philippines on March 18 was likely related to a fiber cut. In an advisory posted by SKY on social media, they stated that “SKY services in several areas in Marikina, Pasig and Quezon City are currently affected by a cut-fiber issue”, listing 45 affected areas. Traffic was most significantly impacted between 20:00 – 21:00 local time (12:00 – 13:00 UTC), although full recovery took several more hours. Only a minor impact to routing resulting from the fiber cut was observed.
Multiple African countries
On March 14, damage to multiple submarine cables off the west coast of Africa impacted Internet connectivity across multiple countries in West and Southern Africa. The damage was reportedly caused by underwater rock falls, and in addition to disrupting Internet connectivity, also caused availability issues for Microsoft Azure and Office 365 cloud services.
Comparatively brief disruptions were observed in Niger, Guinea, and Gambia, lasting from under an hour to approximately two hours.
However, the disruptions stretched out across multiple days in countries including Togo, Liberia, and Ghana, where it took several weeks for traffic to return to previously observed peak levels.
Operators in impacted countries attempted to maintain availability by shifting traffic to Google’s Equiano submarine cable, which reportedly experienced a 4x increase in traffic, and Morocco’sMaroc Telecom West Africa submarine cable. Service on the SAT-3 cable was fully restored as of April 6, with repairs on ACE completed on April 17, repairs to WACS and MainOne expected to be done by April 28.
On February 24, three submarine cables that run through the Red Sea were damaged: the Seacom/Tata cable, the Asia Africa Europe-1 (AAE-1), and the Europe India Gateway (EIG). It is believed that the cables were cut by the anchor of the Rubymar, a cargo ship that was damaged by a ballistic missile on February 18. At the time of the disruption, Seacom confirmed the damage to their cable, while the owners of the other two cables did not publish similar confirmations.
On February 2, Cloudflare observed a loss of traffic at AS15706 (Sudatel) and AS36972 (MTN Sudan), with a similar loss occurring on February 7 at AS36998 (Zain Sudan / SDN Mobitel). The disruption at MTN Sudan aligns with a social media post from the provider, in which they stated (translated) “We regret the interruption of all services due to circumstances beyond our control. While we apologize for the inconvenience caused by this interruption, we assure you of our endeavor to restore the service as soon as possible, and you will be notified of the return of the service.” On February 5, several days after their outage started, Zain Sudan published a social media post that stated (translated) “Zain Sudan has been constantly striving to maintain communication and Internet service to serve its valued subscribers, and we would like to point out that the current network outage is due to circumstances beyond its control, with our hopes that safety will prevail, and that service will be restored as soon as possible.” Sudatel did not share any information about the status of its network. On February 4, Digital Rights Lab – Sudan posted on social media that “Our sources confirmed that @RSFSudan forces tookover data centers of ISPs in Khartoum, #Sudan.” It is likely that the Internet outages observed across these providers are related to these takeovers, part of the military conflict that has been underway in the country since April 15, 2023.
The disruptions on these networks varied in length. At Sudatel, traffic started to return on February 11. At Zain Sudan, traffic began to return on March 3, corroborated by a social media post that stated (translated) “Zain network is gradually returning to work and allows its subscribers to communicate for free for a limited time. Zain promises to continue working to restore its network in the rest of the states.” Traffic had not yet returned on MTN Sudan by the end of the first quarter.
Ukraine
In February, the Ukraine/Russia war reached the two-year mark, and over that time, we have covered a number of Internet outages in Ukraine caused by conflict-related attacks. On February 22, Russian air strikes on critical infrastructure in Ukraine damaged energy facilities across the country, resulting in widespread power outages. These power outages caused Internet disruptions across multiple regions in Ukraine, including Kharkiv, Zaporizhzhia, Odessa, Dnipropetrovsk Oblast, and Khmelnytskyi Oblast. Traffic initially dropped around 05:00 local time (03:00 UTC), falling as much as 68% in Kharkiv. However, all regions saw lower traffic levels for several days as compared to the week before.
The associated outages during the quarter varied in length, from just a few hours to over a week. Each outage is shown in the graphs below, which show Paltel traffic within four Palestinian governorates in the Gaza Strip region. While it appears that the Gaza governorate suffered a disruption to traffic as connectivity remained available, complete outages occurred in the Khan Yunis, Rafah, and Deir al-Balah governorates.
January 12-19
January 22-24
March 5
Cyberattacks
In addition to the previously discussed cyberattack that impacted connectivity for AS327802 (Moov Africa Tchad / Millicom) on January 11, several other observed Internet disruptions were caused by cyberattacks in the first quarter.
Anonymous Sudan also reportedly targeted AS31452 (Zain Bahrain) with a cyber attack. This attack appeared to be less severe than the one that targeted HotNet in Israel, but it also lasted significantly longer, with traffic disrupted between 20:45 on March 3 and 18:15 on March 4 local time (17:45 on March 3 to 15:15 on March 4 UTC). No impact to announced IP address space was observed. Zain Bahrain acknowledged the connectivity disruption in a social media post on March 4, noting (translated) “We would like to inform you that some customers may encounter difficulties in using some of our services. Our technical team works to avoid these difficulties as quickly as possible.”
Multiple networks in Ukraine
On March 13, an attack targeted a number of Ukrainian telecommunications providers, including AS16066 (Triangulum), AS34359 (Link Telecom Ukraine), AS197522 (Kalush Information Network), AS52074 (Mandarun), and AS29013 (LinkKremen). Triangulum appeared to be the most significantly impacted, experiencing a near complete loss of traffic between March 13 and March 20, as seen below. Triangulum posted a notice on its website, noting in part “On March 13, 2024, a hacker attack was carried out on a number of Ukrainian providers. At 10:28 a.m. on March 13, 2024, a large-scale technical failure occurred on our Company’s network, as a result of which it became impossible to provide electronic communication services. The Company’s employees, together with employees of the Cyber Police and the National Cyber Security Coordination Center, are taking comprehensive measures around the clock aimed at restoring the entire range of services as soon as possible. Services are being restored gradually. Full recovery may take several days.”
Other affected providers experienced comparatively shorter connectivity disruptions. The near complete outage at Mandarun lasted approximately a day, while the others saw outages lasting around seven hours, starting around 11:30 local time (09:30 UTC) on March 13, with connectivity returning to typical levels around 08:00 local time (06:00 UTC) on March 14.
Government directed
Comoros
Following protests against the re-election of President Azali Assoumani, authorities in Comorosreportedly shut down Internet connectivity on January 17. While some disruption was visible to traffic at a country level between 12:00 local time on January 17 (09:00 UTC) and 17:30 local time on January 19 (14:30 UTC), it was significantly more noticeable in the traffic from AS36939 (Comores Telecom), which saw several periods of near-complete outage across the two-day span. Although Comores Telecom announces a limited amount of IPv4 address space, it saw significant volatility on January 17 & 18, dropping to zero several times.
Sudatel Senegal/Expresso Telecom and Tigo/Free (Senegal)
On February 4, the Minister of Communication, Telecommunications, and Digital Affairs in Senegalordered the suspension of mobile Internet connectivity starting at 22:00 local time (22:00 UTC). The suspension followed protests that erupted in the wake of the postponement of the presidential election. Traffic from AS37196 (Sudatel Senegal/Expresso Telecom) fell sharply at the time the suspension went into effect, recovering around 07:30 local time (07:30 UTC) on February 7. Traffic from AS37649 (Tigo/Free) fell at around 09:30 local time (09:30 UTC) on February 5, with the provider notifying subscribers of the suspension via social media. Traffic on Tigo/Free recovered around midnight local time (00:00 UTC) on February 7, and the provider again used social media to inform subscribers of service availability. No changes were observed to announced IP address space for either provider, indicating that the suspension of mobile Internet connectivity was not done at a routing level.
A little more than a week later, on February 13, the government in Senegal again ordered the suspension of mobile Internet connectivity in an effort to prevent “the spread of hateful and subversive messages online.” ahead of a march planned by activist groups which aimed to express dissent against the postponement of the presidential election. The mobile Internet shutdown was most visible on Tigo/Free, which saw a significant disruption between 10:15 and 19:45 local time (10:15 – 19:45 UTC).
Pakistan
According to a published report, The Pakistan Telecommunication Authority (PTA) said that Internet services would remain available as citizens went to the polls on February 8 to elect a new government. However, on that day, Pakistani authorities cut mobile Internet access across the country as the nation’s voters went to cast their ballots, with the authorities attributing the move “to maintain law and order” in the wake of the violence that occurred the previous day. The impact of the ordered shutdown was visible across multiple Internet providers in Pakistan, including AS59257 (Zong/CMPak), AS24499 (Telenor Pakistan), and AS45669 (Jazz/Mobilink), lasting from 07:00 until 20:00 (02:00 – 15:00 UTC), with traffic returning to expected levels approximately nine hours later. A post on the Internet Society’s Pulse blog estimated that the shutdown cost Pakistan nearly USD $18.5M in lost Gross Domestic Product.
Chad
Several Internet disruptions were observed in Chad between February 28 and March 7. The first one started at 10:45 local time on February 28 and lasted until 18:00 local time on March 1 (09:45 on February 28 – 17:00 on March 1). Shorter disruptions lasting just a few hours each were also observed on March 3, 4, and 7. The apparent shutdowns came in the wake of political violence in the country. Notable drops in announced IPv4 address space aggregated across networks in the country were observed coincident with the February 28, March 3, and March 4 shutdowns, although it isn’t clear why a similar drop did not occur on March 7.
Power outages
Tajikistan
According to a published report, a widespread multi-hour power outage occurred in Tajikistan on March 1, possibly related to increased electricity usage by electric heaters as temperatures across the country neared freezing. The outage began around 11:00 local time (06:00 UTC), and lasted for approximately three hours. The impact on Internet traffic from the country is visible in the graph below. Although power was restored around 14:00 local time (09:00 UTC), Internet traffic did not return to expected levels until around 05:00 local time the next day (midnight UTC on March 2).
Although power outages most often have the biggest impact on Internet traffic, as computers and home/office routers shut down, this outage also appeared to impact network infrastructure within the country, as the aggregate volume of announced IPv4 address space across the country dipped slightly when the power was out.
Tanzania
On March 4, the Tanzania Electricity Corporation (TANESCO)posted a notice on social media regarding an ongoing power outage. It stated (translated) “The Tanzania Electricity Corporation (TANESCO) has notified the public that there has been an error in the National Grid system, resulting in a lack of electricity service in some areas of the country including Zanzibar. Our experts are continuing their efforts to ensure that the electricity service returns to its normal state. The organization apologizes for any inconvenience caused.” The power outage disrupted Internet connectivity in Tanzania, causing an observed drop in traffic between 13:30 and 23:00 local time (10:30 – 20:00 UTC).
Technical problems
Orange España
Network routing is the process of selecting a path across one or more networks, and on the Internet, routing relies on the Border Gateway Protocol (BGP). Historically, the exchange of BGP routing information was based on trust between providers, but over time, security mechanisms such as Resource Public Key Infrastructure (RPKI) have been developed to prevent abuse of the system by bad actors. RPKI is a cryptographic method of signing records that associate a BGP route announcement with the correct originating AS number. ROA (Route Origin Authorization) records provide a means of verifying that an IP address block holder has authorized an AS (Autonomous System) to originate routes to that one or more prefixes within the address block. Cloudflare has published a number of blog posts over the years about the importance of, and our support for, RPKI. Properly implemented and configured, RPKI and ROAs help support routing security, effectively preventing behavior like BGP hijacking.
Because Cloudflare enforces RPKI validation, we also rejected the RPKI-invalid routes. We would have started trying to reach Orange España over our default route toward some of our transit providers, but because they also perform RPKI validation, traffic would have been dropped within those provider networks as well. Because of this, from Cloudflare’s perspective, this incident caused a drop in traffic from Orange España between 16:45 and 19:45 local time (14:45 – 17:45 UTC) as well as a notable drop in announced IPv4 address space from AS12479.
On January 11, subscribers of AS34700 (MaxNet) in Ukraine experienced a nine-hour Internet outage. Initial traffic loss occurred around 16:00 local time (14:00 UTC), and recovered around 01:00 local time on January 12 (23:00 UTC on January 11). An initial social media post from the provider explained the reason for the outage, noting (translated) “Dear subscribers! Due to the flooding of one of the hub sites due to a utility malfunction, some areas of the city may be without services, partially or completely. We are doing our best to restore services, but it takes time. Further information regarding the opening times will be published as soon as the emergency works have been completed.” A subsequent post informed subscribers that Internet connectivity had been restored. The flooding apparently impacted core routing infrastructure as well, as the volume of IPv4 address space announced by MaxNet also fell to zero between 16:00 and 22:00 local time (14:00 – 20:00 UTC).
Plusnet (United Kingdom)
A traffic disruption observed on AS6871 (Plusnet) in the United Kingdom on January 15 was initially characterized as a “mass outage” by the provider in replies to customer complaints on social media. However, the underlying cause of the disruption turned out to be significantly less sensational – it was apparently linked to problems with their DNS servers. Because subscribers were unable to successfully resolve hostnames using Plusnet’s default DNS resolvers, this ultimately manifested itself as a drop in traffic from the network for approximately two hours, between 16:00 and 18:00 local time (and UTC). Users that had configured their systems to use a third-party DNS resolver, such as Cloudflare’s 1.1.1.1 service, did not experience a service disruption.
Russia
DNS issues also impacted users in Russia during January, though in a different way than Plusnet subscribers in the UK experienced. A reportedDNSSEC failure on January 30 resulted in .ru domains becoming inaccessible for several hours. (DNSSEC creates a secure domain name system by adding cryptographic signatures to existing DNS records. By checking its associated signature, you can verify that a requested DNS record comes from its authoritative name server and wasn’t altered en-route, as opposed to a fake record injected in a man-in-the-middle attack.)
The DNSSEC validation failure resulted in SERVFAIL responses to DNS lookups against Cloudflare’s 1.1.1.1 resolver for hostnames in the .ru country code top level domain (ccTLD). At peak, 68.4% of requests received SERVFAIL responses. The Coordination Center for the .ru ccTLD confirmed that it was working on the “technical problem affecting the .ru zone associated with the global DNSSEC infrastructure” but didn’t provide any additional details around the root cause of the problem, such as a potential issue with a DNSSEC key rollover. The .ru ccTLD experienced a similar DNSSEC-related outage for several hours on August 16, 2019, as well.
AT&T (United States)
Starting just before 04:00 Eastern / 03:00 Central (09:00 UTC) on February 22, AT&T subscribers in several cities across the United States experienced mobile service interruptions. Impacted cities included Atlanta, Houston, and Chicago, with connectivity disrupted for approximately eight hours. Cloudflare data showed that as the problem began, AT&T (AS7018) traffic dropped as much as 45% in Chicago and 18% in Dallas, as compared with the previous week.
According to a “network update” published by AT&T, “Based on our initial review, we believe that today’s outage was caused by the application and execution of an incorrect process used as we were expanding our network, not a cyber attack.”
Just before noon local time (05:15 UTC) on March 12, a significant drop in traffic was observed on AS136442 (Ocean Wave), a consumer fiber and business Internet service provider in Myanmar. A (translated) social media post from the provider noted “Ocean Wave customers, please be informed that there will be no internet/ slow connection due to network maintenance.” The connectivity disruption lasted approximately seven hours, with traffic returning to typical levels just before 19:00 local time (12:15 UTC).
Conclusion
Two notable submarine cable damage events during the first quarter again highlighted the importance of protecting submarine cables, and the risks associated with them passing through/near geopolitically sensitive areas. Given the reliance on submarine cables for carrying Internet traffic, this will continue to be an issue for many years to come.
The Orange España incident also shed light on the importance of securing operationally important resources with multi-factor authentication, a topic that Cloudflare has written about in the past. Organizations like RIPE play a critically important behind-the-scenes role in functioning of the Internet, arguably obligating them to take all practical precautions when it comes to securing their systems in order to prevent malicious actors from taking actions that could broadly disrupt Internet connectivity.
During 2021’s Birthday Week, we announced our Email Routing service, which allows users to direct different types of email messages (such as marketing, transactional, or administrative) to separate accounts based on criteria such as the recipient’s address or department. Its capabilities and the volume of messages routed have grown significantly since launch.
Just a few months later, on February 23, 2022, we announced our intent to acquire Area 1 Security to protect users from phishing attacks in email, web, and network environments. Since the completion of the acquisition on April 1, 2022, Area 1’s email security capabilities have been integrated into Cloudflare’s secure access service edge (SASE) solution portfolio, and now processes tens of millions of messages daily.
Processing millions of email messages each day on behalf of our customers gives us a unique perspective on the threats posed by malicious emails, spam volume, the adoption of email authentication methods like SPF, DMARC, and DKIM, and the use of IPv4/IPv6 and TLS by email servers. Today, we are launching a new Email Security section on Cloudflare Radar to share these perspectives with you. The insights in this new section can help you better understand the state of email security as viewed across various metrics, as well as understanding real-time trends in email-borne threats. (For instance, correlating an observed increase within your organization in messages containing malicious links with a similar increase observed by Cloudflare.) Below, we review the new metrics that are now available on Radar.
Tracking malicious email
As Cloudflare’s email security service processes email messages on behalf of customers, we are able to identify and classify offending messages as malicious. As examples, malicious emails may attempt to trick recipients into sharing personal information like login details, or the messages could attempt to spread malware through embedded images, links, or attachments. The new Email Security section on Cloudflare Radar now provides insight at a global level into the aggregate share of processed messages that we have classified as malicious over the selected timeframe. During February 2024, as shown in the figure below, we found that an average of 2.1% of messages were classified as being malicious. Spikes in malicious email volume were seen on February 10 and 11, accounting for as much as 29% of messages. These spikes occurred just ahead of the Super Bowl, in line with previous observations of increases in malicious email volume in the week ahead of the game. Other notable (but lower) spikes were seen on February 13, 15, 17, 24, and 25. The summary and time series data for malicious email share are available through the Radar API.
Threat categorization
The Cloudflare Radar 2023 Year in Review highlighted some of the techniques used by attackers when carrying out attacks using malicious email messages. As noted above, these can include links or attachments leading to malware, as well as approaches like identity deception, where the message appears to be coming from a trusted contact, and brand impersonation, where the message appears to be coming from a trusted brand. In analyzing malicious email messages, Cloudflare’s email security service categorizes the threats that it finds these messages contain. (Note that a single message can contain multiple types of threats — the sender could be impersonating a trusted contact while the body of the email contains a link leading to a fake login page.)
Based on these assessments, Cloudflare Radar now provides insights into trends observed across several different groups of threat types including “Attachment”, “Link”, “Impersonation”, and “Other”. “Attachment” groups individual threat types where the attacker has attached a file to the email message, “Link” groups individual threat types where the attacker is trying to get the user to click on something, and “Impersonation” groups individual threat types where the attacker is impersonating a trusted brand or contact. The “Other” grouping includes other threat types not covered by the previous three.
During February 2024 for the “Link” grouping, as the figure below illustrates, link-based threats were unsurprisingly the most common, and were found in 58% of malicious emails. Since the display text for a link (i.e., hypertext) in HTML can be arbitrarily set, attackers can make a URL appear as if it links to a benign site when, in fact, it is actually malicious. Nearly a third of malicious emails linked to something designed to harvest user credentials. The summary and time series data for these threat categories are available through the Radar API.
For the “Attachment” grouping, during February 2024, nearly 13% of messages were found to have a malicious attachment that when opened or executed in the context of an attack, includes a call-to-action (e.g. lures target to click a link) or performs a series of actions set by an attacker. The share spiked several times throughout the month, reaching as high as 70%. The attachments in nearly 6% of messages attempted to download additional software (presumably malware) once opened.
If an email message appears to be coming from a trusted brand, users may be more likely to open it and take action, like checking the shipping status of a package or reviewing a financial transaction. During February 2024, on average, over a quarter of malicious emails were sent by attackers attempting to impersonate well-known brands. Similar to other threat categories, this one also saw a number of significant spikes, reaching as high as 88% of February 17. Just over 18% of messages were found to be trying to extort users in some fashion. It appears that such campaigns were very active in the week ahead of Valentine’s Day (February 14), although the peak was seen on February 15, at over 95% of messages.
Identity deception occurs when an attacker or someone with malicious intent sends an email claiming to be someone else, whether through use of a similar-looking domain or display name manipulation. This was the top threat category for the “Other” grouping, seen in over 36% of malicious emails during February 2024. The figure below shows three apparent “waves” of the use of this technique — the first began at the start of the month, the second around February 9, and the third around February 20. Over 11% of messages were categorized as malicious because of the reputation of the network (autonomous system) that they were sent from; some network providers are well-known sources of malicious and unwanted email.
Dangerous domains
Top-level domains, also known as TLDs, are found in the right-most portion of a hostname. For example, radar.cloudflare.com is in the .comgeneric Top Level Domain (gTLD), while bbc.co.uk is in the .ukcountry code Top Level Domain (ccTLD). As of February 2024, there are nearly 1600 Top Level Domains listed in the IANA Root Zone Database. Over the last 15 years or so, several reports have been published that look at the “most dangerous TLDs” — that is, which TLDs are most favored by threat actors. The “top” TLDs in these reports are often a mix of ccTLDs from smaller counties and newer gTLDs. On Radar, we are now sharing our own perspective on these dangerous TLDs, highlighting those where we have observed the largest shares of malicious and spam emails. The analysis is based on the sending domain’s TLD, found in the From: header of an email message. For example, if a message came from [email protected], then example.com is the sending domain, and .com is the associated TLD.
On Radar, users can view shares of spam and malicious email, and can also filter by timeframe and “type” of TLD, with options to view all (the complete list), ccTLDs (country codes), or “classic” TLDs (the original set of gTLDs specified in RFC 1591). Note that spam percentages shown here may be lower than those published in other industry analyses. Cloudflare cloud email security customers may be performing initial spam filtering before messages arrive at Cloudflare for processing, resulting in a lower percentage of messages characterized as spam by Cloudflare.
Looking back across February 2024, we found that new gTLD associates and the ccTLD zw (Zimbabwe) were the TLDs with domains originating the largest shares of malicious email, at over 85% each. New TLDs academy, directory, and bar had the largest shares of spam in email sent by associated domains, at upwards of 95%.
TLDs with the highest percentage of malicious email in February 2024TLDs with the highest percentage of spam email in February 2024
The figure below breaks out ccTLDs, where we found that at least half of the messages coming from domains in zw (Zimbabwe, at 85%) and bd (Bangladesh, at 50%) were classified as malicious. While the share of malicious email vastly outweighed the share of spam seen from zw domains, it was much more balanced in bd and pw (Palau). A total of 80 ccTLDs saw fewer than 1% of messages classified as malicious in February 2024.
ccTLDs with the highest percentage of malicious email in February 2024
Among the “classic” TLDs, we can see that the shares of both malicious emails and spam are relatively low. Perhaps unsurprisingly, as the largest TLD, com has the largest shares of both in February 2024. Given the restrictions around registering int and gov domains, it is interesting to see that even 2% of the messages from associated domains are classified as malicious.
Classic TLDs with the highest percentage of malicious email in February 2024.
The reasons that some TLDs are responsible for a greater share of malicious and/or spam email vary — some may have loose or non-existent registration requirements, some may be more friendly to so-called “domain tasting”, and some may have particularly low domain registration fees.The malicious and spam summary shares per TLD are available through the Radar API.
Sender Policy Framework (SPF) is a way for a domain to list all the servers they send emails from, with SPF records in the DNS listing the IP addresses of all the servers that are allowed to send emails from the domain. Mail servers that receive an email message can check it against the SPF record before passing it on to the recipient’s inbox. DomainKeys Identified Mail (DKIM) enables domain owners to automatically “sign” emails from their domain with a digital “signature” that uses cryptography to mathematically verify that the email came from the domain. Domain-based Message Authentication Reporting and Conformance (DMARC) tells a receiving email server what to do, given the results after checking SPF and DKIM. A domain’s DMARC policy, stored in DMARC records, can be set in a variety of ways, instructing mail servers to quarantine emails that fail SPF or DKIM (or both), to reject such emails, or to deliver them.
These authentication methods have recently taken on increased importance, as both Google and Yahoo! have announced that during the first quarter of 2024, as part of a more aggressive effort to reduce spam, they will require bulk senders to follow best practices that include implementing stronger email authentication using standards like SPF, DKIM, and DMARC. When a given email message is evaluated against these three methods, the potential outcomes are PASS, FAIL, and NONE. The first two are self-explanatory, while NONE means that there was no associated SPF/DKIM/DMARC policy associated with the message’s sending domain.
Reviewing the average shares across February 2024, we find that over 93% of messages passed SPF authentication, while just 2.7% failed. When considering this metric, FAIL is the outcome of greater interest because SPF is easier to spoof than DKIM, and also because failure may be driven by “shadow IT” situations, such as when a company’s Marketing department uses a third party to send email on behalf of the company, but fails to add that third party to the associated SPF records. An average of 88.5% of messages passed DKIM evaluation in February, while just 2.1% failed. For DKIM, the focus should be on PASS, as there are potential non-malicious reasons that a given signature may fail to verify. For DMARC, 86.5% of messages passed authentication, while 4.2% failed, and the combination of PASS and FAIL is the focus, as the presence of an associated policy is of greatest interest for this metric, and whether the message passed or failed less so. For all three methods in this section, NONE indicates the lack of an associated policy. SPF (summary, time series), DKIM (summary, time series), and DMARC (summary, time series) data is available through the Radar API.
Protocol usage
Cloudflare has long evangelized IPv6 adoption, although it has largely been focused on making Web resources available via this not-so-new version of the protocol. However, it’s also important that other Internet services begin to support and use IPv6, and this is an area where our recent research shows that providers may be lacking.
Through analysis of inbound connections from senders’ mail servers to Cloudflare’s email servers, we can gain insight into the distribution of these connections across IPv4 and IPv6. Looking at this distribution for February 2024, we find that 95% of connections were made over IPv4, while only 5% used IPv6. This distribution is in sharp contrast to the share of IPv6 requests for IPv6-capable (dual stacked) Web content, which was 37% for the same time period. The summary and time series data for IPv4/v6 distribution are available through the Radar API.
Cloudflare has also been a long-time advocate for secure connections, launching Universal SSL during 2014’s Birthday Week, to enable secure connections between end users and Cloudflare for all of our customers’ sites (which numbered ~2 million at the time). Over the last 10 years, SSL has completed its evolution to TLS, and although many think of TLS as only being relevant for Web content, possibly due to years of being told to look for the 🔒 padlock in our browser’s address bar, TLS is also used to encrypt client/server connections across other protocols including SMTP (email), FTP (file transfer), and XMPP (messaging).
Similar to the IPv4/v6 analysis discussed above, we can also calculate the share of inbound connections to Cloudflare’s email servers that are using TLS. Messages are encrypted in transit when the connection is made over TLS, while messages sent over unencrypted connections can potentially be read or modified in transit. Fortunately, the vast majority of messages received by Cloudflare’s email servers are made over encrypted connections, with just 6% sent unencrypted during February 2024. The summary and time series data for TLS usage are available through the Radar API.
Conclusion
Although younger Internet users may eschew email in favor of communicating through a variety of messaging apps, email remains an absolutely essential Internet service, relied on by individuals, enterprises, online and offline retailers, governments, and more. However, because email is so ubiquitous, important, and inexpensive, it has also become an attractive threat vector. Cloudflare’s email routing and security services help customers manage and secure their email, and Cloudflare Radar’s new Email Security section can help security researchers, email administrators, and other interested parties understand the latest trends around threats found in malicious email, sources of spam and malicious email, and the adoption of technologies designed to prevent abuse of email.
The 2023 Cloudflare Radar Year in Review is our fourth annual review of Internet trends and patterns observed throughout the year at both a global and country/region level across a variety of metrics. Below, we present a summary of key findings, and then explore them in more detail in subsequent sections.
Global Internet traffic grew 25%, in line with peak 2022 growth. Major holidays, severe weather, and intentional shutdowns clearly impacted Internet traffic. 🔗
Google was again the most popular general Internet service, with 2021 leader TikTok falling to fourth place. OpenAI was the most popular service in the emerging Generative AI category, and Binance remained the most popular Cryptocurrency service. 🔗
Globally, over two-thirds of mobile device traffic was from Android devices. Android had a >90% share of mobile device traffic in over 25 countries/regions; peak iOS mobile device traffic share was 66%. 🔗
Global traffic from Starlink nearly tripled in 2023. After initiating service in Brazil in mid-2022, Starlink traffic from that country was up over 17x in 2023. 🔗
Google Analytics, React, and HubSpot were among the most popular technologies found on top websites. 🔗
Globally, nearly half of web requests used HTTP/2, with 20% using HTTP/3. 🔗
NodeJS was the most popular language used for making automated API requests. 🔗
Googlebot was responsible for the highest volume of request traffic to Cloudflare in 2023. 🔗
Over 180 Internet outages were observed around the world in 2023, with many due to government-directed regional and national shutdowns of Internet connectivity. 🔗
Aggregated across 2023, only a third of IPv6-capable requests worldwide were made over IPv6. In India, however, that share reached 70%. 🔗
The top 10 countries all had measured average download speeds above 200 Mbps, with Iceland showing the best results across all four measured Internet quality metrics. 🔗
Over 40% of global traffic comes from mobile devices. In more than 80 countries/regions, the majority of traffic comes from mobile devices. 🔗
Just under 6% of global traffic was mitigated by Cloudflare’s systems as being potentially malicious or for customer-defined reasons. In the United States, 3.65% of traffic was mitigated, while in South Korea, it was 8.36%. 🔗
A third of global bot traffic comes from the United States, and over 11% of global bot traffic comes from Amazon Web Services. 🔗
Globally, Finance was the most attacked industry, but the timing of spikes in mitigated traffic and the target industries varied widely throughout the year and around the world. 🔗
Even as an older vulnerability, Log4j remained a top target for attacks during 2023. However, HTTP/2 Rapid Reset emerged as a significant new vulnerability, beginning with a flurry of record-breaking attacks. 🔗
1.7% of TLS 1.3 traffic is using post-quantum encryption. 🔗
Deceptive links and extortion attempts were two of the most common types of threats found in malicious email messages. 🔗
Routing security, measured as the share of RPKI valid routes, improved globally during 2023. Significant growth was observed in countries including Saudi Arabia, the United Arab Emirates, and Vietnam. 🔗
Introduction
Cloudflare Radar launched in September 2020, and in the blog post that announced its availability, we talked about how its intent was to “shine a light on the Internet’s patterns”. Cloudflare’s network currently spans more than 310 cities in over 120 countries/regions, serving an average of over 50 million HTTP(S) requests per second for millions of Internet properties, in addition to handling over 70 million DNS requests per second on average. The data generated by this massive global footprint and scale, combined with data from complementary Cloudflare tools, enables Radar to provide unique near-real time perspectives on the patterns and trends we observe across the Internet. For the last several years (2020, 2021, 2022), we’ve been aggregating these insights into an annual Year In Review, shining a light on the Internet’s patterns over the course of that year. The new Cloudflare Radar 2023 Year In Review continues that tradition, featuring interactive charts, graphs, and maps you can use to explore notable Internet trends observed throughout this past year.
The 2023 Year In Review is organized into three sections: Traffic Insights & Trends, Connectivity & Speed, and Security. We have incorporated several new metrics this year, and have endeavored to keep underlying methodologies consistent with last year wherever possible. Website visualizations shown at a weekly granularity cover the period from January 2 through November 26, 2023. Trends for over 180 countries/regions are available on the website, with some smaller or less populated locations excluded due to insufficient data. Note that some of the metrics are presented only as a worldwide view, and will not be shown if a country/region is selected. Because of the control plane and analytics outage that occurred November 2-4, traffic data for relevant metrics has been interpolated for that three-day period.
Below, we provide an overview of the content contained within the major Year In Review sections (Traffic Insights & Trends, Connectivity & Speed, and Security), along with notable observations and key findings. In addition, we have also published a companion blog post that specifically explores trends seen across Top Internet Services.
However, the notable observations and key findings contained within this post only skim the surface of the unique insights that can be found in the Year in Review website, which we strongly encourage you to visit to explore the data in more detail and look at trends for your country/region. As you do so, we encourage you to consider how the trends presented within these blog posts and the website’s various sections impact your business or organization, and to think about how these insights can inform actions that you can take to improve user experience or enhance your security posture in the future.
Traffic Insights & Trends
Global Internet traffic grew 25%, in line with peak 2022 growth. Major holidays, severe weather, and intentional shutdowns clearly impacted Internet traffic.
Twenty-five years ago, Worldcom executives claimed that Internet traffic was doubling every 100 days (3.5 months). A quarter-century later, we know that these claims were unrealistically aggressive, but it is clear that the Internet is growing quickly as more and more devices are connected, consuming content from a growing universe of websites, applications, and services.
To determine the traffic trends over time, we first established a baseline, calculated as the average daily traffic volume (excluding bot traffic) over the second full calendar week (January 8-14) of 2023. We chose the second calendar week to allow time for people to get back into their “normal” routines (school, work, etc.) after the winter holidays and New Year’s Day. The percent change shown in our traffic trends chart is calculated relative to the baseline value, and represents a seven-day trailing average — it does not represent absolute traffic volume for a country/region. The seven-day averaging is done to smooth the sharp changes seen with a daily granularity. A trend line for 2022 is shown for comparison purposes.
Our data shows that globally, Internet traffic grew 25% in 2023, with nominal initial growth accelerating during the second half of the year. Overall, the pattern is similar to that observed in 2022 (excepting last year’s late February spike), and peak growth for the year is just slightly above the peak growth level seen in 2022. Traffic patterns in Canada were also rather consistent year-over-year, exhibiting similar seasonality, and peak growth above 30% in both 2022 and 2023. In many countries, the 2022 trend line shows a clear drop in traffic heading into the Christmas holiday, with a slight rebound ahead of New Year’s Day. It will be interesting to see if traffic follows this pattern in 2023 as well.
Global Internet traffic growth in 2023, compared with 2022Internet traffic growth in Canada in 2023, compared with 2022
Comparisons with 2022 traffic trends helps make the impact of major holidays on Internet traffic more visible. For example, in Muslim countries including Indonesia, Turkey, and the United Arab Emirates, the celebration of Eid-Ul-Fitr, the festival marking the end of the fast of Ramadan, is visible as a noticeable drop in traffic around April 21-23, 2023, just before a similar drop visible in the 2022 trend line during last year’s celebration on May 2-3. In Italy, a drop in traffic is clearly visible around Pasqua di Resurrezione and Lunedì dell’Angelo (Easter Sunday and Monday) on April 9-10, one week ahead of a similar drop in traffic in 2022
Internet traffic growth in Indonesia in 2023, compared with 2022
In addition, extended disruptions to Internet connectivity are also clearly visible within the traffic trend charts. Examples include Mauritania, where government-directed shutdowns occurred from March 6-12 and May 30 – June 6, and Gabon, where a shutdown was in place from August 26-30, as well as Guam, where Super Typhoon Mawar caused a multi-week drop in traffic starting on May 24.
Internet traffic growth in Mauritania in 2023, compared with 2022 Internet traffic growth in Guam in 2023, compared with 2022
Google was again the most popular general Internet service, with 2021 leader TikTok falling to fourth place. OpenAI was the most popular service in the emerging Generative AI category, and Binance remained the most popular Cryptocurrency service.
One of the most popular sections of the Year In Review over the last several years has been the exploration of the most popular Internet services, both generally and across a number of categories. These rankings of service popularity are based on analysis of anonymized query data of traffic to our 1.1.1.1 public DNS resolver from millions of users around the world. Although DNS resolution operates at a domain level, domains that belong to a single Internet service are grouped together for the purposes of these rankings.
In the overall category, Google once again held the top spot, owing in part to its broad portfolio of services as well as the popularity of the Android mobile operating system. In addition to perennial categories like e-commerce, video streaming, and messaging, this year we also looked at Generative AI, which has been on a meteoric rise in 2023. In this category, OpenAI held the top spot, building on the success and popularity of ChatGPT, which it launched only a year ago. And despite the turmoil seen in the cryptocurrency space this year, Binance remained the most popular service in that category.
We explore these categorical rankings, as well as trends seen by specific services, in more detail in a separate blog post.
Globally, over two-thirds of mobile device traffic was from Android devices. Android had a >90% share of mobile device traffic in over 25 countries/regions; peak iOS mobile device traffic share was 66%.
Apple’s iOS and Google’s Android are the two leading operating systems used on mobile devices, and analysis of information in the user agent reported with each request allows us to gain insight into the distribution of traffic by client operating system throughout the year. Given the wide range of both devices and price points for Android devices, it is not surprising that Android is responsible for the majority of mobile device traffic when aggregated globally.
Globally, over two-thirds of mobile device traffic was from Android devices. The split is in line with Android/iOS usage observed in 2022. When looking at the countries/regions with the highest levels of Android usage, we find Bangladesh and Papua New Guinea at the top of the list, both with over 95% of mobile device traffic coming from Android devices. Looking more closely at other countries that see particularly high levels of Android usage, it is interesting to note that they are largely in Africa, Oceania/Asia, and South America, and that many have lower levels of gross national income per capita. This is presumably where the availability of lower priced “budget” phones plays to Android’s advantage from an adoption perspective.
In contrast, while the share of mobile device traffic from iOS at a country/region level never tops 70%, many of the countries with an iOS share over 50%, including Denmark, Australia, Japan, and Canada, have comparatively higher gross national income per capita, which likely speaks to a greater ability to afford higher priced devices.
Mobile device traffic operating system distribution across selected countries
Global traffic from Starlink nearly tripled in 2023. After initiating service in Brazil in mid-2022, Starlink traffic from that country was up over 17x in 2023.
SpaceX’s Starlink high-speed satellite Internet service has continued to rapidly grow its footprint since launching in 2019, making high performance Internet connections available in many countries/regions that were previously unserved or underserved by traditional wired or wireless broadband. The current leader in the space, in the future it will be joined by Amazon’s Project Kuiper service, which launched its first two test satellites this year, as well as Eutelsat OneWeb, which grew its satellite constellation in 2023 as well.
To track the growth in usage and availability of Starlink’s service, we analyzed aggregate Cloudflare traffic volumes associated with the service’s autonomous system (AS14593) throughout 2023. Although Starlink is not yet available globally, we did see traffic growth across a number of countries/regions. The request volume shown on the trend line in the chart represents a seven-day trailing average. A trend line for 2022 is shown for comparison purposes, and is scaled to the maximum value across 2022 and 2023.
Globally, we saw Starlink traffic more than triple this year. In the United States, traffic from Starlink was up over 2.5x, and grew over 17x in Brazil. In countries where Starlink turned up service in 2023, including Kenya, the Philippines, and Zambia, we saw traffic grow rapidly once the service became available.
Starlink traffic growth in Brazil, compared with 2022
Google Analytics, React, and HubSpot were among the most popular technologies found on top websites.
Modern websites are complex productions, relying on a mix of frameworks, platforms, services, and tools, and the developer community is responsible for making them coexist with one another to deliver a seamless experience. Using the Cloudflare Radar URL Scanner, which we launched in March 2023, we scanned websites associated with the top 5000 domains to identify the most popular technologies and services used across a dozen different categories, including (but not limited to) Analytics, where Google Analytics was by far the most widely used; JavaScript Frameworks, where React had a commanding lead; and Marketing Automation providers, where leader HubSpot was closely followed by several competitors.
Top website technologies, JavaScript frameworks category
Globally, nearly half of web requests used HTTP/2, with 20% using HTTP/3.
HTTP (HyperText Transfer Protocol) is the core protocol that the web relies upon. HTTP/1.0 was first standardized in 1996, HTTP/1.1 in 1999, and HTTP/2 in 2015. The most recent version, HTTP/3, was completed in 2022, and runs on top of QUIC, a new transport protocol. On the client side, HTTP/3 support is enabled by default in the latest versions of desktop and mobile Google Chrome and Mozilla Firefox, and for a portion of Apple Safari users. HTTP/3 is available for free for all Cloudflare customers, though not every customer chooses to enable it.
Using QUIC allows HTTP/3 to deliver improved performance by mitigating the effects of packet loss and network changes, as well as establishing connections more quickly. It also provides encryption by default, mitigating the risk of attacks. Websites and applications that remain on older versions of HTTP miss out on these benefits.
Analysis of the HTTP version negotiated for each request allows us to gain insight into the distribution of traffic by the various versions of the protocol aggregated throughout the year. (“HTTP/1.x” aggregates requests made over HTTP/1.0 and HTTP/1.1.) At a global level, 20% of requests were made over the latest version, HTTP/3. Another third of requests were made over the comparatively ancient HTTP/1.x versions, while HTTP/2 remained dominant, and accounted for the 47% balance.
Global HTTP version traffic distribution
Looking at the version distribution geographically, we found a number of Asian countries, including Nepal, Thailand, Malaysia, and Sri Lanka among those with highest rates of HTTP/3 usage, although these rates did not exceed 35%. In contrast, more than half of the requests from ten countries, including Ireland, Albania, Finland, and China, were made over HTTP/1.x during 2023.
NodeJS was the most popular language used for making automated API requests.
In addition, as developers increasingly use automated API calls to power dynamic websites and applications, we can use our unique visibility into Web traffic to identify the top languages these API clients are written in. Looking at API-related requests determined to not be coming from a person using a browser or native mobile application, we applied heuristics to help identify the language used to build the client.
Our analysis found that almost 15% of automated API requests are made by NodeJS clients, with Go, Java, Python, and .NET holding smaller shares.
Top languages used to make automated API calls
Googlebot was responsible for the highest volume of request traffic to Cloudflare in 2023.
Cloudflare Radar enables users to see Internet traffic trends at a country/region or network level over a selected period of time. However, we wanted to zoom out a bit, and look at the traffic Cloudflare saw from the entire IPv4 Internet over the course of the entire year. Hilbert curves, as “continuous space-filling curves”, have properties that are useful for visualizing the Internet’s IPv4 address space.
Using a Hilbert curve visualization, we can visualize aggregated request traffic (over IPv4) to Cloudflare from January 1st through November 26th, 2023. In order to make the amount of data used for the visualization manageable, IP addresses are aggregated at a /20 level, meaning that at the highest zoom level, each cell represents traffic from 4096 IPv4 addresses. (The sheer size of the IPv6 address space would make associated traffic very hard to see in such a visualization, especially as such a small amount has been allocated for assignment by the Regional Internet Registries.)
Within the visualization, IP addresses are grouped by ownership, and for much of the IP address space shown there, a mouseover at the default zoom level will show the Regional Internet Registry (RIR) that the address block belongs to. However, there are also a number of blocks that were assigned prior to the existence of the RIR system, and for these, they are labeled with the name of the organization that owns them. Progressive zooming ultimately shows the autonomous system and country/region that the IP address block is associated with, as well as its share of traffic relative to the maximum. (If a country/region is selected, only the IP address blocks associated with that location are visible.) Overall traffic shares are indicated by shading based on a color scale, and although a number of large unshaded blocks are visible, this does not necessarily mean that the associated address space is unused, but rather that it may be used in a way that does not generate traffic to Cloudflare.
Hilbert curve showing aggregated 2023 traffic to Cloudflare across the IPv4 Internet
Areas of higher request volume, indicated by warmer orange/red shading, are visibly scattered throughout the plot, but the IP address block that had the maximum request volume to Cloudflare during 2023 was 66.249.64.0/20, which belongs to Google. This IP address block is one of several used by the Googlebot web crawler, which is a likely explanation for the high request volume, given the number of web properties on Cloudflare’s network.
Zoomed view of the Hilbert curve showing the IPv4 address block that generated the highest volume of requests
It is hard to do this visualization justice with a short summary and static screenshot. To explore it in more detail, we encourage you to go to the Year in Review website and explore it by dragging and zooming to move around the IPv4 Internet.
Connectivity & Speed
Over 180 Internet outages were observed around the world in 2023, with many due to government-directed regional and national shutdowns of Internet connectivity.
During 2023, we have written frequently about Internet outages, whether due to technical issues, government-directed shutdowns, or geopolitical conflict, as well as infrastructure resilience issues (including fiber cuts, power outages, and severe weather) highlighted in our quarterly summaries. The impacts of these outages can be significant, including significant economic losses and severely limited communications. The Cloudflare Radar Outage Center tracks these Internet outages, and uses Cloudflare traffic data for insights into their scope and duration.
Some of these outages seen through the year were short-lived, lasting just a couple of hours, while others have stretched on for multiple months. In the latter category, localized government-directed shutdowns in Manipur, India and Amhara, Ethiopia have lasted over seven and four months respectively (as of early December). In the former category, Iraq frequently experienced multi-hour nationwide Internet shutdowns intended to prevent cheating on academic exams — these contribute to the clustering visible in the timeline during June, July, and August.
Within the timeline on the Year in Review website, mousing over a dot will display metadata about that outage, and clicking on it will open a page with additional information. If a country/region is selected, only outages for that country will be displayed.
Internet outages were observed around the world during 2023
Aggregated across 2023, only a third of IPv6-capable requests worldwide were made over IPv6. In India, however, that share reached 70%.
IPv6 has been around in some fashion since 1998, with an expanded address space that better supports the universe of Internet-connected devices that has grown exponentially over the last quarter-century. And over that time, available IPv4 space has been exhausted, leading connectivity providers to resort to solutions like Network Address Translation, and cloud and hosting providers to acquire blocks of IPv4 address space for as much as $50 per address. IPv6 also brings a number of other benefits to network providers, and if implemented correctly, adoption should be transparent from an end user perspective.
Cloudflare has been a vocal and active advocate for IPv6 stretching all the way back to our first birthday in 2011, when we announced our Automatic IPv6 Gateway, which enabled free IPv6 support for all of our customers. Just a few years later, we enabled IPv6 support by default for all of our customers. (Although it is enabled by default, not all customers choose to keep it enabled for a variety of reasons.) However, this support is only half of the equation for driving IPv6 adoption, as end user connections need to support it as well. (Technically, it is a bit more complex than that, but those are the two foundational requirements.) Analysis of the IP version used for each request made to Cloudflare allows us to gain insight into the distribution of traffic by the various versions of the protocol, aggregated throughout the year.
Thanks to near-complete IPv6 adoption by Indian telecommunications provider Reliance Jio, 70% of dual-stacked requests from Indian users were made via IPv6. India was followed closely by Malaysia, where 66% of dual-stacked requests were made over IPv6 during 2023, thanks to strong IPv6 adoption rates across leading Internet providers within the country. Other countries that saw more than half of dual-stacked requests, on average, made over IPv6 include Saudi Arabia, Vietnam, Greece, France, Uruguay, and Thailand. In contrast, there were on the order of 40 countries/regions where less than 1% of dual-stacked requests were made over IPv6 during 2023. Lagging adoption across such a large cohort of countries/regions 25 years after IPv6 was first published as a draft standard is quite surprising.
IPv4/IPv6 traffic distribution in India
The top 10 countries all had measured average download speeds above 200 Mbps, with Iceland showing the best results across all four measured Internet quality metrics.
Even when they are not facing Internet outages, users around the world are often contending with poor performance on their Internet connections, whether due to low speeds, high latency, or a combination of these factors. Although Internet providers continue to evolve their service portfolios to offer increased connection speeds and reduced latency in order to support growth in use cases like online gaming and videoconferencing, consumer adoption is often mixed due to cost, availability, or other issues. By aggregating the results of speed.cloudflare.com tests taken during 2023, we can get a geographic perspective on connection quality metrics including average download and upload speeds, and average idle and loaded latencies, as well as the distribution of the measurements.
In Iceland, over 85% of all Internet connections are over fiber, and this is reflected in its ranking as the country with the best overall Internet quality metrics, as speed test results show that providers there deliver the highest average speeds (282.5 Mbps download, 179.9 Mbps upload) and lowest average latencies (9.6 ms idle, 77.1 ms loaded). The histogram below shows that while there is a large cluster of download speeds between 0–100 Mbps, there were also a significant number of tests that measured even higher speeds, including some in excess of 1 Gbps.
Western European countries including Spain, Portugal, and Denmark also ranked among the top 10 across multiple Internet quality metrics.
Download and upload speed test result distribution in Iceland
Over 40% of global traffic comes from mobile devices. In more than 80 countries/regions, the majority of traffic comes from mobile devices.
Over the last 15 years or so, mobile devices have become increasingly ubiquitous, becoming indispensable in both our personal and professional lives, thanks in large part to their ability to enable us to access the Internet from nearly anywhere at any time. In some countries/regions, mobile devices primarily connect to the Internet via Wi-Fi, while others are “mobile first”, where Internet access is primarily through 4G/5G services.
Analysis of information contained with the user agent reported with each request to Cloudflare enables us to categorize it as coming from a mobile, desktop, or other type of device. Aggregating this categorization throughout the year at a global level, we found that 42% of traffic came from mobile devices, with 58% coming from desktop devices such as laptops and “classic” PCs. These traffic shares were in line with those measured in 2022. 79% of traffic came from mobile devices in Zambia, making it the country with the largest mobile device traffic share in 2023. Other countries/regions that had more than 50% of traffic come from mobile devices were concentrated in the Middle East/Africa, the Asia Pacific Region, and South/Central America. In contrast, Finland had one of the highest shares of desktop device traffic, at 80%.
Desktop and mobile device traffic distribution across selected countries
Security
Just under 6% of global traffic was mitigated by Cloudflare’s systems as being potentially malicious or for customer-defined reasons. In the United States, 3.65% of traffic was mitigated, while in South Korea, it was 8.36%.
Malicious bots are often used to attack websites and applications. To protect customers from these threats, Cloudflare mitigates (blocks) this attack traffic using DDoS mitigation techniques or Web Application Firewall (WAF) Managed Rules. However, customers may also choose to have Cloudflare mitigate traffic using other techniques for a variety of other reasons, such as rate-limiting requests, or blocking all traffic from a given location, even if it isn’t malicious. Analyzing traffic to Cloudflare’s network seen throughout 2023, we looked at the overall share that was mitigated (for any reason), as well as the share that was mitigated as a DDoS attack or by WAF Managed Rules.
Overall, just under 6% of global traffic was mitigated by Cloudflare’s systems as being potentially malicious or for customer-defined reasons, while only around 2% of it saw DDoS/Managed WAF mitigations. Some countries, such as Bermuda, saw the percentages for the two metrics track very closely, while other countries, like Pakistan and South Africa showed much larger gaps between their trend lines.
Mitigated traffic trends in Pakistan
A third of global bot traffic comes from the United States, and over 11% of global bot traffic comes from Amazon Web Services.
Bot traffic describes any non-human Internet traffic, and monitoring bot traffic levels can help site and application owners spot potentially malicious activity. Of course, bots can be helpful too, and Cloudflare maintains a list of verified bots to help keep the Internet healthy. Verified bots include those used for things like search engine indexing, performance testing, and availability monitoring. Regardless of intent, we wanted to look at where bot traffic was coming from, and we can use the IP address of a request to identify the network (autonomous system) and country/region associated with the bot making the request. Perhaps unsurprisingly, we found that cloud platforms were among the leading sources of bot traffic. This is likely due to the ease of automating the provisioning/teardown of compute resources and the relatively low cost of doing so, the distributed geographic footprint of cloud platforms, and the availability of high-bandwidth connections.
Globally, nearly 12% of bot traffic comes from Amazon Web Services, and over 7% from Google. Some of it comes from consumer ISPs as well, with U.S. broadband provider Comcast originating over 1.5% of global bot traffic. A disproportionate amount of bot traffic originates from the United States, responsible for nearly a third of global bot traffic, four times that of Germany, which originates just 8%. Within the United States, Amazon’s total share of bot traffic just edges out Google’s.
Global bot traffic distribution by source networkGlobal bot traffic distribution by source country
Globally, Finance was the most attacked industry, but the timing of spikes in mitigated traffic and the target industries varied widely throughout the year and around the world.
The industries targeted by attacks often shift over time, depending on the intent of the attackers. They may be trying to cause financial harm by attacking ecommerce sites during a busy shopping period, or they may be trying to make a political statement by attacking government-related sites. To identify industry-targeted attack activity during 2023, we analyzed mitigated traffic for customers that had an associated industry and vertical within their customer record. Mitigated traffic was aggregated weekly by source country/region across 18 target industries.
At a global level, Finance organizations were the most attacked over the course of the year, though we saw a significant amount of volatility from week-to-week. Interestingly, some clustering was evident, as Finance, which includes organizations that provide websites and applications for mobile payments, investments/trading, and cryptocurrency, was also a top target for a number of European countries, including Austria, Switzerland, France, the United Kingdom, Ireland, Italy, and the Netherlands, as well as in North America, for Canada, the United States, and Mexico. The Health industry, which includes companies that make exercise equipment, as well medical testing device manufacturers, was a top target across multiple African countries, including Benin, Côte d’Ivoire, Cameroon, Ethiopia, Senegal, and Somalia.
Overall, however, the year started slowly, with no industry seeing more than 8% of traffic being mitigated. As the first quarter progressed, Professional Services and News/Media/Publications organizations saw spikes in the share of mitigated traffic later in January, with Health jumping in mid-February and Law & Government organizations seeing a sharp increase in mitigated traffic in early March. Customers in the Arts/Entertainment/Recreation industry classification were apparently targeted by a multi-week attack campaign, with more than 20% of traffic mitigated during the weeks of March 26, April 2, and April 9. The overall peak during the year was experienced by the Professional Services industry, which saw a mitigated traffic share of 38.4% for the week of August 6, nearly twice its January spike. The timing of spikes and the industries experiencing those spikes varied widely across countries/regions.
Global mitigated traffic share by industry, week of August 6, 2023
Even as an older vulnerability, Log4j remained a top target for attacks during 2023. However, HTTP/2 Rapid Reset emerged as a significant new vulnerability, beginning with a flurry of record-breaking attacks.
In August 2023, we published a blog post that explored traffic seen by Cloudflare for the most commonly exploited vulnerabilities of 2022, as listed in a joint Cybersecurity Advisory. These included vulnerabilities in the Log4j Java-based logging utility, Microsoft Exchange, Atlassian’s Confluence platform, VMWare, and F5’s BIG-IP traffic management system. Although these are older vulnerabilities, attackers continued to actively target and exploit them throughout 2023, in part because organizations are frequently slow to follow the recommendations outlined in the Cybersecurity Advisory. We updated the analysis done for our blog post to include just the attack activity seen in 2023.
Attack activity by vulnerability varied by location, and in some, attacks targeted only a subset of the vulnerabilities. Aggregated worldwide, attack volume targeting Log4j consistently dwarfed that seen for the other vulnerabilities, and saw spikes during the last week of October and mid-late November; attack activity targeting Atlassian vulnerabilities increased in late July and trended slowly higher through the rest of the year. At a country/region level, Log4j was generally the most targeted vulnerability. In countries including France, Germany, India, and the United States, associated attack volume remained at a significant level throughout the year, while in other countries/regions, these attacks are most visible as infrequent, short-lived spikes within a country/region’s graphs, punctuating otherwise low levels of attack volume.
Global attack activity trends for commonly exploited vulnerabilities
We also expect that through 2024, attackers will continue to target the HTTP/2 Rapid Reset vulnerability disclosed in October. The vulnerability (see CVE-2023-44487 for details) abuses an underlying weakness in the request cancellation feature of the HTTP/2 protocol, leading to resource exhaustion on the target web/proxy server. Between the end of August and the beginning of October, we saw a number of attacks targeting this vulnerability. Across this set of attacks, the average attack rate was 30M requests per second (rps), with nearly 90 peaking above 100M rps, and the largest one hitting 201M rps. This largest attack was nearly 3x bigger than our previous biggest attack on record.
One notable concern about this vulnerability is that the attacker was able to generate such a large attack with a botnet consisting of just 20,000 compromised systems. This is much smaller than some of the largest botnets today, which comprise hundreds of thousands or millions of hosts. With average web traffic estimated to be between 1–3 billion requests per second, attacks using this method could conceivably focus an entire web’s worth of requests on a few unsuspecting targets.
HTTP/2 Rapid Reset campaign of hyper-volumetric DDoS attacks
1.7% of TLS 1.3 traffic is using post-quantum encryption
Post-quantum refers to a new set of cryptographic techniques that can protect data from adversaries with the ability to capture and store today’s data for decryption by sufficiently powerful quantum computers in the future. The Cloudflare Research team has been exploring post-quantum cryptography since 2017.
In October 2022, we enabled post-quantum key agreement at our edge by default, but use of it requires that the browser support it as well. Google’s Chrome browser started to slowly enable support in August 2023, and we expect its support will continue to grow in 2024, and that other browsers will add support over time as well. In September 2023, we announced general availability of post-quantum cryptography for both inbound and outbound connections and for many internal services, and expect to finish upgrading all internal services by the end of 2024.
After first enabling support in August, Chrome began ramping the number of browsers (version 116 and later) that use post-quantum cryptography, resulting in gradual growth leading to the significant increase seen on November 8. These actions helped push the share of TLS 1.3 traffic using post-quantum encryption to 1.7% at the end of November. As this ramp continues with future Chrome updates, and as other browsers add support for post-quantum encryption, we expect this share to continue to grow rapidly in 2024.
Growth trends in post-quantum encrypted TLS 1.3 traffic
Deceptive links and extortion attempts were two of the most common types of threats found in malicious email messages.
As the #1 business application, email represents a very attractive entry point into enterprise networks for attackers. Targeted malicious emails may attempt to impersonate an otherwise legitimate sender, try to get the user to click on a deceptive link, or contain a dangerous attachment, among other types of threats. Cloudflare Area 1 Email Security protects customers from email-based attacks, including those carried out through targeted malicious email messages. Over the course of 2023, an average of 2.65% of emails analyzed by Cloudflare Area 1 were found to be malicious. Aggregated at a weekly level, spikes to over 3.5%, 4.5%, and over 5% were seen in early February, early September, and late October respectively.
Global malicious email share trends
When carrying out attacks using malicious email messages, attackers use a variety of techniques, which we refer to as threat categories. These categories are defined and explored in detail in Cloudflare’s 2023 phishing threats report. Analysis of malicious emails shows that messages may contain multiple types of threats, highlighting the need for a comprehensive email security solution. Exploring threat activity trends for these categories, aggregated weekly across the year, we found that as much as 80% of them contained deceptive links.
However, it appears that attackers may have started to shift strategies in August, as the percentage of emails containing deceptive links began to fall while the share proposing to extort the recipient began to increase. By the end of October, and into November, the two threat categories had traded places, with nearly 80% of analyzed malicious emails containing an extortion threat, while only 20% contained deceptive links, as seen towards the right side of the graph below. However, this extortion campaign may have been short-lived, as its percentage fell almost as quickly as it rose. Identity deception and credential harvesting were also commonly identified threats, though the share of emails they were found in gradually declined over the course of the year.
Global threat category trends for malicious emails
Routing security, measured as the share of RPKI valid routes, improved globally during 2023. Significant growth was observed in countries including Saudi Arabia, the United Arab Emirates, and Vietnam.
Border Gateway Protocol (BGP) is the routing protocol for the Internet, communicating routes between networks, enabling traffic to flow between source and destination. However, because it relies on trust between networks, incorrect information shared between peers, whether done so intentionally or not, can send traffic to the wrong place, potentially with malicious results. Resource Public Key Infrastructure (RPKI) is a cryptographic method of signing records that associate a BGP route announcement with the correct originating AS number. In simple terms, it provides a way of ensuring that the information being shared originally came from a network that is allowed to do so. (Note that this is only half of the challenge of implementing routing security, as network providers also need to validate these signatures and filter out invalid announcements.) In the United States, the federal government recognizes the importance of routing security, with the Federal Communications Commission holding a “Border Gateway Protocol Security Workshop” on July 31.
Cloudflare has been a strong proponent of routing security, from being a founding participant in the MANRS CDN and Cloud Programme, to releasing an RPKI toolkit for network operators, to providing a public tool that enables users to test whether their Internet provider has implemented BGP safely, to presenting at this summer’s FCC workshop.
Building on the July release of the new Routing page on Cloudflare Radar, we analyzed data from RIPE NCC’s RPKI daily archive to determine the share of RPKI valid routes (as opposed to those route announcements that are invalid or whose status is unknown) and how that share has changed over the course of 2023. Since the start of the year, the global share of RPKI valid routes grew to nearly 45%, up six percentage points from the end of 2022. At a country/region level, we are looking at routes announced by autonomous systems associated with the given country/region. In the United States, the increased FCC attention on routing security is arguably warranted, as less than a third of the routes are RPKI valid. Although this is significantly better than South Korea, where less than 1% of announced routes are RPKI valid, it trails Vietnam significantly, where the share increased 35 percentage points during the first half of the year to 90%.
RPKI valid route growth in Vietnam
Conclusion
In the Cloudflare Radar 2023 Year In Review, we have attempted to provide a snapshot of the Internet, as dynamic as it is, through trend graphs and summary statistics, providing unique perspectives on Internet traffic, Internet quality, and Internet security, and how key metrics across these areas vary around the world.
As we said in the introduction, we strongly encourage you to visit the Cloudflare Radar 2023 Year In Review website and explore the trends relevant to metrics, countries/regions, and industries of interest, and to consider how they impact your organization so that you are appropriately prepared for 2024.
As we noted last year, it truly is a team effort to produce the data, website, and content for our annual Year in Review, and I’d like to acknowledge those team members that contributed to this year’s effort. Thank you to: Sabina Zejnilovic, Jorge Pacheco, Carlos Azevedo (Data Science); Arun Chintalapati, Reza Mohammady (Design); Vasco Asturiano, Nuno Pereira, Tiago Dias (Front End Development); João Tomé (Most popular Internet services); and Davide Marquês, Paula Tavares, Celso Martinho (Project/Engineering Management) as well as countless other colleagues for their answers, edits, and ideas.
Cloudflare operates in more than 300 cities in over 100 countries, where we interconnect with over 12,500 network providers in order to provide a broad range of services to millions of customers. The breadth of both our network and our customer base provides us with a unique perspective on Internet resilience, enabling us to observe the impact of Internet disruptions.
We have been publishing these summaries since the first quarter of 2022, and over that time, the charts on Cloudflare Radar have evolved. Many of the traffic graphs in early editions of this summary were screenshots from the relevant traffic pages on Radar. Late last year, we launched the ability to download graphs, and earlier this year, to embed dynamic graphs, and these summaries have taken advantage of those capabilities where possible. Sharp-eyed readers may notice an additional evolution in some of the graphs below: yellow highlighting indicating an observed “traffic anomaly”. Identification of such anomalies, along with the ability to be notified about them, as well as a timeline enhancement (embedded below) to the Cloudflare Radar Outage Center, were launched as part of Birthday Week at the end of September. More information on these new features can be found in our announcement blog post.
As we have seen in previous quarters, Iraq pursued an aggressive plan of government-directed Internet shutdowns intended to prevent cheating on exams, and several other African countries implemented politically motivated shutdowns. Damage to several submarine cables, as well as planned maintenance to others, caused Internet disruptions across a number of countries during the third quarter. Natural disasters, including wildfires and an earthquake, caused issues with connectivity, as did power outages in multiple countries. An acknowledged cyberattack resulted in a major US university intentionally disconnecting from the Internet, while a number of other major Internet providers acknowledged problems on their networks without ever disclosing the root cause of those problems.
(Note that the Internet disruptions related to the Israel/Palestine conflict are not covered in this post, as they began on October 7 in Q4 of 2023. Disruptions related to this conflict are being tracked, with additional insights found on the Cloudflare blog and @CloudflareRadar on X/Twitter.)
Government directed
Because the Internet has become a critical communications tool, Internet shutdowns are often used by governments as a means of controlling communication both within a country and with the outside world. These government-directed shutdowns are imposed for a variety of reasons, including during periods of civil unrest and protests around elections, and as a deterrent against cheating during exams.
Iraq
As we have discussed in past summaries, Internet shutdowns are used by some governments in an attempt to prevent cheating on national high school or baccalaureate exams. These shutdowns have a nationwide impact, and it isn’t clear whether they are ultimately successful at mitigating cheating. As we have also discussed in the past, such shutdowns frequently occur in Iraq, and that was certainly the case during the third quarter, with rounds of shutdowns occurring during all three months.
The first round of exam-related Internet shutdowns during the quarter in Iraq was a continuation of a set that started in June, and continued on into July, targeting cheating on 9th and 12th grade exams. On ten days between July 4 and July 17, Internet connectivity was shut down on AS203214 (HulumTele), AS59588 (ZAINAS-IQ), AS199739 (Earthlink), AS203735 (Capacities-LTD), AS51684 (ASIACELL), and AS58322 (Halasat) in Iraq (except for the Kurdistan Region) between 04:00 – 08:00 local time (01:00 – 05:00 UTC).
A second round of 9th grade exams in August drove a week of Internet shutdowns across Iraq (except the Kurdistan region) between August 21 and August 29. Connectivity was shut down between 04:00 – 08:00 local time (01:00 – 05:00 UTC) across the same networks impacted by the shutdowns implemented in July.
Following the second round of 9th grade exams in August, the second round of 12th grade exams in Iraq (except the Kurdistan region) occurred in September, and with these exams, came yet another round of Internet shutdowns. Impacting the same set of network providers as the previous two months, these shutdowns occurred between September 17-30. However, while they started at the same time (04:00 local time, 01:00 UTC), they were shorter than previous rounds, ending an hour earlier (07:00 local time, 04:00 UTC).
As shown in the graphs below, the shutdowns began mid-morning local time, generally between 08:00 and 10:00, from July 31 through August 5, and ended early the next morning, generally between midnight and 02:00. The final shutdown on August 5 was an exception, ending at 22:00 local time on both networks. (Senegal is UTC+0, so the local times are the same as UTC.)
Ethiopia
Following days of clashes between the federal military and local militia, mobile Internet connectivity was shut down in Amhara, Ethiopia. Cloudflare saw traffic to the region drop around 21:00 local time (18:00 UTC) on August 2. This is the second time that authorities have shut down mobile Internet connectivity in Amhara in 2023 — the first time was on April 6 after protests broke out following the federal government’s move to disband regional security forces. (Note that the country is no stranger to Internet shutdowns, as they have taken such action multiple times over the last several years.) Despite calls to restore connectivity, mobile Internet remained unavailable through the end of the third quarter, as seen in the figure below.
Gabon
On August 26, following contentious presidential elections in Gabon, Internet connectivity was shut down in order to “prevent the spread of calls for violence”. As shown in the figure below, traffic began to fall just before 17:00 local time (16:00 UTC), and remained at zero through approximately 07:30 local time (06:30 UTC) on August 30. Connectivity was restored hours after military officers seized power in the country, placing President Ali Bongo under house arrest and naming a new leader after the country’s election body announced Bongo had won a third term.
Cable cuts
Cameroon
On July 7, an X/Twitter post from Cameroon Telecommunications alerted subscribers to disruptions to voice and data services, with a subsequent post nearly six hours later noting that services had been re-established. Although these posts did not provide details on the cause of the disruption, a Facebook post from the operator included an attached communiqué explaining that “The optical fibre has been severed by road maintenance operations, causing major disruptions in the delivery of our services.” The figure below shows the impact of this fiber damage, with traffic from AS15964 (CAMNET-AS) dropping sharply around 11:30 local time (10:30 UTC), and returning to expected levels by 18:00 local time (17:00 UTC).
Liberia
Damage to the Africa Coast to Europe (ACE) submarine cable disrupted Internet connectivity in Liberia on July 28. A Facebook post from the Liberia Telecommunications Authority (LTA) noted “The Liberia Telecommunications Authority(LTA) announces the temporary interruption of all nationwide Internet services due to the breakdown of the Africa Coast to Europe Cable in Ivory Coast.” and also highlighted that the ACE cable serves as the “sole source of internet connectivity between Europe and Liberia”. The figure below shows a near complete loss of traffic starting at 13:00 local time (13:00 UTC) and gradually recovering over the next several hours, returning to expected levels by 17:00 local time (17:00 UTC).
Togo, Benin, Namibia, and the Republic of Congo (Brazzaville)
As such, the graphs below illustrate that there was not a complete loss of traffic for impacted countries. To that end, traffic in Togo appeared to recover several weeks before the cable repairs were completed. The full impact is harder to see in the graphs for Benin, Namibia, and the Republic of Congo (Brazzaville) because the selected timeframe is long enough to force data aggregation at a daily level, but it is clearly visible in graphs covering shorter periods of time (with data aggregation at an hourly level) during the weeks after the cable cut occurred.
South Sudan
Highlighting the interconnected nature of the Internet, fiber cuts in Uganda caused a brief Internet disruption for customers on MTN South Sudan (AS37594) on August 14, occurring between 13:00 – 15:00 local time (11:00 – 13:00 UTC), and impacting an estimated 438,000 users. An X/Twitter post from the provider that afternoon told subscribers “We sincerely apologize for the network issues experienced over the last couple of hours. It was due to multiple fiber cuts in Uganda.”
Cyberattack
University of Michigan
On August 27, a “significant security concern” led the University of Michigan to shut down the Internet on the Ann Arbor, Flint and Dearborn campuses. Although the shutdown occurred at the start of the new school year, classes continued as scheduled, but an announcement posted by the University detailed the impact of disconnecting from the Internet, including potential delays in financial aid refunds and the unavailability of certain campus systems. The impact of the disconnection can be seen in the figure below, appearing as a significant drop in traffic starting just before 14:00 local time (18:00 UTC) on August 27, and lasting until just after 08:00 local time (12:00 UTC) on August 30 on AS36375 (UMICH-AS-5), the primary autonomous system used by the University of Michigan.
Fire
Lahaina, Hawaii
In early August, a series of wildfires broke out in the state of Hawaii, predominantly on the island of Maui. The town of Lahaina was one of the hardest hit, with the fires killing nearly 100 people, as well as destroying homes, businesses, and infrastructure, causing power outages and disrupting Internet connectivity. The graph below shows traffic to Cloudflare from Lahaina dropping to near zero around 21:00 local time on August 7 (07:00 UTC on August 8), and remaining at minimal levels through August 30. Some recovery of Internet traffic can be seen through the end of September as cleanup and repairs progressed, and as wireless operators deployed temporary network assets in an effort to restore some service capacity.
Earthquake
Morocco
At 23:11 local time on September 8 (22:11 UTC), a magnitude 6.8 earthquake occurred in Morocco, centered 79 kilometers (49 miles) southwest of Marrakesh. Nearly 3,000 deaths were reported as a result of the quake, and significant damage was reported, including the collapse of schools, houses, and historic buildings. Power outages and infrastructure damage also impacted Internet connectivity in the region, leading to largely localized disruptions.
The country-level graph below shows a nominal loss of traffic in Morocco after the earthquake, remaining slightly lower than expected for approximately four days. However, the impacts are more evident at a regional level, with the earthquake causing an immediate 64% drop in traffic in Marrkesh-Safi, a 64% loss in Souss-Massa, and a 49% decline in Casablanca-Settat. Peak traffic levels in these regions remained slightly lower than those seen in previous weeks for several days after the earthquake occurred.
Power outages
Curaçao
On July 27, a malfunction at a major Aqualectra Utility power distribution center resulted in 70% of neighborhoods in Curaçao losing power. The power outage resulted in an island-wide Internet disruption. As seen in the graph below, Internet traffic fell sharply at around 12:30 local time (16:30 UTC), remaining largely flat for approximately five hours before starting to recover around 17:30 local time (21:30 UTC). The start of the recovery aligns with the timing of a Facebook post made at 18:00 local time by Aqualectra Utility noting that “55% of Curaçao’s power supply has been restored.” The ongoing traffic increase is in line with additional neighborhoods having power restored, with traffic returning to expected levels by around 22:00 local time (2:00 UTC on July 28).
Brazil
A widespread power outage in Brazil starting at 08:30 local time (11:30 UTC) on August 15 resulted in a nominal disruption to Internet traffic within the country. Although the power outage represented a loss of approximately 27% of the total electric load at the time it occurred, the impact to the country’s Internet traffic was much lower, as seen in the graph below. Traffic returned to expected levels by around 11:30 local time (14:30 UTC).
Kenya
A “system disturbance” at 21:45 local time (18:45 UTC) on August 25 led to “loss of bulk power supply to various parts of the country” in Kenya, according to an X/Twitter post from Kenya Power. The impact of the power outage is visible in the graph below, with traffic dropping as power is lost. Subsequent updates from Kenya Power on August 26 (1, 2, 3) highlighted the progress made in restoring electricity across the country. Internet traffic from the country returned to expected levels by 03:00 on August 27 (00:00 UTC).
French Guiana
An 11-hour Internet disruption in French Guiana on August 27 was the result of a power outage caused by “a problem that occurred at the energy evacuation station which connects Petit-Saut to the Kourou-Saint-Laurent line”. The power outage caused a nationwide drop in Internet traffic between 11:00 local time (14:00 UTC) and 22:00 local time (01:00 UTC on August 28), visible in the graph below.
Tunisia
A fire at the Tunisian Company of Electricity and Gas power station in Rades, Ben Arous Governorate caused a widespread power outage in Tunisia, resulting in an Internet disruption starting at 01:00 local time (00:00 UTC) on September 20. Traffic remained lower than expected for approximately five hours, as shown in the graph below, in line with a published report that noted “The unexpected outage lasted for over four hours in some areas of the country.”
Barbados
A September 21 Facebook post from The Barbados Light & Power Company Limited noted that the company was aware of an outage affecting customers, and that they were “working to promptly and safely restore power in the shortest time possible.” This outage resulted in a significant drop in Internet traffic from the country starting at 11:30 local time (15:30 UTC). A subsequent Facebook post from the utility company at 20:00 local time (00:00 UTC on September 22) noted that power had been restored to all customers. Ahead of full power restoration, Internet traffic had returned to expected levels around 17:00 local time (21:00 UTC).
Maintenance
Guinea
La Guinéenne de la Large Bande, also known as GUILAB, is the company responsible for managing the capacity allocated to the country of Guinea on the Africa Coast to Europe (ACE) submarine cable. According to a (translation of the) communiqué posted by the company on Facebook, planned maintenance on the cable would be taking place between 22:00 on July 14 and 06:18 “sharp” on July 15 (22:00 on July 14 and 06:18 on July 15 UTC). This maintenance resulted in a complete Internet outage in Guinea, as seen in the graph below. It appears that the ACE submarine cable is Guinea’s sole international Internet connection, with no other backup submarine or terrestrial connectivity.
Palau
Just a few days later, planned maintenance to another submarine cable took Palau, an island country in the western Pacific, completely offline for several days. According to a press release from the Palau National Communications Corporation (PNCC) posted to their Facebook page, “BSCC (Belau Submarine Cable Corporation) has been notified that an emergency repair will be undertaken on the SEA – US cable network in Guam from Tuesday, July 18th 7:00 a.m. Palau time, and expected to be completed 5:00 p.m. Saturday, July 22nd. … For safety reasons, repairs can only be undertaken when the cable is not powered. Since BSCC’s Palau Cable Network No 1 connects to SEA – US for onward transport to Guam, BSCC will be unable to provide service for the duration of the repair. BSCC will be unable to provide any international connectivity for Palau. The only available international connection will be via PNCC satellite connection, which will provide limited capacity compared to normal cable service.”
The graph below shows that Cloudflare did not see any appreciable traffic from Palau’s backup satellite connection during the duration of the repairs, as traffic dropped to zero at 07:00 local time on July 18 (22:00 UTC on July 17), and remained there until around 18:00 local time on July 21 (09:00 UTC), as the repairs were completed earlier than expected. A PNCC press release confirmed this early completion, noting “PNCC is pleased to inform the public that Internet and Mobile Data services for our customers have been restored, due to the early completion today of the emergency repairs on the SEA-US Submarine Cable System, our main off-island internet connection.”
Unspecified issues
Spectrum (Charter Communications)
At 14:03 Eastern Time (18:03 UTC) on August 17, the X/Twitter support account for Spectrum, a brand of US-based Internet service provider Charter Communications, posted a statement that noted “We are aware of an outage affecting customers in Alabama, Georgia and Tennessee. We apologize for the inconvenience and are working to resolve as quickly as possible. Thank you.” The graphs below show the varied impacts to traffic seen from Spectrum (AS20115) across the listed states, as well as Texas, which wasn’t initially cited by Spectrum as having an issue, though customers quickly called it out.
A near complete outage was observed in Tennessee between 12:30 – 14:00 local time (17:30 – 19:00 UTC), while a brief drop in traffic at 12:00 local time (17:00 UTC) and quick recovery ahead of another drop at 13:30 local time (18:30 UTC) was seen in Alabama. Georgia also saw an initial drop in traffic at 13:00 local time (17:00 UTC) ahead of a larger fall at 14:30 local time (18:30 UTC), while traffic from Texas only experienced a decline at 13:30 local time (18:30 UTC). Traffic volumes from all four impacted states recovered within several hours — approximately three hours after the initial post, Spectrum’s support account stated “We have received confirmation repairs have been completed and services have been restored to affected customers in the Alabama, Georgia and Tennessee area.”
Starlink
On September 12, satellite Internet service provider SpaceX Starlink experienced a brief but complete outage. The graph below shows traffic from AS14593 (SPACEX-STARLINK) dropping at 23:15 UTC, but quickly recovering, returning to normal within 90 minutes. At 00:33 UTC on September 13, Starlink shared an X/Twitter post stating “Starlink is currently in a network outage, and we are actively implementing a solution. We appreciate your patience, we’ll share an update once this issue is resolved” and just over an hour later, posted “The network issue has been fully resolved”.
Sky UK
During the evening (UTC) of September 19, numerous complaints could be found on social media about a nationwide outage across the United Kingdom on Sky Broadband (AS5607). A sharp drop in traffic from Sky Broadband can be seen in the graph below starting at 21:00 UTC, but a full outage did not appear to have taken place. Traffic volumes below expected levels lasted until approximately 01:00 UTC on September 20. While the issue was acknowledged by Sky’s support account on X/Twitter, no root cause for the disruption was ever provided.
Conclusion
As we’ve noted in past quarterly summaries, this report is intended as a summary overview of observed disruptions, and not an exhaustive or complete list of issues that have occurred during the quarter. Some disruptions not covered here were visible in our data, but never acknowledged by the impacted provider, while others were reported by industry colleagues based on their measurement methodologies, but not clearly obvious in our traffic graphs.
As we indicated above, the Cloudflare Radar Outage Center now includes information on observed traffic anomalies as well as verified outages. Interested users can subscribe to notifications for both anomalies and outages — our blog post includes more information on how to do so.
We launched the Cloudflare Radar Outage Center (CROC) during Birthday Week 2022 as a way of keeping the community up to date on Internet disruptions, including outages and shutdowns, visible in Cloudflare’s traffic data. While some of the entries have their genesis in information from social media posts made by local telecommunications providers or civil society organizations, others are based on an internal traffic anomaly detection and alerting tool. Today, we’re adding this alerting feed to Cloudflare Radar, showing country and network-level traffic anomalies on the CROC as they are detected, as well as making the feed available via API.
Building on this new functionality, as well as the route leaks and route hijacks insights that we recently launched on Cloudflare Radar, we are also launching new Radar notification functionality, enabling you to subscribe to notifications about traffic anomalies, confirmed Internet outages, route leaks, or route hijacks. Using the Cloudflare dashboard’s existing notification functionality, users can set up notifications for one or more countries or autonomous systems, and receive notifications when a relevant event occurs. Notifications may be sent via e-mail or webhooks — the available delivery methods vary according to plan level.
Traffic anomalies
Internet traffic generally follows a fairly regular pattern, with daily peaks and troughs at roughly the same volumes of traffic. However, while weekend traffic patterns may look similar to weekday ones, their traffic volumes are generally different. Similarly, holidays or national events can also cause traffic patterns and volumes to differ significantly from “normal”, as people shift their activities and spend more time offline, or as people turn to online sources for information about, or coverage of, the event. These traffic shifts can be newsworthy, and we have covered some of them in past Cloudflare blog posts (King Charles III coronation, Easter/Passover/Ramadan, Brazilian presidential elections).
However, as you also know from reading our blog posts and following Cloudflare Radar on social media, it is the more drastic drops in traffic that are a cause for concern. Some are the result of infrastructure damage from severe weather or a natural disaster like an earthquake and are effectively unavoidable, but getting timely insights into the impact of these events on Internet connectivity is valuable from a communications perspective. Other traffic drops have occurred when an authoritarian government orders mobile Internet connectivity to be shut down, or shuts down all Internet connectivity nationwide. Timely insights into these types of anomalous traffic drops are often critical from a human rights perspective, as Internet shutdowns are often used as a means of controlling communication with the outside world.
Over the last several months, the Cloudflare Radar team has been using an internal tool to identify traffic anomalies and post alerts for followup to a dedicated chat space. The companion blog post Gone Offline: Detecting Internet Outages goes into deeper technical detail about our traffic analysis and anomaly detection methodologies that power this internal tool.
Many of these internal traffic anomaly alerts ultimately result in Outage Center entries and Cloudflare Radar social media posts. Today, we’re extending the Cloudflare Radar Outage Center and publishing information about these anomalies as we identify them. As shown in the figure below, the new Traffic anomalies table includes the type of anomaly (location or ASN), the entity where the anomaly was detected (country/region name or autonomous system), the start time, duration, verification status, and an “Actions” link, where the user can view the anomaly on the relevant entity traffic page or subscribe to a notification. (If manual review of a detected anomaly finds that it is present in multiple Cloudflare traffic datasets and/or is visible in third-party datasets, such as Georgia Tech’s IODA platform, we will mark it as verified. Unverified anomalies may be false positives, or related to Netflows collection issues, though we endeavor to minimize both.)
In addition to this new table, we have updated the Cloudflare Radar Outage Center map to highlight where we have detected anomalies, as well as placing them into a broader temporal context in a new timeline immediately below the map. Anomalies are represented as orange circles on the map, and can be hidden with the toggle in the upper right corner. Double-bordered circles represent an aggregation across multiple countries, and zooming in to that area will ultimately show the number of anomalies associated with each country that was included in the aggregation. Hovering over a specific dot in the timeline displays information about the outage or anomaly with which it is associated.
Internet outage information has been available via the Radar API since we launched the Outage Center and API in September 2022, and traffic anomalies are now available through a Radar API endpoint as well. An example traffic anomaly API request and response are shown below.
Timely knowledge about Internet “events”, such as drops in traffic or routing issues, are potentially of interest to multiple audiences. Customer service or help desk agents can use the information to help diagnose customer/user complaints about application performance or availability. Similarly, network administrators can use the information to better understand the state of the Internet outside their network. And civil society organizations can use the information to inform action plans aimed at maintaining communications and protecting human rights in areas of conflict or instability. With the new notifications functionality also being launched today, you can subscribe to be notified about observed traffic anomalies, confirmed Internet outages, route leaks, or route hijacks, at a country or autonomous system level. In the following sections, we discuss how to subscribe to and configure notifications, as well as the information contained within the various types of notifications.
Subscribing to notifications
Note that you need to log in to the Cloudflare dashboard to subscribe to and configure notifications. No purchase of Cloudflare services is necessary — just a verified email address is required to set up an account. While we would have preferred to not require a login, it enables us to take advantage of Cloudflare’s existing notifications engine, allowing us to avoid having to dedicate time and resources to building a separate one just for Radar. If you don’t already have a Cloudflare account, visit https://dash.cloudflare.com/sign-up to create one. Enter your username and a unique strong password, click “Sign Up”, and follow the instructions in the verification email to activate your account. (Once you’ve activated your account, we also suggest activating two-factor authentication (2FA) as an additional security measure.)
Once you have set up and activated your account, you are ready to start creating and configuring notifications. The first step is to look for the Notifications (bullhorn) icon – the presence of this icon means that notifications are available for that metric — in the Traffic, Routing, and Outage Center sections on Cloudflare Radar. If you are on a country or ASN-scoped traffic or routing page, the notification subscription will be scoped to that entity.
Look for this icon in the Traffic, Routing, and Outage Center sections of Cloudflare Radar to start setting up notifications.In the Outage Center, click the icon in the “Actions” column of an Internet outages table entry to subscribe to notifications for the related location and/or ASN(s). Click the icon alongside the table description to subscribe to notifications for all confirmed Internet outages.In the Outage Center, click the icon in the “Actions” column of a Traffic anomalies table entry to subscribe to notifications for the related entity. Click the icon alongside the table description to subscribe to notifications for all traffic anomalies.On country or ASN traffic pages, click the icon alongside the description of the traffic trends graph to subscribe to notifications for traffic anomalies or Internet outages impacting the selected country or ASN.On country or ASN routing pages, click the icon alongside the description to subscribe to notifications for route leaks or origin hijacks related to the selected country or ASN.Within the Route Leaks or Origin Hijacks tables on the routing pages, click the icon in a table entry to subscribe to notifications for route leaks or origin hijacks for referenced countries and/or ASNs.
After clicking a notification icon, you’ll be taken to the Cloudflare login screen. Enter your username and password (and 2FA code if required), and once logged in, you’ll see the Add Notification page, pre-filled with the key information passed through from the referring page on Radar, including relevant locations and/or ASNs. (If you are already logged in to Cloudflare, then you’ll be taken directly to the Add Notification page after clicking a notification icon on Radar.) On this page, you can name the notification, add an optional description, and adjust the location and ASN filters as necessary. Enter an email address for notifications to be sent to, or select an established webhook destination (if you have webhooks enabled on your account).
Click “Save”, and the notification is added to the Notifications Overview page for the account.
You can also create and configure notifications directly within Cloudflare, without starting from a link on Radar a Radar page. To do so, log in to Cloudflare, and choose “Notifications” from the left side navigation bar. That will take you to the Notifications page shown below. Click the “Add” button to add a new notification.
On the next page, search for and select “Radar” from the list of Cloudflare products for which notifications are available.
On the subsequent “Add Notification” page, you can create and configure a notification from scratch. Event types can be selected in the “Notify me for:” field, and both locations and ASNs can be searched for and selected within the respective “Filtered by (optional)” fields. Note that if no filters are selected, then notifications will be sent for all events of the selected type(s). Add one or more emails to send notifications to, or select a webhook target if available, and click “Save” to add it to the list of notifications configured for your account.
It is worth mentioning that advanced users can also create and configure notifications through the Cloudflare API Notification policies endpoint, but we will not review that process within this blog post.
Notification messages
Example notification email messages are shown below for the various types of events. Each contains key information like the type of event, affected entities, and start time — additional relevant information is included depending on the event type. Each email includes both plaintext and HTML versions to accommodate multiple types of email clients. (Final production emails may vary slightly from those shown below.)
Internet outage notification emails include information about the affected entities, a description of the cause of the outage, start time, scope (if available), and the type of outage (Nationwide, Network, Regional, or Platform), as well as a link to view the outage in a Radar traffic graph.Traffic anomaly notification emails simply include information about the affected entity and a start time, as well as a link to view the anomaly in a Radar traffic graph.BGP hijack notification emails include information about the hijacking and victim ASNs, affected IP address prefixes, the number of BGP messages (announcements) containing leaked routes, the number of peers announcing the hijack, detection timing, a confidence level on the event being a true hijack, and relevant tags, as well as a link to view details of the hijack event on Radar.BGP route leak notification emails include information about the AS that the leaked routes were learned from, the AS that leaked the routes, the AS that received and propagated the leaked routes, the number of affected prefixes, the number of affected origin ASes, the number of BGP route collector peers that saw the route leak, and detection timing, as well as a link to view details of the route leak event on Radar.
If you are sending notifications to webhooks, you can integrate those notifications into tools like Slack. For example, by following the directions in Slack’s API documentation, creating a simple integration took just a few minutes and results in messages like the one shown below.
Conclusion
Cloudflare’s unique perspective on the Internet provides us with near-real-time insight into unexpected drops in traffic, as well as potentially problematic routing events. While we’ve been sharing these insights with you over the past year, you had to visit Cloudflare Radar to figure out if there were any new “events”. With the launch of notifications, we’ll now automatically send you information about the latest events that you are interested in.
We encourage you to visit Cloudflare Radar to familiarize yourself with the information we publish about traffic anomalies, confirmed Internet outages, BGP route leaks, and BGP origin hijacks. Look for the notification icon on the relevant graphs and tables on Radar, and go through the workflow to set up and subscribe to notifications. (And don’t forget to sign up for a Cloudflare account if you don’t have one already.) Please send us feedback about the notifications, as we are constantly working to improve them, and let us know how and where you’ve integrated Radar notifications into your own tools/workflows/organization.
Cloudflare operates in more than 300 cities in over 100 countries, where we interconnect with over 12,000 network providers in order to provide a broad range of services to millions of customers. The breadth of both our network and our customer base provides us with a unique perspective on Internet resilience, enabling us to observe the impact of Internet disruptions.
As we have noted in the past, this post is intended as a summary overview of observed disruptions, and is not an exhaustive or complete list of issues that have occurred during the quarter.
Government directed
Late spring often marks the start of a so-called “exam season” in several Middle Eastern and African countries, where students sit for a series of secondary school exams. In an attempt to prevent cheating on these exams, governments in the countries have taken to implementing wide-scale Internet shutdowns covering time periods just before and during the exams. We have covered these shutdowns in the past, including Sudan and Syria in 2021 and Syria, Sudan, and Algeria in 2022. This year, we saw governments in Iraq, Algeria, and Syria taking such actions.
Iraq
In the weeks prior to the start of this year’s shutdowns, it was reported that the Iraqi Ministry of Communications had announced it had refused a request from the Ministry of Education to impose an Internet shutdown during the exams as part of efforts to prevent cheating. Unfortunately, this refusal was short-lived, with shutdowns ultimately starting two weeks later.
In Iraq, two sets of shutdowns were observed: one impacted networks nationwide, except for the Kurdistan Region, while the other impacted networks within the Kurdistan Region. The former set of shutdowns were related to 9th and 12th grade exams, and were scheduled to occur from June 1 through July 15, between 04:00 and 08:00 local time (01:00 – 05:00 UTC). The graphs below show that during June, shutdowns took place on June 1, 4, 6, 8, 11, 13, 15, 17, 21, 22, 24, 25, and 26, resulting in significant disruptions to Internet connectivity. The shutdowns were implemented across a number of network providers, including AS203214 (HulumTele), AS59588 (Zain), AS199739 (Earthlink), AS203735 (Net Tech), AS51684 (Asiacell), and AS58322 (Halasat). The orange-highlighted areas in the graphs below show traffic on each network provider dropping to zero during the shutdowns.
As noted above, exam-related Internet shutdowns were also implemented in the Kurdistan region of Iraq. One report quoted the Minister of Education of the Kurdistan Regional Government as stating "The Internet will be turned off as needed during exams, but just like in previous years, the period of the Internet shutdown will not be lengthy, but rather short.” To that end, the observed shutdowns generally lasted about two hours, occurring between 06:30 and 08:30 local time (03:30 – 05:30 UTC) on June 3, 6, 10, 13, 17, and 24. The graphs below show the impact across three network providers in the region: AS21277 (Newroz Telecom), AS48492 (IQ Online), and AS59625 (KorekTel).
2023 marks the sixth year that Algeria has disrupted Internet connectivity to prevent cheating during nationwide exams. In 2022, we noted that “it appears that the Algerian government has shifted to a content blocking-based approach, instead of a wide-scale Internet shutdown.” It appears that the same approach was taken this year, as we again observed two nominal drops in traffic during each of the exam days, rather than a complete loss of traffic. These traffic shifts were observed on mobile network providers AS33779 (Ooredoo/Wataniya), AS327931 (Djezzy/Optimum), and AS327712 (Mobilis/Telecom Algeria). The first disruption takes place between 08:00 – 12:00 local time (07:00 – 11:00 UTC), with the second occurring between 14:00 – 17:00 local time (13:00 – 16:00 UTC).
Syria
After implementing four exam-related Internet shutdowns in 2022, this year saw just two. On June 25 and 26, Internet shutdowns took place between 05:00 – 08:30 local time (02:00 – 05:30 UTC). Syrian Telecom (AS29256), the government-affiliated telecommunications company, informed subscribers in a Facebook post that the Internet would be cut off at the request of the Ministry of Education.
Senegal
In Senegal, violent protests over the sentencing of opposition leader Ousmane Sonko to jail led the government to restrict access to platforms including WhatsApp, Facebook, Twitter, Instagram, TikTok, Telegram, Signal, and YouTube. On June 4, the Senegal Ministry of Communication issued a statement temporarily suspending mobile Internet access, with a followup statement on June 6 ending the suspension. These disruptions to mobile Internet access were visible on two network providers within the country: AS37196 (Sudatel Senegal) and AS37649 (Tigo/Free).
As shown in the graphs below, the shutdowns on Sudatel Senegal occurred from 15:00 local time on June 3 through 01:00 local time on June 5, and then again from 13:00 local time on June 5 until 01:00 local time on June 6. The three shutdowns seen on Tigo/Free took place between 15:30 – 19:00 local time on June 3, from 13:45 local time on June 4 until 02:05 local time on June 5, and from 13:05 local time on June 5 through 01:00 local time on June 6. (Senegal is UTC+0, so the local times are the same as UTC.)
Mauritania
In Mauritania, authorities cut off mobile Internet services after protests over the death of a young man in police custody. The shutdown began at 23:00 local time on May 30, and lasted six days, with connectivity returning at 23:00 local time on June 6. (Mauritania is UTC+0, so the local times are the same as UTC.) The graphs below show a near complete loss of Internet traffic during that period from AS37541 (Chinguitel) and AS37508 (Mattel), two mobile network providers within the country.
Pakistan
On May 9, Imran Khan, former Prime Minister of Pakistan was arrested on corruption charges. Following the arrest, violent protests erupted in several cities, leading the government of Pakistan to order the shutdown of mobile Internet services, as well as the blocking of several social media platforms. The figures below show the impact of the ordered shutdown to traffic on four mobile network providers within the country: AS24499 (Telenor Pakistan), AS59257 (China Mobile Pak), AS45669 (Mobilink/Jazz), and AS56167 (Ufone/PTML). The ordered shutdown caused a complete loss of Internet traffic from these networks that started at 22:00 local time (17:00 UTC) on May 9 at Telenor and China Mobile Pakistan, 18:00 local time (13:00 UTC) on Mobilink/Jazz, and 01:00 local time on May 10 (20:00 UTC on May 9) at Ufone/PTML. Traffic was restored at 22:00 local time (17:00 UTC) on May 12.
Looking at Cloudflare Radar’s recently launchedInternet Quality page for Pakistan during the duration of the shutdown, we observed that median latency within Pakistan dropped slightly after mobile networks were shut down, shown in the graph below. Prior to the shutdown, median latency (as observed to Cloudflare and a set of other providers) was in the 90-100ms range, while afterward, it averaged closer to 75ms. This may be a result of users shifting to lower latency fixed broadband connections – several fixed broadband providers in the country experienced increased traffic volumes while the mobile networks were unavailable.
Additional details about the mobile network shutdowns, content blocking, and the impact at an administrative unit and city level can be found in our May 12 blog post Cloudflare’s view of Internet disruptions in Pakistan.
India
Internet shutdowns are unfortunately frequent in India, with digital rights organization Access Now reporting at least 84 shutdowns within the country in 2022. The shutdowns are generally implemented at a more local level, and often last for a significant amount of time. One such shutdown took place in the northeastern Indian state of Manipur starting on May 3 after the escalation of ethnic conflict, and was reportedly intended to “thwart the design and activities of anti-national and anti-social elements… by stopping the spread of disinformation and false rumours'' and the likelihood of “serious disturbances to the entire peaceful coexistence of the communities and maintenance of public order”. Mobile data services were initially suspended for a five-day period, with the suspension continually extended through additional templated orders issued every five days.
The graphs below show the impact of the ordered shutdown to traffic from two major network providers in Manipur. Traffic from both AS45609 (Airtel) and AS9829 (BSNL) fell significantly around 18:00 local time (12:30 UTC) on May 4. Traffic on Airtel has remained low, and continued to drop further through the end of June. Traffic on BSNL showed slight signs of recovery starting in early June, but remains extremely low.
The shutdown order remains in place as of the time of this writing (late July).
Severe weather
Guam
On May 24, “Super Typhoon” Mawar wreaked havoc on the US territory of Guam, causing widespread physical damage after it made landfall, taking down trees, buildings, power lines, and communications infrastructure across the island. One result of this damage was a significant disruption to Internet connectivity, as shown in the country-level graph below. Restoration efforts started almost immediately, with Guam Power Authority, Docomo Pacific, and GTA Teleguam all posting regular status updates on their websites and/or social media accounts.
Among the two Internet providers, GTA Teleguam (AS9246) was largely able to complete service restoration in June, with traffic returning to pre-storm levels around June 17, as seen in the graph below. In fact, in a June 20 Facebook post they noted that “As of today, a majority of our wireless network cell sites are operational.” However, recovery at Docomo Pacific (AS3605) is taking significantly longer. The graph below shows that as of the end of June, traffic remained significantly below pre-storm levels.
Cable damage
Bolivia
On June 19, COTAS, a Bolivian telecommunications company, posted an update on their Facebook page that alerted users that a fiber optic cable had been cut in the town of Pongo. As seen in the graphs below, this cut significantly disrupted Internet connectivity across COTAS and two other network providers in the country: AS25620 (COTAS), AS27839 (Comteco), and AS52495 (Cotel) between 13:00 – 18:00 local time (17:00 – 22:00 UTC).
The Gambia
Gamtel, the state telecommunications company in The Gambia, notified subscribers via a Twitter post on June 7 of a localized fiber cut, and then of additional cable cuts on June 8. These fiber cuts disrupted Internet connectivity on AS25250 (Gamtel) between 14:00 local time on June 7 and 00:00 local time on June 9, with traffic volumes down as much as 80% as compared to the previous period. (The Gambia is UTC+0, so the local times are the same as UTC.)
Philippines
An advisory posted on Twitter by Philippines telecommunications provider PLDT at 18:43 local time (10:43 UTC) on June 5 stated “One of our submarine cable partners confirms a loss in some of its internet bandwidth capacity, and thus causing slower Internet browsing. We are working with our partners to provide alternate capacity that would restore the browsing experience in the next few hours.” The traffic graph below shows a minor disruption to Internet traffic for AS9299 (PLDT) starting around 14:00 local time (06:00 UTC), and the “slower Internet browsing” noted by PLDT is evident in the Internet quality graphs below, with increased latency and decreased bandwidth evident around that same time. PLDT stated in a subsequent tweet that as of 06:22 local time on June 6 (22:22 UTC on June 5), “Our submarine cable partner confirms supplementing additional capacity, restoring browser experience.”
Power outages
Curaçao
Aqualectra is the primary utility company in Curaçao, providing water and power services. On June 8, they posted a series of alerts to their Facebook page (1, 2, 3, 4) regarding a power outage impacting “all neighborhoods”, caused by a malfunction in one of the main power cables connected to the substation at Parera. This loss of power impacted Internet connectivity on the island, with a significant loss of traffic observed at a country level, as seen in the graph below, as well as across several Internet service providers, including AS11081 (UTS), AS52233 (Columbus Communications), and AS27660 (Curaçao Telecom). A followup Facebook post dated 01:25 local time on June 9 (05:25 UTC) confirmed the restoration of power to all neighborhoods.
Portugal
A power outage at an Equinix data center in Prior Velho (near Lisbon) on the afternoon of June 6 affected local utilities, banking services, and court networks, according to published reports (1, 2). Portuguese Internet service provider MEO was also impacted by the power outage, which caused a drop in traffic for AS3243 (MEO-RESIDENCIAL) and AS15525 (MEO-EMPRESAS), seen in the graphs below. The disruptions caused by the power outage also impacted connectivity quality within Portugal, as the Radar Internet quality graphs below highlight – a concurrent drop in bandwidth and increase in latency is visible, indicating that end users likely experienced poorer performance during that period of time.
Botswana
A countrywide power outage in Botswana on May 19 caused an Internet disruption that lasted about 90 minutes, from 10:45 until 12:15 local time (08:45 – 10:15 UTC), visible in the graph below. A tweet from Botswana Power Corporation provided public notice of the incident, but did not include a root cause.
Barbados
On April 4, The Barbados Light & Power Company tweeted an “Outage Notice”, stating “We are aware that our customers across the island are currently without electricity.” Posted at 11:46 local time (15:46 UTC), the notice comes shortly after a significant drop in traffic was observed country-wide, indicating that the power outage also impacted Internet connectivity across the country. After posting several additional updates throughout the day, a final update posted at 18:00 local time (22:00 UTC) indicated that power had been restored to 100% of impacted customers. The graph below shows that traffic took several additional hours to return to normal levels. (Note that the orange highlighting in the graph represents the duration of the disruption, and the red shading is related to an internal data collection issue.)
Technical problems
Namibia
A seven-hour Internet disruption observed in Namibia on June 15 and 16 was caused by unspecified “technical challenges” faced by Telecom Namibia. According to a tweet from the provider, “Telecom Namibia experienced technical challenges on its fixed and mobile data services on Thursday leading to intermittent Internet connectivity.” The impact of these challenges is visible in both the country- and network-level traffic graphs below.
Solomon Islands
Unspecified “technical reasons” also disrupted mobile Internet connectivity for Our Telekom customers in the Solomon Islands on April 26 and 27. An April 26 Facebook post from Our Telekom simply stated “Our mobile data network is currently down due to technical reasons.” The graphs below show a significant drop in traffic for AS45891 (Our Telekom/SBT) between 06:30 local time on April 27 (19:30 UTC on April 26) and 17:00 local time on April 27 (06:00 UTC). The loss of mobile traffic from Our Telekom also impacted traffic at a country level, as the graph shows a similar disruption for the Solomon Islands.
SpaceX Starlink
With an increasingly global service footprint, disruptions observed on SpaceX Starlink potentially impact users across multiple countries around the world. Just before midnight UTC on April 7, Internet traffic seen from AS14593 (SpaceX-Starlink) began to decline significantly. The disruption was short-lived, with traffic returning to expected levels within two hours. According to a Twitter post from Elon Musk, CEO of SpaceX, the problem was “Caused by expired ground station cert” (an expired digital certificate associated with one or more Starlink ground stations, likely preventing communication between the satellite constellation and the ground station(s)).
Madagascar
In Madagascar, a “problem with the backbone”, reported by Telma Madagascar, caused a loss of as much as two-thirds of Internet traffic between 09:15 – 14:00 local time (06:15 – 11:00 UTC) on April 7. The graphs below show that the backbone issue disrupted traffic at a national level, as well as for AS37054 (Telma Madagascar).
United Kingdom
On April 4, UK Internet provider Virgin Media suffered multiple service disruptions that impacted Internet connectivity for broadband customers. The first outage started just before 01:00 local time (midnight UTC)l, lasting until approximately 09:00 local time (08:00 UTC). The second outage started around 16:00 local time (15:00 UTC), with traffic volumes going up and down over the next several hours before appearing to stabilize around 21:30 local time (20:30 UTC).
Virgin Media’s Twitter account acknowledged the early morning disruption several hours after it began, posting “We’re aware of an issue that is affecting broadband services for Virgin Media customers as well as our contact centres. Our teams are currently working to identify and fix the problem as quickly as possible and we apologise to those customers affected.” A subsequent post after service restoration noted “We’ve restored broadband services for customers but are closely monitoring the situation as our engineers continue to investigate. We apologise for any inconvenience caused.”
However, the second disruption was acknowledged on Virgin Media’s Twitter account much more rapidly, with a post at 16:25 UTC stating “Unfortunately we have seen a repeat of an earlier issue which is causing intermittent broadband connectivity problems for some Virgin Media customers. We apologise again to those impacted, our teams are continuing to work flat out to find the root cause of the problem and fix it.”
Although no additional details have been shared via social media by Virgin Media about the outages or their resolution, some additional information was shared via Twitter by an apparent customer, who posted “Virgin Media engineers re-seated fibre cards and reset hub equipment to restore service. TTL was extended as a workaround to maintain stability whilst a permanent fix is implemented.”
A little more than a year later, on May 26, we observed an Internet outage at Miranda Media. Traffic started to fall around 16:30 local time (13:30 UTC), dropping to zero around 18:15 local time (15:15 UTC). The outage disrupted connectivity on the Crimean Peninsula and parts of occupied Ukraine and lasted until approximately 06:00 local time on May 27 (03:00 UTC). Published reports (1,2) suggest that the outage was due to a cyberattack targeting Miranda Media, reportedly carried out by Ukrainian hacktivists.
Russia
Russian satellite provider Dozor Teleport, whose customers include Russia’s Ministry of Defense, ships of the Northern Fleet, Russian energy firm Gazprom, remote oil fields, the Bilibino nuclear power plant, the Federal Security Service (FSB), Rosatom, and other organizations, experienced a multi-hour outage on June 29. The outage, which occurred between 01:30 – 17:30 UTC, was reportedly the result of a cyberattack that at least two groups claimed responsibility for.
Military action
Chad
Multiple Internet disruptions occurred in Chad on April 23 and 24, impacting several Internet providers, and were ultimately visible at a country level as well. As seen in the graphs below, the outages occurred from 04:30 – 06:00 local time (03:30 – 05:00 UTC) and 15:00 – 20:00 local time (14:00 – 19:00 UTC) on April 23, and 04:00 – 08:30 local time (03:00 – 07:30 UTC) on April 24. The impacted network providers in Chad included AS327802 (Millicom Chad), AS327756 (Airtel Chad), AS328594 (Sudat Chad), and AS327975 (ILNET-TELECOM). The outages were reportedly caused by damage to fiber infrastructure that links Chad with neighboring Cameroon and Sudan, with the latter experiencing Internet service disruptions amid clashes between the Sudanese Armed Forces (SAF) and Rapid Support Forces (RSF).
Sudan
As noted above, military action in Sudan disrupted Internet connectivity in Chad in April. Starting in mid-April, multiple Internet outages were observed at major Sudanese Internet providers, three of which are shown in the graphs below. The fighting in the country has led to fuel shortages and power cuts, ultimately disrupting Internet connectivity.
AS15706 (Sudatel) experienced complete Internet outages from 03:00 on April 23 to 17:00 on April 24 local time (01:00 on April 23 to 15:00 on April 24 UTC) and again from 03:00 on April 25 until 01:00 on April 28 local time (01:00 on April 25 to 23:00 on April 27 UTC). Internet connectivity on AS36972 (MTN) was disrupted between 03:00 and 12:00 local time on April 16 (01:00 – 10:00 UTC) and again between 20:00 on April 27 until 02:00 on April 29 (18:00 on April 27 to 00:00 on April 29). After a nominal multi-day recovery, a long-term near complete outage started on May 5, lasting for multiple weeks. Similar to MTN, multiple extended outages were also observed on AS33788 (Kanar Telecommunication). After seeing a significant drop in traffic midday on April 19, a near complete outage is visible between 12:00 on April 21 and 01:00 on April 29 (10:00 on April 21 to 23:00 on April 28 UTC), with a very brief minor recovery late in the day on April 24. A longer duration outage began around 00:00 local time on May 11 (22:00 on May 10 UTC), also lasting for multiple weeks.
Togo, Republic of Congo (Brazzaville), Burkina Faso
Repair work on the West Africa Cable System (WACS) submarine cable disrupted Internet connectivity across multiple countries, including Togo, Republic of Congo (Brazzaville), and Burkina Faso on April 6. According to the Google translation of a Facebook post from Canalbox Congo, the repair work was likely to cause “very strong disruptions on Internet connections with the risk of a total outage”. (Canalbox (GVA) is an African telecommunications operator that provides services across multiple countries in Africa.)
The graph below for AS36924 (GVA-Canalbox) shows three overlapping outage annotations, with each related to a disruption observed on that autonomous system (network) in one of the impacted countries. In the Republic of Congo (Brazzaville), a significant traffic disruption is visible between 16:15 – 23:15 local time (15:15 – 22:15 UTC). In Burkina Faso, the disruption happened earlier and was less severe, taking place between 09:15 – 18:00 local time (09:15 – 18:00 UTC), with a similar impact in Togo, where traffic was disrupted between 11:00 – 23:15 local time (11:00 – 23:15 UTC).
Conclusion
Because of how tightly interwoven the Internet has become with commerce, financial services, and everyday life around the world, any disruption to Internet connectivity ultimately carries an economic impact. The providers impacted by disruptions caused by unexpected or unavoidable events such as cable cuts or severe weather generally try to minimize the scope and duration of such disruptions, ultimately limiting the economic impact. However, in the case of government-directed Internet shutdowns, the damage to the economy is ultimately self-inflicted. The Internet Society’s new Net Loss Calculator now provides a way to quantify this damage, enabling the public, advocacy groups, and governments themselves to understand the potential cost of an Internet shutdown from gross domestic product (GDP), foreign direct investment (FDI), and unemployment perspectives.
Internet connections are most often marketed and sold on the basis of "speed", with providers touting the number of megabits or gigabits per second that their various service tiers are supposed to provide. This marketing has largely been successful, as most subscribers believe that "more is better”. Furthermore, many national broadband plans in countries around the world include specific target connection speeds. However, even with a high speed connection, gamers may encounter sluggish performance, while video conference participants may experience frozen video or audio dropouts. Speeds alone don't tell the whole story when it comes to Internet connection quality.
Additional factors like latency, jitter, and packet loss can significantly impact end user experience, potentially leading to situations where higher speed connections actually deliver a worse user experience than lower speed connections. Connection performance and quality can also vary based on usage – measured average speed will differ from peak available capacity, and latency varies under loaded and idle conditions.
The new Cloudflare Radar Internet Quality page
A little more than three years ago, as residential Internet connections were strained because of the shift towards working and learning from home due to the COVID-19 pandemic, Cloudflare announced the speed.cloudflare.com speed test tool, which enabled users to test the performance and quality of their Internet connection. Within the tool, users can download the results of their individual test as a CSV, or share the results on social media. However, there was no aggregated insight into Cloudflare speed test results at a network or country level to provide a perspective on connectivity characteristics across a larger population.
Today, we are launching these long-missing aggregated connection performance and quality insights on Cloudflare Radar. The new Internet Quality page provides both country and network (autonomous system) level insight into Internet connection performance (bandwidth) and quality (latency, jitter) over time. (Your Internet service provider is likely an autonomous system with its own autonomous system number (ASN), and many large companies, online platforms, and educational institutions also have their own autonomous systems and associated ASNs.) The insights we are providing are presented across two sections: the Internet Quality Index (IQI), which estimates average Internet quality based on aggregated measurements against a set of Cloudflare & third-party targets, and Connection Quality, which presents peak/best case connection characteristics based on speed.cloudflare.com test results aggregated over the previous 90 days. (Details on our approach to the analysis of this data are presented below.)
Users may note that individual speed test results, as well as the aggregate speed test results presented on the Internet Quality page will likely differ from those presented by other speed test tools. This can be due to a number of factors including differences in test endpoint locations (considering both geographic and network distance), test content selection, the impact of “rate boosting” by some ISPs, and testing over a single connection vs. multiple parallel connections. Infrequent testing (on any speed test tool) by users seeking to confirm perceived poor performance or validate purchased speeds will also contribute to the differences seen in the results published by the various speed test platforms.
And as we announced in April, Cloudflare has partnered with Measurement Lab (M-Lab) to create a publicly-available, queryable repository for speed test results. M-Lab is a non-profit third-party organization dedicated to providing a representative picture of Internet quality around the world. M-Lab produces and hosts the Network Diagnostic Tool, which is a very popular network quality test that records millions of samples a day. Given their mission to provide a publicly viewable, representative picture of Internet quality, we chose to partner with them to provide an accurate view of your Internet experience and the experience of others around the world using openly available data.
Connection speed & quality data is important
While most advertisements for fixed broadband and mobile connectivity tend to focus on download speeds (and peak speeds at that), there’s more to an Internet connection, and the user’s experience with that Internet connection, than that single metric. In addition to download speeds, users should also understand the upload speeds that their connection is capable of, as well as the quality of the connection, as expressed through metrics known as latency and jitter. Getting insight into all of these metrics provides a more well-rounded view of a given Internet connection, or in aggregate, the state of Internet connectivity across a geography or network.
The concept of download speeds are fairly well understood as a measure of performance. However, it is important to note that the average download speeds experienced by a user during common Web browsing activities, which often involves the parallel retrieval of multiple smaller files from multiple hosts, can differ significantly from peak download speeds, where the user is downloading a single large file (such as a video or software update), which allows the connection to reach maximum performance. The bandwidth (speed) available for upload is sometimes mentioned in ISP advertisements, but doesn’t receive much attention. (And depending on the type of Internet connection, there’s often a significant difference between the available upload and download speeds.) However, the importance of upload came to the forefront in 2020 as video conferencing tools saw a surge in usage as both work meetings and school classes shifted to the Internet during the COVID-19 pandemic. To share your audio and video with other participants, you need sufficient upload bandwidth, and this issue was often compounded by multiple people sharing a single residential Internet connection.
Latency is the time it takes data to move through the Internet, and is measured in the number of milliseconds that it takes a packet of data to go from a client (such as your computer or mobile device) to a server, and then back to the client. In contrast to speed metrics, lower latency is preferable. This is especially true for use cases like online gaming where latency can make a difference between a character’s life and death in the game, as well as video conferencing, where higher latency can cause choppy audio and video experiences, but it also impacts web page performance. The latency metric can be further broken down into loaded and idle latency. The former measures latency on a loaded connection, where bandwidth is actively being consumed, while the latter measures latency on an “idle” connection, when there is no other network traffic present. (These specific loaded and idle definitions are from the device’s perspective, and more specifically, from the speed test application’s perspective. Unless the speed test is being performed directly from a router, the device/application doesn't have insight into traffic on the rest of the network.) Jitter is the average variation found in consecutive latency measurements, and can be measured on both idle and loaded connections. A lower number means that the latency measurements are more consistent. As with latency, Internet connections should have minimal jitter, which helps provide more consistent performance.
Our approach to data analysis
The Internet Quality Index (IQI) and Connection Quality sections get their data from two different sources, providing two different (albeit related) perspectives. Under the hood they share some common principles, though.
IQI builds upon the mechanism we already use to regularly benchmark ourselves against other industry players. It is based on end user measurements against a set of Cloudflare and third-party targets, meant to represent a pattern that has become very common in the modern Internet, where most content is served from distribution networks with points of presence spread throughout the world. For this reason, and by design, IQI will show worse results for regions and Internet providers that rely on international (rather than peering) links for most content.
IQI is also designed to reflect the traffic load most commonly associated with web browsing, rather than more intensive use. This, and the chosen set of measurement targets, effectively biases the numbers towards what end users experience in practice (where latency plays an important role in how fast things can go).
For each metric covered by IQI, and for each ASN, we calculate the 25th percentile, median, and 75th percentile at 15 minute intervals. At the country level and above, the three calculated numbers for each ASN visible from that region are independently aggregated. This aggregation takes the estimated user population of each ASN into account, biasing the numbers away from networks that source a lot of automated traffic but have few end users.
The Connection Quality section gets its data from the Cloudflare Speed Test tool, which exercises a user's connection in order to see how well it is able to perform. It measures against the closest Cloudflare location, providing a good balance of realistic results and network proximity to the end user. We have a presence in 285 cities around the world, allowing us to be pretty close to most users.
Similar to the IQI, we calculate the 25th percentile, median, and 75th percentile for each ASN. But here these three numbers are immediately combined using an operation called the trimean — a single number meant to balance the best connection quality that most users have, with the best quality available from that ASN (users may not subscribe to the best available plan for a number of reasons).
Because users may choose to run a speed test for different motives at different times, and also because we take privacy very seriously and don’t record any personally identifiable information along with test results, we aggregate at 90-day intervals to capture as much variability as we can.
At the country level and above, the calculated trimean for each ASN in that region is aggregated. This, again, takes the estimated user population of each ASN into account, biasing the numbers away from networks that have few end users but which may still have technicians using the Cloudflare Speed Test to assess the performance of their network.
Navigating the Internet Quality page
The new Internet Quality page includes three views: Global, country-level, and autonomous system (AS). In line with the other pages on Cloudflare Radar, the country-level and AS pages show the same data sets, differing only in their level of aggregation. Below, we highlight the various components of the Internet Quality page.
Global
The top section of the global (worldwide) view includes time series graphs of the Internet Quality Index metrics aggregated at a continent level. The time frame shown in the graphs is governed by the selection made in the time frame drop down at the upper right of the page, and at launch, data for only the last three months is available. For users interested in examining a specific continent, clicking on the other continent names in the legend removes them from the graph. Although continent-level aggregation is still rather coarse, it still provides some insight into regional Internet quality around the world.
Further down the page, the Connection Quality section presents a choropleth map, with countries shaded according to the values of the speed, latency, or jitter metric selected from the drop-down menu. Hovering over a country displays a label with the country’s name and metric value, and clicking on the country takes you to the country’s Internet Quality page. Note that in contrast to the IQI section, the Connection Quality section always displays data aggregated over the previous 90 days.
Country-level
Within the country-level page (using Canada as an example in the figures below), the country’s IQI metrics over the selected time frame are displayed. These time series graphs show the median bandwidth, latency, and DNS response time within a shaded band bounded at the 25th and 75th percentile and represent the average expected user experience across the country, as discussed in the Our approach to data analysis section above.
Below that is the Connection Quality section, which provides a summary view of the country’s measured upload and download speeds, as well as latency and jitter, over the previous 90 days. The colored wedges in the Performance Summary graph are intended to illustrate aggregate connection quality at a glance, with an “ideal” connection having larger upload and download wedges and smaller latency and jitter wedges. Hovering over the wedges displays the metric’s value, which is also shown in the table to the right of the graph.
Below that, the Bandwidth and Latency/Jitter histograms illustrate the bucketed distribution of upload and download speeds, and latency and jitter measurements. In some cases, the speed histograms may show a noticeable bar at 1 Gbps, or 1000 ms (1 second) on the latency/jitter histograms. The presence of such a bar indicates that there is a set of measurements with values greater than the 1 Gbps/1000 ms maximum histogram values.
Autonomous system level
Within the upper-right section of the country-level page, a list of the top five autonomous systems within the country is shown. Clicking on an ASN takes you to the Performance page for that autonomous system. For others not displayed in the top five list, you can use the search bar at the top of the page to search by autonomous system name or number. The graphs shown within the AS level view are identical to those shown at a country level, but obviously at a different level of aggregation. You can find the ASN that you are connected to from the My Connection page on Cloudflare Radar.
Exploring connection performance & quality data
Digging into the IQI and Connection Quality visualizations can surface some interesting observations, including characterizing Internet connections, and the impact of Internet disruptions, including shutdowns and network issues. We explore some examples below.
Characterizing Internet connections
Verizon FiOS is a residential fiber-based Internet service available to customers in the United States. Fiber-based Internet services (as opposed to cable-based, DSL, dial-up, or satellite) will generally offer symmetric upload and download speeds, and the FiOS plans page shows this to be the case, offering 300 Mbps (upload & download), 500 Mbps (upload & download), and “1 Gig” (Verizon claims average wired speeds between 750-940 Mbps download / 750-880 Mbps upload) plans. Verizon carries FiOS traffic on AS701 (labeled UUNET due to a historical acquisition), and in looking at the bandwidth histogram for AS701, several things stand out. The first is a rough symmetry in upload and download speeds. (A cable-based Internet service provider, in contrast, would generally show a wide spread of download speeds, but have upload speeds clustered at the lower end of the range.) Another is the peaks around 300 Mbps and 750 Mbps, suggesting that the 300 Mbps and “1 Gig” plans may be more popular than the 500 Mbps plan. It is also clear that there are a significant number of test results with speeds below 300 Mbps. This is due to several factors: one is that Verizon also carries lower speed non-FiOS traffic on AS701, while another is that erratic nature of in-home WiFi often means that the speeds achieved on a test will be lower than the purchased service level.
Traffic shifts drive latency shifts
On May 9, 2023, the government of Pakistan ordered the shutdown of mobile network services in the wake of protests following the arrest of former Prime Minister Imran Khan. Our blog post covering this shutdown looked at the impact from a traffic perspective. Within the post, we noted that autonomous systems associated with fixed broadband networks saw significant increases in traffic when the mobile networks were shut down – that is, some users shifted to using fixed networks (home broadband) when mobile networks were unavailable.
Examining IQI data after the blog post was published, we found that the impact of this traffic shift was also visible in our latency data. As can be seen in the shaded area of the graph below, the shutdown of the mobile networks resulted in the median latency dropping about 25% as usage shifted from higher latency mobile networks to lower latency fixed broadband networks. An increase in latency is visible in the graph when mobile connectivity was restored on May 12.
Bandwidth shifts as a potential early warning sign
On April 4, UK mobile operator Virgin Media suffered several brief outages. In examining the IQI bandwidth graph for AS5089, the ASN used by Virgin Media (formerly branded as NTL), indications of a potential problem are visible several days before the outages occurred, as median bandwidth dropped by about a third, from around 35 Mbps to around 23 Mbps. The outages are visible in the circled area in the graph below. Published reports indicate that the problems lasted into April 5, in line with the lower median bandwidth measured through mid-day.
Submarine cable issues cause slower browsing
On June 5, Philippine Internet provider PLDTTweeted an advisory that noted “One of our submarine cable partners confirms a loss in some of its internet bandwidth capacity, and thus causing slower Internet browsing.” IQI latency and bandwidth graphs for AS9299, a primary ASN used by PLDT, shows clear shifts starting around 06:45 UTC (14:45 local time). Median bandwidth dropped by half, from 17 Mbps to 8 Mbps, while median latency increased by 75% from 37 ms to around 65 ms. 75th percentile latency also saw a significant increase, nearly tripling from 63 ms to 180 ms coincident with the reported submarine cable issue.
Conclusion
Making network performance and quality insights available on Cloudflare Radar supports Cloudflare’s mission to help build a better Internet. However, we’re not done yet – we have more enhancements planned. These include making data available at a more granular geographical level (such as state and possibly city), incorporating AIM scores to help assess Internet quality for specific types of use cases, and embedding the Cloudflare speed test directly on Radar using the open source JavaScript module.
In the meantime, we invite you to use speed.cloudflare.com to test the performance and quality of your Internet connection, share any country or AS-level insights you discover on social media (tag @CloudflareRadar on Twitter or @[email protected] on Mastodon), and explore the underlying data through the M-Lab repository or the Radar API.
Over the last several years, governments in a number of countries in the Middle East/Northern Africa (MENA) region have taken to implementing widespread nationwide shutdowns in an effort to prevent cheating on nationwide academic exams. Although it is unclear whether such shutdowns are actually successful in curbing cheating, it is clear that they take a financial toll on the impacted countries, with estimated losses in the millions of US dollars.
During the first two weeks of June 2023, we’ve seen Iraq implementing a series of multi-hour shutdowns that will reportedly occur through mid-July, as well as Algeria taking similar actions to prevent cheating on baccalaureate exams. Shutdowns in Syria were reported to begin on June 7, but there’s been no indication of them in traffic data as of this writing (June 13). These actions echo those taken in Iraq, Syria, Sudan, and Algeria in 2022 and in Syria and Sudan in 2021.
(Note: The interactive graphs below have been embedded directly into the blog post using a new Cloudflare Radar feature. This post is best viewed in landscape mode when on a mobile device.)
Iraq
Iraq had reportedly committed on May 15 to not implementing Internet shutdowns during the 2023 exam season, with a now unavailable page on the Iraqi Ministry of Communications web site (although captured in the Internet Archive’s Wayback Machine) noting (via Google Translate) “Her Excellency the Minister of Communications, Dr. Hayam Al-Yasiri: We rejected a request to cut off the internet service during the ministerial exams.” However, that commitment was apparently short-lived, as the first shutdown was implemented just a couple of weeks later, on June 1. The shutdowns observed across Iraq thus far have impacted networks and localities nationwide, with the exception of the autonomous Kurdistan region. However, networks in that region have experienced their own set of connectivity restrictions due to local exams.
In Iraq, the impact of the shutdowns between 04:00 – 08:00 local time (01:00 – 05:00 UTC) is clearly visible at a country level, as seen in the figure below.
The impact is, of course, also visible in the network-level graphs shown below, with traffic dropping to or near zero during each of the four-hour shutdown windows.
The shutdowns are also visible in the BGP announcement activity from the impacted networks, with spikes in announcement volume clearly visible around the shutdown windows each day that they have occurred. The announcement activity represents withdrawals ahead of the shutdown, removing routes to prefixes within the network, effectively cutting them off from the Internet, and updates after the shutdown period has ended, restoring the previously withdrawn routes, effectively reconnecting the prefixes to the Internet. (Additional announcement activity may also be visible for periods outside of the scheduled shutdowns, and is likely unrelated.)
While the shutdowns discussed above didn’t impact the Kurdistan region of Iraq, that region has also chosen to implement their own shutdowns. In the Kurdistan region, exams started June 3, we saw shorter traffic disruptions across three local network providers on June 3, 6, 10, and 13. The disruptions lasted from 06:30 – 07:30 local time (03:30 to 04:30 UTC) on the 3rd, and 06:40 – 08:30 local time (03:30 to 05:30 UTC) on the 6th, 10th, and 13th). Impacted regions include Erbil, Sulaymaniyah, and Duhok.
BGP announcement activity for the impacted networks in the Kurdistan region did not show the same patterns as those observed on the other Iraqi network providers discussed above.
Both sets of shutdowns in Iraq are also visible in traffic to Cloudflare’s 1.1.1.1 DNS resolver, although they highlight a difference in usage between the autonomous Kurdistan region and the rest of the country. The “totalTcpUdp” graph (blue line) below shows requests made to the resolver over UDP or TCP on port 53, the standard port used for DNS requests. The “totalDoHDoT” graph (orange line) below shows requests made to the resolver using DNS-over-HTTPS or DNS-over-TLS using port 443 or 853 respectively.
In the “totalTcpUdp” graph, we can see noticeable drops in traffic that align with the dates and times where we observed the traffic disruptions across Kurdistani networks. This drop in DNS traffic, combined with the lack of BGP announcement activity, suggests that the Internet disruptions in the Kurdistan region may be implemented as widespread blocking of Internet traffic, rather than routing-based shutdowns.
In the “totalDoHDoT” graph below, we can see noticeable drops in traffic that align with the dates and times where we observed the traffic disruptions in the rest of Iraq.
It isn’t immediately clear why there is a difference in the use of 1.1.1.1 between the two parts of the country.
Algeria
In Algeria, it appears that the country is following a similar pattern as that seen in 2021 and 2022, with two multi-hour Internet disruptions each day. Also similar to last year, it appears that they are pursuing a content blocking-based approach, instead of the wide-scale Internet shutdowns implemented in 2021, as impacted networks are not experiencing complete outages, nor do they show patterns of BGP announcement activity.
A published report indicates that two Internet disruptions will be implemented each day between June 11 and June 15. The first takes place between 08:00 – 12:00 local time (07:00 – 11:00 UTC), with the second occurring between 14:00 – 17:00 local time (13:00 – 16:00 UTC). These disruptions are visible in the shaded areas of the network-level graphs below as two distinct drops in traffic each day.
Conclusion
In cooperation with the Internet Society and Lebanese digital rights organization SMEX, digital rights organization Access Now is coordinating a #NoExamShutdown campaign across social media platforms. The campaign calls on MENA governments to end the practice of Internet shutdowns during exams, and aims to highlight how these shutdowns undermine human rights and disrupt essential social, political, economic, and cultural activities. Cloudflare Radar will continue to bring visibility to these, and other similar Internet disruptions, as they occur. You can follow them through the Cloudflare Radar Outage Center, or by following Cloudflare Radar on Twitter or Mastodon.
Our previous post reviewed usage trends for HTTP/1.1, HTTP/2, and HTTP/3 observed across Cloudflare’s network between May 2021 and May 2022, broken out by version and browser family, as well as for search engine indexing and social media bots. At the time, we found that browser-driven traffic was overwhelmingly using HTTP/2, although HTTP/3 usage was showing signs of growth. Search and social bots were mixed in terms of preference for HTTP/1.1 vs. HTTP/2, with little-to-no HTTP/3 usage seen.
Between May 2022 and May 2023, we found that HTTP/3 usage in browser-retrieved content continued to grow, but that search engine indexing and social media bots continued to effectively ignore the latest version of the web’s core protocol. (Having said that, the benefits of HTTP/3 are very user-centric, and arguably offer minimal benefits to bots designed to asynchronously crawl and index content. This may be a key reason that we see such low adoption across these automated user agents.) In addition, HTTP/3 usage across API traffic is still low, but doubled across the year. Support for HTTP/3 is on by default for zones using Cloudflare’s free tier of service, while paid customers have the option to activate support.
HTTP/1.1 and HTTP/2 use TCP as a transport layer and add security via TLS. HTTP/3 uses QUIC to provide both the transport layer and security. Due to the difference in transport layer, user agents usually require learning that an origin is accessible using HTTP/3 before they'll try it. One method of discovery is HTTP Alternative Services, where servers return an Alt-Svc response header containing a list of supported Application-Layer Protocol Negotiation Identifiers (ALPN IDs). Another method is the HTTPS record type, where clients query the DNS to learn the supported ALPN IDs. The ALPN ID for HTTP/3 is "h3" but while the specification was in development and iteration, we added a suffix to identify the particular draft version e.g., "h3-29" identified draft 29. In order to maintain compatibility for a wide range of clients, Cloudflare advertised both "h3" and "h3-29". However, draft 29 was published close to three years ago and clients have caught up with support for the final RFC. As of late May 2023, Cloudflare no longer advertises h3-29 for zones that have HTTP/3 enabled, helping to save several bytes on each HTTP response or DNS record that would have included it. Because a browser and web server typically automatically negotiate the highest HTTP version available, HTTP/3 takes precedence over HTTP/2.
In the sections below, “likely automated” and “automated” traffic based on Cloudflare bot score has been filtered out for desktop and mobile browser analysis to restrict analysis to “likely human” traffic, but it is included for the search engine and social media bot analysis. In addition, references to HTTP requests or HTTP traffic below include requests made over both HTTP and HTTPS.
Overall request distribution by HTTP version
Aggregating global web traffic to Cloudflare on a daily basis, we can observe usage trends for HTTP/1.1, HTTP/2, and HTTP/3 across the surveyed one year period. The share of traffic over HTTP/1.1 declined from 8% to 7% between May and the end of September, but grew rapidly to over 11% through October. It stayed elevated into the new year and through January, dropping back down to 9% by May 2023. Interestingly, the weekday/weekend traffic pattern became more pronounced after the October increase, and remained for the subsequent six months. HTTP/2 request share saw nominal change over the year, beginning around 68% in May 2022, but then starting to decline slightly in June. After that, its share didn’t see a significant amount of change, ending the period just shy of 64%. No clear weekday/weekend pattern was visible for HTTP/2. Starting with just over 23% share in May 2022, the percentage of requests over HTTP/3 grew to just over 30% by August and into September, but dropped to around 26% by November. After some nominal loss and growth, it ended the surveyed time period at 28% share. (Note that this graph begins in late May due to data retention limitations encountered when generating the graph in early June.)
API request distribution by HTTP version
Although API traffic makes up a significant amount of Cloudflare’s request volume, only a small fraction of those requests are made over HTTP/3. Approximately half of such requests are made over HTTP/1.1, with another third over HTTP/2. However, HTTP/3 usage for APIs grew from around 6% in May 2022 to over 12% by May 2023. HTTP/3’s smaller share of traffic is likely due in part to support for HTTP/3 in key tools like curl still being considered as “experimental”. Should this change in the future, with HTTP/3 gaining first-class support in such tools, we expect that this will accelerate growth in HTTP/3 usage, both for APIs and overall as well.
Mitigated request distribution by HTTP version
The analyses presented above consider all HTTP requests made to Cloudflare, but we also thought that it would be interesting to look at HTTP version usage by potentially malicious traffic, so we broke out just those requests that were mitigated by one of Cloudflare’s application security solutions. The graph above shows that the vast majority of mitigated requests are made over HTTP/1.1 and HTTP/2, with generally less than 5% made over HTTP/3. Mitigated requests appear to be most frequently made over HTTP/1.1, although HTTP/2 accounted for a larger share between early August and late November. These observations suggest that attackers don’t appear to be investing the effort to upgrade their tools to take advantage of the newest version of HTTP, finding the older versions of the protocol sufficient for their needs. (Note that this graph begins in late May 2022 due to data retention limitations encountered when generating the graph in early June 2023.)
HTTP/3 use by desktop browser
As we noted last year, support for HTTP/3 in the stable release channels of major browsers came in November 2020 for Google Chrome and Microsoft Edge, and April 2021 for Mozilla Firefox. We also noted that in Apple Safari, HTTP/3 support needed to be enabled in the “Experimental Features” developer menu in production releases. However, in the most recent releases of Safari, it appears that this step is no longer necessary, and that HTTP/3 is now natively supported.
Looking at request shares by browser, Chrome started the period responsible for approximately 80% of HTTP/3 request volume, but the continued growth of Safari dropped it to around 74% by May 2023. A year ago, Safari represented less than 1% of HTTP/3 traffic on Cloudflare, but grew to nearly 7% by May 2023, likely as a result of support graduating from experimental to production.
Removing Chrome from the graph again makes trends across the other browsers more visible. As noted above, Safari experienced significant growth over the last year, while Edge saw a bump from just under 10% to just over 11% in June 2022. It stayed around that level through the new year, and then gradually dropped below 10% over the next several months. Firefox dropped slightly, from around 10% to just under 9%, while reported HTTP/3 traffic from Internet Explorer was near zero.
As we did in last year’s post, we also wanted to look at how the share of HTTP versions has changed over the last year across each of the leading browsers. The relative stability of HTTP/2 and HTTP/3 seen over the last year is in some contrast to the observations made in last year’s post, which saw some noticeable shifts during the May 2021 – May 2022 timeframe.
In looking at request share by protocol version across the major desktop browser families, we see that across all of them, HTTP/1.1 share grows in late October. Further analysis indicates that this growth was due to significantly higher HTTP/1.1 request volume across several large customer zones, but it isn’t clear why this influx of traffic using an older version of HTTP occurred. It is clear that HTTP/2 remains the dominant protocol used for content requests by the major browsers, consistently accounting for 50-55% of request volume for Chrome and Edge, and ~60% for Firefox. However, for Safari, HTTP/2’s share dropped from nearly 95% in May 2022 to around 75% a year later, thanks to the growth in HTTP/3 usage.
HTTP/3 share on Safari grew from under 3% to nearly 18% over the course of the year, while its share on the other browsers was more consistent, with Chrome and Edge hovering around 40% and Firefox around 35%, and both showing pronounced weekday/weekend traffic patterns. (That pattern is arguably the most pronounced for Edge.) Such a pattern becomes more, yet still barely, evident with Safari in late 2022, although it tends to vary by less than a percent.
HTTP/3 usage by mobile browser
Mobile devices are responsible for over half of request volume to Cloudflare, with Chrome Mobile generating more than 25% of all requests, and Mobile Safari more than 10%. Given this, we decided to explore HTTP/3 usage across these two key mobile platforms.
Looking at Chrome Mobile and Chrome Mobile Webview (an embeddable version of Chrome that applications can use to display Web content), we find HTTP/1.1 usage to be minimal, topping out at under 5% of requests. HTTP/2 usage dropped from 60% to just under 55% between May and mid-September, but then bumped back up to near 60%, remaining essentially flat to slightly lower through the rest of the period. In a complementary fashion, HTTP/3 traffic increased from 37% to 45%, before falling just below 40% in mid-September, hovering there through May. The usage patterns ultimately look very similar to those seen with desktop Chrome, albeit without the latter’s clear weekday/weekend traffic pattern.
Perhaps unsurprisingly, the usage patterns for Mobile Safari and Mobile Safari Webview closely mirror those seen with desktop Safari. HTTP/1.1 share increases in October, and HTTP/3 sees strong growth, from under 3% to nearly 18%.
Search indexing bots
Exploring usage of the various versions of HTTP by search engine crawlers/bots, we find that last year’s trend continues, and that there remains little-to-no usage of HTTP/3. (As mentioned above, this is somewhat expected, as HTTP/3 is optimized for browser use cases.) Graphs for Bing & Baidu here are trimmed to a period ending April 1, 2023 due to anomalous data during April that is being investigated.
GoogleBot continues to rely primarily on HTTP/1.1, which generally comprises 55-60% of request volume. The balance is nearly all HTTP/2, although some nominal growth in HTTP/3 usage sees it peaking at just under 2% in March.
Through January 2023, around 85% of requests from Microsoft’s BingBot were made via HTTP/2, but dropped to closer to 80% in late January. The balance of the requests were made via HTTP/1.1, as HTTP/3 usage was negligible.
Looking at indexing bots from search engines based outside of the United States, Russia’s YandexBot appears to use HTTP/1.1 almost exclusively, with HTTP/2 usage generally around 1%, although there was a period of increased usage between late August and mid-November. It isn’t clear what ultimately caused this increase. There was no meaningful request volume seen over HTTP/3. The indexing bot used by Chinese search engine Baidu also appears to strongly prefer HTTP/1.1, generally used for over 85% of requests. However, the percentage of requests over HTTP/2 saw a number of spikes, briefly reaching over 60% on days in July, November, and December 2022, as well as January 2023, with several additional spikes in the 30% range. Again, it isn’t clear what caused this spiky behavior. HTTP/3 usage by BaiduBot is effectively non-existent as well.
Social media bots
Similar to Bing & Baidu above, the graphs below are also trimmed to a period ending April 1.
Facebook’s use of HTTP/3 for site crawling and indexing over the last year remained near zero, similar to what we observed over the previous year. HTTP/1.1 started the period accounting for under 60% of requests, and except for a brief peak above it in late May, usage of HTTP/1.1 steadily declined over the course of the year, dropping to around 30% by April 2023. As such, use of HTTP/2 increased from just over 40% in May 2022 to over 70% in April 2023. Meta engineers confirmed that this shift away from HTTP/1.1 usage is an expected gradual change in their infrastructure's use of HTTP, and that they are slowly working towards removing HTTP/1.1 from their infrastructure entirely.
In last year’s blog post, we noted that “TwitterBot clearly has a strong and consistent preference for HTTP/2, accounting for 75-80% of its requests, with the balance over HTTP/1.1.” This preference generally remained the case through early October, at which point HTTP/2 usage began a gradual decline to just above 60% by April 2023. It isn’t clear what drove the week-long HTTP/2 drop and HTTP/1.1 spike in late May 2022. And as we noted last year, TwitterBot’s use of HTTP/3 remains non-existent.
In contrast to Facebook’s and Twitter’s site crawling bots, HTTP/3 actually accounts for a noticeable, and growing, volume of requests made by LinkedIn’s bot, increasing from just under 1% in May 2022 to just over 10% in April 2023. We noted last year that LinkedIn’s use of HTTP/2 began to take off in March 2022, growing to approximately 5% of requests. Usage of this version gradually increased over this year’s surveyed period to 15%, although the growth was particularly erratic and spiky, as opposed to a smooth, consistent increase. HTTP/1.1 remained the dominant protocol used by LinkedIn’s bots, although its share dropped from around 95% in May 2022 to 75% in April 2023.
Conclusion
On the whole, we are excited to see that usage of HTTP/3 has generally increased for browser-based consumption of traffic, and recognize that there is opportunity for significant further growth if and when it starts to be used more actively for API interactions through production support in key tools like curl. And though disappointed to see that search engine and social media bot usage of HTTP/3 remains minimal to non-existent, we also recognize that the real-time benefits of using the newest version of the web’s foundational protocol may not be completely applicable for asynchronous automated content retrieval.
On Tuesday, May 9, Imran Khan, former Prime Minister of Pakistan was arrested on corruption charges. Following the arrest, violent protests erupted in several cities, leading the government of Pakistan to order the shutdown of mobile Internet services, as well as the blocking of several social media platforms. Below, we examine the impact of these shutdowns at a national and local level, as seen through Cloudflare traffic data. In addition, we illustrate how Pakistanis appear to be turning to Cloudflare’s 1.1.1.1 resolver in an attempt to maintain access to the open Internet.
General traffic trends
Since Tuesday, May 9, peak traffic levels aggregated at a country level (as measured by HTTP request volume) have been declining, down nearly 30% during the first several days of the mobile Internet shutdowns. The lowest traffic levels (nadirs of the graph) have also declined, dropping by as much as one-third as well. In the sections below, we drill down into this traffic loss, looking at outages at a network level, and the impact of those outages at an administrative unit and city level.
The mobile network shutdowns have also impacted the profile of traffic that Cloudflare sees from Pakistan. In analyzing traffic from desktop devices vs. mobile devices, we observed a 60% drop in request volume from mobile devices, while desktop traffic request volume remained fairly consistent. Peak mobile device traffic share dropped from 70% to 43%.
Cloudflare uses a bot score assigned to each request to indicate how likely it is that the request came from a bot or a human user. Since these shutdowns began, peak human request volume has dropped by 40%, while bot traffic has remained relatively consistent.
Mobile network shutdowns
On Wednesday, May 10, the Pakistan Telecommunication Authority (PTA) announced that Internet services would remain suspended across the country for an “indefinite” period, responding to a directive from the Ministry of the Interior to block mobile broadband services. As a result of the shutdowns associated with this directive, Cloudflare observed outages on the four major mobile providers within the country:
Although Pakistan has high mobile Internet usage, it appears that fixed broadband Internet connections are readily used as a backup when mobile connectivity becomes unavailable. Autonomous systems associated with fixed broadband networks saw significant increases in traffic when the mobile networks were shut down.
Interestingly, median latency within Pakistan also dropped slightly after mobile networks were shut down. Prior to the shutdown, median latency (as observed to Cloudflare and a set of other providers) was in the 90-100ms range, while afterwards, it has averaged closer to 75ms. This may be a result of users shifting to lower latency fixed broadband connections, as discussed above.
Administrative unit-level disruptions
Because the mobile network providers that were affected by the shutdown directive provide services nationwide, we also observed an impact to traffic across multiple administrative units within the country. None of these locations has experienced a complete outage, but peak traffic levels have clearly been declining in comparison to previous days.
Gilgit-Baltistan experienced the largest loss, where peak traffic has fallen nearly 60%. In Sindh, peak traffic is down around 35%, followed by Khyber Pakhtunkhwa, where it is down 30%. Islamabad and Azad Jammu and Kashmir have seen peak traffic declines of ~20%.
City-level disruptions
The impact of the mobile network shutdowns is also visible at a more local level, with lower peak traffic levels clearly visible in four cities. The significant traffic loss has been in Peshawar (Khyber Pakhtunkhwa), which has dropped nearly 55% from prior days. Faisalabad (Punjab), Karachi (Sindh), and Multan (Punjab) have all seen peak traffic drop approximately 40%.
Content blocking
In addition to the government-directed mobile network shutdowns, Pakistan’s authorities have also ordered Internet service providers to block access to social media platforms including Facebook, Instagram, YouTube, and Twitter. Testing by the Open Observatory for Network Interference (OONI), an Internet censorship measurement organization, suggests that this blocking is using a combination of TLS-level interference and DNS-based blocking. When the latter occurs in a country, Cloudflare’s 1.1.1.1 DNS resolver often sees an increase in request volume from the country as users seek ways to continue to access the open Internet.
Over the last several days, as expected, 1.1.1.1 request volume from Pakistan has increased, up approximately 40%. Peak request volume for the blocked social media platforms has also increased. Traffic for facebook.com saw a significant increase starting around 14:00 UTC (19:00 local time) on May 9, with peak request volume more than doubling. Request volume for instagram.com, also owned by Facebook parent Meta, also began to increase around the same time, and has grown nearly 50%. Requests for twitter.com began to spike around 08:00 UTC (13:00 local time) on May 9, growing as much as 150% that afternoon. Request volume for youtube.com also spiked on May 9, increasing by approximately 40%. And like twitter.com, request volume on May 10 was higher than earlier in the week, but lower than the spike seen the previous day.
Conclusion
Because of the ubiquity of Internet connectivity and social media tools in everyday life, Internet shutdowns and website blocking ultimately come with a significant human and financial cost. The mobile network shutdowns in Pakistan have impacted tens of thousands of “gig workers” and freelancers that depend on mobile connectivity. Many point-of-sale terminals in the country also depend on mobile connectivity, with transactions through Pakistan’s main digital payment systems fell by around 50% after the shutdowns were put into place. Telecommunications operators within Pakistan have estimated the extent of the financial damage thus far to be Rs. 820 million (approximately $2.8 million USD).
Use Cloudflare Radar to monitor the impact of such government-directed Internet disruptions, and follow @CloudflareRadar on Twitter for updates on Internet disruptions as they occur.
Cloudflare operates in more than 285 cities in over 100 countries, where we interconnect with over 11,500 network providers in order to provide a broad range of services to millions of customers. The breadth of both our network and our customer base provides us with a unique perspective on Internet resilience, enabling us to observe the impact of Internet disruptions.
We entered 2023 with Internet disruptions due to causes that ran the gamut, including several government-directed Internet shutdowns, cyclones, a massive earthquake, power outages, cable cuts, cyberattacks, technical problems, and military action. As we have noted in the past, this post is intended as a summary overview of observed disruptions, and is not an exhaustive or complete list of issues that have occurred during the quarter.
Government directed
Iran
Over the last six-plus months, government-directed Internet shutdowns in Iran have largely been in response to protests over the death of Mahsa Amini while in police custody. While these shutdowns are still occurring in a limited fashion, a notable shutdown observed in January was intended to prevent cheating on academic exams. Internet shutdowns with a similar purpose have been observed across a number of other countries, and have also occurred in Iran in the past. Access was restricted across AS44244 (Irancell) and AS197207 (MCCI), with lower traffic levels observed in Alborz Province, Fars, Khuzestan, and Razavi Khorasan between 08:00 to 11:30 local time (04:30 to 08:00 UTC) on January 19.
A shutdown of mobile Internet connectivity in Punjab, India began on March 19, ordered by the local government amid concerns of protest-related violence. Although the initial shutdown was ordered to take place between March 18, 12:00 local time and March 19, 12:00 local time, it was extended several times, ultimately lasting for three days. Traffic for AS38266 (Vodafone India), AS45271 (Idea Cellular Limited), AS45609 (Bharti Mobility), and AS55836 (Reliance Jio Infocomm) began to fall around 12:30 local time (07:00 UTC) on March 18, recovering around 12:30 local time (07:00 UTC) on March 21. However, it was subsequently reported that connectivity remained shut down in some districts until March 23 or 24.
Cable cuts
Bolivia
Bolivian ISP Cometco (AS27839)reported on January 12 that problems with international fiber links were causing degradation of Internet service. Traffic from the network dropped by approximately 80% starting around 16:00 local time (20:00 UTC) before returning to normal approximately eight hours later. It isn’t clear whether the referenced international fiber links were terrestrial connections to neighboring countries, or issues with submarine cables several network hops upstream. As a landlocked country, Bolivia is not directly connected to any submarine cables.
A brief connectivity disruption was observed on Bangladeshi provider Grameenphone on February 23, between 11:45-14:00 local time (05:45-08:00 UTC). According to a Facebook post from Grameenphone, the outage was caused by fiber cuts due to road maintenance.
Venezuela
Venezuela, and more specifically, AS8048 (CANTV), are no stranger to Internet disruptions, seeing several (Q1, Q2) during 2022, as well as others in previous years. During the last couple of days in February, a few small outages were observed on CANTV’s network in several Venezuelan states. However, a more significant near-complete outage occurred on February 28, starting around midnight local time (04:00 UTC), and lasting for the better part of the day, with traffic recovering at 17:30 local time (21:30 UTC). A Tweet posted the morning of February 28 by CANTV referenced an outage in their fiber optic network, which was presumably the cause.
Power outages
Pakistan
A country-wide power outage in Pakistan on January 23 impacted more than 220 million people, and resulted in a significant drop in Internet traffic being observed in the country. The power outage began at 07:34 local time (02:34 UTC), with Internet traffic starting to drop almost immediately. The figure below shows that traffic volumes dropped as much as 50% from normal levels before recovering around 04:15 local time on January 24 (23:15 UTC on January 23). This power outage was reportedly due to a “sudden drop in the frequency of the power transmission system”, which led to a “widespread breakdown”. Nationwide power outages have also occurred in Pakistan in January 2021, May 2018, and January 2015.
Bermuda
BELCO, the power company servicing the island of Bermuda, tweeted about a mass power outage affecting the island on February 3, and linked to their outage map so that customers could track restoration efforts. BELCO’s tweet was posted at 16:10 local time (20:10 UTC), approximately one hour after a significant drop was observed in Bermuda’s Internet traffic. The power outage, and subsequent Internet disruption, lasted over five hours, as BELCO later tweeted that “As of 9.45 pm [00:45 UTC, February 4], all circuits have been restored.”
Argentina
Soaring temperatures in Argentina triggered a large-scale power outage across the country that resulted in multi-hour Internet disruption on March 1. Internet traffic dropped by approximately one-third during the disruption, which lasted from 16:30 to 19:30 local time (19:30 to 22:30 UTC). Cities that experienced visible impacts to Internet traffic during the power outage included Buenos Aires, Cordoba, Mendoza, and Tucuman.
Kenya
Just a few days later on March 4, Kenya Power issued a Customer Alert at 18:25 local time (15:25 UTC) regarding a nationwide power outage, noting that it had “lost bulk power supply to various parts of the country due to a system disturbance.” The alert came approximately an hour after the country’s Internet traffic dropped significantly. A subsequent tweet dated midnight local time (21:00 UTC) claimed that “electricity supply has been restored to all areas countrywide” and the figure below shows that traffic levels returned to normal levels shortly thereafter.
Earthquake
Turkey
On February 5, a magnitude 7.8 earthquake occurred 23 km east of Nurdağı, Turkey, leaving many thousands dead and injured. The quake, which occurred at 04:17 local time (01:17 UTC), was believed to be the strongest to hit Turkey since 1939. The widespread damage and destruction resulted in significant disruptions to Internet connectivity in multiple areas of the country, as shown in the figures below. Although Internet traffic volumes were relatively low because it was so early in the morning, the graphs show it dropping even further at or around the time of the earthquake. Nearly half a day later, traffic volumes in selected locations were between 63-94% lower than at the same time the previous week. A month later, after several aftershocks, traffic volumes had mostly recovered, although some regions were still struggling to recover.
Weather
New Zealand
Called the “country’s biggest weather event in a century”, Cyclone Gabrielle wreaked havoc on northern New Zealand, including infrastructure damage and power outages impacting tens of thousands of homes. As a result, regions including Gisborne and Hawkes Bay experienced Internet disruptions that lasted several days, starting at 00:00 local time on February 14 (11:00 UTC on February 13). The figures below show that in both regions, peak traffic volume returned to pre-cyclone levels around February 19.
Vanuatu
Later in February, Cyclone Judy hit the South Pacific Ocean nation of Vanuatu, the South Pacific Ocean nation made up of roughly 80 islands that stretch 1,300 kilometers. The Category 4 cyclone damaged homes and caused power outages, resulting in a significant drop in the country’s Internet traffic. On February 28, Vanuatu’s traffic dropped by nearly 80% as the cyclone struck, and as seen in the figure below, it took nearly two weeks for traffic to recover to the levels seen earlier in February.
Malawi
Cyclone Freddy, said to be the longest-lasting, most powerful cyclone on record, hit Malawi during the weekend of March 11-12, and into Monday, March 13. The resulting damage disrupted Internet connectivity in the east African country, with traffic dropping around 11:00 local time (09:00 UTC) on March 13. The disruption lasted for over two days, with traffic levels recovering around 21:00 local time (19:00 UTC) on March 15.
Technical problems
South Africa
Just before 07:00 local time (05:00 UTC) on February 1, South African service provider RSAWEB initially tweeted about a problem that they said was impacting their cloud and VOIP platforms. However, in several subsequent tweets, they noted that the problem was also impacting internal systems, as well as fiber and mobile connectivity. The figure below shows traffic for RSAWEB dropping at 06:30 local time (04:30 UTC), a point at which it would normally be starting to increase for the day. Just before 16:00 local time (14:00 UTC), RSAWEB tweeted“…engineers are actively working on restoring services post the major incident. Customers who experienced no connectivity may see some services restoring.” The figure below shows a sharp increase in traffic around that time with gradual growth through the evening. However, full restoration of services across all of RSAWEB’s impacted platforms took a full week, according to a February 8 tweet.
Italy
An unspecified “international interconnectivity problem” impacting Telecom Italia caused a multi-hour Internet disruption in Italy on February 5. At a country level, a nominal drop in traffic is visible in the figure below starting around 11:45 local time (10:45 UTC) with some volatility visible in the lower traffic through 17:15 local time (16:15 UTC). However, the impact of the problem is more obvious in the traffic graphs for AS3269 and AS16232, both owned by Telecom Italia. Both graphs show a more significant loss of traffic, as well as greater volatility through the five-plus hour disruption.
Myanmar
A fire at an exchange office of MPT (Myanma Posts and Telecommunications) on February 7 disrupted Internet connectivity for customers of the Myanmar service provider. A Facebook post from MPT informed customers that “We are currently experiencing disruptions to our MPT’s services including MPT’s call centre, fiber internet, mobile internet and mobile and telephone communications.” The figure below shows the impact of this disruption on MPT-owned AS9988 and AS45558, with traffic dropping significantly at 10:00 local time (03:30 UTC). Significant recovery was seen by 22:00 local time (15:30 local time).
Republic of the Congo (Brazzaville)
Congo Telecomtweeted a “COMMUNIQUÉ” on March 15, alerting users to a service disruption due to a “network incident”. The impact of this disruption is clearly visible at a country level, with traffic dropping sharply at 00:45 local time (23:45 on March 14 UTC), driven by complete outages at MTN Congo and Congo Telecom, as seen in the graphs below. While traffic at MTN Congo began to recover around 08:00 local time (07:00 UTC), Congo Telecom’s recovery took longer, with traffic beginning to increase around 17:00 local time (16:00 UTC). Congo Telecom tweeted on March 16 that the nationwide Internet outage had been resolved. MTN Congo did not acknowledge the disruption on social media, and neither company provided more specific information about the reported “network incident”.
Lebanon
Closing out March, disruptions observed at AS39010 (Terranet) and AS42334 (Mobi) in Lebanon may have been related to a strike at upstream provider Ogero Telecom, common to both networks. A published report quoted the Chairman of Ogero commenting on the strike, “We are heading to a catastrophe if a deal is not found with the government: the network will completely stop working as our generators will gradually run out of fuel. Lebanon completely relies on Ogero for its bandwidth, leaving no one exempt from a blackout.” Traffic at both Terranet and Mobi dropped around 05:00 local time (03:00 UTC) on March 29, with the disruption lasting approximately 4.5 hours, as traffic recovered at 09:30 local time (07:30 UTC).
Cyberattacks
South Korea
On January 29, South Korean Internet provider LG Uplus suffered two brief Internet disruptions which were reportedly caused by possible DDoS attacks. The first disruption occurred at 03:00 local time (18:00 UTC on January 28), and the second occurred at 18:15 local time (09:15 UTC). The disruptions impacted traffic on AS17858 and AS3786, both owned by LG. The company was reportedly hit by a second pair of DDoS attacks on February 4.
Guam
In a March 17 tweet posted at 11:30 local time (01:30 UTC), Docomo Pacific reported an outage affecting multiple services, with a subsequent tweet noting that “Early this morning, a cyber security incident occurred and some of our servers were attacked”. This outage is visible at a country level in Guam, seen as a significant drop in traffic starting around 10:00 local time (00:00 UTC) in the figure below. However, in the graph below for AS3605 (ERX-KUENTOS/Guam Cablevision/Docomo Pacific), the cited outage results in a near-complete loss of traffic starting around 05:00 local time (19:00 on March 16 UTC). Traffic returned to normal levels by 18:00 local time on March 18 (08:00 UTC).
Ukraine/Military Action
In February, the conflict in Ukraine entered its second year, and over this past year, we have tracked its impact on the Internet, highlighting traffic shifts, attacks, routing changes, and connectivity disruptions. In the fourth quarter of 2022, a number of disruptions were related to attacks on electrical infrastructure, and this pattern continued into the first quarter of 2023.
One such disruption occurred in Odessa on January 27, amid news of Russian airstrikes on local energy infrastructure. As seen in the figure below, Internet traffic in Odessa usually begins to climb just before 08:00 local time (06:00 UTC), but failed to do so that morning after several energy infrastructure facilities near Odessa were hit and damaged. Traffic remained lower than levels seen the previous week for approximately 18 hours.
Power outages resulting from Russian attacks on energy generation and distribution facilities on March 9 resulted in disruptions to Internet connectivity in multiple locations around Ukraine. As seen in the figures below, traffic dropped below normal levels after 02:00 local time (00:00 UTC) on March 9. Traffic in Kharkiv fell over 50% as compared to previous week, while in Odessa, traffic fell as much as 60%. In Odessa, Mykolaiv, and Kirovohrad Oblast, traffic recovered by around 08:00 local time (06:00 UTC), while in Kharkiv, the disruption lasted nearly two days, returning to normal levels around 23:45 local time (21:45 UTC) on Friday, March 10.
Conclusion
The first quarter of 2023 seemed to be particularly active from an Internet disruption perspective, but hopefully it is not a harbinger of things to come through the rest of the year. This is especially true of government-directed shutdowns, which occurred fairly regularly through 2022. To that end, civil society organization Access Now recently published their Internet shutdowns in 2022 report, finding that In 2022, governments and other actors disrupted the internet at least 187 times across 35 countries. Cloudflare Radar is proud to support Access Now’s #KeepItOn initiative, using our data to help illustrate the impact of Internet shutdowns and other disruptions.
By continuing to use the site, you agree to the use of cookies. more information
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.
