All posts by Mike Conlow

Making progress on routing security: the new White House roadmap

Post Syndicated from Mike Conlow original https://blog.cloudflare.com/white-house-routing-security

The Internet can feel like magic. When you load a webpage in your browser, many simultaneous requests for data fly back and forth to remote servers. Then, often in less than one second, a website appears. Many people know that DNS is used to look up a hostname, and resolve it to an IP address, but fewer understand how data flows from your home network to the network that controls the IP address of the web server.

The Internet is an interconnected network of networks, operated by thousands of independent entities. To allow these networks to communicate with each other, in 1989, on the back of two napkins, three network engineers devised the Border Gateway Protocol (BGP). It allows these independent networks to signal directions for IP prefixes they own, or that are reachable through their network. At that time, Internet security wasn’t a big deal — SSL, initially developed to secure websites, wasn’t developed until 1995, six years later. So BGP wasn’t originally built with security in mind, but over time, security and availability concerns have emerged.

Today, the White House Office of the National Cyber Director issued the Roadmap to Enhancing Internet Routing Security, and we’re excited to highlight their recommendations. But before we get into that, let’s provide a quick refresher on what BGP is and why routing security is so important.

BGP: pathways through the Internet

BGP is the core signaling protocol used on the Internet. It’s fully distributed, and managed independently by all the individual operators of the Internet. With BGP, operators will send messages to their neighbors (other networks they are directly connected with, either physically or through an Internet Exchange) that indicate their network can be used to reach a specific IP prefix. These IP prefixes can be resources the network owns themselves, such as 104.16.128.0/20 for Cloudflare, or resources that are reachable through their network, by transiting the network.

By exchanging all of this information between peers, each individual network on the Internet can form a full map of what the Internet looks like, and ideally, how to reach each IP address on the Internet. This map is in an almost constant state of flux: networks disappear from the Internet for a wide variety of reasons, ranging from scheduled maintenance to catastrophic failures, like the Facebook incident in 2021. On top of this, the ideal path to take from point A (your home ISP) to point B (Cloudflare) can change drastically, depending on routing decisions made by your home ISP, and any or all intermediate networks between your home ISP and Cloudflare (here’s an example from 2019). These routing decisions are entirely arbitrary, and left to the owners of the networks. Performance and security can be considered, but neither of these have been historically made visible through BGP itself.

As all the networks can independently make their own routing decisions, there are a lot of individual points where things can go wrong. Going wrong can have multiple meanings here: this can range from routing loops, causing Internet traffic to go back and forth repeatedly between two networks, never reaching its destination, to more malicious problems, such as traffic interception or traffic manipulation.

As routing security wasn’t accounted for in that initial two-napkin draft, it is easy for a malicious actor on the Internet to pretend to either be an originating network (where they claim to own the IP prefix, positioning themselves as the destination network), or they can pretend to be a viable middle network, getting traffic to transit through their network.

In either of these examples, the actor can manipulate the Internet traffic of unsuspecting end users and potentially steal passwords, cryptocurrency, or any other data that can be of value. While transport security (TLS for HTTP/1.x and HTTP/2, QUIC for HTTP/3) has reduced this risk significantly, there’s still ways this can be bypassed. Over time, the Internet community has acknowledged the security concerns with BGP, and has built infrastructure to mitigate some of these problems.

BGP security: The RPKI is born

This journey is now coming to a final destination with the development and adoption of the Resource Public Key Infrastructure (RPKI). The RPKI is a PKI, just like the Web PKI which provides security certificates for the websites we browse (the “s” in https). The RPKI is a PKI specifically with the Internet in mind: it provides core constructs for IP addresses and Autonomous System Numbers (ASNs), the numbers used to identify these individual operating networks mentioned earlier.

Through the RPKI, it’s possible for an operator to establish a cryptographically secure relationship between the IP prefixes they originate, and their ASN, through the issuance of Route Origin Authorization records (ROAs). These ROAs can be used by all other networks on the Internet to validate that the IP prefix update they just received for a given origin network actually belongs to that origin network, a process called Route Origin Validation (ROV). If a malicious party tries to hijack an IP prefix that has a ROA to their (different) origin network, validating networks would know this update is invalid and reject it, maintaining the origin security and ensuring reachability.

Why does BGP security matter? Examples of route hijacks and leaks

But why should you care about BGP? And more importantly, why does the White House care about BGP? Put simply: BGP (in)security can cost people and companies millions of dollars and cause widespread disruptions for critical services.

In February 2022, Korean crypto platform KLAYswap was the target of a malicious BGP hijack, which was used to steal $1.9 million of cryptocurrency from their customers. The attackers were able to serve malicious code that mimicked the service KLAYswap was using for technical support. They were able to do this by announcing the IP prefix used to serve the JavaScript SDK KLAYswap was using. When other networks accepted this announcement, end user traffic loading the technical support page instead received malicious JavaScript, which was used to drain customer wallets. As the attackers hijacked the IP address, they were also able to register a TLS certificate for the domain name used to serve the SDK. As a result, nothing looked out of the ordinary for Klayswap’s customers until they noticed their wallets had been drained.

However, not all BGP problems are intentional hijacks. In March 2022, RTComm (AS8342), a Russian ISP, announced itself as the origin of 104.244.42.0/24, which is an IP prefix actually owned by Twitter (now X) (AS13414). In this case, all researchers have drawn a similar conclusion: RTComm wanted to block its users from accessing Twitter, but inadvertently advertised the route to its peers and upstream providers. Thankfully, the impact was limited, in large part due to Twitter issuing ROA records for their IP prefixes, which meant the hijack was blocked at all networks that had implemented ROV and were validating announcements.

Inadvertent incorrect advertisements passing from one network to another, or route leaks, can happen to anyone, even Cloudflare. Our 1.1.1.1 public DNS service — used by millions of consumers and businesses — is often the unintended victim. Consider this situation (versions of which have happened numerous times): a network engineer running a local ISP is testing a configuration on their router and announces to the Internet that you can reach the IP address 1.1.1.1 through their network. They will often pick this address because it’s easy to input on the router and observe in network analytics. They accidentally push that change out to all their peer networks — the networks they’re connected to — and now, if proper routing security isn’t in place, users on multiple networks around the Internet trying to reach 1.1.1.1 might be directed to this local ISP where there is no DNS service to be found. This can lead to widespread outages.

The types of routing security measures in the White House roadmap can prevent these issues. In the case of 1.1.1.1, Cloudflare has ROAs in place that tell the Internet that we originate the IP prefix that contains 1.1.1.1. If someone else on the Internet is advertising 1.1.1.1, that’s an invalid route, and other networks should stop accepting it. In the case of KLAYswap, had there been ROAs in place, other networks could have used common filtering techniques to filter out the routes pointing to the attackers malicious JavaScript. So now let’s talk more about the plan the White House has to improve routing security on the Internet, and how the US government developed its recommendations.

Work leading to the roadmap

The new routing security roadmap from the Office of the National Cyber Director (ONCD) is the product of years of work, throughout both government and industry. The National Institute of Standards and Technology (NIST) has been a longstanding proponent of improving routing security, developing test and measurement tools and publishing special publication 1800-14 on Protecting the Integrity of Internet Routing, among many other initiatives. They are active participants in the Internet community, and an important voice for routing security.

Cloudflare first started publicly advocating for adoption of security measures like RPKI after a massive BGP route leak took down a portion of the Internet, including websites using Cloudflare’s services, in 2019. 

Since that time, the federal government has increasingly recognized the need to elevate efforts to secure Internet routing, a process that Cloudflare has helped support along the way. The Cyberspace Solarium Commission report, published in 2020, encouraged the government to develop a strategy and recommendations to define “common, implementable guidance for securing the DNS and BGP.”    

In February 2022, the Federal Communication Commission launched a notice of inquiry to better understand Internet routing. Cloudflare responded with a detailed explanation of our history with RPKI and routing security. In July 2023, the FCC, jointly with the Director of the Cybersecurity and Infrastructure Security Agency, held a workshop for stakeholders, with Cloudflare as one of the presenters. In June 2024, the FCC issued a Notice of Proposed Rulemaking that would require large service providers to develop security risk management plans and report on routing security efforts, including RPKI adoption. 

The White House has been involved as well. In March 2023, they cited the need to secure the technical foundation of the Internet, from issued such as BGP vulnerabilities, as one of the strategic objectives of the National Cybersecurity Strategy. Citing those efforts, in May 2024, the Department of Commerce issued ROAs signing some of its IP space, and this roadmap strongly encourages other departments and agencies to do the same. All of those efforts and the focus on routing security have resulted in increased adoption of routing security measures. 

Report observations and recommendations

The report released by the White House Office of the National Cyber Director details the current state of BGP security, and the challenges associated with Resource Public Key Infrastructure (RPKI) Route Origin Authorization (ROA) issuance and RPKI Route Origin Validation (ROV) adoption. It also provides network operators and government agencies with next steps and recommendations for BGP security initiatives. 

One of the first recommendations is for all networks to create and publish ROAs. It’s important that every network issues ROAs for their IP prefixes, as it’s the only way for other networks to validate they are the authorized originator of those prefixes. If one network is advertising an IP address as their own, but a different network issued the ROA, that’s an important sign that something might be wrong!

As shown in the chart below from NIST’s RPKI Monitor, as of September 2024, at least 53% of all the IPv4 prefixes on the Internet have a valid ROA record available (IPv6 reached this milestone in late 2023), up from only 6% in 2017. (The metric is even better when measured as a percent of Internet traffic: data from Kentik, a network observability company, shows that 70.3% of Internet traffic is exchanged with IP prefixes that have a valid ROA.) This increase in the number of signed IP prefixes (ROAs) is foundational to secure Internet routing.

Unfortunately, the US is lagging behind: Only 39% of IP prefixes originated by US networks have a valid ROA. This is not surprising, considering the US has significantly more Internet address resources than other parts of the world. However, the report highlights the need for the US to overcome the common barriers network operators face when implementing BGP security measures. Administrative challenges, the perception of risk, and prioritization and resourcing constraints are often cited as the problems networks face when attempting to move forward with ROV and RPKI.

A related area of the roadmap highlights the need for networks that allow their customers to control IP address space to still create ROAs for those addresses. The reality of how every ISP, government, and large business allocates its IP address space is undoubtedly messy, but that doesn’t reduce the importance of making sure that the correct entity is identified in the official records with a ROA. 

A network signing routes for its IP addresses is an important step, but it isn’t enough. To prevent incorrect routes — malicious or not — from spreading around the Internet, networks need to implement Route Origin Validation (ROV) and implement other BGP best practices, outlined by MANRS in their Zen Guide to Routing Security Policy. If one network incorrectly announces itself as the origin for 1.1.1.1, that won’t have any effect beyond its own borders if no other networks pick up that invalid route. The Roadmap calls out filtering invalid routes as another action for network service providers. 

As of 2022, our data showed that around 15 percent of networks were validating routes. Ongoing measurements from APNIC show progress: this year about 20 percent of APNIC probes globally correctly filter invalid routes with ROV. In the US, it’s 70 percent. Continued growth of ROV is a critical step towards achieving better BGP security.

Filtering out invalid routes is prominently highlighted in the report’s recommendations. While recognizing that there’s been dramatic improvement in filtering by the large transit networks, the first report recommendation is for network service providers — large and small —  to fully deploy ROV. 

In addition, the Roadmap proposes using the federal government’s considerable weight as a purchaser, writing, “[Office of Management and Budget] should require the Federal Government’s contracted service providers to adopt and deploy current commercially-viable Internet routing security technologies.” It goes on to say that grant programs, particularly broadband grants, “should require grant recipients to incorporate routing security measures into their projects.”

The roadmap doesn’t only cover well-established best practices, but also highlights emerging security technologies, such as Autonomous System Provider Authorization (ASPA) and BGPsec. ROAs only cover part of the BGP routing ecosystem, so additional work is needed to ensure we secure everything. It’s encouraging to see the work being done by the wider community to address these concerns is acknowledged, and more importantly, actively followed.

What’s next for the Internet community

The new roadmap is an important step in outlining actions that can be taken today to improve routing security. But as the roadmap itself recognizes, there’s more work to be done both in making sure that the steps are implemented, and that we continue to push routing security forward.

From an implementation standpoint, our hope is that the government’s focus on routing security through all the levers outlined in the roadmap will speed up ROA adoption, and encourage wider implementation of ROV and other best practices. At Cloudflare, we’ll continue to report on routing practices on Cloudflare Radar to help assess progress against the goals in the roadmap.

At a technical level, the wider Internet community has made massive strides in adopting RPKI ROV, and have set their sights on the next problem: we are securing the IP-to-originating network relationship, but what about the relationships between the individual networks?

Through the adoption of BGPsec and ASPA, network operators are able to not only validate the destination of a prefix, but also validate the path to get there. These two new technical additions within the RPKI will combine with ROV to ultimately provide a fully secure signaling protocol for the modern Internet. The community has actively undertaken this work, and we’re excited to see it progress!

Outside the RPKI, the community has also ratified the formalization of customer roles through RFC9234: Route Leak Prevention and Detection Using Roles in UPDATE and OPEN Messages. As this new BGP feature gains support, we’re hopeful that this will be another helpful tool in the operator toolbox in preventing route leaks of any kind.

How you can help keep the Internet secure

If you’re a network operator, you’ll need to sign your routes, and validate incoming prefixes. This consists of signing Route Origin Authorization (ROA) records, and performing Route Origin Validation (ROV). Route signing involves creating records with your local Regional Internet Registry (RIR) and signing to their PKI. Route validation involves only accepting routes that are signed with a ROA. This will help ensure that only secure routes get through. You can learn more about that here.

If you’re not a network operator, head to isbgpsafeyet.com, and test your ISP. If your ISP is not keeping BGP safe, be sure to let them know how important it is. If the government has pointed out prioritization is a consistent problem, let’s help increase the priority of routing security.

A secure Internet is an open Internet

As the report points out, one of the keys to keeping the Internet open is ensuring that users can feel safe accessing any site they need to without worrying about attacks that they can’t control. Cloudflare wholeheartedly supports the US government’s efforts to bolster routing security around the world and is eager to work to ensure that we can help create a safe, open Internet for every user.

Making home Internet faster has little to do with “speed”

Post Syndicated from Mike Conlow original https://blog.cloudflare.com/making-home-internet-faster/

Making home Internet faster has little to do with “speed”

Making home Internet faster has little to do with “speed”

More than ten years ago, researchers at Google published a paper with the seemingly heretical title “More Bandwidth Doesn’t Matter (much)”. We published our own blog showing it is faster to fly 1TB of data from San Francisco to London than it is to upload it on a 100 Mbps connection. Unfortunately, things haven’t changed much. When you make purchasing decisions about home Internet plans, you probably consider the bandwidth of the connection when evaluating Internet performance. More bandwidth is faster speed, or so the marketing goes. In this post, we’ll use real-world data to show both bandwidth and – spoiler alert! – latency impact the speed of an Internet connection. By the end, we think you’ll understand why Cloudflare is so laser focused on reducing latency everywhere we can find it.

First, we should quickly define bandwidth and latency. Bandwidth is the amount of data that can be transmitted at any single time. It’s the maximum throughput, or capacity, of the communications link between two servers that want to exchange data. Usually, the bottleneck – the place in the network where the connection is constrained by the amount of bandwidth available – is in the “last mile”, either the wire that connects a home, or the modem or router in the home itself.

If the Internet is an information superhighway, bandwidth is the number of lanes on the road. The wider the road, the more traffic can fit on the highway at any time. Bandwidth is useful for downloading large files like operating system updates and big game updates. While bandwidth, throughput and capacity are synonyms, confusingly “speed” has come to mean bandwidth when talking about Internet plans. More on this later.

We use bandwidth when streaming video, though probably less than you think. Netflix recommends 15 Mbps of bandwidth to watch a stream in 4K/Ultra HD. A 1 Gbps connection could stream more than 60 Netflix shows in 4K at the same time!

Latency, on the other hand, is how fast data is moving through the Internet. To extend our analogy, tatency is the speed at which traffic is moving on the highway. If traffic is moving quickly, you’ll get to your destination faster. Latency is measured in the number of milliseconds that it takes a packet of data to go from a client (such as your laptop computer) to a server, and then back to the client. If you’re practicing tennis against a wall, round-trip latency is the time the ball was in the air. On the Internet “backbone” data is traveling at almost 200,000 kilometers per second as it bounces off the glass on the inside of fiber optic wires. That’s fast!

While we can’t make light travel through glass much faster, we can improve latency by moving the content closer to users, shortening the distance data needs to travel. That’s the effect of our presence in more than 285 cities globally: when you’re on the Internet superhighway trying to reach Cloudflare, we want to be just off the next exit.

We know that low-latency (low delay; high speed) connections are important for gaming, where tiny bits of data, such as the change in position of players in a game, need to reach another computer quickly. And increasingly, we’re becoming aware of high latency when it makes our videoconferencing choppy and unpleasant.

But we don’t use the Internet only to play games, nor only watch streaming video. We do those, and we visit a lot of normal web pages in between.

In the 2010 paper from Google, the author simulated loading web pages while varying the throughput and latency of the connection. What he finds is that above about 5 Mbps, the page doesn’t load much faster. Increasing bandwidth from 1 Mbps to 2 Mbps is almost a 40 percent improvement in page load time. From 5 Mbps to 6 Mbps is less than a 5 percent improvement.

When he varies the latency (the Round Trip Time, or RTT), something interesting happens: there is a linear return to better latency. For every 20 milliseconds of reduced latency, the page load time improves by about 10%.

Let’s see what this looks like in real life with empirical data. Below is a chart from an excellent recent paper by two researchers from MIT. Using data from the FCC’s Measuring Broadband America program, these researchers produce a similar chart to what we expect from the 2010 simulation. Though the point of diminishing returns to more bandwidth has moved higher – to about 20 Mbps – the overall trend is exactly the same.

Making home Internet faster has little to do with “speed”

For the latency relationship, we use aggregate and anonymized data from Cloudflare’s own delivery network. It’s a familiar pattern. For every 200 milliseconds of latency we can save, we cut the page load time by over 1 second. That relationship applies when the latency is 950 milliseconds. And it applies when the latency is 50 milliseconds.

Making home Internet faster has little to do with “speed”

Let’s try to understand why this relationship exists. When you connect to a website, the first thing that your browser does is encrypt the connection and make sure the site you want to access is authenticated to be the site you think you’re accessing. This process is called the TLS establishment and SSL handshake. Before you even see a pixel of content, your browser and the website will exchange information up to seven times: three to perform TLS establishment, and three more to perform the SSL handshake.

Making home Internet faster has little to do with “speed”

On top of that, when we load a webpage after we establish encryption and verify website authority, we might be asking the browser to load hundreds of different files across dozens of different domains. Some of these files can be loaded in parallel, but others need to be loaded sequentially. As the browser races to compile all these different files, it’s the speed at which it can get to the server and back that determines how fast it can put the page together. The files are often quite small, but there’s a lot of them.

The chart below shows the beginning of what the browser does when it loads cnn.com. After a 301 redirect, the browser loads the main HTML page in step two. Only then, more than 1 second into the load, does it know about all the javascript files it needs to render the page. Files 3-19 become unblocked once the browser has the HTML file in step two, but we see more blocking later on. Files 20-27 are all blocked on earlier files. They can’t start until the browser has the previous file back from the server. There are 650 assets in this page load, and the blocking happens all the way through the page load. Here’s why this matters: better latency makes every file load faster, which in turn unblocks other files faster, and so on.

Making home Internet faster has little to do with “speed”

The browser will do its best to use all the bandwidth available, but often is using just a fraction of what’s available. It’s no wonder then that adding more bandwidth doesn’t speed up the page load, but better latency does. While developments like Early Hints help this by allowing browsers to pre-fetch files and content that don’t need to be serialized, this is still a problem for many websites on the Internet today.

Making home Internet faster has little to do with “speed”

Recently, Internet researchers have turned their attention to using our understanding of the relationship between throughput and latency to improve Internet Quality of Experience (QoE). A paper from the Broadband Internet Technical Advisory Group (BITAG) summarizes:

But we now recognize that it is not just greater throughput that matters, but also consistently low latency. Unfortunately, the way that we’ve historically understood and characterized latency was flawed, and our latency measurements and metrics were not aligned with end-user QoE.

There’s a difference between latency on an idle Internet connection and latency measured in working conditions, which we call “working latency” or “responsiveness”. Since responsiveness is what the user experiences as the speed of their Internet connection, it’s important to understand and measure this particular latency.

An Internet connection can suffer from poor responsiveness (even if it has good idle latency) when data is delayed in buffers. If you download a large file, for example an operating system update, the server sending the file might send the file with higher throughput than the Internet connection can accept. That’s ok. Extra bits of the file will sit in a buffer until it’s their turn to go through the funnel. Adding extra lanes to the highway allows more cars to pass through, and is a good strategy if we aren’t particularly concerned with the speed of the traffic.

Say for example, Christabel is watching a stream of the news while on a video meeting. When Christabel starts watching the video, her browser fetches a bunch of content and stores it in various buffers on the way from the content host to the browser. Those same buffers also contain data packets pertaining to the video meeting Christabel is currently in. If the data generated as part of a video conference sits in the same buffer as the video files, the video files will fill up the buffer and cause delay for the video meeting packets as well. That type of buffering delay is called bufferbloat, and leads to jittery video calls and delays where people speak over each other and get frustrated.

Making home Internet faster has little to do with “speed”

To help users understand the strengths and weaknesses of their connection, we recently added Aggregated Internet Measurement (AIM) scores to our own “Speed” Test. These scores remove the technical metrics and give users a real-world, plain-English understanding of what their connection will be good at, and where it might struggle. We’d also like to collect more data from our speed test to help track Page Load Times (PLT) and see how they are correlated with the reduction of lower working latency. You’ll start seeing those numbers on our speed test soon!

Making home Internet faster has little to do with “speed”

We all use our Internet connections in slightly different ways, but we share the desire for our connections to be as fast as possible. As more and more services move into the cloud – word documents, music, websites, communications, etc – the speed at which we can access those services becomes critical. While bandwidth plays a part, the latency of the connection – the real Internet “speed” – is more important.

At Cloudflare, we’re working every day to help build a more performant Internet. Want to help? Apply for one of our open engineering roles here.

The Montgomery, Alabama Internet Exchange is making the Internet faster. We’re happy to be there.

Post Syndicated from Mike Conlow original https://blog.cloudflare.com/montgomery-alabama-ix/

The Montgomery, Alabama Internet Exchange is making the Internet faster. We’re happy to be there.

The Montgomery, Alabama Internet Exchange is making the Internet faster. We’re happy to be there.

Part of the magic of the Internet is in tens of thousands of networks connecting to each other all across the world in an effort to share information more efficiently. Cloudflare is a member of 279 Internet Exchanges (IX for short), but today we want to highlight one such dot on the global Internet map: the Montgomery, Alabama Internet Exchange, called MGMix. Thanks to the hard work of local leaders and the participation of dozens of networks (including Cloudflare), the Internet in Alabama works better today than it did before the IX launched.

Understanding IXs

Before we talk more about Alabama in particular, let’s take a step back to understand the critical role that Internet Exchanges play in our global Internet. In a simple model of exchanging Internet traffic, one person is on their laptop and requests content on a website, uses a video conferencing application, or wants to securely connect to their workplace from home. The person, or “client” in technical terms, is generally using a traditional Internet Service Provider, who they pay to access everything on the Internet. On the other hand, whatever the user is trying to reach – the website, API endpoint, or security service – or “server” in technical terms, is usually on a different network. How the data gets from the client’s network to the server’s network is not something Internet users think much about, but at Cloudflare, we think about it a lot.

One way that a network can reach another network is by paying a 3rd party network to deliver the traffic. This is called “transit” and it’s an appealing option because it’s simple. One “Tier 1” transit provider can reach the entire Internet. Of course, the tradeoff is that convenience comes at a cost – networks pay transit providers based on the quantity of traffic passed over the connection.

At the other end, larger networks often connect directly with what are called Private Network Interconnections (PNI). If one network is consistently sending large volumes of traffic to another network, it will be less expensive to use a PNI than to send the traffic over a transit provider. In this case, the two networks string a fiber cable across the ceiling of a data center where both networks have a presence, from one network’s cage to the other’s.

The Montgomery, Alabama Internet Exchange is making the Internet faster. We’re happy to be there.

Right in the Goldilocks zone between transit providers and PNIs are Internet Exchanges. An IX brings networks together in one place, and lets them freely exchange traffic. Sometimes they’re literally called “meeting rooms”. Once a network joins an IX, they might be able to reach hundreds of other networks without incurring 3rd party transit fees. Thriving IX communities are a power-up for the Internet: they reduce the cost of delivering Internet traffic, incentivizing more networks to join, while making the Internet faster through better interconnection.

Montgomery Internet Exchange (MGMix)

Back to Alabama. Unfortunately, Alabama, and the “Deep South” in general, has some of the worst performing Internet in the country. In Alabama, 15% of locations don’t have access to home Internet with download throughput of 25 Mbps and 3 Mbps upload according to the latest FCC data. In Mississippi, it’s 20%. The national average is 7%. In terms of latency, which is how we measure the speed of the Internet, the Deep South is also well above average.

50th percentile TCP Connect Time (ms) to Major Content Delivery Networks

The Montgomery, Alabama Internet Exchange is making the Internet faster. We’re happy to be there.

One of the reasons for the poor performance is that requests for content often travel to Atlanta, Dallas, or other Internet hubs even farther away before coming all the way back to the user in Alabama or Mississippi. That’s why an IX in Montgomery is so exciting: if networks can exchange traffic in Montgomery, the data doesn’t need to travel as far, and the Internet will be faster.

A few years ago, local leaders in Montgomery started to build up the Montgomery Internet Exchange (MGMix). With the support of the mayor, and the help of city staff, and a cooperative that included the city, county, state, and a nearby Air Force base, they launched the IX in 2016.  Later they formed a technical committee and upgraded to 100 Gbps of capacity.

With a donated switch from Packet Clearing House, MGMix estimated their initial costs at $1,000 per month for data center space and connection to the Internet. At their core, an IX is just a Layer 2 switch where all the networks plug in and advertise their presence to each other. That’s not to say it’s easy. One of the hardest parts is the work to attract networks.

IX’s have a hard chicken-and-egg problem. The first network at an IX doesn’t have anyone to exchange traffic with. Conversely, once there are a lot of networks at an IX, it becomes easy to attract new ones. Additionally, networks like Cloudflare need certain types of networks – transits – to be present. In almost all cases, Cloudflare doesn’t actually host the website or service an Internet user is trying to reach; we protect them, but aren’t the original source. To get content from the original source, we need access to transit networks. The City of Montgomery did the hard work of building up the IX network by network.

MGMix now has a who’s-who of the Internet in Alabama as members. Some are ISPs like Charter, Wide Open West, Uniti Fiber, and Troy Cablevision. Some are big institutions like the State of Alabama, Alabama State University, the City of Montgomery. And still others are the providers of content and services, like Cloudflare, Meta, and Akamai.

From Cloudflare’s perspective, it was an easy decision to join MGMix. We followed the development closely, and joined soon after it opened. After all, it means better Internet performance for a group of southern states that have been historically underserved. Now that it’s established, it’s essentially maintenance-free. It’s set-it-and-forget-it for better Internet performance.

Below is a chart of our traffic through MGMix over the course of November. We see daily spikes in traffic outbound from Cloudflare to other networks that are members of the IX. Interestingly, the traffic is lower from the 20th of November through the 27th of November which is the week of Thanksgiving in the US. It looks like Internet users in Alabama were enjoying a restful week with their families and not using the Internet (as much as usual).

The Montgomery, Alabama Internet Exchange is making the Internet faster. We’re happy to be there.

It has apparently been going so well that MGMix just announced they’re expanding to Auburn, Alabama.

Steven Reed, the current mayor of Montgomery, said of the expansion: “This is a step forward to achieving digital equity across the region, benefiting individuals who live in underserved rural communities. By extending our network fabric to a datacenter in Auburn, the MGMix will improve the efficiency and resiliency of the Internet for the Montgomery area, colleges and businesses along the I-85 corridor, and the entire River Region.”

We couldn’t have said it better. IXs are a critical part of a strong Internet interconnection ecosystem. We’re proud members of the MGMix, and will continue to join IXs globally where we can reach Internet users more efficiently and effectively.

The US government is working on an “Internet for all” plan. We’re on board.

Post Syndicated from Mike Conlow original https://blog.cloudflare.com/internet-for-all-us/

The US government is working on an “Internet for all” plan. We’re on board.

The US government is working on an “Internet for all” plan. We’re on board.

Recently, the United States Department of Commerce announced that all 50 states and every eligible territory had signed on to the “Internet for All” initiative. Internet for All is the US government’s $65 billion initiative to close the Digital Divide once and for all through new broadband deployment and digital equity programs. Cloudflare is on a mission to help build a better Internet, and we support initiatives like this because we want more people using the Internet on high-throughput, low-latency, resilient and affordable Internet connections. It’s been written often since the start of the pandemic because it’s true: it isn’t acceptable that students need to go to a Taco Bell parking lot to do their homework, and a good Internet connection is increasingly important for doing adult jobs as well.

The Internet for All initiative is the result of $65 billion in broadband-related funding appropriated by the US Congress as part of the Infrastructure Investment and Jobs Act (IIJA). It’s been called a “once in a generation” funding opportunity, and compared with the Rural Electrification Act which brought power lines to rural America in the 1930s. The components of the broadband portion of the Infrastructure bill are:

  • \$42.5 billion for broadband deployment – new wires and wireless radios in places that don’t have them – called the Broadband Equity, Access, and Deployment Program (BEAD).
  • \$14.2 billion to make permanent a $30 per month subsidy for low-income families to purchase a home Internet subscription.
  • \$2.75 billion to establish a grant program that will improve digital equity, which means teaching Americans how to make the most of the Internet and their home connection.
  • \$2 billion for new connectivity on tribal lands.
  • \$1 billion to establish new “middle-mile” capacity, which will connect rural communities to the Internet “backbone”.

The US should be applauded for making this kind of investment in broadband infrastructure. By appropriating federal funds, the government is able to ensure the money is used as it’s intended. For example, federal rules will require that areas with no infrastructure and disadvantaged urban areas will receive priority funding. Individual states will have the option of adding their own rules.

There’s significant work to do. According to the latest numbers from the Federal Communications Commission, 12% of Americans lack access to home broadband with throughput of at least 100 Mbps download and 20 Mbps upload.

There’s another way to think about access to broadband. A wire running near your house doesn’t do any good if the residents can’t afford it, or don’t know how to use the Internet. According to Pew Research, 23% of Americans say they don’t have an Internet connection at home. Those aren’t just rural areas without broadband infrastructure, it’s also urban areas where the connection is too expensive.

Cloudflare isn’t a disinterested observer. When Internet users don’t have access to good broadband, their experience with our services – the websites, APIs and security products we offer – won’t work as well as they should. In the map below, we use the Resource Timing API to measure the latency between Internet users and the major Content Delivery Networks (CDNs), including Cloudflare. We see rural and southern states have worse performance than the northeastern United States, with Hawaii and Alaska being off the charts in terms of their poor speed.

50th percentile TCP Connect Time (ms) to Major Content Delivery Networks

The US government is working on an “Internet for all” plan. We’re on board.
*Alaska and Hawaii have TCP Connect times of 263 and 160 respectively. 

Access technology, which is how Internet users connect to the Internet (cable, fiber, DSL, wireless, satellite), is one important part of the overall quality of their connection, but there are other, less talked about factors. Another factor is how close geographically the user is to the content and services they are accessing. Midwestern states where requests for data need to travel to Internet hubs in Chicago or Dallas are going to be slower than requests for data from Washington, DC, served by the giant Internet hub around Ashburn, Virginia. To be as close as possible to users geographically, Cloudflare has servers in 51 locations across 28 states in the US, and is still growing.

Programs that provide funding for deployment are one piece of the puzzle, but there are important non-financial initiatives as well. For example, the IIJA directed the Federal Communications Commission to come up with “broadband nutrition labels” that will be shown to consumers at the point of purchase for any Internet service. Just a few weeks ago, the FCC announced their implementation. Cloudflare filed comments with the FCC with our suggestions for how to make these labels informative, future-proof, and easy for consumers to understand. We also wrote about it here.

The US government is working on an “Internet for all” plan. We’re on board.

We’d be remiss to not also mention our own contribution to digital divide initiatives – Project Pangea. For community and non-profit networks that have invested in last-mile infrastructure but need a connection to the Internet – “transit” in industry terms – the network can connect to Cloudflare, and we’ll provide that Internet transit at no charge to the network. It’s one piece of the puzzle, and we’re always looking for additional ways to help.

One thing everyone can do is help the FCC build the most accurate broadband map possible by going to the map, entering your address, and verifying the data. The map will show your individual location and all ISPs that claim to serve your address. If there’s a problem – and there can be, it’s a new map and new process – you can file a challenge right from the FCC’s mapping site.

It’s laudable that the US government is stepping up with billions of dollars in funding for broadband networks and digital equity programs. In the shared project of helping build a better Internet, this is an important and big step.

Bringing Zero Trust to mobile network operators

Post Syndicated from Mike Conlow original https://blog.cloudflare.com/zero-trust-for-mobile-operators/

Bringing Zero Trust to mobile network operators

Bringing Zero Trust to mobile network operators

At Cloudflare, we’re excited about the quickly-approaching 5G future. Increasingly, we’ll have access to high throughput and low-latency wireless networks wherever we are. It will make the Internet feel instantaneous, and we’ll find new uses for this connectivity such as sensors that will help us be more productive and energy-efficient. However, this type of connectivity doesn’t have to come at the expense of security, a concern raised in this recent Wired article. Today we’re announcing the creation of a new partnership program for mobile networks—Zero Trust for Mobile Operators—to jointly solve the biggest security and performance challenges.

SASE for Mobile Networks

Every network is different, and the key to managing the complicated security environment of an enterprise network is having lots of tools in the toolbox. Most of these functions fall under the industry buzzword SASE, which stands for Secure Access Service Edge. Cloudflare’s SASE product is Cloudflare One, and it’s a comprehensive platform for network operators.  It includes:

  • Magic WAN, which offers secure Network-as-a-Service (NaaS) connectivity for your data centers, branch offices and cloud VPCs and integrates with your legacy MPLS networks
  • Cloudflare Access, which is a Zero Trust Network Access (ZTNA) service requiring strict verification for every user and every device before authorizing them to access internal resources.
  • Gateway, our Secure Web Gateway, which operates between a corporate network and the Internet to enforce security policies and protect company data.
  • A Cloud Access Security Broker, which monitors the network and external cloud services for security threats.
  • Cloudflare Area 1, an email threat detection tool to scan email for phishing, malware, and other threats.
Bringing Zero Trust to mobile network operators

We’re excited to partner with mobile network operators for these services because our networks and services are tremendously complementary. Let’s first think about SD-WAN (Software-Defined Wide Area Network) connectivity, which is the foundation on which much of the SASE framework rests. As an example, imagine a developer working from home developing a solution with a Mobile Network Operator’s (MNO) Internet of Things APIs. Maybe they’re developing tracking software for the number of drinks left in a soda machine, or want to track the routes for delivery trucks.

The developer at home and their fleet of devices should be on the same wide area network, securely, and at reasonable cost. What Cloudflare provides is the programmable software layer that enables this secure connectivity. The developer and the developer’s employer still need to have connectivity to the Internet at home, and for the fleet of devices. The ability to make a secure connection to your fleet of devices doesn’t do any good without enterprise connectivity, and the enterprise connectivity is only more valuable with the secure connection running on top of it. They’re the perfect match.

Once the connectivity is established, we can layer on a Zero Trust platform to ensure every user can only access a resource to which they’ve been explicitly granted permission. Any time a user wants to access a protected resource – via ssh, to a cloud service, etc. – they’re challenged to authenticate with their single-sign-on credentials before being allowed access. The networks we use are growing and becoming more distributed. A Zero Trust architecture enables that growth while protecting against known risks.

Bringing Zero Trust to mobile network operators

Edge Computing

Given the potential of low-latency 5G networks, consumers and operators are both waiting for a “killer 5G app”. Maybe it will be autonomous vehicles and virtual reality, but our bet is on a quieter revolution: moving compute – the “work” that a server needs to do to respond to a request – from big regional data centers to small city-level data centers, embedding the compute capacity inside wireless networks, and eventually even to the base of cell towers.

Cloudflare’s edge compute platform is called Workers, and it does exactly this – execute code at the edge. It’s designed to be simple. When a developer is building an API to support their product or service, they don’t want to worry about regions and availability zones. With Workers, a developer writes code they want executed at the edge, deploys it, and within seconds it’s running at every Cloudflare data center globally.

Some workloads we already see, and expect to see more of, include:

  • IoT (Internet of Things) companies implementing complex device logic and security features directly at the edge, letting them add cutting-edge capabilities without adding cost or latency to their devices.
  • eCommerce platforms storing and caching customized assets close to their visitors for improved customer experience and great conversion rates.
  • Financial data platforms, including new Web3 players, providing near real-time information and transactions to their users.
  • A/B testing and experimentation run at the edge without adding latency or introducing dependencies on the client-side.
  • Fitness-type devices tracking a user’s movement and health statistics can offload compute-heavy workloads while maintaining great speed/latency.
  • Retail applications providing fast service and a customized experience for each customer without an expensive on-prem solution.

The Cloudflare Case Studies section has additional examples from NCR, Edgemesh, BlockFi, and others on how they’re using the Workers platform. While these examples are exciting, we’re most excited about providing the platform for new innovation.

You may have seen last week we announced Workers for Platforms is now in General Availability. Workers for Platforms is an umbrella-like structure that allows a parent organization to enable Workers for their own customers. As an MNO, your focus is on providing the means for devices to send communication to clients. For IoT use cases, sending data is the first step, but the exciting potential of this connectivity is the applications it enables. With Workers for Platforms, MNOs can expose an embedded product that allows customers to access compute power at the edge.

Network Infrastructure

The complementary networks between mobile networks and Cloudflare is another area of opportunity. When a user is interacting with the Internet, one of the most important factors for the speed of their connection is the physical distance from their handset to the content and services they’re trying to access. If the data request from a user in Denver needs to wind its way to one of the major Internet hubs in Dallas, San Jose, or Chicago (and then all the way back!), that is going to be slow. But if the MNO can link to the service locally in Denver, the connection will be much faster.

One of the exciting developments with new 5G networks is the ability of MNOs to do more “local breakout”. Many MNOs are moving towards cloud-native and distributed radio access networks (RANs) which provides more flexibility to move and multiply packet cores. These packet cores are the heart of a mobile network and all of a subscriber’s data flows through one.

For Cloudflare – with a data center presence in 275+ cities globally – a user never has to wait long for our services. We can also take it a step further. In some cases, our services are embedded within the MNO or ISP’s own network. The traffic which connects a user to a device, authorizes the connection, and securely transmits data is all within the network boundary of the MNO – it never needs to touch the public Internet, incur added latency, or otherwise compromise the performance for your subscribers.

We’re excited to partner with mobile networks because our security services work best when our customers have excellent enterprise connectivity underneath. Likewise, we think mobile networks can offer more value to their customers with our security software added on top. If you’d like to talk about how to integrate Cloudflare One into your offerings, please email us at [email protected], and we’ll be in touch!

New cities on the Cloudflare global network: March 2022 edition

Post Syndicated from Mike Conlow original https://blog.cloudflare.com/mid-2022-new-cities/

New cities on the Cloudflare global network: March 2022 edition

If you follow the Cloudflare blog, you know that we love to add cities to our global map. With each new city we add, we help make the Internet faster, more reliable, and more secure. Today, we are announcing the addition of 18 new cities in Africa, South America, Asia, and the Middle East, bringing our network to over 270 cities globally. We’ll also look closely at how adding new cities improves Internet performance, such as our new locations in Israel, which reduced median response time (latency) from 86ms to 29ms (a 66% improvement) in a matter of weeks for subscribers of one Israeli Internet service provider (ISP).

The Cities

Without further ado, here are the 18 new cities in 10 countries we welcomed to our global network: Accra, Ghana; Almaty, Kazakhstan; Bhubaneshwar, India; Chiang Mai, Thailand; Joinville, Brazil; Erbil, Iraq; Fukuoka, Japan; Goiânia, Brazil; Haifa, Israel; Harare, Zimbabwe; Juazeiro do Norte, Brazil; Kanpur, India; Manaus, Brazil; Naha, Japan; Patna, India; São José do Rio Preto, Brazil; Tashkent, Uzbekistan; Uberlândia, Brazil.

Cloudflare’s ISP Edge Partnership Program

But let’s take a step back and understand why and how adding new cities to our list helps make the Internet better. First, we should reintroduce the Cloudflare Edge Partnership Program. Cloudflare is used as a reverse proxy by nearly 20% of all Internet properties, which means the volume of ISP traffic trying to reach us can be significant. In some cases, as we’ll see in Israel, the distance data needs to travel can also be significant, adding to latency and reducing Internet performance for the user. Our solution is partnering with ISPs to embed our servers inside their network. Not only does the ISP avoid lots of back haul traffic, but their subscribers also get much better performance because the website is served on-net, and close to them geographically. It is a win-win-win.

Consider a large Israeli ISP we did not peer with locally in Tel Aviv. Last year, if a subscriber wanted to reach a website on the Cloudflare network, their request had to travel on the Internet backbone – the large carriers that connect networks together on behalf of smaller ISPs – from Israel to Europe before reaching Cloudflare and going back. The map below shows where they were able to find Cloudflare content before our deployment went live: 48% in Frankfurt, 33% in London, and 18% in Amsterdam. That’s a long way!

New cities on the Cloudflare global network: March 2022 edition

In January and March 2022, we turned up deployments with the ISP  in Tel Aviv and Haifa. Now live, these two locations serve practically all requests from their subscribers locally within Israel. Instead of traveling 3,000 km to reach one of the millions of websites on our network, most requests from Israel now travel 65 km, or less. The improvement has been dramatic: now we’re serving 66% of requests in under 50ms; before the deployment we couldn’t serve any in under 50ms because the distance was too great. Now, 85% are served in under 100ms; before, we served 66% of requests in under 100ms.

New cities on the Cloudflare global network: March 2022 edition

As we continue to put dots on the map, we’ll keep putting updates here on how Internet performance is improving. As we like to say, we’re just getting started.

If you’re an ISP that is interested in hosting a Cloudflare cache to improve performance and reduce back haul, get in touch on our Edge Partnership Program page. And if you’re a software, data, or network engineer – or just the type of person who is curious and wants to help make the Internet better – consider joining our team.