Tag Archives: Birthday Week

The secret to Cloudflare’s pace of Innovation

Post Syndicated from Jen Taylor original https://blog.cloudflare.com/the-secret-to-cloudflare-pace-of-innovation/

The secret to Cloudflare’s pace of Innovation

The secret to Cloudflare’s pace of Innovation

We are 11! And we also may be a little bleary-eyed and giddy from a week of shipping.

The secret to Cloudflare’s pace of Innovation

Our Birthday Weeks are one of my favorite Cloudflare traditions — where we release innovations that help to build a better Internet. Just this week we tackled email security, expanded our network into office buildings, and entered into the Web3 world.

The secret to Cloudflare’s pace of Innovation

But these weeks also precipitate the most common questions I’m asked from my product and engineering peers across the industry: how do we do it? How do we get so much stuff out so quickly? That we are able to innovate — and innovate so quickly — is no happy accident. In fact, this capability has been very deliberately built into the DNA of Cloudflare. I want to touch on three of the reasons unique to us: one relates to our people, one relates to our technology, and one relates to our customers.

Cultivating curiosity

The seeds of innovative ideas start with our team. One of the core things we look for when hiring in every role at Cloudflare — be it engineering and product or sales or account — is curiosity. We seek people who approach a situation with curiosity — who seek to understand the what, the how, and perhaps most importantly, the why of the world around them. Innovation at its core is finding new ways to solve existing problems, and this curiosity is jet fuel for innovation. It pushes us to challenge the status quo. It is through this inquiry that we identify assumptions and unnecessary points of friction.

The secret to Cloudflare’s pace of Innovation

We turn our curiosity towards the world in front of us, the problems that “bug” us about our workflows and the business of the Internet. This is as much about innovating with code as it is about upending business models that don’t make sense to us. It frustrated us that hackers used DDoS attacks to perpetrate financial terrorism by driving up the cost of their victim’s web hosting and delivery bills, so as part of Birthday week in 2017 we made our DDoS protection free and accessible to all. Interestingly, we also have seen a correlating drop in DDoS attacks against many types of sites since then. This year we turned our attention to the egregious egress fees charged by cloud providers and delivered R2, our object storage solution which offers the ability to store large amounts of data — both expanding what developers can build on Cloudflare while slashing the egress bandwidth fees associated with cloud provider storage to zero. We also continue to expand our at-cost registrar service to more TLDs.

The secret to Cloudflare’s pace of Innovation

There’s an important attribute that partners with curiosity, however. To innovate, we listen. One of the things I screen for when we hire product managers is their ability to listen and synthesize the information they are hearing and then their ability to distill it into actionable problems for us to tackle. We ship early and often with initial concepts. We must listen closely to the feedback from customers — both what they tell us through community forums, support tickets, and meetings and through what they show us in how they adopt and use a feature — and quickly fold this back into the roadmap and the next stage of development. Like some of what you saw this week? Watch our blog for updates and new features as we round out and mature the products.

The best dog food

We like to say Cloudflare was built on Cloudflare and for Cloudflare. This means we eat our own dog food with a glass of our own champagne on the side. We wanted a way to connect with our customers and the community, so we built ourselves a real-time streaming service and launched CloudflareTV. Over the past 18 months we ironed out the kinks and this week we made that available to all of you to stand up your own network. We care deeply not only about the capabilities we deliver but how they look and feel. This focus leads to big innovations, but also leads to smaller improvements in how our tools look and feel, like Dark Mode, which was a passion project of a few of our front-end developers and one of the most requested features of all time by our own team.

The secret to Cloudflare’s pace of Innovation

So that’s an example of what happens. But how were we able to do this?

It’s driven by our Workers platform. This isn’t just something that’s making our customers’ lives easier. No containers to manage; no scaling to handle. We use it, too. It’s a layer of abstraction on top of our network that enables a massive amount of development velocity. Our unified architecture provides a single, scalable platform on which to innovate and allows us to roll things out in a matter of seconds across our entire network (and roll them back even more quickly if things don’t go to plan, which also sometimes happens). We’re constantly striving to make this network faster, safer, and more reliable. We’re constantly evolving our network to make it better. A couple of weeks ago we announced we’d expanded to 250 cities and are now within 50 milliseconds of 95% of the Internet-connected population. This week we expanded the footprint of our network into office buildings, getting even closer to end users.

The secret to Cloudflare’s pace of Innovation

This abstraction enables us to have smaller teams to build any new product or service. While others in the industry talk about building “pizza box” teams — teams small enough that they can all share a pizza together — many of our innovation efforts start with teams even smaller. At Cloudflare, it is not unusual for an initial product idea to start with a team small enough to split a pack of Twinkies and for the initial proof of concept to go from whiteboard to rolled out in days. We intentionally staff and structure our teams and our backlogs so that we have flexibility to pivot and innovate. Our Emerging Technology and Incubation team is a small group of product managers and engineers solely dedicated to exploring new products for new markets. Our Research team is dedicated to thinking deeply and partnering with organizations across the globe to define new standards and new ways to tackle some of the hardest challenges. These efforts drove our Web3 announcements and I could not be more excited to see where you all take them.

The secret to Cloudflare’s pace of Innovation

We ship software for businesses like a consumer software company. Traditional B2B software development typically follows longer development cycles and focuses on delivering more fully featured and deeply integrated offerings out of the gate. This cautious approach yields fewer more fully featured offerings shipped less frequently. Consumer software, on the other hand, is typically built in a highly iterative process — where teams develop initial concepts quickly and roll them out frequently in small pieces to subsets of users to test and understand how people use them and then use these observations to guide development. Many of the seeds that shaped the email security offerings this week started as small experiments by the DNS team, Emerging Technology, and our Customer Support team.

There’s one more aspect to all of this — because we’ve innovated in the way we’ve built our network — making it highly scalable and cost-effective — we’re able to pass these savings on to our customers. In a lot of instances, these savings amount to a pretty good price for our products and services: nothing at all.

Born to be free

Typically, in B2B enterprise software companies, the big customers and the multi-million dollar contracts get 99.99% of the focus. Here’s what I would say: our free customers are the secret sauce to our innovation. Today, millions of websites and applications  across the globe use our service for free. Free is invaluable to innovation.

One of the most difficult parts of building B2B software is getting the first 100 users. These early users are critical to assess the quality, scale, and capabilities of a product. But B2B folks are notoriously risk averse: their application or users are their lifeblood, and they aren’t necessarily willing to make them guinea pigs. They would rather sit on the sidelines and wait for someone else to go first. So when you are building B2B software, you are stuck in a conundrum — how do you get the users you need to prove out the capabilities?  We’re fortunate to have millions of free sites and applications on our network. Like a consumer app developer, we typically roll out new capabilities to pods of these users first. These users give us the volume and scale to get confident in what we’ve delivered. The more confident we get, the more broadly we roll out, such that by the time it hits broad rollout, our customers can feel confident in the quality and stability.

The large and diverse free customer base also helps fuel innovation. Through servicing a large and diverse base — from a personal blog in Bolivia to a small business in Chennai to a start-up in Munich — we observe unique traffic and threat patterns. We take these learning and fold the insights back into our product to dynamically route traffic across the fastest route and stop emerging threats before they scale.

The other thing I love about Birthday Week? It is the place critical innovations start that will ultimately transform the way we work. Universal SSL in 2014 made SSL free and available to all customers, doubling the number of active sites that use encryption within 24 hours of launch.   Seedling ideas, such as Workers, which we launched in 2017, have become the foundation of a whole new generation of applications.

I mentioned earlier that two of the most common questions I’m asked about Cloudflare relate to how we do it. Well, there’s often a third, and that too comes up quite a lot during Birthday Week. And that question is: “are you hiring?!” The answer is a resounding yes! We have opportunities across a variety of roles in Cloudflare: Legal, Product, Engineering, Sales, IT. Good ideas come from everywhere. I’d love nothing more than to engage your curiosity to help us build a better Internet.

The secret to Cloudflare’s pace of Innovation

Two Weeks Later: Finding and Eliminating Long Tail Latencies

Post Syndicated from Simona Pop original https://blog.cloudflare.com/two-weeks-later-finding-and-eliminating-long-tail-latencies/

Two Weeks Later: Finding and Eliminating Long Tail Latencies

Two Weeks Later: Finding and Eliminating Long Tail Latencies

A little over two weeks ago, we shared extensive benchmarking results of edge networks all around the world.  It showed that on a range of tests (TCP connection time, time to first byte, time to last byte), and on a range of measurements (p95, mean), that Cloudflare had some impressive network performance. But we weren’t the fastest everywhere. So we made a commitment: we would improve in at least 10% of networks where we were not #1.

Today, we’re happy to tell you that we’ve delivered as promised. Of the networks where our average latency exceeded 100ms behind the leading provider during Speed Week, we’ve dramatically improved our performance. There were such 61 networks; now, we’re the fastest in 29 of them. Of course, we’re not done yet — but we wanted to share with you the latest results, and explain how we did it.

Measuring What Matters

In the process of quantifying network performance, it became clear where we were not the fastest everywhere. There were 61 country/network pairs where we more than 100ms behind the leading provider:

Two Weeks Later: Finding and Eliminating Long Tail Latencies

Once that was done, the fun began: we needed to go through the process of figuring out why we were slow — and then improve. The challenges we faced were unique to each network and highlighted a variety of different issues that are prevalent on the Internet. We’re going to deep dive into a couple of networks, and show how we diagnosed and then improved performance.

But before we do, here are the results of our efforts in the past two weeks: of the 61 networks where our performance was over 100ms behind the leader, we are now the #1 network in 29 of them.

Two Weeks Later: Finding and Eliminating Long Tail Latencies

And it’s not that we just focused on those 29 networks, either. We’ve dramatically improved our performance in almost all the networks where we were over 100ms behind the leader.

Two Weeks Later: Finding and Eliminating Long Tail Latencies

With the results out of the way, let’s share the story of chasing peak performance in three very different geographies — each with three very different sets of challenges. Before we begin: a lot of Cloudflare’s internal network performance is automatically tuned. However, by its very nature, the Internet is a network of networks — and that inherently relies on us talking to other network operators to maximize performance. That’s often what we had to do here.

Rectifying Route Advertisement in Brazil

One particular network that was flagged for improvement during Speed Week stood out: we’ll refer to it as Network-A. This network was well known to our edge team (the team that looks after our network connectivity in Cloudflare data centers) for frequently congesting the dedicated interconnection we have with the network in São Paulo. This type of dedicated connection is called a Private Network Interconnect (PNI), or private peering, and it helps Cloudflare talk to Network-A without any intermediaries using the BGP protocol.

At a first look, we noticed that a significant chunk of traffic to Network-A was not using the PNI, but instead was being sent through one of our transit providers. A transit provider is an intermediary network that provides connectivity to the rest of the Internet.

This is not uncommon. The most likely reason for this behavior is that at some point in the past traffic was shifted away from the PNI due to capacity issues mentioned earlier.

We then started to take a more in-depth look at the path from this particular transit provider and identified that traffic was routed all the way to the USA before coming back to Brazil. The transit provider was exhibiting behavior known as tromboning: traffic from one location destined to a network in the same location travels vast distances only to be exchanged and then returns again. Tromboning typically occurs as a result of networks preferring paths that are farther away from the best possible path. This can happen due to peering preferences, BGP configurations, or the presence of direct interconnection farther away from end users. This explained the higher latency on this network we saw during Speed Week.

Two Weeks Later: Finding and Eliminating Long Tail Latencies

The next step we took was to look into alternatives to this transit connection. We have a nearby data center in Rio de Janeiro — where we also had a PNI with Network-A. Moreover, São Paulo and Rio de Janeiro are connected via our backbone network. After making the necessary checks to ensure we had room to carry traffic towards Network-A through our backbone and out through the PNI in Rio, we proceeded to prepare the network configuration changes.

We first started announcing Network-A IP addresses out of our backbone in Rio and then accepting them into São Paulo. We then ensured we preferred the path via our backbone over the PNI by changing the BGP behavior through the LOCAL_PREF path attribute. We then removed all the configuration specifying that transit provider as the preferred route for the previously identified traffic from Network-A.

Two Weeks Later: Finding and Eliminating Long Tail Latencies

The result was as expected. Traffic moved away from the transit provider onto our backbone network. We confirmed we achieved a decrease in latency by monitoring our p95 TCP RTTs, which went from 175ms to 90ms.

We currently rank #1 with Network-A, moving up from #5 during Speed Week, as seen in the chart below.

Two Weeks Later: Finding and Eliminating Long Tail Latencies

Immaculate Ingress in Spain

Another network that stood out was a European ISP with a global presence. We’ll refer to it as Network-B. Our RUM measurements showed that we were experiencing high latencies in several parts of the world, including Spain.

The first thing we did was to check how we handled traffic from Spain for Network-B. Our data showed that we had several data centers outside the country which were serving users from Network-B: Milan in Italy and Marseille in France. This obviously raised a question: why is traffic not staying locally in Spain?

The traffic was not staying local because Network-B had not peered with us in Madrid. If private peering describes a connection between exactly two networks using a dedicated circuit, public peering allows multiple networks to interconnect, if they wish so, at an Internet Exchange Point (IXP) location using a shared infrastructure. We looked at our Peering Portal to identify any potential peering opportunities with Network-B and established peering sessions in various locations where we saw high latency, including Madrid.

We looked at the traffic breakdown for these locations and identified the top destinations not being advertised in-country. We then checked whether our Spanish data centers were advertising these destinations and found that the corresponding anycast IP addresses were not enabled in Barcelona. We enabled the additional anycast IP addresses in Barcelona, and this change resulted in traffic for Network-B to be handled locally, which helped reduce latency.

Since we were looking into public peering status with Network-B, we also noticed that they had turned off their public peering session with Cloudflare in Milan. Our logs showed that the session with Network-B was down because it thought Cloudflare was sending more IP prefixes than allowed. We contacted Network-B and advised them to update the configuration according to the data we publish in PeeringDB. While it is a public peering session which comes with its own pros and cons, it still represents a more direct path than using a transit provider.

These changes pushed us up from ranking #2 during Speed Week to #1, as shown by the graph below:

Two Weeks Later: Finding and Eliminating Long Tail Latencies

You may notice that going from #2 to #1 still means we have a latency of about 300ms.  We want to ensure that every network has amazing performance, but we can’t control all network providers and how they connect with the rest of the Internet. We’re constantly working to ensure that end users see the best experience possible.

Upstream Selection in Africa

We’ve previously discussed private peering and transit providers and how a direct connection is better than a transit connection which usually routes through intermediary networks. However, sometimes this might not be true. This was the case for a network in Africa, which we will call Network-C.

As before, we started by looking at the locations from where we serve traffic to Network-C. This was mostly from our data centers in Western Europe. Looking at the parent ASNs for Network-C, we expected this outcome since we don’t peer with either of them anywhere in Africa.

Let’s take our data center in London. There, we had a private peering connection with Parent-1 and a transit connection with Parent-2. We were receiving IP addresses belonging to Network-C from both parents, however we were only sending traffic to Parent-1 since that was our private peer.

As Parent-2 also provided a direct path to Network-C and, moreover, they belonged to the same organization, we decided to test the latency via Parent-2. It is generally tricky to identify potential upstream bottlenecks especially for transit providers, as each network has its own internal mechanisms for routing. However, in this case we were directly connected.

Once again, we modified the BGP behaviour. Let’s go into more detail this time. Our routing policies are configured differently depending on the type of network we peer with and the type of connection we use. Our policies configure the BGP LOCAL_PREF path attribute, which is the first decisive step in the selection of a path. In our case, Network-C prefixes from Parent-1 had a higher associated value than the same prefixes learned from Parent-2 and were thus chosen for routing. In order to steer traffic away from the private peer and towards the transit provider, we needed to adjust our transit policy to set a higher LOCAL_PREF value only for Network-C prefixes. We also had to use a regular expression to match the desired prefixes by filtering Network-C ASN in the AS-path in a way that would not affect traffic to the other networks from the transit provider.

This change produced better results in terms of latency. We were #2 during Speed Week. We are now #1, as seen by this chart:

Two Weeks Later: Finding and Eliminating Long Tail Latencies

Update on Speed Week

Two weeks ago, when we first reported our measurements, there were two charts that stuck out where Cloudflare was not #1 in terms of number of networks where we had the lowest connection time or TTLB.

The first was the mean TCP connection time in the top 1,000 networks by number of IP addresses. Since then, we’ve been optimizing and have measured our performance again, and we’ve now moved into the #1 spot.

Two Weeks Later: Finding and Eliminating Long Tail Latencies

The other measurement where we were #2 was mean TTLB in the top 1,000 networks by IP count. We’ve moved into the #1 spot, but there’s still work to do. Which makes sense because the work we’ve been doing over the last two weeks optimized network performance and not our software platform. Hence, connection times got a lot better while TTLB improved less dramatically.

Two Weeks Later: Finding and Eliminating Long Tail Latencies

Getting ever faster

Improving performance on the Internet is a long tail problem: each issue requires a different solution because every network is unique and covers different end users. As we continue to grow our network and interconnect with more of the world, it’s important that we constantly examine our performance to ensure that we’re the fastest.

The efforts of our team have yielded great improvements for our customers, but we’re not just stopping because Speed Week and Birthday Week are over. We’re automating the discovery process of poor performance on networks like these, and are working hard to also automate the remediation processes in order to deliver more incredible performance for our customers.

And we have two more innovation weeks coming in 2021. We’ll be back each week to report on further progress on optimizing our performance globally.

Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone

Post Syndicated from Wesley Evans original https://blog.cloudflare.com/announcing-web3-gateways/

Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone

Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone

It’s cliché to say that the Internet has undergone massive changes in the last five years. New technologies like distributed ledgers, NFTs, and cross-platform metaverses have become all the rage. Unless you happen to hang out with the Web3 community in Hong Kong, San Francisco, and London, these technologies have a high barrier to entry for the average developer. You have to understand how to run distributed nodes, set up esoteric developer environments, and keep up with the latest chains just to get your app to run. That stops today. Today you can sign up for the private beta of our Web3 product suite starting with our Ethereum and IPFS gateway.

Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone

Before we go any further, a brief introduction to blockchain (Ethereum in our example) and the InterPlanetary FileSystem (IPFS). In a Web3 setting, you can think of Ethereum as the compute layer, and IPFS as the storage layer. By leveraging decentralised ledger technology, Ethereum provides verifiable decentralised computation. Publicly available binaries, called “smart contracts”, can be instantiated by users to perform operations on an immutable set of records. This set of records is the state of the blockchain. It has to be maintained by every node on the network, so they can verify, and participate in the computation. Performing operations on a lot of data is therefore expensive. A common pattern is to use IPFS as an external storage solution. IPFS is a peer-to-peer network for storing content on a distributed file system. Content is identified by its hash, making it inexpensive to reference from a blockchain context.

If you want an even deeper understanding of how Web3 works check out our other blog posts on what is Web3 and creating Web3 Dapps with Cloudflare Workers.

Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone

Web3 and the Metaverse

Over the last four years, while we have been working to mature the technology required to provide access to Web3 services at a global scale, the idea of the Metaverse has come back into vogue. Popularized by novels like “Snowcrash,” and “Ready Player One,” the idea is a simple one. Imagine an Internet where you can hop into an app and have access to all of your favorite digital goods available for you to use regardless of where you purchased them. You could sell your work on social media without granting them a worldwide license, and the buyer could use it on their online game. The Metaverse is a place where copyright and ownership can be managed through NFTs (Non-Fungible Tokens) stored on IPFS, and accessed trustlessly through Ethereum. It is a place where everyday creators can easily monetize their content, and have it be used by everyone, regardless of platform, since content is not being stored in walled gardens but decentralised ecosystems with open standards.

Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone
Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone

This shifts the way users and content creators think about the Internet. Questions like: “Do you actually need a Model View Controller system with a server to build an application?” “What is the best way to provide consistent naming of web resources across platforms?” “Do we actually need to keep our data locked behind another company’s systems or can the end-user own their data?”. This builds different trust assumptions. Instead of trusting a single company because they are the only one to have your users’ data, trust is being built leveraging a source verifiable by all participants. This can be people you physically interact with for messaging applications, X.509 certificates logged in a public Certificate Transparency Log for websites, or public keys that interact with blockchains for distributed applications.

Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone

It’s an exciting time. Unlike the emergence of the Internet however, there are large established companies that want to control the shape and direction of Web3 and this Metaverse. We believe in a future of a decentralised and private web. An open, standards-based web independent of any one company or centralizing force. We believe that we can be one of the many technical platforms that supports Web3 and the growing Metaverse ecosystem. It’s why we are so excited to be announcing the private beta of our Ethereum and IPFS gateways. Technologies that are at the forefront of Web3 and its emerging Metaverse.

Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone

Time and time again over the last year we have been asked by our customers to support their exploration of Web3, and oftentimes their core product offering. At Cloudflare, we are committed to helping build a better Internet for everyone, regardless of their preferred tech stack. We want to be the pickaxes and shovels for everyone. We believe that Web3 and the Metaverse is not just an experiment, but an entirely new networking paradigm where many of the next multi-billion dollar businesses are going to be built. We believe that the first complete metaverse could be built entirely on Cloudflare today using systems like Ethereum, IPFS, RTC, R2 storage, and Workers. Maybe you will be the one to build it…

We are excited to be on this journey with our Web3 community members, and can’t wait to show you what else we have been working on.

Introducing the Cloudflare Web3 Gateways!

A gateway is a computer that sits between clients (such as your browser or mobile device) and a number of other systems and helps translate traffic from one protocol to another, so the systems powering an application required to handle the request can do so properly. But there are different types of gateways that exist today.

You have probably heard mention of an API gateway, which is responsible for accepting API calls inbound to an application and aggregating the appropriate services to fulfill those requests and return a proper response to the end user. You utilize gateways every time you watch Netflix! Their company leverages an API gateway to ensure the hundreds of different devices that access their streaming service can receive a successful and proper response, allowing end users to watch their shows. Gateways are a critical component of how Web3 is being enabled for every end user on the planet.

Remember that Web3 or the distributed web is a set of technologies that enables hosting of content and web applications in a serverless manner by leveraging purely distributed systems and consensus protocols. Gateways let you use these applications in your browser without having to install plugins or run separate pieces of software called nodes. The distributed web community runs into the same problem of needing a stable, reliable, and resilient method to translate HTTP requests into the correct Web3 functions or protocols.

Today, we are introducing the Cloudflare Ethereum and IPFS Gateways to help Web3 developers do what they do best, develop applications, without having to worry about also running the infrastructure required to support Ethereum (Eth) or IPFS nodes.

Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone

What’s the problem with existing Eth or IPFS Web Gateways?

Traditional web technologies such as HTTP have had decades to develop standards and best practices that make sites fast, secure, and available. These haven’t been developed on the distributed web side of the Internet, which focuses more on redundancy. We identified an opportunity to bring the optimizations and infrastructure of the web to the distributed web by building a gateway — a service that translates HTTP API calls to IPFS or Ethereum functions, while adding Cloudflare added-value services on the HTTP side. The ability for a customer to operate their entire network control layer with a single pane of glass using Cloudflare is huge. You can manage the DNS, Firewall, Load Balancing, Rate Limiting, Tunnels, and more for your marketing site, your distributed application (Dapp), and corporate security, all from one location.

For many of our customers, the existing solutions for Web3 gateway do not have a large enough network to handle the growing amount of requests within the Ethereum and IPFS networks, but more importantly do not have the degree of resilience and redundancy that businesses expect and require operating at scale. The idea of the distributed web is to do just that… stay distributed, so no single actor can control the overall market. Speed, security, and reliability are at the heart of what we do. We are excited to be part of the growing Web3 infrastructure community so that we can help Dapp developers have more choice, scalability, and reliability from their infrastructure providers.

A clear example of this is when existing gateways have an outage. With too few gateways to handle the traffic, the result of this outage is pre-process transactions falling behind the blockchain they are accessing, thus leading to increased latency for the transaction, potentially leading to it failing. Worse, when decentralised application (Dapp) developers use IPFS to power their front end, it can lead to their entire application falling over. Overall, this leads to massive amounts of frustration from businesses and end users alike — not being able to collect revenue for products or services, thus putting a portion of the business at a halt and breaking trust with end users who depend on the reliability of these services to manage their Web3 assets.

Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone

How is Cloudflare solving this problem?

We found that there was a unique opportunity in a segment of the Web3 community that closely mirrored Cloudflare’s traditional customer base: the distributed web. This segment has some major usability issues that Cloudflare could help solve around reliability, performance, and caching. Cloudflare has an advantage that no other company in this space — and very few in the industry — have: a global network. For instance, content fetched through our IPFS Gateway can be cached near users, allowing download latency in the milliseconds. Compare this with up to seconds per asset using native IPFS. This speed enables services based on IPFS to go hybrid. Content can be served over the source decentralised protocols while browsers and tools are maturing to access them, and served to regular web users through a gateway like Cloudflare. We do provide a convenient, fast and secure option to browse this distributed content.

On Ethereum, users can be categorised in two ways. Application developers that operate smart contracts, and users that want to interact with the said contracts. While smart contracts operate autonomously based on their code, users have to fetch data and send transactions. As part of the chain, smart contracts do not have to worry about the network or a user interface to be online. This is why decentralised exchanges have had the ability to operate continuously across multiple interfaces without disruptions. Users on the other hand do need to know the state of the chain, and be able to interact with it. Application developers therefore have to require the users to run an Ethereum node, or can point them to use remote nodes through a standardised JSON RPC API. This is where Cloudflare comes in. Cloudflare Ethereum gateway relies on Ethereum nodes and provides a secure and fast interface to the Ethereum network. It allows application developers to leverage Ethereum in front-facing applications. The gateway can interact with any content part of the Ethereum chain. This includes NFT contracts, DeFi exchanges, or name services like ENS.

Announcing The Cloudflare Distributed Web Gateways Private Beta: Unlocking the Web3 Metaverse and Decentralized Finance for Everyone

How are the gateways doing so far?

Since our alpha release to very early customers as research experiments, we’ve seen a staggering number of customers wanting to leverage the new gateway technology and benefit from the availability, resiliency, and caching benefits of Cloudflare’s network.

Our current alpha includes companies that have raised billions of dollars in venture capital, companies that power the decentralised finance ecosystem on Ethereum, and emerging metaverses that make use of NFT technology.

In fact, we have over 2,000 customers leveraging our IPFS gateway lending to over 275TB of traffic per month. For Ethereum, we have over 200 customers transacting over 13TB, including 1.6 billion requests per month. We’ve seen extremely stable results from these customers and fully expect to see these metrics continue to ramp up as we add more customers to use this new product.

We are now very happy to announce the opening of our private beta for both the Ethereum and IPFS gateways. Sign up to participate in the private beta and our team will reach out shortly to ensure you are set up!

P.S. We are hiring for Web3! If you want to come work on it with us, check out our careers page.

Get started Building Web3 Apps with Cloudflare

Post Syndicated from Kristian Freeman original https://blog.cloudflare.com/get-started-web3/

Get started Building Web3 Apps with Cloudflare

Get started Building Web3 Apps with Cloudflare

For many developers, the term Web3 feels like a buzzword — it’s the sort of thing you see on a popular “Things you need to learn in 2021” tweet. As a software developer, I’ve spent years feeling the same way. In the last few months, I’ve taken a closer look at the Web3 ecosystem, to better understand how it works, and why it matters.

Web3 can generally be described as a decentralized evolution of the Internet. Instead of a few providers acting as the mediators of how your interactions and daily life on the web should work, a Web3-based future would liberate your data from proprietary databases and operate without centralization via the incentive structure inherent in blockchains.

The Web3 space in 2021 looks and feels much different from what it did a few years ago. Blockchains like Ethereum are handling incredible amounts of traffic with relative ease — although some improvements are needed — and newer blockchains like Solana have entered the space as genuine alternatives that could alleviate some of the scaling issues we’ve seen in the past few years.

Cloudflare is incredibly well-suited to empower developers to build the future with Web3. The announcement of Cloudflare’s Ethereum gateway earlier today will enable developers to build scalable Web3 applications on Cloudflare’s reliable network. Today, we’re also releasing an open-source example showing how to deploy, mint, and render NFTs, or non-fungible tokens, using Cloudflare Workers and Cloudflare Pages. You can try it out here, or check out the open-source codebase on GitHub to get started deploying your own NFTs to production.

The problem Web3 solves

When you begin to read about Web3 online, it’s easy to get excited about the possibilities. As a software developer, I found myself asking: “What actually is a Web3 application? How do I build one?“

Most traditional applications make use of three pieces: the database, a code interface to that database, and the user interface. This model — best exemplified in the Model-View-Controller (MVC) architecture — has served the web well for decades. In MVC, the database serves as the storage system for your data models, and the controller determines how clients interact with that data. You define views with HTML, CSS and JavaScript that take that data and display it, as well as provide interactions for creating and updating that data.

Imagine a social media application with a billion users. In the MVC model, the data models for this application include all the user-generated content that are created daily: posts, friendships, events, and anything else. The controllers written for that application determine who can interact with that data internally; for instance, only the two users in a private conversation can access that conversation. But those controllers — and the application as a whole — don’t allow external access to that data. The social media application owns that data and leases it out “for free” in exchange for viewing ads or being tracked across the web.

This was the lightbulb moment for me: understanding how Web3 offers a compelling solution to these problems. If the way MVC-based, Web 2.0 applications has presented itself is as a collection of “walled gardens” — meaning disparate, closed-off platforms with no interoperability or ownership of data — Web3 is, by design, the exact opposite.

In Web3 applications, there are effectively two pieces. The blockchain (let’s use Ethereum as our example), and the user interface. The blockchain has two parts: an account, for a user, a group of users, or an organization, and the blockchain itself, which serves as an immutable system of record of everything taking place on the network.

One crucial aspect to understand about the blockchain is the idea that code can be deployed to that blockchain and that users of that blockchain can execute the code. In Ethereum, this is called a “smart contract”. Smart contracts executed against the blockchain are like the controller of our MVC model. Instead of living in shrouded mystery, smart contracts are verifiable, and the binary code can be viewed by anyone.

For our hypothetical social media application, that means that any actions taken by a user are not stored in a central database. Instead, the user interacts with the smart contract deployed on the blockchain network, using a program that can be verified by anyone. Developers can begin building user interfaces to display that information and easily interact with it, with no walled gardens or platform lock-in. In fact, another developer could come up with a better user interface or smart contract, allowing users to move between these interfaces and contracts based on which aligns best with their needs.

Operating with these smart contracts happens via a wallet (for instance, an Ethereum wallet managed by MetaMask). The wallet is owned by a user and not by the company providing the service. This means you can take your wallet (the final authority on your data) and do what you want with it at any time. Wallets themselves are another programmable aspect of the blockchain — while they can represent a single user, they can also be complex multi-signature wallets that represent the interests of an entire organization. Owners of that wallet can choose to make consensus decisions about what to do with their data.


The rise of non-fungible tokens

One of the biggest recent shifts in the Web3 space has been the growth of NFTs — non-fungible tokens. Non-fungible tokens are unique assets stored on the blockchain that users can trade and verify ownership of. In 2019, Cloudflare was already writing about NFTs, as part of our announcement of the Cloudflare Ethereum Gateway. Since then, NFTs have exploded in popularity, with projects like CryptoPunks and Bored Ape Yacht Club trading millions of dollars in volume monthly.

NFTs are a fascinating addition to the Web3 space because they represent how ownership of data and community can look in a post-walled garden world. If you’ve heard of NFTs before, you may know them as a very visual medium: CryptoPunks and Bored Ape Yacht Club are, at their core, art. You can buy a Punk or Ape and use it as your profile picture on social media. But underneath that, owning an Ape isn’t just owning a profile picture; they also have exclusive ownership of a blockchain-verified asset.

It should be noted that the proliferation of NFT contracts led to an increase in the number of scams. Blockchain-based NFTs are a medium of conveying ownership, based on a given smart contract. This smart contract can be deployed by anyone, and associated with any content. There is no guarantee of authenticity, until you verify the trustworthiness and identity of the contract you are interacting with. Some platforms may support Verified accounts, while others are only allowing a set of trusted partners to appear on their platform. NFTs are flexible enough to allow multiple approaches, but these trust assumptions have to be communicated clearly.

That asset, tied to a smart contract deployed on Ethereum, can be traded, verified, or used as a way to gate access to programs. An NFT developer can hook into the trade event for their NFTs and charge a royalty fee, or when “minting”, or creating an NFT, they can charge a mint price, generating revenue on sales and trades to fund their next big project. In this way, NFTs can create strong incentive alignment between developers and community members, more so than your average web application.

What we built

To better understand Web3 (and how Cloudflare fits into the puzzle), we needed to build something using the Web3 stack, end-to-end.

To allow you to do the same, we’re open-sourcing a full-stack application today, showing you how to mint and manage an NFT from start to finish. The smart contract for the application is deployed and verified on Ethereum’s Rinkeby network, which is a testing environment for Ethereum projects and smart contracts. The Rinkeby test network allows you to test the smart contract off of the main blockchain, using the exact same workflow, without using real ethers. When your project is ready to be deployed on Ethereum’s Mainnet, you can take the same contract, deploy and verify it, and begin using it in production.

Once deployed, the smart contract will provide the ability to manage your NFT project, compliant with the ERC-721 spec, that can be minted by users, displayed on NFT marketplaces like OpenSea and your own web applications. We also provided a web interface and example code for minting these NFTs — as a user, you can visit the web application with a compatible Ethereum wallet installed and claim a NFT.

Once you’ve minted the NFT, the example user interface will render the metadata for each claimed NFT. According to the ERC-721 (NFT) spec, a deployed token must have a corresponding URL that provides JSON metadata. This JSON endpoint, which we’ve built with Cloudflare Workers, returns a name and description for each unique NFT, as well as an image. To host this image, we’ve used Infura to pin the service, and Cloudflare IPFS Gateway to serve it. Our NFT identifies the content via its hash, making it not replaceable with something different in the future.

This open-source project provides all the tools that you need to build an NFT project. By building on Workers and Pages, you have all the tools you need to scale a successful NFT launch, and always provide up-to-date metadata for your NFT assets as users mint and trade them between wallets.

Get started Building Web3 Apps with Cloudflare
Architecture diagram of Cloudflare’s open-source NFT project

Cloudflare + Web3

Cloudflare’s developer platform — including Workers, Pages, and the IPFS gateway — works together to provide scalable solutions at each step of your NFT project’s lifecycle. When you move your NFT project to production, Cloudflare’s Ethereum and IPFS gateways are available to handle any traffic that your project may have.

We’re excited about Web3 at Cloudflare. The world is shifting back to a decentralized model of the Internet, the kind envisioned in the early days of the World Wide Web. As we say a lot around Cloudflare, The Network is the Computer — we believe that whatever form Web3 may take, whether through projects like Metaverses, DAOs (decentralized autonomous organizations) and NFTs for community and social networking, DeFi (decentralized finance) applications for managing money, and a whole class of decentralized applications that we probably haven’t even thought of…  Cloudflare will be foundational to that future.

Web3 — A vision for a decentralized web

Post Syndicated from Thibault Meunier original https://blog.cloudflare.com/what-is-web3/

Web3 — A vision for a decentralized web

Web3 — A vision for a decentralized web

By reading this, you are a participant of the web. It’s amazing that we can write this blog and have it appear to you without operating a server or writing a line of code. In general, the web of today empowers us to participate more than we could at any point in the past.

Last year, we mentioned the next phase of the Internet would be always on, always secure, always private. Today, we dig into a similar trend for the web, referred to as Web3. In this blog we’ll start to explain Web3 in the context of the web’s evolution, and how Cloudflare might help to support it.

Going from Web 1.0 to Web 2.0

When Sir Tim Berners-Lee wrote his seminal 1989 document “Information Management: A Proposal”, he outlined a vision of the “web” as a network of information systems interconnected via hypertext links. It is often assimilated to the Internet, which is the computer network it operates on. Key practical requirements for this web included being able to access the network in a decentralized manner through remote machines and allowing systems to be linked together without requiring any central control or coordination.

Web3 — A vision for a decentralized web
The original proposal for what we know as the web, fitting in one diagram – Source: w3

This vision materialized into an initial version of the web that was composed of interconnected static resources delivered via a distributed network of servers and accessed primarily on a read-only basis from the client side — “Web 1.0”. Usage of the web soared with the number of websites growing well over 1,000% in the ~2 years following the introduction of the Mosaic graphical browser in 1993, based on data from the World Wide Web Wanderer.

The early 2000s marked an inflection point in the growth of the web and a key period of its development, as technology companies that survived the dot-com crash evolved to deliver value to customers in new ways amidst heightened skepticism around the web:

  • Desktop browsers like Netscape became commoditized and paved the way for native web services for discovering content like search engines.
  • Network effects that were initially driven by hyperlinks in web directories like Yahoo! were hyperscaled by platforms that enabled user engagement and harnessed collective intelligence like review sites.
  • The massive volume of data generated by Internet activity and the growing realization of its competitive value forced companies to become experts at database management.

O’Reilly Media coined the concept of Web 2.0 in an attempt to capture such shifts in design principles, which were transformative to the usability and interactiveness of the web and continue to be core building blocks for Internet companies nearly two decades later.

However, in the midst of the web 2.0 transformation, the web fell out of touch with one of its initial core tenets — decentralization.

Decentralization: No permission is needed from a central authority to post anything on the web, there is no central controlling node, and so no single point of failure … and no “kill switch”!
— History of the web by Web Foundation

A new paradigm for the Internet

This is where Web3 comes in. The last two decades have proven that building a scalable system that decentralizes content is a challenge. While the technology to build such systems exists, no content platform achieves decentralization at scale.

There is one notable exception: Bitcoin. Bitcoin was conceptualized in a 2008 whitepaper by Satoshi Nakamoto as a type of distributed ledger known as a blockchain designed so that a peer-to-peer (P2P) network could transact in a public, consistent, and tamper-proof manner.

That’s a lot said in one sentence. Let’s break it down by term:

  • A peer-to-peer network is a network architecture. It consists of a set of computers, called nodes, that store and relay information. Each node is equally privileged, preventing one node from becoming a single point of failure. In the Bitcoin case, nodes can send, receive, and process Bitcoin transactions.
  • A ledger is a collection of accounts in which transactions are recorded. For Bitcoin, the ledger records Bitcoin transactions.
  • A distributed ledger is a ledger that is shared and synchronized among multiple computers. This happens through a consensus, so each computer holds a similar replica of the ledger. With Bitcoin, the consensus process is performed over a P2P network, the Bitcoin network.
  • A blockchain is a type of distributed ledger that stores data in “blocks” that are cryptographically linked together into an immutable chain that preserves their chronological order. Bitcoin leverages blockchain technology to establish a shared, single source of truth of transactions and the sequence in which they occurred, thereby mitigating the double-spending problem.

Bitcoin — which currently has over 40,000 nodes in its network and processes over $30B in transactions each day — demonstrates that an application can be run in a distributed manner at scale, without compromising security. It inspired the development of other blockchain projects such as Ethereum which, in addition to transactions, allows participants to deploy code that can verifiably run on each of its nodes.

Today, these programmable blockchains are seen as ideal open and trustless platforms to serve as the infrastructure of a distributed Internet. They are home to a rich and growing ecosystem of nearly 7,000 decentralized applications (“Dapps”) that do not rely on any single entity to be available. This provides them with greater flexibility on how to best serve their users in all jurisdictions.

The web is for the end user

Distributed systems are inherently different from centralized systems. They should not be thought about in the same way. Distributed systems enable the data and its processing to not be held by a single party. This is useful for companies to provide resilience, but it’s also useful for P2P-based networks where data can stay in the hands of the participants.

For instance, if you were to host a blog the old-fashioned way, you would put up a server, expose it to the Internet (via Cloudflare 😀), et voilà. Nowadays, your blog would be hosted on a platform like WordPress, Ghost, Notions, or even Twitter. If these companies were to have an outage, this affects a lot more people. In a distributed fashion, via IPFS for instance, your blog content can be hosted and served from multiple locations operated by different entities.

Web3 — A vision for a decentralized web
Web 1.0
Web3 — A vision for a decentralized web
Web 2.0
Web3 — A vision for a decentralized web
Web3

Each participant in the network can choose what they host/provide and can be home to different content. Similar to your home network, you are in control of what you share, and you don’t share everything.

This is a core tenet of decentralized identity. The same cryptographic principles underpinning cryptocurrencies like Bitcoin and Ethereum are being leveraged by applications to provide secure, cross-platform identity services. This is fundamentally different from other authentication systems such as OAuth 2.0, where a trusted party has to be reached to assess one’s identity. This materializes in the form of “Login with <Big Cloud provider>” buttons. These cloud providers are the only ones with enough data, resources, and technical expertise.

In a decentralised web, each participant holds a secret key. They can then use it to identify each other. You can learn about this cryptographic system in a previous blog. In a Web3 setting where web participants own their data, they can selectively share these data with applications they interact with. Participants can also leverage this system to prove interactions they had with one another. For example, if a college issues you a Decentralized Identifier (DID), you can later prove you have been registered at this college without reaching out to the college again. Decentralized Identities can also serve as a placeholder for a public profile, where participants agree to use a blockchain as a source of trust. This is what projects such as ENS or Unlock aim to provide: a way to verify your identity online based on your control over a public key.

This trend of proving ownership via a shared source of trust is key to the NFT craze. We have discussed NFTs before on this blog. Blockchain-based NFTs are a medium of conveying ownership. Blockchain enables this information to be publicly verified and updated. If the blockchain states a public key I control is the owner of an NFT, I can refer to it on other platforms to prove ownership of it. For instance, if my profile picture on social media is a cat, I can prove the said cat is associated with my public key. What this means depends on what I want to prove, especially with the proliferation of NFT contracts. If you want to understand how an NFT contract works, you can build your own.

Web3 — A vision for a decentralized web

How does Cloudflare fit in Web3?

Decentralization and privacy are challenges we are tackling at Cloudflare as part of our mission to help build a better Internet.

In a previous post, Nick Sullivan described Cloudflare’s contributions to enabling privacy on the web. We launched initiatives to fix information leaks in HTTPS through Encrypted Client Hello (ECH), make DNS even more private by supporting Oblivious DNS-over-HTTPS (ODoH), and develop OPAQUE which makes password breaches less likely to occur. We have also released our data localization suite to help businesses navigate the ever evolving regulatory landscape by giving them control over where their data is stored without compromising performance and security. We’ve even built a privacy-preserving attestation that is based on the same zero-knowledge proof techniques that are core to distributed systems such as ZCash and Filecoin.

It’s exciting to think that there are already ways we can change the web to improve the experience for its users. However, there are some limitations to build on top of the exciting infrastructure. This is why projects such as Ethereum and IPFS build on their own architecture. They are still relying on the Internet but do not operate with the web as we know it. To ease the transition, Cloudflare operates distributed web gateways. These gateways provide an HTTP interface to Web3 protocols: Ethereum and IPFS. Since HTTP is core to the web we know today, distributed content can be accessed securely and easily without requiring the user to operate experimental software.

Where do we go next?

The journey to a different web is long but exciting. The infrastructure built over the last two decades is truly stunning. The Internet and the web are now part of 4.6 billion people’s lives. At the same time, the top 35 websites had more visits than all others (circa 2014). Users have less control over their data and are even more reliant on a few players.

The early Web was static. Then Web 2.0 came to provide interactiveness and service we use daily at the cost of centralisation. Web3 is a trend that tries to challenge this. With distributed networks built on open protocols, users of the web are empowered to participate.

At Cloudflare, we are embracing this distributed future. Applying the knowledge and experience we have gained from running one of the largest edge networks, we are making it easier for users and businesses to benefit from Web3. This includes operating a distributed web product suite, contributing to open standards, and moving privacy forward.

If you would like to help build a better web with us, we are hiring.

Real-Time Communications at Scale

Post Syndicated from Matt Silverlock original https://blog.cloudflare.com/announcing-our-real-time-communications-platform/

Real-Time Communications at Scale

Real-Time Communications at Scale

For every successful technology, there is a moment where its time comes. Something happens, usually external, to catalyze it — shifting it from being a good idea with promise, to a reality that we can’t imagine living without. Perhaps the best recent example was what happened to the cloud as a result of the introduction of the iPhone in 2007. Smartphones created a huge addressable market for small developers; and even big developers found their customer base could explode in a way that they couldn’t handle without access to public cloud infrastructure. Both wanted to be able to focus on building amazing applications, without having to worry about what lay underneath.

Last year, during the outbreak of COVID-19, a similar moment happened to real time communication. Being able to communicate is the lifeblood of any organization. Before 2020, much of it happened in meeting rooms in offices all around the world. But in March last year — that changed dramatically. Those meeting rooms suddenly were emptied. Fast-forward 18 months, and that massive shift in how we work has persisted.

While, undoubtedly, many organizations would not have been able to get by without the likes of Slack, Zoom and Teams as real time collaboration tools, we think today’s iteration of communication tools is just the tip of the iceberg. Looking around, it’s hard to escape the feeling there is going to be an explosion in innovation that is about to take place to enable organizations to communicate in a remote, or at least hybrid, world.

With this in mind, today we’re excited to be introducing Cloudflare’s Real Time Communications platform. This is a new suite of products designed to help you build the next generation of real-time, interactive applications. Whether it’s one-to-one video calling, group audio or video-conferencing, the demand for real-time communications only continues to grow.

Running a reliable and scalable real-time communications platform requires building out a large-scale network. You need to get your network edge within milliseconds of your users in multiple geographies to make sure everyone can always connect with low latency, low packet loss and low jitter. A backbone to route around Internet traffic jams. Infrastructure that can efficiently scale to serve thousands of participants at once. And then you need to deploy media servers, write business logic, manage multiple client platforms, and keep it all running smoothly. We think we can help with this.

Launching today, you will be able to leverage Cloudflare’s global edge network to improve connectivity for any existing WebRTC-based video and audio application, with what we’re calling “WebRTC Components”.  This includes scaling to (tens of) thousands of participants, leveraging our DDoS mitigation to protect your services from attacks, and enforce IP and ASN-based access policies in just a few clicks.

How Real Time is “Real Time”?

Real-time typically refers to communication that happens in under 500ms: that is, as fast as packets can traverse the fibre optic networks that connect the world together. In 2021, most real-time audio and video applications use WebRTC, a set of open standards and browser APIs that define how to connect, secure, and transfer both media and data over UDP. It was designed to bring better, more flexible bi-directional communication when compared to the primary browser-based communication protocol we rely on today, HTTP. And because WebRTC is supported in the browser, it means that users don’t need custom clients, nor do developers need to build them: all they need is a browser.

Importantly, we’ve seen the need for reliable, real-time communication across time-zones and geographies increase dramatically, as organizations change the way they work (yes, including us).

So where is real-time important in practice?

  • One-to-one calls (think FaceTime). We’re used to almost instantaneous communication over traditional telephone lines, and there’s no reason for us to head backwards.
  • Group calling and conferencing (Zoom or Google Meet), where even just a few seconds of delay results in everyone talking over each other.
  • Social video, gaming and sports. You don’t want to be 10 seconds behind the action or miss that key moment in a game because the stream dropped a few frames or decided to buffer.
  • Interactive applications: from 3D modeling in the browser, Augmented Reality on your phone, and even game streaming need to be in real-time.

We believe that we’ve only collectively scratched the surface when it comes to real-time applications — and part of that is because scaling real-time applications to even thousands of users requires new infrastructure paradigms and demands more from the network than traditional HTTP-based communication.

Enter: WebRTC Components

Today, we’re launching our closed beta WebRTC Components, allowing teams running centralized WebRTC TURN servers to offload it to Cloudflare’s distributed, global network and improve reliability, scale to more users, and spend less time managing infrastructure.

TURN, or Traversal Using Relays Around NAT (Network Address Translation), was designed to navigate the practical shortcomings of WebRTC’s peer-to-peer origins. WebRTC was (and is!) a peer-to-peer technology, but in practice, establishing reliable peer-to-peer connections remains hard due to Carrier-Grade NAT, corporate NATs and firewalls. Further, each peer is limited by its own network connectivity — in a traditional peer-to-peer mesh, participants can quickly find their network connections saturated because they have to receive data from every other peer. In a mixed environment with different devices (mobile, desktops), networks (high-latency 3G through to fast fiber), scaling to more than a handful of peers becomes extremely challenging.

Real-Time Communications at Scale

Running a TURN service at the edge instead of your own infrastructure gets you a better connection. Cloudflare operates an anycast network spanning 250+ cities, meaning we’re very close to wherever your users are. This means that when users connect to Cloudflare’s TURN service, they get a really good connection to the Cloudflare network. Once it’s on there, we leverage our network and private backbone to get you superior connectivity, all the way back to the other user on the call.

But even better: stop worrying about scale. WebRTC infrastructure is notoriously difficult to scale: you need to make sure you have the right capacity in the right location. Cloudflare’s TURN service scales automatically and if you want more endpoints they’re just an API call away.

Real-Time Communications at Scale

Of course WebRTC Components is built on the Cloudflare network, benefiting from the DDoS protection that it’s 100 Tbps network offers. From now on deploying scalable, secure, production-grade WebRTC relays globally is only a couple of API calls away.

A Developer First Real-Time Platform

But, as we like to say at Cloudflare: we’re just getting started. Managed, scalable TURN infrastructure is a critical building block to building real-time services for one-to-one and small group calling, especially for teams who have been managing their own infrastructure, but things become rapidly more complex when you start adding more participants.

Whether that’s managing the quality of the streams (“tracks”, in WebRTC parlance) each client is sending and receiving to keep call quality up, permissions systems to determine who can speak or broadcast in large-scale events, and/or building signalling infrastructure with support chat and interactivity on top of the media experience, one thing is clear: it there’s a lot to bite off.

With that in mind, here’s a sneak peek at where we’re headed:

  • Developer-first APIs that abstract the need to manage and configure low-level infrastructure, authentication, authorization and participant permissions. Think in terms of your participants, rooms and channels, without having to learn the intricacies of ICE, peer connections and media tracks.
  • Integration with Cloudflare for Teams to support organizational access policies: great for when your company town hall meetings are now conducted remotely.
  • Making it easy to connect any input and output source, including broadcasting to traditional HTTP streaming clients and recording for on-demand playback with Stream Live, and ingesting from RTMP sources with Stream Connect, or future protocols such as WHIP.
  • Embedded serverless capabilities via Cloudflare Workers, from triggering Workers on participant events (e.g. join, leave) through to building stateful chat and collaboration tools with Durable Objects and WebSockets.

… and this is just the beginning.

We’re also looking for ambitious engineers who want to play a role in building our RTC platform. If you’re an engineer interested in building the next generation of real-time, interactive applications, join us!

If you’re interested in working with us to help connect more of the world together, and are struggling with scaling your existing 1-to-1 real-time video & audio platform beyond a few hundred or thousand concurrent users, sign up for the closed beta of WebRTC Components. We’re especially interested in partnering with teams at the beginning of their real-time journeys and who are keen to iterate closely with us.

Serverless Live Streaming with Cloudflare Stream

Post Syndicated from Zaid Farooqui original https://blog.cloudflare.com/stream-live/

Serverless Live Streaming with Cloudflare Stream

Serverless Live Streaming with Cloudflare Stream

We’re excited to introduce the open beta of Stream Live, an end-to-end scalable live-streaming platform that allows you to focus on growing your live video apps, not your codebase.

With Stream Live, you can painlessly grow your streaming app to scale to millions of concurrent broadcasters and millions of concurrent users. Start sending live video from mobile or desktop using the industry standard RTMPS protocol to millions of viewers instantly. Stream Live works with the most popular live video broadcasting software you already use, including ffmpeg, OBS or Zoom. Your broadcasts are automatically recorded, optimized and delivered using the Stream player.

When you are building your live infrastructure from scratch, you have to answer a few critical questions:

  1. “Which codec(s) are we going to use to encode the videos?”
  2. “Which protocols are we going to use to ingest and deliver videos?”
  3. “How are the different components going to impact latency?”

We built Stream Live, so you don’t have to think about these questions and spend considerable engineering effort answering them. Stream Live abstracts these pesky yet important implementation details by automatically choosing the most compatible codec and streaming protocol for the client device. There is no limit to the number of live broadcasts you can start and viewers you can have on Stream Live. Whether you want to make the next viral video sharing app or securely broadcast all-hands meetings to your company, Stream will scale with you without having to spend months building and maintaining video infrastructure.

Built-in Player and Access Control

Every live video gets an embed code that can be placed inside your app, enabling your users to watch the live stream. You can also use your own player with included support for the two major HTTP streaming formats — HLS and DASH — for a granular control over the user experience.

You can limit who can view your live videos with self-expiring tokenized links for each viewer. When generating the tokenized links, you can define constraints including time-based expiration, geo-fencing and IP restrictions. When building an online learning site or a video sharing app, you can put videos behind authentication, so only logged-in users can view your videos. Or if you are building a live concert platform, you may have agreements to only allow viewers from specific countries or regions. Stream’s signed tokens help you comply with complex and custom rulesets.

Instant Recordings

With Stream Live, you don’t have to wait for a recording to be available after the live broadcast ends. Live videos automatically get converted to recordings in less than a second. Viewers get access to the recording instantly, allowing them to catch up on what they missed.

Instant Scale

Whether your platform has one active broadcaster or ten thousand, Stream Live scales with your use case. You don’t have to worry about adding new compute instances, setting up availability zones or negotiating additional software licenses.

Legacy live video pipelines built in-house typically ingest and encode the live stream continents away in a single location. Video that is ingested far away makes video streaming unreliable, especially for global audiences. All Cloudflare locations run the necessary software to ingest live video in and deliver video out. Once your video broadcast is in the Cloudflare network, Stream Live uses the Cloudflare backbone and Argo to transmit your live video with increased reliability.

Serverless Live Streaming with Cloudflare Stream

Broadcast with 15 second latency

Depending on your video encoder settings, the time between you broadcasting and the video displaying on your viewer’s screens can be as low as fifteen seconds with Stream Live. Low latency allows you to build interactive features such as chat and Q&A into your application. This latency is good for broadcasting meetings, sports, concerts, and worship, but we know it doesn’t cover all uses for live video.

We’re on a mission to reduce the latency Stream Live adds to near-zero. The Cloudflare network is now within 50ms for 95% of the world’s population. We believe we can significantly reduce the delay from the broadcaster to the viewer in the coming months. Finally, in the world of live-streaming, latency is only meaningful once you can assume reliability. By using the Cloudflare network spanning over 250 locations, you get unparalleled reliability that is critical for live events.

Simple and predictable pricing

Stream Live is available as a pay-as-you-go service based on the duration of videos recorded and duration of video viewed.

  • It costs $5 per 1,000 minutes of video storage capacity per month. Live-streamed videos are automatically recorded. There is no additional cost for ingesting the live stream.
  • It costs $1 per 1,000 minutes of video viewed.
  • There are no surprises. You never have to pay hidden costs for video ingest, compute (encoding), egress or storage found in legacy video pipelines.
  • You can control how much you spend with Stream using billing alerts and restrict viewing by creating signed tokens that only work for authorized viewers.

Cloudflare Stream encodes the live stream in multiple quality levels at no additional cost. This ensures smooth playback for your viewers with varying Internet speed. As your viewers move from Wi-Fi to mobile networks, videos continue playing without interruption. Other platforms that offer live-streaming infrastructure tend to add extra fees for adding quality levels that caters to a global audience.

If your use case consists of thousands of concurrent broadcasters or millions of concurrent viewers, reach out to us for volume pricing.

Go live with Stream

Stream works independent of any domain on Cloudflare. If you already have a Cloudflare account with a Stream subscription, you can begin using Stream Live by clicking on the “Live Input” tab on the Stream Dashboard and creating a new input:

Serverless Live Streaming with Cloudflare Stream

If you are new to Cloudflare, sign up for Cloudflare Stream.

Announcing Cloudflare TV as a Service

Post Syndicated from Fallon Blossom original https://blog.cloudflare.com/cloudflare-tv-as-a-service/

Announcing Cloudflare TV as a Service

Announcing Cloudflare TV as a Service

In June 2020, Cloudflare TV made its debut: a 24/7 streaming video channel, focused on topics related to building a better Internet (and the people working toward that goal). Today, over 1,000 live shows later, we’re excited to announce that we’re making the technology we used to build Cloudflare TV available to any other business that wants to run their own 24×7 streaming network. But, before we get to that, it’s worth reflecting on what it’s been like for us to run one ourselves.

Let’s take it from the top.

Cloudflare TV began as an experiment in every way you could think of, one we hoped would help capture the serendipity of in-person events in a world where those were few and far between. It didn’t take long before we realized we had something special on our hands. Not only was the Cloudflare team thriving on-screen, showcasing an amazing array of talent and expertise — they were having a great time doing it. Cloudflare TV became a virtual watercooler, spiked with the adrenaline rush of live TV.

One of the amazing things about Cloudflare TV has been the breadth of content it’s inspired. Since launching, CFTV has hosted over 1,000 live sessions, featuring everything from marquee customer events with VIP speakers to game shows and DJ sets. Cloudflare’s employee resource groups have hosted hundreds of sessions speaking to their unique experiences, sharing a wealth of advice with the next generation of technology leaders. All told, we’ve welcomed over 650 Cloudflare employees and interns — and over 500 external guests, including the likes of Intel CEO Pat Gelsinger, Gradient Ventures board partner Bonita Stewart, Broadcom CTO Andy Nallappan, and Zendesk SVP Christina Liu.

Tune In, Geek Out: A CFTV Montage

This is Cloudflare TV, so of course we put an emphasis on technical content for viewers of all stripes. When we announce a new product or protocol on the Cloudflare Blog, we often host live sessions on CFTV the same day, featuring the engineers who wrote the code that just shipped. Every week, we broadcast episodes on cryptography, on learning how to code, and on the hardware that powers Cloudflare’s network in over 250 cities around the world.

Whether you’re new to Cloudflare TV or a longtime viewer, we encourage you to pay a visit to the just-launched Discover page, where you’ll find many of our most-loved shows on demand, ranging from Latest from Product and Engineering, to perennial favorite Silicon Valley Squares, to Yes We Can, featuring women leaders from across the tech industry. You can also browse upcoming Live segments and easily add them to your calendar.

One of the most promising indicators that we’re on the right track has been the feedback we’ve gotten, not just from viewers — but from companies eager to know which platform we were using to power CFTV. To date we haven’t had much to offer them other than our sincere thanks, but as of today we’re able to share something much more exciting.

But first: a look behind the scenes.

The Production Stack

We didn’t initially set out to build Cloudflare TV from scratch. But as we explored our available options, we quickly realized that few solutions were designed for 24/7 linear streaming, and fewer still were optimized to be managed by a globally-distributed team. Thankfully, at Cloudflare, we like to build.

Our engineers worked at a blazing pace to build our own homegrown system, tapping open-source projects where we could, and inventing the things that didn’t yet exist. Among the starring components:

  • Brave (BBC) — Brave is an open-source project named for a highly descriptive acronym: Basic Real-Time Audio Video Editor. It serves as the Cloudflare TV switchboard, allowing us to jump from live content to commercial to a pre-recorded session and back automatically, based on our broadcast schedule. The only issue with Brave is that, as the BBC put it: it’s a prototype. One that hasn’t been updated since 2018…
Announcing Cloudflare TV as a Service
The CFTV Switchboard (Now streaming: Latest from Product & Engineering)
  • Zoom — When we first designed Cloudflare TV, there was one directive that stood above the others: it had to be easy. If presenters had to deal with installing a browser plugin or unfamiliar app, we knew we’d lose many of them — especially external guests. Zoom emerged as the clear answer, and thanks to its RTMP broadcast feature, it’s worked seamlessly to facilitate live content on Cloudflare TV. In most cases, participating in a CFTV session is as simple as joining a Zoom meeting.
  • Cloudflare Workers — Put simply, Cloudflare TV wouldn’t exist were it not for Cloudflare Workers. Workers is the glue that brings together each of the disparate components of the platform — handling authentication, application logic, securely relaying data from our backend to our frontend, and sprinkling SEO optimizations across the site. It’s the first tool we reach for, and often the only one we need.
  • Cloudflare Stream — With over 1,000 episodes in our content library, we have a lot of assets to manage. Thankfully Stream makes it easy: episodes are uploaded and automatically transcoded to the appropriate bitrate, and we use Stream embeds to power Video on Demand across the entire platform. We also use the Stream API to deliver recordings to our backend switchboard so that they can be seamlessly rebroadcast alongside our Live sessions.
  • Cloudflare for Teams — Cloudflare TV is obviously public-facing, but there are an array of dashboards and admin interfaces that are only accessible to select members of the Cloudflare team. Thankfully the Cloudflare for Teams suite, including Cloudflare Access, makes it easy for us to set up custom rulesets that keep everything secure, without any cumbersome VPNs or authentication hurdles.

We Get By With a Little Help from Our Engs

We knew from the beginning that it wasn’t enough for Cloudflare TV to be easy for presenters — we needed to be able to run it with a relatively small team, working remotely, most of whom were juggling other responsibilities.

A special shoutout goes to the members of Cloudflare’s office and executive admin teams, whose roles were dramatically impacted by the pandemic. Each of them has stepped up and taken on the mantle of Cloudflare TV Producer, providing technical support, calming nerves, and facilitating each one of our live sessions. We couldn’t do it without them, nor would we want to.

Even so, running a TV station is a lot of work, and we had little choice but to make the platform as efficient as possible — automating away our pain points, and developing intuitive admin tools to empower our team. Here are some of the key contributors to the system’s efficiency:

The Auto-Switcher — CFTV’s schedule features hundreds of sessions every week, including weekends, which would be prohibitive if any manual switching were involved. Thankfully the system operates essentially on auto-pilot. This is no simple playlist: every minute, a program running on Cloudflare Workers syncs with the CFTV backend to queue up recordings and inputs for upcoming sessions, deleting those belonging to sessions that have already aired. If we take a week off over the holidays, Cloudflare TV will keep on humming.

The Auto-Scheduler — Scheduling CFTV content by hand (well over 250 segments per week) quickly went from a meaningful exercise to a perverse task. By week two we knew we had to figure something else out. And so the auto-scheduler was born, allowing us to select an arbitrary window of time and populate it with recordings from our content library, filling in any time slots between live segments.

Segments can be dragged, dropped, added, and removed in a couple of clicks; one person can schedule the entire week in less than an hour. The auto-scheduler intelligently rotates through each episode in the catalog to ensure they all get airtime — and we see plenty of opportunities for it to get smarter.

The Broadcasting Center — The lifeblood of Cloudflare TV is our live segments, so we naturally spend a lot of time trying to improve the experience for presenters. The Broadcasting Center is their home base: a page that loads automatically for each session’s host, providing them a countdown timer and other essentials. And because viewer engagement is a crucial part of what makes live programming special, it features a section for viewer questions — including a call-in feature, which records and automatically transcribes questions phoned in by viewers.

Announcing Cloudflare TV as a Service
Broadcasting Center — Presenter View

Meanwhile, our CFTV Producers use an administrative view of the same tool, where they check to make sure the stream is coming through clearly before each session begins. A set of admin controls allow them to troubleshoot if needed, and they can moderate viewer questions as well.

For both producers and presenters, the Broadcasting Center provides a single control plane to manage a live session. This ease-of-use goes a long way toward keeping the system running smoothly with a lean team.

Announcing Cloudflare TV as a Service
Broadcasting Center — Admin View

There’s a sequel? There’s a sequel.

One reason we’ve invested in Cloudflare TV is that it serves as fantastic platform for dogfooding — not only are we leveraging a broad array of Cloudflare’s media products, but our 24/7 linear content makes us a particularly demanding customer, with no appetite for arbitrary constraints like time limits or maintenance downtime.

With that in mind, we’re excited to integrate many of the new technologies Cloudflare is introducing this week, which will combine to power an overhauled version of the CFTV platform that we’re calling Cloudflare TV 2.0. Namely:

  • Real Time Communications Platform — Today, Cloudflare announced its new Real Time Communications Platform, powered by WebRTC. In the near future, Cloudflare TV will leverage this platform to handle many of our live sessions. CFTV will continue to support Zoom, OBS, and any other application capable of outputting a RTMP stream, because convenience is one of the essential pillars in helping our presenters engage with the platform. But we see opportunities to push our creativity to new heights with custom, programmatically-controlled media streams — powered by Cloudflare’s Real-Time Communications Platform.
  • Stream Live — CFTV’s backend server currently handles video encoding for our live broadcast, generating a stream that is relayed to a video.js embed. Replacing this setup with Stream Live will yield several key benefits: first, we will offload video encoding to Cloudflare’s global network, resulting in improved speed, reliability, and redundancy. It also means we’ll be able to generate multiple renditions of the broadcast at different bitrates, allowing us to offer streams that are optimized for mobile devices with limited bandwidth, and to dynamically switch between bitrates as a user’s network conditions change.
  • Stream Connect — Today, the only way to watch Cloudflare TV is from the platform’s homepage — but there’s no reason we can’t syndicate it to other popular video platforms like YouTube. Stream Connect will become the primary endpoint for our backend mixer, and will in turn generate multiple copies of that stream, outputting to YouTube, the main broadcast, and any number of additional platforms.

We’re also actively working on a fresh implementation of our switchboard — one that is designed to be more reliable, scalable, and customizable. This switchboard will power the core of Cloudflare TV 2.0.

Announcing Cloudflare TV as a Service

It’s not TV. It’s Cloudflare TV.

Cloudflare TV 2.0 will represent a major step forward for the platform, one that leverages over a year of insights as we rearchitect the system from its core to take full advantage of the Cloudflare network. And we’re doing it with you in mind: the same technology will be used to power Cloudflare TV as a Service.

Most products at Cloudflare are designed to scale from individuals up to the largest businesses. This is not one of those. Running a 24×7 streaming network takes a lot of time and effort. While we’ve made it easier than ever before, this is a product really designed for businesses that are willing to make a commitment similar to what we have at Cloudflare. But, if you are, we’re here to tell you that running a streaming service is incredibly rewarding, and we want to enable more companies to do it.

Interested? Fill out this form and, if it looks like you’d be a good fit, we’ll reach out and work with you to help build your own streaming service.

In the meantime, don’t miss out on Stream Live and the new Real Time Communications Platform. There’s no reason you can’t start building today.

How Cloudflare provides tools to help keep IPFS users safe

Post Syndicated from Thibault Meunier original https://blog.cloudflare.com/cloudflare-ipfs-safe-mode/

How Cloudflare provides tools to help keep IPFS users safe

How Cloudflare provides tools to help keep IPFS users safe

Cloudflare’s journey with IPFS started in 2018 when we announced a public gateway for the distributed web. Since then, the number of infrastructure providers for the InterPlanetary FileSystem (IPFS) has grown and matured substantially. This is a huge benefit for users and application developers as they have the ability to choose their infrastructure providers.

Today, we’re excited to announce new secure filtering capabilities in IPFS. The Cloudflare IPFS module is a tool to protect users from threats like phishing and ransomware. We believe that other participants in the network should have the same ability. We are releasing that software as open source, for the benefit of the entire community.

Its code is available on github.com/cloudflare/go-ipfs. To understand how we built it and how to use it, read on.

A brief introduction on IPFS content retrieval

Before we get to understand how IPFS filtering works, we need to dive a little deeper into the operation of an IPFS node.

The InterPlanetary FileSystem (IPFS) is a peer-to-peer network for storing content on a distributed file system. It is composed of a set of computers called nodes that store and relay content using a common addressing system.

Nodes communicate with each other over the Internet using a Peer-to-Peer (P2P) architecture, preventing one node from becoming a single point of failure. This is even more true given that anyone can operate a node with limited resources. This can be light hardware such as a Raspberry Pi, a server at a cloud provider, or even your web browser.

How Cloudflare provides tools to help keep IPFS users safe

This creates a challenge since not all nodes may support the same protocols, and networks may block some types of connections. For instance, your web browser does not expose a TCP API and your home router likely doesn’t allow inbound connections. This is where libp2p comes to help.

libp2p is a modular system of protocols, specifications, and libraries that enable the development of peer-to-peer network applications – libp2p documentation

That’s exactly what four IPFS nodes need to connect to the IPFS network. From a node point of view, the architecture is the following:

How Cloudflare provides tools to help keep IPFS users safe

Any node that we maintain a connection with is a peer. A peer that does not have 🐱 content can ask their peers, including you, they WANT🐱. If you do have it, you will provide the 🐱 to them. If you don’t have it, you can give them information about the network to help them find someone who might have it. As each node chooses the resources they store, it means some might be stored on a limited number of nodes.

For instance, everyone likes 🐱, so many nodes will dedicate resources to store it. However, 🐶 is less popular. Therefore, only a few nodes will provide it.

How Cloudflare provides tools to help keep IPFS users safe

This assumption does not hold for public gateways like Cloudflare. A gateway is an HTTP interface to an IPFS node. On our gateway, we allow a user of the Internet to retrieve arbitrary content from IPFS. If a user asks for 🐱, we provide 🐱. If they ask for 🐶, we’ll find 🐶 for them.

How Cloudflare provides tools to help keep IPFS users safe

Cloudflare’s IPFS gateway is simply a cache in front of IPFS. Cloudflare does not have the ability to modify or remove content from the IPFS network. However, IPFS is a decentralized and open network, so there is the possibility of users sharing threats like phishing or malware. This is content we do not want to provide to the P2P network or to our HTTP users.

In the next section, we describe how an IPFS node can protect its users from such threats.

If you would like to learn more about the inner workings of libp2p, you can go to ProtoSchool which has a great tutorial about it.

How IPFS filtering works

As we described earlier, an IPFS node provides content in two ways: to its peers through the IPFS P2P network and to its users via an HTTP gateway.

Filtering content of the HTTP interface is no different from the current protection Cloudflare already has in place. If 🐶 is considered malicious and is available at cloudflare-ipfs.com/ipfs/🐶, we can filter these requests, so the end user is kept safe.

The P2P layer is different. We cannot filter URLs because that’s not how the content is requested. IPFS is content-addressed. This means that instead of asking for a specific location such as cloudflare-ipfs.com/ipfs/🐶, peers request the content directly using its Content IDentifiers (CID), 🐶.

More precisely, 🐶 is an abstraction of the content address. A CID looks like QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy (QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy happens to be the hash of a .txt file containing the string “I’m trying out IPFS”). CID is a convenient way to refer to content in a cryptographically verifiable manner.

This is great, because it means that when peers ask for malicious 🐶 content, we can prevent our node from serving it. This includes both the P2P layer and the HTTP gateway.

In addition, the working of IPFS makes it, so content can easily be reused. On directories for instance, the address is a CID based on the CID of its files. This way, a file can be shared across multiple directories, and still be referred to by the same CID. It allows IPFS nodes to efficiently store content without duplicating it. This can be used to share docker container layers for example.

In the filtering use case, it means that if 🐶 content is included in other IPFS content, our node can also prevent content linking to malicious 🐶 content from being served. This results in 😿, a mix of valid and malicious content.

How Cloudflare provides tools to help keep IPFS users safe

This cryptographic method of linking content together is known as MerkleDAG. You can learn more about it on ProtoSchool, and Consensys did an article explaining the basic cryptographic construction with bananas 🍌.

How to use IPFS secure filtering

By now, you should have an understanding of how an IPFS node retrieves and provides content, as well as how we can protect peers and users from shared nodes accessing threats. Using this knowledge, Cloudflare went on to implement IPFS Safemode, a node protection layer on top of go-ipfs. It is up to every node operator to build their own list of threats to be blocked based on their policy.

To use it, we are going to follow the instructions available on cloudflare/go-ipfs repository.

First, you need to clone the git repository

git clone https://github.com/cloudflare/go-ipfs.git
cd go-ipfs/

Then, you have to check out the commit where IPFS safemode is implemented. This version is based on v0.9.1 of go-ipfs.

git checkout v0.9.1-safemode

Now that you have the source code on your machine, we need to build the IPFS client from source.

make build

Et voilà. You are ready to use your IPFS node, with safemode capabilities.

# alias ipfs command to make it easier to use
alias ipfs=’./cmd/ipfs/ipfs’
# run an ipfs daemon
ipfs daemon &
# understand how to use IPFS safemode
ipfs safemode --help
USAGE
ipfs safemode - Interact with IPFS Safemode to prevent certain CIDs from being provided.
...

Going further

IPFS nodes are running in a diverse set of environments and operated by parties at various scales. The same software has to accommodate configuration in which it is accessed by a single-user, and others where it is shared by thousands of participants.

At Cloudflare, we believe that decentralization is going to be the next major step for content networks, but there is still work to be done to get these technologies in the hands of everyone. Content filtering is part of this story. If the community aims at embedding a P2P node in every computer, there needs to be ways to prevent nodes from serving harmful content. Users need to be able to give consent on the content they are willing to serve, and the one they aren’t.

By providing an IPFS safemode tool, we hope to make this protection more widely available.

Cloudflare for Offices

Post Syndicated from James Allworth original https://blog.cloudflare.com/cloudflare-for-offices/

Cloudflare for Offices

Cloudflare for Offices

Cloudflare’s network is one of the biggest, most connected, and fastest in the world. It extends to more than 250 cities. In those cities, we’re often present in multiple data centers in order to connect to as many networks and bring our services as close to as many users as possible. We’re always asking ourselves: how can we get closer to even more of the world’s Internet users?

Today, we’re taking a big step toward that goal.

Introducing Cloudflare for Offices. We are creating strategic partnerships that will enable us to extend Cloudflare’s network into over 1,000 of the world’s busiest office buildings and multi-dwelling units. These buildings span the globe, and are where millions of people work every day; now, they’re going to be microseconds away from our global network. Our first deployments will include 30 Hudson Yards, 4 Times Square, and 520 Madison in New York; Willis Tower in Chicago; John Hancock Tower in Boston; and the Embarcadero Center and Salesforce Tower in San Francisco.

And we’re not done. We’ve built custom secure hardware and partnered with fiber providers to scale this model globally. It will bring a valuable new resource to the literal doorstep of building tenants.

Cloudflare has built a mutually beneficial relationship with the world’s ISPs by reducing their operational costs and improving customer performance. Similarly, we expect a mutually beneficial relationship as we roll out Cloudflare for Offices. Real estate operators & service offices upgraded with this amenity increase the value and occupancy of their portfolio. IT teams can enforce a consistent security posture while enabling flexible work environments from any location their employees prefer. And employees in these smart spaces, experiencing faster Internet performance, can be more productive, seamlessly working as they choose, be it at the office, at home, or on the go.

Why offices?

There’s no disputing the fact that the nature of work has undergone a tremendous shift over the past 18 months. While we still don’t know what the future of work will look like exactly, here’s what we do know: it’s going to require more flexibility, all while maintaining security and performance standards that are a prerequisite for operating on today’s Internet. Enabling flexibility, and improving performance AND security (as opposed to trading one off for the other) has been a long held belief of Cloudflare. Alongside, of course, driving value for organizations.

Cloudflare for Offices — by connecting directly with enterprises — enables us to now do that for commercial office space.

No More Band-Aid Boxes in the Basement

There are a variety of advantages to Cloudflare for Offices. First and foremost, it eliminates the need to rely on the costly, rigid hardware solutions and multiple, regional, third parties that are often required to provide secure and performant branch office connectivity. Businesses have maintained expensive and hardware-intensive office networks since the dawn of the modern Internet.

Never have they gotten less return on that investment than through the pandemic.

The hybrid future of work will only exacerbate the high costs and complexity of maintaining and securing this outdated infrastructure. MPLS links. WANs. Hardware firewalls. VPNs. All these remain mainstays of the modern office. In the same way that we look back on maintaining server rooms for compute and storage as complete anachronisms, so too will we soon look back on maintaining all these boxes in an office. We’ve spoken to customers who now have over half of their workforce remote, and who are considering giving up their office space or increasing their presence in shared workspaces. Some are being hamstrung because of a need for MPLS to make their network operate securely. But it’s not just customers. This is a problem that we ourselves have been facing. Setting up new offices, or securing and optimizing shared workspaces, is a huge lift, physically as well as technologically.

Cloudflare for Offices simplifies this: a direct connection to Cloudflare’s network puts all office traffic behind Cloudflare’s services. Now, creating an office is as simple as plugging a cable into our box, and all the security and performance features that an office typically needs are microseconds away. It also enables the creation of custom topologies on Cloudflare’s network, dramatically increasing the flexibility of your physical footprint.

“Throughout the pandemic, we’ve supported our over 12,000 employees to work safely and seamlessly from home or from our offices. Cloudflare solutions have been critical, and we’re excited to continue to partner on efficient and strong solutions.”
– Mark Papermaster, CTO and Executive Vice President, Technology and Engineering, AMD

Zero (Trust) to 100 performance

COVID-19 hasn’t just driven a paradigm shift in where people work, however. It’s also driven a paradigm shift in how organizations think about IT security.

The old model — castle and moat — was designed during the desktop era, when most computing happened on premises. Everyone within the walls of the enterprise was considered authenticated; if you were outside the office, you needed to “tunnel” in through the moat in the castle of the office. As more and more users entered the portable era — through laptops and smartphones — then more tunnels were created.

The pandemic made it so that everyone was outside the moat, tunneling into an empty castle. Nobody was in the office anymore. The paradigm has been stretched to a parody.

Google was one of the first organizations to start to think about how things could be done differently: it proposed a model called BeyondCorp, which treated internal employees to an organization similar to how it treated external customers or suppliers to an organization. To put it simply: nobody is trusted, no matter if they’re in the office or not. If you want access to something, be prepared to prove you are who you say you are.

Fast-forward to 2021, and this model — otherwise known as Zero Trust — has become the gold standard of enterprise security, to which more and more organizations are implementing. Cloudflare’s Zero Trust solution — Cloudflare for Teams — has become increasingly popular for not just its advanced functionality and its ease of use, but because, when coupled with our enterprise connectivity offerings, allows you to run more and more of your traffic across Cloudflare’s network. We call this holistic solution Cloudflare One, and it provides your organization a virtual private network in the cloud, with all the associated security and visibility benefits.

Cloudflare for Offices

Cloudflare for Offices is the onramp for offices onto Cloudflare One. It’s a fast, private onramp for your office network traffic straight onto the Cloudflare network — with all the security and visibility benefits that running your traffic over our network provides.

We also realize that for many organizations, Zero Trust is a journey. Not every customer is ready to go from MPLS and built-out networks to trusting the public Internet overnight. Cloudflare for Offices is a great start in the journey — by building out your own networks on top of Cloudflare, you reduce your threat vectors while being able to keep your existing topologies. This gives you the privacy and security of Cloudflare One, but with the flexibility to build Zero Trust any way you choose.

But security and visibility are not the only benefits. One of the common complaints we hear from customers about competing solutions is that performance can be extremely variable. The proximity Cloudflare has to so many people around the world is important because when employees connect using a Zero Trust solution, at least a subset (but often all) the traffic going from an end-user device needs to connect to the Zero Trust provider. Having Cloudflare equipment close means that the performance of the user device will be vastly increased as opposed to having to connect to a far off data center. You’ve probably read about what happens when Cloudflare takes control of your Last Mile connectivity and your network to your data centers. And you know that connecting to a Cloudflare data center in the same city increases performance, but imagine what happens when you’re connecting to Cloudflare in your office basement. And when you think about all the employees that you have are running on a zero trust model, that performance difference sums up to a lot of additional employee productivity.

Up until now, something like this has been extremely expensive, complicated, and oftentimes, slow.

“We see a lot of potential in the way Cloudflare is bringing its network directly to our office locations. It’s critical that we empower our employees to work productively and securely, and this makes it that much easier for us to do so no matter where our teams are working from in the future–and reducing our network costs along the way.”
– Aaron Dearinger, Edge Architect, Garmin International

Cloudflare for Offices allows for customers to choose their Network as a Service: let us manage your footprint and build your network out however you like.

Living on the Edge

But it’s not just zero trust that gets a boost. Workers, Cloudflare’s serverless platform, runs on the edge from the nearest data center to the user making the request. As you might have already read: it’s fast. With more and more business and application logic being moved to Workers, your end users stand to benefit.

But it does beg the question: just how fast are we talking?

Cloudflare for Offices
Photo by Denys Nevozhai on Unsplash

One example building we’re planning to enable is Salesforce Tower, in San Francisco. It’s 1,070 feet tall. A light signal running from the top of the building to the basement along a single-mode fiber cable would take no more than 6 µs (6 microseconds) to complete its journey. This puts customers fractions of a millisecond away from Cloudflare’s network.

The edge is becoming indistinguishable in performance from local compute.

Built for Purpose

We’ve written many times before about how Cloudflare designs our hardware. But deploying Cloudflare hardware outside of data centers — and into office basements — presented a new set of challenges. Cooling, energy efficiency, and resiliency were even more important in the design. Similarly, these are going to be deployed to offices all over the world; they needed to be cost-effective. Finally, and perhaps most importantly, there is also a security aspect to this: we could not assume the same level of access control inside a building as we could inside a data center.

Cloudflare for Offices

This is where the inherent advantages of designing and owning the hardware come to the fore. Because of it, we’re able to build exactly what we need for the environment: ranging from how resilient these devices need to be, to an appropriate level of security given where they’re going to be operating. In fact, we have been working on hardware security for the last five years in anticipation of the launch of Cloudflare for Offices. We’re starting with switching, and we plan to add compute and storage capabilities in short order. Stay tuned for more details.

Join the Revolution

If you’re an organization (tenant) in a large office building, an owner/operator of multi-tenant (or multi-dwelling) real estate, or a co-working space looking to bring Cloudflare to your doorstep — with all the flexibility, performance and security enhancements, and cost savings that would entail — then we’d love for you to get in touch with us.

Dark Mode for the Cloudflare Dashboard

Post Syndicated from Garrett Galow original https://blog.cloudflare.com/dark-mode/

Dark Mode for the Cloudflare Dashboard

Dark Mode for the Cloudflare Dashboard

Today, dark mode is available for the Cloudflare Dashboard in beta! From your user profile, you can configure the Cloudflare Dashboard in light mode, dark mode, or match it to your system settings.

For those unfamiliar, dark mode, or light on dark color schemes, uses light text on dark backgrounds instead of the typical dark text on light (usually white) backgrounds. In low-light environments, this can help reduce eyestrain and actually reduce power consumption on OLED screens. For many though, dark mode is simply a preference supported widely by applications and devices.

Dark Mode for the Cloudflare Dashboard
Side by side comparing the Cloudflare dashboard in dark mode and in light mode

How to enable dark mode

  1. Log into Cloudflare.
  2. Go to your user profile.
  3. Under Appearance, select an option: Light, Dark, or Use system setting. For the time being, your choice is saved into local storage.
Dark Mode for the Cloudflare Dashboard
The appearance card in the dashboard for modifying color themes

There are many primers and how-tos on implementing dark mode, and you can find articles talking about the general complications of implementing a dark mode including this straightforward explanation. Instead, we will talk about what enabled us to be able to implement dark mode in only a matter of weeks.

Cloudflare’s Design System – Our Secret Weapon

Before getting into the specifics of how we implemented dark mode, it helps to understand the system that underpins all product design and UI work at Cloudflare – the Cloudflare Design System.

Dark Mode for the Cloudflare Dashboard
The six pillars of the design system: logo, typography, color, layout, icons, videos

Cloudflare’s Design System defines and documents the interface elements and patterns used to build products at Cloudflare. The system can be used to efficiently build consistent experiences for Cloudflare customers. In practice, the Design System defines primitives like typography, color, layout, and icons in a clear and standard fashion. What this means is that anytime a new interface is designed, or new UI code is written, an easily referenceable, highly detailed set of documentation is available to ensure that the work matches previous work. This increases productivity, especially for new employees, and prevents repetitious discussions about style choices and interaction design.

Built on top of these design primitives, we also have our own component library. This is a set of ready to use components that designers and engineers can combine to form the products our customers use every day. They adhere to the design system, are battle tested in terms of code quality, and enhance the user experience by providing consistent implementations of common UI components. Any button, table, or chart you see looks and works the same because it is the same underlying code with the relevant data changed for the specific use case.

So, what does all of this have to do with dark mode? Everything, it turns out. Due to the widespread adoption of the design system across the dashboard, changing a set of variables like background color and text color in a specific way and seeing the change applied nearly everywhere at once becomes much easier. Let’s take a closer look at how we did that.

Turning Out the Lights

The use of color at Cloudflare has a well documented history. When we originally set out to build our color system, the tools we built and the extensive research we performed resulted in a ten-hue, ten-luminosity set of colors that can be used to build digital products. These colors were built to be accessible — not just in terms of internal use, but for our customers. Take our blue hue scale, for example.

Dark Mode for the Cloudflare Dashboard
Our blue color scale, as used on the Cloudflare Dashboard. This shows color-contrast accessible text and background pairings for each step in the scale.

Each hue in our color scale contains ten colors, ordered by luminosity in ten increasing increments from low luminosity to high luminosity. This color scale allows us to filter down the choice of color from the 16,777,216 hex codes available on the web to a much simpler choice of just hue and brightness. As a result, we now have a methodology where designers know the first five steps in a scale have sufficient color contrast with white or lighter text, and the last five steps in a scale have sufficient contrast with black or darker text.

Color scales also allow us to make changes while designing in a far more fluid fashion. If a piece of text is too bright relative to its surroundings, drop down a step on the scale. If an element is too visually heavy, take a step-up. With the Design System and these color scales in place, we’ve been able to design and ship products at a rapid rate.

So, with this color system in place, how do we begin to ship a dark mode? It turns out there’s a simple solution to this, and it’s built into the JS standard library. We call reverse() and flip the luminosity scales.

Dark Mode for the Cloudflare Dashboard
Our blue color scale after calling reverse on it. High luminosity colors are now at the start of the scale, making them contrast accessible with darker backgrounds (and vice-versa).

By performing this small change within our dashboard’s React codebase and shipping a production preview deploy, we were able to see the Cloudflare Dashboard in dark mode with a whole new set of colors in a matter of minutes.

Dark Mode for the Cloudflare Dashboard
An early preview of the Cloudflare Dashboard after flipping our color scales.

While not perfect, this brief prototype gave us an incredibly solid baseline and validated the approach with a number of benefits.

Every product built using the Cloudflare Design System now had a dark mode theme built in for free, with no additional work required by teams.

Our color contrast principles remain sound — just as the first five colors in a scale would be accessible with light text, when flipped, the first five colors in the scale are accessible with dark text. Our scales aren’t perfectly symmetrical, but when using white and black, the principle still holds.

In a traditional approach of “inverting” colors, we face the issue of a color’s hue being changed too. When a color is broken down into its constituent hue, saturation, and luminosity values, inverting it would mean a vibrant light blue would become a dull dark orange. Our approach of just inverting the luminosity of a color means that we retain the saturation and hue of a color, meaning we retain Cloudflare’s brand aesthetic and the associated meaning of each hue (blue buttons as calls-to-action, and so on).

Of course, shipping a dark mode for a product as complex as the Cloudflare Dashboard can’t just be done in a matter of minutes.

Not Quite Just Turning the Lights Off

Although our prototype did meet our initial requirements of facilitating the dashboard in a dark theme, some details just weren’t quite right. The data visualization and mapping libraries we use, our icons, text, and various button and link states all had to be audited and required further iterations. One of the most obvious and prominent examples was the page background color. Our prototype had simply changed the background color from white (#FFFFFF) to black (#000000). It quickly became apparent that black wasn’t appropriate. We received feedback that it was “too intense” and “harsh.” We instead opted for off black, specifically what we refer to as “gray.0” or #1D1D1D. The difference may not seem noticeable, but at larger dimensions, the gray background is much less distracting.

Here is what it looks like in our design system:

Dark Mode for the Cloudflare Dashboard
Black background color contrast for white text
Dark Mode for the Cloudflare Dashboard
Gray background color contrast for white text

And here is a more realistic example:

Dark Mode for the Cloudflare Dashboard
lorem ipsum sample text on black background and on gray background

The numbers at the end of each row represent the contrast of the text color on the background. According to the Web Content Accessibility Guidelines (WCAG), the standard contrast ratio for text should be at least 4.5:1. In our case, while both of the above examples exceed the standard, the gray background ends up being less harsh to use across an entire application. This is not the case with light mode as dark text on white (#FFFFFF) background works well.

Our technique during the prototyping stage involved flipping our color scale; however, we additionally created a tool to let us replace any color within the scale arbitrarily. As the dashboard is made up of charts, icons, links, shadows, buttons and certainly other components, we needed to be able to see how they reacted in their various possible states. Importantly, we also wanted to improve the accessibility of these components and pay particular attention to color contrast.

Dark Mode for the Cloudflare Dashboard
Color picker tool screenshot showing a color scale

For example, a button is made up of four distinct states:

1) Default
2) Focus
3) Hover
4) Active

Dark Mode for the Cloudflare Dashboard
Example showing the various colors for states of buttons in light and dark mode

We wanted to ensure that each of these states would be at least compliant with the AA accessibility standards according to the WCAG. Using a combination of our design systems documentation and a prioritized list of components and pages based on occurrence and visits, we meticulously reviewed each state of our components to ensure their compliance.

Dark Mode for the Cloudflare Dashboard
Side by side comparison of the navbar in light and dark modes

The navigation bar used to select between the different applications was a component we wanted to treat differently compared to light mode. In light mode, the app icons are a solid blue with an outline of the icon; it’s a distinct look and certainly one that grabs your attention. However, for dark mode, the consensus was that it was too bright and distracting for the overall desired experience. We wanted the overall aesthetic of dark mode to be subtle, but it’s important to not conflate aesthetic with poor usability. With that in mind, we made the decision for the navigation bar to use outlines around each icon, instead of being filled in. Only the selected application has a filled state. By using outlines, we are able to create sufficient contrast between the current active application and the rest. Additionally, this provided a visually distinct way to present hover states, by displaying a filled state.

After applying the same methodology as described to other components like charts, icons, and links, we end up with a nicely tailored experience without requiring a substantial overhaul of our codebase. For any new UI that teams at Cloudflare build going forward, they will not have to worry about extra work to support dark mode. This means we get an improved customer experience without any impact to our long term ability to keep delivering amazing new capabilities — that’s a win-win!

Welcome to the Dark Side

We know many of you have been asking for this, and we are excited to bring dark mode to all. Without the investment into our design system by many folks at Cloudflare, dark mode would not have seen the light of day. You can enable dark mode on the Appearance card in your user profile. You can give feedback to shape the future of the dark theme with the feedback form in the card.

If you find these types of problems interesting, come help us tackle them! We are hiring across product, design, and engineering!

Announcing Cloudflare R2 Storage: Rapid and Reliable Object Storage, minus the egress fees

Post Syndicated from Greg McKeon original https://blog.cloudflare.com/introducing-r2-object-storage/

Announcing Cloudflare R2 Storage: Rapid and Reliable Object Storage, minus the egress fees

Announcing Cloudflare R2 Storage: Rapid and Reliable Object Storage, minus the egress fees

We’re excited to announce Cloudflare R2 Storage! By giving developers the ability to store large amounts of unstructured data, we’re expanding what’s possible with Cloudflare while slashing the egress bandwidth fees associated with typical cloud storage services to zero.

Cloudflare R2 Storage includes full S3 API compatibility, working with existing tools and applications as built.

Let’s get into the R2 details.

R2 means “Really Requestable”

Object Storage, sometimes referred to as blob storage, stores arbitrarily large, unstructured files. Object storage is well suited to storing everything from media files or log files to application-specific metadata, all retrievable with consistent latency, high durability, and limitless capacity.

The most familiar API for Object Storage, and the API R2 implements, is Amazon’s Simple Storage Service (S3). When S3 launched in 2006, cloud storage services were a godsend for developers. It didn’t happen overnight, but over the last fifteen years, developers have embraced cloud storage and its promise of infinite storage space.

As transformative as cloud storage has been, a downside emerged: actually getting your data back. Over time, companies have amassed massive amounts of data on cloud provider networks. When they go to retrieve that data, they’re hit with massive egress fees that don’t correspond to any customer value — just a tax developers have grown accustomed to paying.

Enter R2.

Announcing Cloudflare R2 Storage: Rapid and Reliable Object Storage, minus the egress fees

Traditional object storage charges developers for three things: bandwidth, storage size and storage operations.

R2 builds on Cloudflare’s commitment to the Bandwidth Alliance, providing zero-cost egress for stored objects — no matter your request rate.  Egress bandwidth is often the largest charge for developers utilizing object storage and is also the hardest charge to predict.  Eliminating it is a huge win for open-access to data stored in the cloud.

That doesn’t mean we are shifting bandwidth costs elsewhere. Cloudflare R2 will be priced at $0.015 per GB of data stored per month — significantly cheaper than major incumbent providers.

Infrequent access to objects is often trivial for providers to support yet incurs the same per-operation charges. We don’t think it’s fair that typical object storage bills a developer making one request a second the same rate as an enterprise making thousands of requests a second — or frequently a higher rate when considering negotiated volume discounts.

On the flip side, providers designed for infrequent access typically can’t scale to heavy usage.

R2 will zero-rate infrequent storage operations under a threshold — currently planned to be in the single digit requests per second range. Above this range, R2 will charge significantly less per-operation than the major providers. Our object storage will be extremely inexpensive for infrequent access and yet capable of and cheaper than major incumbent providers at scale.

This cheaper price doesn’t come with reduced scalability. Behind the scenes, R2 automatically and intelligently manages the tiering of data to drive both performance at peak load and low-cost for infrequently requested objects.  We’ve gotten rid of complex, manual tiering policies in favor of what developers have always wanted out of object storage: limitless scale at the lowest possible cost.

R2 means “Repositioning Records”

Zero egress means you can get objects out easily, but what about putting objects in? Migrating data across cloud providers, even if they both support the complete S3 API, is error-prone and costly.

To make this easy for you, without requiring you to change any of your tooling, Cloudflare R2 will include automatic migration from other S3-compatible cloud storage services. Migrations are designed to be dead simple. After specifying an existing storage bucket, R2 will serve requests for objects from the existing bucket, egressing the object only once before copying and serving from R2. Our easy-to-use migrator will reduce egress costs from the second you turn it on in the Cloudflare dashboard.

Announcing Cloudflare R2 Storage: Rapid and Reliable Object Storage, minus the egress fees

Our vision for R2 includes multi-region storage that automatically replicates objects to the locations they’re frequently requested from. As with Durable Objects, we plan on introducing jurisdictional restrictions that allow developers to comply with complex data sovereignty requirements via a simple API.

R2 means “Ridiculously Reliable”

The core of what makes Object Storage great is reliability — we designed R2 for data durability and resilience at its core. R2 will provide 99.999999999% (eleven 9’s) of annual durability, which describes the likelihood of data loss. If you store 1,000,000 objects on R2, you can expect to lose one once every 100,000 years — the same level of durability as other major providers. R2 will be resistant to regional failures, replicating objects multiple times for high availability.

R2 is designed with redundancy across a large number of regions for reliability. We plan on starting from automatic global distribution and adding back region-specific controls for when data has to be stored locally, as described above.

R2 means “Radically Reprogrammable”

R2 is fully integrated with the Cloudflare Workers serverless runtime. You can bind a Worker to a specific bucket, dynamically transforming objects as they are written to or read from storage buckets. The deep integration between Workers and R2 makes building data pipelines and manipulating objects incredibly easy.

Cloudflare R2 is designed to easily integrate with the rest of Cloudflare’s products. As a few examples, our plan is to allow Durable Objects to be configured with R2 as a backup target, and provide automatic integration between R2 and Cloudflare cache to greatly extend cache lifetimes for infrequently changing objects.

What will you be able to build with Cloudflare R2?

There’s a lot you can do with long-term storage, especially with access to the Workers compute platform just alongside it.

For example, streaming data from a large number of IoT devices becomes a breeze with R2. Starting with a Worker to transform and manipulate the data, R2 can ingest large volumes of sensor data and store it at low cost. With no egress fees, it becomes simple to migrate volumes of data to multiple databases and analytics solutions as needed, dramatically reducing storage costs. With the ability to run a Worker on the outgoing data as well, the data pipeline itself is more flexible.

R2 is also a great place for CDN assets and large media files. For large files, R2 can significantly extend cache lifetimes while dramatically slashing egress bills. Combined with the Cache API and Workers, content can be dynamically cached for low-latency access around the globe.

More than anything, R2’s lack of egress bandwidth charges makes it ideal for storing content that’s accessed frequently. Today, R2 scales well to handle heavy request loads, dynamically tiering your objects to provide the best performance at the lowest cost. This dynamic tiering allows us to offer the lowest prices while supporting peak performance — with no user configuration required.

Accessing Cloudflare R2

R2 is currently under development — you can sign up here to join the waitlist for access. We’re excited to work with a number of earlier users to refine and test the product. We’ll be announcing an open beta where any user will be able to sign up for the service soon.

We’re excited to continue to build the product and push towards open beta, and we have big ideas for what the future of storage at Cloudflare’s edge could look like. If you’re a distributed systems engineer who wants to help us build the future of state at the edge, come work with us!

Registrar for Everyone

Post Syndicated from Eric Brown original https://blog.cloudflare.com/registrar-for-everyone/

Registrar for Everyone

Registrar for Everyone

Today, we are excited to announce that all Cloudflare customers now have full Registrar access, including the ability to register new domains.

Second, starting today — and over the course of the next few weeks — we will be introducing over 40 new top-level domains (TLDs). We’re starting with .uk, our most requested country code extension. Initially, customers will only be able to transfer in existing .uk domains from other registrars, but support for new registrations will become available within the next few weeks. In keeping with our at-cost model, .uk domains will be priced at the wholesale registry fee.

A short registrar primer

In the domain name world, there are two key players: registrars and registries. Understandably, the two are often confused. One way to look at it is that registries are the wholesalers and registrars are the retailers. Registries host the centralized database of registered domains within a TLD. They are responsible for establishing the policies and business rules for the TLD. They also set the wholesale price. Registrars sell domains to end users and manage those registrations on an ongoing basis. They set the retail fee, collect payment, provide customer support, and ensure registrations are renewed and kept up to date. They often provide complementary services such as DNS, web hosting, and email.

There are various “types” of registrars. Retail registrars primarily sell to SMBs and individuals. Corporate registrars typically provide services to large enterprises, and often offer brand protection and monitoring services. There are also registrars that focus on the reseller market, essentially enabling other companies to act as domain resellers.

Registrars typically interact with registries using a standard protocol called the Extensible Provisioning Protocol (EPP). While EPP is well-defined in various RFCs, each registry often has its own flavor and uses protocol extensions in support of their specific policies.

Where we started

Cloudflare has operated a registrar for many years. Initially, we became a registrar solely to manage and protect our own mission-critical domains. Over time, we began offering highly secure registration services to some of our customers as well. This evolved into our Custom Domain Protection service. This was a high-end niche service for customers with very specific needs. As we learned more about the registrar space, however, we wanted to expand this service to everyone. We believed that we could provide a highly secure, privacy-focused, and cost-effective registrar for everyone. So, in 2018 we announced the launch of Cloudflare Registrar.

There are two ways to have Cloudflare handle your domain registration: through the registration of a brand-new domain or through the transfer of an existing domain from another registrar. Unlike many new registrars starting from scratch, we had a large and sophisticated customer base. Our customers were already using our DNS services for domains they had registered through other registrars. So, we initially focused on helping them transfer existing domains to Cloudflare. At the time, we estimated that if our customers transferred all of their domains to us, they would collectively save over $50 million per year in registration fees.

And we’ve done just that. Since our launch in 2018, we have transferred in hundreds of thousands of domains. Collectively, it’s saved our customers millions of dollars in annual registration fees.

In 2020, new registrations were launched in beta. Access was first provided to our Biz, Pro, and Enterprise customers by default, and then over the following months we enabled several thousand additional customers who had previously expressed interest.

Transfers are not enough

Part of the reason why we launched our beta for new registrations was the excitement we saw around new domain registrations. Though we intentionally started only with domain transfers, folks began asking for new domain functionality almost immediately. We heard this initially from customers who hadn’t yet purchased a domain. Since they didn’t have anything to transfer in, they would have to go through the somewhat cumbersome process of registering a new domain with another provider and only then transfer their domain to Cloudflare.

As time went by, however, we began to hear requests from our existing Registrar customers.

After all, domain portfolios are not static. Companies, large and small, are continually updating their domain assets. Whether through the development of new products, expansion into new markets, M&A activities, or brand protection, the ability to register new domain names is vitally important. In Q2 of this year, there were 11.7 million new registrations in .com and .net alone. Cloudflare customers have registered over 2 million new domains through other registrars in the first half of this year alone. And these are just the ones we know about!

Today, we’re excited to open up new registrations to all of our customers.r. You no longer need to register new domains at another registrar and then transfer those domains to Cloudflare.

Registering a new domain

Registering a new domain is simple. Log into the Cloudflare dashboard and click Add a Site. In addition to adding an existing domain, you can now register a new one.

Registrar for Everyone

Start the registration process by entering the domain name or keyword into the search box, and we’ll provide a suggested list of available domains. After making your selection, you’ll need to select one of the plans (FREE is an option) and provide some basic information. Once you check out, we’ll create the zone and add the domain to your account. The entire process can be completed in less than a minute.

What about pricing? It’s important to note that our registrar pricing is “at-cost.” That means we charge our customers exactly what we pay the registry, plus any applicable ICANN transaction fees. In certain cases, the registry fees are in a currency other than US dollars. In those situations, we convert the price we charge our customer to USD based on the current exchange rate. As the exchange rate changes, we periodically update the USD price — but never more often than once per month.

Registrar for Everyone

It’s a big world

Beyond registering new domains, we’ve also recognized the need to expand the list of supported TLDs. Our customer base is already highly diverse and becoming even more so all the time. While .com is often considered the “king” of TLDs — especially within the U.S. — that’s not necessarily the case in other countries. Today, there are over 1,500 top-level domains. There are the legacy TLDs like .com, .net, and .org. There are also over 1,000 “new” TLDs such as .online, .live, and .cloud. There’s even .horse! And there are the country code TLDs, such as .uk, .in, and .au. In many areas of the world, the local country code TLD is much more popular than .com.

We believe we owe it to our customers to provide them with domains in the TLDs that work best for them. We have spent much of our effort in support of legacy and new TLDs. Now, we will be turning our focus towards supporting more country code TLDs.

What’s next

In the coming weeks, you can expect to see us add over 40 new extensions, including .us, .co, .me, and .tv. You can also check out the full list of TLDs we currently support and dates for upcoming launches here.

In the coming months we will be adding even more new extensions, with a focus on country-codes such as .de, .in, .ca, and .au to name just a few. We’re also planning to support premium (non-standard) priced domains, as well as Internationalized Domain Names (IDNs).

It’s just another step on the road to building a better — and more inclusive — Internet. To learn more about Cloudflare Registrar and how to use it, visit our developer documentation.

A Better Internet with UN Global Compact

Post Syndicated from Patrick Day original https://blog.cloudflare.com/cloudflare-and-un-global-compact/

A Better Internet with UN Global Compact

A Better Internet with UN Global Compact

Every year during Birthday Week, we talk about what we mean by our mission to help build a better Internet. We release support for new standards and products that help the global Internet community and give things like unmitigated DDoS Protection away for free. We also think about our role as an active participant in the global community of individuals, companies and governments that make the Internet what it is.

In 2020, we decided to formalize our commitment to being an active partner in the global community by joining the UN Global Compact (UNGC) as a signatory. We share the view that achievement of the Sustainable Development Goals set out in the UN Global Compact are the blueprint for a better and more sustainable future. Today, we are proud to release our first Communication on Progress, which describes how we are integrating UNGC principles across our company and as part of helping build a better Internet.

Shared values, economy, and Internet

In 1999, then UN Secretary General Kofi Annan shared a sober message with business leaders gathered at the World Economic Forum in Davos. He argued that basic protections like human rights, environmental sustainability, and fair labor practices are not just good for the world or good for business, they are fundamental to the long-term stability of a free and open global market.

Mr. Annan also warned that failure to ensure these basic protections could have dire political and economic consequences. Specifically, if governments, non-governmental organizations, and corporations could not translate the same shared values underlying national markets to the newly-created global market, then the global economy would remain fragile and vulnerable. He described how people feeling victimized would be subject to exploitation, including from “all the ‘isms’ of our post-cold-war world: protectionism; populism; nationalism; ethnic chauvinism; fanaticism; and terrorism,” which prey on misery and insecurity.

More than twenty years later, it’s difficult to find issue with Mr. Annan’s message. In fact, we think that human rights, environmental sustainability, fair labor practices, and anti-corruption are not only fundamental to the global economy, but to building a better Internet as well.

A Global Compact

The UN Global Compact (UNGC) is the world’s largest sustainability initiative with over 14,000 members in 162 countries. The UNGC’s mission is to mobilize companies to align their operations and strategies with UN principles and values.

Participants are required to make three commitments: operating responsibly by adhering to the UN Ten Principles, taking strategic action to help advance the UN Sustainable Development Goals (SDGs), and providing annual public reporting on implementation.

The Ten Principles

The UNGC’s first requirement is that companies operate consistent with fundamental responsibilities embodied in the UN Ten Principles, which include human rights, environmental sustainability, labor protections, and anti-corruption. The principles themselves are derived from a series of related UN treaties like the Universal Declaration of Human Rights, the ILO Fundamental Principles on the Rights at Work, the Rio Declaration on Environment and Development, and the UN Convention Against Corruption.

Sustainable Development Goals

The UNGC’s second requirement is for participants to help advance the UN Sustainable Development Goals (SDGs). The SDGs are an urgent call to action for global development that was adopted by all 193 UN member states in 2015. It builds off a number of previous UN development initiatives, including the Earth Summit in 1992, the Millennium Development Goals, the UN Sustainable Development Summit in 2015, and the Paris Agreement on Climate Change. Each of the 17 SDGs includes a broad goal combined with specific targets and indicators, as well as progress reports and other metrics.

Cloudflare is committed to helping advance all the 17 UN SDGs. However, like many companies, we’ve focused our efforts and our COP reporting on the SDGs that are most relevant to our business.

SDG 5 is focused on achieving gender equality and empowering all women and girls. This goal is particularly relevant right now, given the pandemic’s disproportionate impact on women in the workforce. We have long believed in the importance of encouraging a diverse workforce, and have benefited from partnerships with returnship programs that provide opportunities to mothers or people who have taken a career break to care for a loved one. This year, we’ve also taken steps to begin reporting on pay equity and have signed multiple diversity charters like the EU Charter and UK Tech Talent Charter. In conjunction with International Women’s Day, Cloudflare also hosted a full month of events and programs designed to foster community and support the growth and advancement of those who identify as women.

By offering free services to protect organizations around the world that empower women from denial for service attacks (DDoS) and other online threats, Cloudflare’s Project Galileo also helps advance the goal of gender equality. Through Project Galileo, we’ve been proud to work with organizations like the Women in Media Initiative Somalia (WIMISOM), which works to empower female journalists in Somalia, as well as serving at the forefront of campaigns to end violence against women, girls, and children.

SDG 13 is focused on taking urgent action to combat climate change and its impacts. Although Cloudflare has always had efficiency at our core, we are also committed to reducing our environmental impact and making the Internet as a whole more environmentally friendly. To reduce our greenhouse gas emissions, Cloudflare has committed to power its network by 100 percent renewable energy, which we achieved in 2020. We are also committed to removing or mitigating all of our historic greenhouse gas emissions associated with powering our network by 2025.

Earlier this year, Cloudflare also released new products to help our customers reach their own climate and emissions goals. For example, Cloudflare is directing computing workload to locations on its edge network that result in better climate outcomes, providing customers with real-time information on their individual emissions footprints, and providing developers with the option to build webpages on infrastructure powered by 100 percent renewable energy.

Moving Forward

As part of announcing what would ultimately become the UNGC, Secretary General Annan noted that the rise of transnational corporations had created unprecedented opportunities for private entities to move humanity forward. As Cloudflare celebrates another Birthday Week, we’re proud to share all the ways we are helping move toward a better Internet. And as always, we’re just getting started.

Easily creating and routing email addresses with Cloudflare Email Routing

Post Syndicated from Joao Sousa Botto original https://blog.cloudflare.com/introducing-email-routing/

Easily creating and routing email addresses with Cloudflare Email Routing

Easily creating and routing email addresses with Cloudflare Email Routing

Over four billion people — or half of the world’s population — have at least one email address, many of whom use it as an essential tool to stay on top of their personal and professional matters. More than 300 billion emails are sent and received every day, but seeing email as just a communications tool wouldn’t do it justice. Its impact in our lives goes far beyond being a vessel for messages — its use cases also cover being a common way of validating one’s identity online, and serving as the gateway for other communication platforms.

Today, most people use their email for sensitive purposes, such as logging in to their bank account, or communicating with governmental entities. At the same time, they will use that email to sign up for a 10% off coupon they found online, which will surely spam them for months to come. Despite these two use cases being polar opposites in relation to importance and security, people take the risk, usually for the sake of conveniently managing one account.

Much in the same way, businesses want to have different email addresses for different types of inquiries, such as sales and support, but often find it cumbersome to control who receives these emails. And as the business evolves, matters that were handled by the owner often need to be handed off to other people. But for small businesses it’s not usually straightforward to configure mailboxes and aliases.

And then there are countless individuals and families that juggle multiple mailboxes to handle the Internet identities that they use, to represent their various online presences.

We understand these challenges, and that’s why we’re launching Cloudflare Email Routing, the most straightforward way to create any number of email addresses that are redirected to the mailbox you, your family or your team are already using.

Cloudflare Email Routing

Cloudflare Email Routing is designed to simplify the way you create and manage email addresses, without needing to keep an eye on additional mailboxes.

The process is simple:

  1. You enter the email address you want to create on your domain
  2. You enter the email address you want it forwarded to
Easily creating and routing email addresses with Cloudflare Email Routing

As some of you already know, email consists of the envelope, the header, and its body.

The envelope is part of the SMTP transport protocol and tells the servers where the email is coming from and where it’s supposed to be delivered.

The headers contain structured information like the message traveled path, date, the sender and recipients’ addresses, subject, and other technical metadata like SPF pass results, DKIM signatures, and anti-spam scores. Every time the message goes through a server, it can add new headers or modify the existing ones until it reaches the final inbox.

And finally, there’s the body of the message, where the actual content resides. The body can be plain text, rich HTML, it can contain file attachments, and in some cases, it can be signed or even encrypted.

Here’s a simplified diagram of how the SMTP protocol works and how the three steps of an Email message fit together:

Easily creating and routing email addresses with Cloudflare Email Routing

Cloudflare Email Routing service acts as an intelligent router at the transport layer, handling and modifying the SMTP envelope to deliver the message at its final destination but preserving the original headers and keeping the body intact. This approach ensures that things like SPF, DKIM, and other security or anti-spam protocols don’t break and the recipient stays protected.

Furthermore, following the same privacy-first approach we use in other products, we don’t look into, queue, or store emails at any point. Messages are received, handled according to the configured rules, and delivered to their final destinations in real-time.

Private Beta access

Email Forwarding is now in private beta, and you can save your place in line through this sign-up form.

Then, to start using Email Routing, all you need to do is to add your domain to Cloudflare DNS.

If you don’t currently own a domain, you can buy one right here on our registrar.

Step-by-Step Configuration

Then there are only a few steps to creating a new email address and setting up forwarding:

  1. Go to the email page on the Cloudflare dashboard.
  2. Select Configure.
  3. Enter the email address you want to create (remember, this is for your domain, so you can pick anything you like). Alternatively, you can choose to use a catch-all so that all possible emails addresses in your domain are considered valid and forwarded.
  4. The DNS configuration step is automatic if you don’t have email configured for your domain. Otherwise, we provide straightforward guidance on how to best configure it for your needs.
  5. Lastly, you just need to validate that the destination email belongs to you. Super simple, and exactly the same as you’ve done a million times before.

We did say we made it straightforward!

With efficiency and simplicity in mind, we’re launching Email Routing with support for multiple rules and message forwarding to any upstream inbox of your choice.

However, we feel like the email scene has been long-dormant, and we have exciting new features coming soon that advantage of the Cloudflare platform, resources, and know-how.

We’re also listening. If you have questions, suggestions, or new ideas, share them in the community forum. We’ll be around.

To start using Cloudflare Email Routing just join the waitlist today through this form. We will be opening up this service to more users on a daily basis, and promise the short wait will be worth it!

Tackling Email Spoofing and Phishing

Post Syndicated from Hannes Gerhart original https://blog.cloudflare.com/tackling-email-spoofing/

Tackling Email Spoofing and Phishing

Tackling Email Spoofing and Phishing

Today we’re rolling out a new tool to tackle email spoofing and phishing and improve email deliverability: The new Email Security DNS Wizard can be used to create DNS records that prevent others from sending malicious emails on behalf of your domain. This new feature also warns users about insecure DNS configurations on their domain and shows recommendations on how to fix them. The feature will first be rolled out to users on the Free plan and over the next weeks be made available for Pro, Business and Enterprise customers, as well.

Tackling Email Spoofing and Phishing

Before we dive into what magic this wizard is capable of, let’s take a step back and take a look at the problem: email spoofing and phishing.

What is email spoofing and phishing?

Spoofing is the process of posing as someone else which can be used in order to gain some kind of illicit advantage. One example is domain spoofing where someone hosts a website like mycoolwebpaqe.xyz  to trick users of mycoolwebpage.xyz to provide sensitive information without knowing they landed on a false website. When looking at the address bar side by side in a browser, it’s very hard to spot the difference.

Tackling Email Spoofing and Phishing

Then, there is email spoofing. In order to understand how this works, let’s take a look at a Cloudflare product update email I received on my personal email address. With most email providers you can look at the full source of an email which contains a number of headers and of course the body of the email. 

Date: Thu, 23 Sep 2021 10:30:02 -0500 (CDT)
From: Cloudflare <[email protected]>
Reply-To: [email protected]
To: <my_personal_email_address>

Above you can see four headers of the email, when it was received, who it came from, who I should reply to, and my personal email address. The value of the From header is used to display the sender in my email program.

Tackling Email Spoofing and Phishing

When I receive an email as above, I automatically assume this email has been sent from Cloudflare. However, nobody is stopped from sending an email with a modified From header from their mail server. If my email provider is not performing the right checks, which we will cover later in this blog post, I could be tricked into believing that an email was sent from Cloudflare, but it actually was not.

Tackling Email Spoofing and Phishing

This brings us to the second attack type: phishing. Let’s say a malicious actor has successfully used email spoofing to send emails to your company’s customers that seem to originate from one of your corporate service emails. The content of these emails look exactly like a legitimate email from your company using the same styling and format. The email text could be an urgent message to update some account information including a hyperlink to the alleged web portal. If the receiving mail server of a user does not flag the email as spam or insecure origin, the user might click on the link which could execute malicious code or lead them to a spoofed domain asking for sensitive information.

According to the FBI’s 2020 Internet Crime Report, phishing was the most common cyber crime in 2020 with over 240,000 victims leading to a loss of over $50M. And the number of victims has more than doubled since 2019 and is almost ten times higher than in 2018.

Tackling Email Spoofing and Phishing

In order to understand how most phishing attacks are carried out, let’s take a closer look at the findings of the 2020 Verizon Data Breach Investigations Report. It lies out that phishing accounts for more than 80% of all “social actions”, another word for social engineering attacks, making it by far the most common type of such an attack. Furthermore, the report states that 96% of social actions are sent via email and only 3% through a website and 1% via phone or text.

Tackling Email Spoofing and Phishing

This clearly shows that email phishing is a serious problem causing Internet users a big headache. So let’s see what domain owners can do to stop bad actors from misusing their domain for email phishing.

How can DNS help prevent this?

Luckily, there are three anti-spoofing mechanisms already built into the Domain Name System (DNS):

  • Sender Policy Framework (SPF)
  • DomainKeys Identified Mail (DKIM)
  • Domain-based Message Authentication Reporting and Conformance (DMARC)

However, it is not trivial to configure them correctly, especially for someone less experienced. In case your configuration is too strict, legitimate emails will be dropped or marked as spam. And if you keep your configuration too relaxed, your domain might be misused for email spoofing.

Sender Policy Framework (SPF)

SPF is used to specify which IP addresses and domains are permitted to send email on behalf of your domain. An SPF policy is published as a TXT record on your domain, so everyone can access it via DNS. Let’s inspect the TXT record for cloudflare.com: 

cloudflare.com 	TXT	"v=spf1 ip4:199.15.212.0/22 ip4:173.245.48.0/20 include:_spf.google.com include:spf1.mcsv.net include:spf.mandrillapp.com include:mail.zendesk.com include:stspg-customer.com include:_spf.salesforce.com -all"

An SPF TXT record always needs to start with v=spf1. It usually contains a list of IP addresses of sending email servers using the ip4 or ip6 mechanism. The include mechanism is used to reference another SPF record on another domain. This is usually done if you are relying on other providers that need to send emails on our behalf. You can see a few examples in the SPF record of cloudflare.com above: we’re using Zendesk as customer support software and Mandrill for marketing and transactional emails.

Finally, there is the catch-all mechanism -all which specifies how all incoming, but unspecified emails should be treated The catch-all mechanism is preceded by a qualifier that can be either + (Allow), ~ (Softfail) or – (Fail). Using the Allow qualifier is not recommended as it basically makes the SPF record useless and allows all IP addresses and domains to send email on behalf of your domain. Softfail is interpreted differently by receiving mail servers, marking an email as Spam or insecure, depending on the server. Fail tells a server not to accept any emails originating from unspecified sources.

Tackling Email Spoofing and Phishing

The diagram above shows the steps taken to ensure a received email is SPF compliant.

  1. The email is sent from the IP address 203.0.113.10 and contains a From header with the value of [email protected].
  2. After receiving the email, the receiver queries the SPF record on mycoolwebpage.xyz to retrieve the SPF policy for this domain.
  3. The receiver checks if the sending IP address 203.0.113.10 is listed in the SPF record. If it is, the email succeeds the SPF check. If it is not, the qualifier of the all mechanism defines the outcome.

For a full list of all mechanisms and more details about SPF, refer to RFC7208.

DomainKeys Identified Mail (DKIM)

Okay, with SPF we’ve ensured that only permitted IP addresses and domains can send emails on behalf of cloudflare.com. But what if one of the IPs changes owner without us noticing and updating the SPF record? Or what if someone else who is also using Google’s email server on the same IP tries to spoof emails coming from cloudflare.com?

This is where DKIM comes in. DKIM provides a mechanism to cryptographically sign parts of an email — usually the body and certain headers — using public key encryption. Before we dive into how this works, let’s take a look at the DKIM record used for cloudflare.com:

google._domainkey.cloudflare.com.   TXT   "v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDMxbNxA2V84XMpZgzMgHHey3TQFvHkwlPF2a11Ex6PGD71Sp8elVMMCdZhPYqDlzbehg9aWVwPz0+n3oRD73o+JXoSswgUXPV82O8s8dGny/BAJklo0+y+Bh2Op4YPGhClT6mRO2i5Qiqo4vPCuc6GB34Fyx7yhreDNKY9BNMUtQIDAQAB"

The structure of a DKIM record is <selector>._domainkey.<domain>, where the selector is provided by your email provider. The content of the DKIM TXT record always starts with v=DKIM1 followed by the public key. We can see the public key type, referenced by the k tag, and the public key itself, preceded by the p tag.

Below is a simplified sequence how the signing and DKIM check work:

  1. The sending email server creates a hash from the email content.
  2. The sending email server encrypts this hash with the private DKIM key.
  3. The email is sent, containing the encrypted hash.
  4. The receiving email server retrieves the public key from the DKIM TXT record published on the email domain.
  5. The receiving email server decrypts the hash using the public DKIM key.
  6. The receiving email server generates the hash from the email content.
  7. If the decrypted hash and the generated hash match, email authenticity is proven. Otherwise, the DKIM check fails.

The full DKIM specification can be found in RFC4871 and RFC5672.

Domain-based Message Authentication Reporting and Conformance (DMARC)

Domain-based Message Authentication Reporting and Conformance, that’s definitely a mouthful. Let’s focus on two words: Reporting and Conformance. DMARC provides exactly that. Regular reports let the email sender know how many emails are non-conforming and potentially spoofed. Conformance helps provide a clear signal to the email receiver how to treat non-conforming emails. Email receivers might impose their own policies for emails that fail SPF or DKIM checks even without a DMARC record. However, the policy configured on the DMARC record is an explicit instruction by the email sender, so it increases the confidence for email receivers what to do with non-conforming emails.

Here is the DMARC record for cloudflare.com

_dmarc.cloudflare.com.   TXT   "v=DMARC1; p=reject; pct=100; rua=mailto:[email protected]"

The DMARC TXT record is always set on the _dmarc subdomain of the email domain and — similar to SPF and DKIM — the content needs to start with v=DMARC1. Then we see three additional tags:

The policy tag (p) indicates how email receivers should treat emails that fail SPF or DKIM checks. Possible values are none, reject, and quarantine. The none policy is also called monitoring only and allows emails failing the checks to still be accepted. By specifying quarantine, email receivers will put SPF or DKIM non-conforming emails in the Spam folder. With reject, emails are dropped and not delivered at all if they fail SPF or DKIM.

The percentage tag (pct) can be used to apply the specified policy to a subset of incoming emails. This is helpful if you’re just rolling out DMARC and want to make sure everything is configured correctly by testing on a subset.

The last tag we can find on the record is the reporting URI (rua). This is used to specify an email address that will receive aggregate reports (usually daily) about non-conforming emails.

Tackling Email Spoofing and Phishing

Above, we can see how DMARC works step by step.

  1. An email is sent with a From header containing [email protected].
  2. The receiver queries SPF, DKIM and DMARC records from the domain mycoolwebpage.xyz to retrieve the required policies and the DKIM public key.
  3. The receiver performs SPF and DKIM checks as outlined previously. If both succeed, the email is accepted and delivered to the inbox. If either SPF or DKIM check fails, the DMARC policy will be followed and determines if the email is still accepted, dropped or sent to the spam folder.
  4. Finally, the receiver sends back an aggregate report. Depending on the email specified in the rua tag this report could also be sent to a different email server which is responsible for that email address.

Other optional tags and the complete DMARC specification is described in RFC7489.

A few numbers on the current adoption

Now that we’ve learned what the problem is and how to tackle it using SPF, DKIM, and DMARC, let’s see how widely they’re adopted.

Dmarc.org has published the adoption of DMARC records of domains in a representative dataset. It shows that by the end of 2020, still less than 50% of domains even had a DMARC record. And remember, without a DMARC record there is no clear enforcement of SPF and DKIM checks. The study further shows that, of the domains that have a DMARC record, more than 65% are using the monitoring only policy (p=none), so there is a significant potential to drive this adoption higher. The study does not mention if these domains are sending or receiving emails, but even if they didn’t, a secure configuration should include a DMARC record (more about this later).

Tackling Email Spoofing and Phishing

Another report from August 1, 2021 tells a similar story for domains that belong to entities in the banking sector. Of 2,881 banking entities in the United States, only 44% have published a DMARC record on their domain. Of those that have a DMARC record, roughly 2 out of 5 have set the DMARC policy to None and another 8% are considered “Misconfigured”. Denmark has a very high adoption of DMARC on domains in the banking sector of 94%, in contrast to Japan where only 13% of domains are using DMARC. SPF adoption is on average significantly higher than DMARC, which might have to do with the fact that the SPF standard was first introduced as experimental RFC in 2006 and DMARC only became a standard in 2015.

Country Number of entities SPF present DMARC present
Denmark 53 91% 94%
UK 83 88% 76%
Canada 24 96% 67%
USA 2,881 91% 44%
Germany 39 74% 36%
Japan 90 82% 13%

This shows us there is quite some room for improvement.

Enter: Email Security DNS Wizard

So what are we doing to increase the adoption of SPF, DKIM, and DMARC and tackle email spoofing and phishing? Enter the new Cloudflare Email Security DNS Wizard.

Starting today, when you’re navigating to the DNS tab of the Cloudflare dashboard, you’ll see two new features:

  1. A new section called Email Security
Tackling Email Spoofing and Phishing
  1. New warnings about insecure configurations
Tackling Email Spoofing and Phishing

In order to start using the Email Security DNS Wizard, you can either directly click the link in the warning which brings you to the relevant section of the wizard or click Configure in the new Email Security section. The latter will show you the following available options:

Tackling Email Spoofing and Phishing

There are two scenarios. You’re using your domain to send email, or you don’t. If you do, you can configure SPF, DKIM, and DMARC records by following a few simple steps. Here you can see the steps for SPF:

If your domain is not sending email, there is an easy way to configure all three records with just one click:

Tackling Email Spoofing and Phishing

Once you click Submit, this will create all three records configured in such a way that all emails will fail the checks and be rejected by incoming email servers.

example.com			TXT	"v=spf1 -all"
*._domainkey.example.com.	TXT	"v=DKIM1; p="
_dmarc.example.com.		TXT	"v=DMARC1; p=reject; sp=reject; adkim=s; aspf=s;"

Help tackle email spoofing and phishing

Out you go and make sure your domain is secured against email spoofing and phishing. Just head over to the DNS tab in the Cloudflare dashboard or if you are not yet using Cloudflare DNS, sign up for free in just a few minutes on cloudflare.com.

If you want to read more about SPF, DKIM and DMARC, go check out our new learning pages about email related DNS records.

Cloudflare’s Annual Founders’ Letter

Post Syndicated from Matthew Prince original https://blog.cloudflare.com/cloudflares-annual-founders-letter-2021/

Cloudflare’s Annual Founders’ Letter

Cloudflare’s Annual Founders’ Letter

This week we celebrate Cloudflare’s birthday. We launched the company 11 years ago tomorrow: September 27, 2010. It has been our tradition, since our first birthday, to use this week to launch innovative new products that we think of as our gift back to the Internet.

Since going public, it’s also been an opportunity for us to update our Annual Founders’ Letter and share what’s on our mind. Recently we’ve been thinking about three things: team, the Internet, and innovation.

Team

When anyone asks us the key to Cloudflare’s success, we always say the same thing: the team we’ve been able to attract to help us achieve our mission of helping build a better Internet. In the last year we’ve had more than 250,000 people apply to work for us and extended offers to less than one half of one percent of them. We continue to attract great people.

It’s incredible to realize that more than half of Cloudflare’s team today started since March 13, 2020, when we closed all our physical offices due to the pandemic. In the last several months, as we’ve started to see a light at the end of the COVID tunnel, we’ve been hosting what we called Summer Socials with our team. Getting together outside, often over a picnic lunch, it’s been fun to meet face-to-face people we’d only video conferenced with before. And even more fun to watch people from across the team get to know each other outside the confines of a Brady Bunch-like on-screen box.

Cloudflare’s Annual Founders’ Letter

As a company that was very much a work-from-office culture before the pandemic, we were terrified of what would happen to our culture when we switched to fully remote work. Eighteen months into this forced experiment on a new way of working we’re happy to report: it’s working. Really well.

It turns out what we all suspected is in fact true. Culture has little to do with fun offices, plentiful snacks, or adjustable desks. Instead, for us, it starts with hiring people who are relentlessly curious and, at the same time, empathetic. Curious people want to learn. Empathetic people love to teach. And if you put a group of them together, whether in a swanky office or on Zoom, great things will happen.

As we come out the other side of COVID, we have an opportunity to help build a better way to work. It would be naive to insist that we go back to the way we did things before. We’ve been more productive, and on average our team has been happier in their jobs, than any time in the company’s history. At the same time, we know there can be considerable value in coming together in person to solve hard problems, brainstorm about the future, and build relationships that make the company stronger.

We don’t have all the answers on what the future of work looks like, but we’ve begun to formulate a place to start our experiments as people come back. We hope we can use the times we get together as ways to better collaborate and learn. But, at the same time, give our team the flexibility to work how and wherever they are the most productive.

The Internet

Cloudflare’s mission is to help build a better Internet. We always capitalize the I in Internet, in spite of what the AP style guide has said since 2016, because it’s a proper noun, we believe there is and only should be one, and we have an enduring respect for what a miracle it is that it exists.

Right around the same time that the AP started to say that you needn’t capitalize the I in Internet anymore, something seemed to change. The world shifted from seeing the Internet and what it enabled as an irreproachable good to a source of great danger.

We’ve watched the same thing. Since 2016 it’s often felt like a connection to the Internet only brings cyberattacks, toxic social media, threats to democracy, increasing polarization, and a declining disdainful discourse.

We have real challenges ahead as some of the technologies that ride on top of the Internet have broken down traditional gatekeepers without sufficient concern for addressing the harms they previously protected against. But, at the same time, the Internet itself remains a miracle.

A mere 11 years before Cloudflare’s founding, long distance phone calls still cost a fortune, sharing a photograph with someone in another country took weeks, and the idea that you could access the sum total of human knowledge from a device in your pocket was beyond even the fantasies of science fiction.

Cloudflare’s Annual Founders’ Letter

The last 18 months of the pandemic have reaffirmed our faith in the miracle that is the Internet. Imagine just how much worse it would have been had the pandemic happened just 11 years ago, let alone 22. The Internet allowed many of us to continue to work, connect with our loved ones, exercise our creativity, and stay connected to the world.

We’re proud of what we’ve done to live up to our mission and help build a better Internet during this time. And, as we come out the other side, we will continue to engage with policy makers to address the new harms an interconnected world has brought while preserving the miracle that is the Internet itself.

Innovation

The Internet may seem static, but it is not. 11 years ago, watching a video online was an exercise in frustration. Today, it seems almost automatic that you can push play on your TV and access nearly any movie ever made instantly. That’s possible because the Internet isn’t static; it gets better through innovation.

Cloudflare’s Annual Founders’ Letter

At Cloudflare, we’re optimized to catalyze exactly that innovation. It starts with our mission: to help build a better Internet. The word “help” is important, because we know we can’t do it alone. So, wherever we can, we work with others across the Internet ecosystem to push it forward and make it better.

Sometimes people outside the company are surprised by the products we build. In fact, predicting our roadmap is pretty easy. We look at all the steps that are required to load a web page, send an email, stream a video, login to a workstation, or anything else you do online and ask: can we make that more secure, more reliable, or faster?

What’s exciting is that the pace at which the Internet is getting better is accelerating. And, in turn, the pace at which we are able to launch innovative new products is accelerating along with it. As the Internet grows and acquires more capabilities, we believe we will continue to grow with it. An investment in Cloudflare is, fundamentally, we feel an investment in the Internet itself.

Cloudflare’s Annual Founders’ Letter

And so, this week, we have an incredible series of announcements that are designed to help build a better Internet. We’re entering a new area to close one of the last network security risks that we haven’t historically protected our customers from, driving down costs of core cloud services, pushing the boundary of our network to our customers’ doorsteps, and investing in new technologies that may someday disrupt the web as we know it today.

Thank you to our team, our customers, and our investors. Happy 11th birthday to Cloudflare. And, even as we pick up steam, we continue to believe: we’re just getting started.

Cloudflare’s Annual Founders’ Letter

Diversity and The Digital Divide: Thoughts From Tech Leaders

Post Syndicated from Jason Kincaid original https://blog.cloudflare.com/diversity-and-the-digital-divide-thoughts-from-tech-leaders/

Diversity and The Digital Divide: Thoughts From Tech Leaders

Leaders from across the tech industry and beyond recently joined us for Cloudflare’s Birthday Week, helping us celebrate Cloudflare’s 10th birthday. Many of them touched on the importance of diversity and making the Internet accessible to everyone.

Here are some of the highlights.

On the value of soliciting feedback

Selina Tobaccowala
Chief Digital Officer at Openfit, Co-Founder of Gixo
Former President & CTO of SurveyMonkey

Diversity and The Digital Divide: Thoughts From Tech Leaders

When you think about diversity and inclusion, unfortunately, it’s often only the loudest voice, the squeakiest wheel [who gets heard]. And what a survey allows you to do is let people’s voices be heard who are not always willing to raise their hand or speak the loudest.

So at SurveyMonkey, we always made sure that when we were thinking about user testing and we were thinking about usability testing — that it was that broad swath of the customer because you wanted people across all different segments to submit their opinion.

I think that collecting data in a way that can be anonymized, collecting data in a way that lets people have a thoughtful versus always off the cuff conversation is really important. And what we also provided was a benchmarking product, because if you don’t know how you rank and stack against other people, you don’t know if you’re doing well or not.

Watch the full interview

On the importance of immersion

Bonita Stewart
Vice President, Global Partnerships & Americas Partnerships Solutions of Google

Diversity and The Digital Divide: Thoughts From Tech Leaders

It’s been part of my mission to make sure that technology is introduced particularly into the African-American community, so that people see it as a viable career and not something that’s on a path that requires a different risk profile or certain level of education. It should be accessible. So one of the things that I did — I was doing some research and I found that close to 25% of the STEM grads come from historically Black colleges. And there are many education programs we [Google] work with, but there was never anything for the students to have an immersive experience.

And the thought was, what if we had Howard West at Google? So we had a partnership with Howard University, and worked with Dr. Frederick (President of Howard University) and said: what if your students could actually spend time in the valley so that they could have an immersive experience? So they brought their faculty, along with their students. And there was just an outpouring from Google of volunteers saying, “I’d love to teach the students, is there a role for me that I can play?”

And that was in 2017. Now we have over ten schools — historically Black colleges, as well as historically Hispanic colleges and universities.

Watch the full interview

On making the Internet accessible to those who can’t afford the expense

Erik Hersman
Co-founder and CEO of BRCK

Diversity and The Digital Divide: Thoughts From Tech Leaders

BRCK makes rugged, portable devices that provide free Wi-Fi access to areas throughout Kenya and Rwanda.

We install our devices in buses and public transportation in Kenya and Rwanda. We also put them in fixed locations across the two countries. And we have a platform on it that’s much like what you’d see at an airport, where you get you get a dashboard that pops up, you watch an ad, you do a survey, you do something to earn your time and get online — which in East Africa is really important because people have time, but they don’t have money.

And so if you want to hit this demographic and allow them to have equal access to that kind of global digital ecosystem that’s out there, that we all take part in, you need to find a way that they can do so without going into their wallet. And this is the only way we found that we could do that. And so we have businesses who end up paying us [to serve advertisements, surveys, and microwork tasks] and that’s what subsidizes that cost.

Watch the full interview

On defying expectations

Shellye Archambeau
Former CEO of MetricStream
Board member for Verizon, Okta, Nordstrom, and Roper Technologies

Diversity and The Digital Divide: Thoughts From Tech Leaders

When I first came to Silicon Valley, I was shocked. I was shocked because I’m thinking, OK, I’m going to Silicon Valley — the place with innovation, new ideas, creativity, et cetera — I just knew it had to be diverse and… [it wasn’t]. And so that part was really a shock. And you know, I’m sure some things were more challenging for me. I wasn’t in anybody else’s shoes, so I don’t know if it was easier for them, but…

I’ve been in tech my entire career so I always approach things the same way. I assume that people are going to think that I’m not quite capable. Not quite competent, not quite… Just that little — I know people are going to think that.

So I try to go in the same way each time. It’s like I have to prove myself both to the people who I’m working for and to the people who are working for me. And I’ve always found that using a servant leader approach is the most effective way. To really go in and focus on the team. If I can help the team be successful, then I will be successful. So that has worked for me over and over again.

Watch the full interview

On turning good intentions into results

Pam Kostka
CEO of All Raise

Diversity and The Digital Divide: Thoughts From Tech Leaders

Be intentional about expanding your networks. So get out there and meet a Black investor, get out there and meet a Black founder, get out there and meet a female founder, get out there and be intentional. Don’t sit in your chair. They’re not going to come to you. Somebody gave me a beautiful analogy once and said, “It’s like fishing in the forest. There are plenty of fish there over there in the lake.”

So if you’re fishing in the forest and you’re shocked and surprised to find that there’s no fish on your hook, well, get yourself over to the lake. And you’re going to have to get up out of your chair and walk over — especially if your company or your firm doesn’t look diverse, because it’s not welcoming. And so you have to be intentional about expanding your network.

And you’re not going to get there if you just think you’ll do it. You need to treat it like OKRs, you need to make it a strategic imperative. You need to tie executive compensation to it, and do what you need to do in order to keep the focus and make sure it is appropriately resourced.

Watch the full interview

*Quotes have been lightly edited for clarity and length.

Tech Leaders on the Future of Remote Work

Post Syndicated from Jason Kincaid original https://blog.cloudflare.com/tech-leaders-on-the-future-of-remote-work/

Tech Leaders on the Future of Remote Work

Dozens of top leaders and thinkers from the tech industry and beyond recently joined us for a series of fireside chats commemorating Cloudflare’s 10th birthday. Over the course of 24 hours of conversation, many of these leaders touched on how the workplace has evolved during the pandemic, and how these changes will endure into the future.

Here are some of the highlights.

On the competition for talent

Stewart Butterfield
Co-founder and CEO, Slack

Tech Leaders on the Future of Remote Work

The thing that I think people don’t appreciate or realize is that this is not a choice that companies are really going to make on an individual basis. I’ve heard a lot of leaders say, “we’re going back to the office after the summer.”

If we say we require you to be in the office five days a week and, you know, Twitter doesn’t, Salesforce doesn’t — and those offers are about equal — they’ll take those ones. I think we would also lose existing employees if they didn’t believe that they had the flexibility. Once you do that, it affects the market for talent. If half of the companies support distributed work or flexible hours and flexible time in the office, you can compensate for that, but I think you’ve got to pay a lot more or something like that because that optionality is valuable to people.

Watch the full interview

On the harnessing the benefits of remote work

Hayden Brown
President and CEO, Upwork

Tech Leaders on the Future of Remote Work

I think a lot of these things are here to stay. What’s fleeting is this idea that we have our children home from school and we don’t have a social system around child care and things like that, because that’s not sustainable.

What’s here to stay are really companies finding, and workers finding, a new balance. It’s not about, “let’s all lock ourselves in our homes forever.” This is about being very intentional. How can we be intentional to really recognize the benefits that a distributed, more work from home-oriented culture and set of practices can give workers and businesses?

Those benefits include some very powerful tools towards addressing some of the diversity challenges that all of our companies face, because it suddenly opens up pools of talent that we can tap into, outside of the places where we’ve traditionally hired, and we can tap into those people — and they’re not second class citizens, because they’re not the only ones working remotely while everyone else is back at the office.

Watch the full interview

On capturing the serendipity of in-person meetings

Brett Hautop
VP of Global Design + Build at LinkedIn

Tech Leaders on the Future of Remote Work

That might be the single hardest thing to figure out. Because the big decision that’s made right after the meeting, after you heard everything but you wanted to say it to one person and not everybody else. Or the thing that happens serendipitously on the way into a meeting, just because you’re talking about your weekends and then you remember something — that is really unfair to the people who are on the team (working remotely).

And unless you go back to technology like the telepresence person driving around, or each of us having our own drone in the office that follows people around serving as my ability to see — these creepy things — it’s really hard to recreate. So it’s about changing a cultural norm and getting people to be more thoughtful about how to include people who aren’t there, to go out of their way to include them. And that’s something that could take years for us to teach ourselves.

Watch the full interview

On securing a hybrid work environment

Chris Young
Former CEO, McAfee

Tech Leaders on the Future of Remote Work

We saw a huge rise in phishing attacks that were directly correlated to the move to work from home. Cyber attackers understand that all of a sudden you’ve got probably millions of workers across different organizations that are not supervised in the same way — new systems, new protocols for how they work. And they preyed upon that very quickly… there’s a whole litany of attacks that have been levied against the work from home model.

It’s prudent to make sure that if you’re going to have people working from home, that you take some steps to protect the home networking infrastructure because we could find ourselves in a situation where, if we don’t pay attention to that over the long run, you start to see an uptick of attackers going after the home networking infrastructure. We always know the attackers will find the path of least resistance. It’s like water on a roof: it will find the hole and go right there.

And I think it means a few things for us in a cybersecurity landscape. I think it’s going to continue to shift and put a premium on the identity based architecture. The zero trust model authentication is going to be key. It’s really the combination of: can I trust the user and can I trust the device in order to make a decision of do I trust this session? Do I trust this transaction?

Watch the full interview

On the opportunity for digital transformation

Bret Taylor
President and Chief Operating Officer at Salesforce

Tech Leaders on the Future of Remote Work

I hear across every industry that people aren’t going to come back to the office full time. Maybe they’ll come in a couple days a week. But that means our offices are probably going to be a little bit more for on-sites than they are for desks. And I think about: how does that change the shape of our employee engagement? And more importantly, how does it change the shape of our business models?

I think that the companies who were treating their digital initiatives as something sort of on the side are probably suffering right now. And there’s an urgency around these shifts now that is more powerful than ever before.

I think a lot of these trends will remain. And that’s where the opportunity is for great companies, whether it’s technology companies or other companies, who will lean into these changes and transform themselves. I think the ones that do will benefit from it. And I think there’s going to be a lot of business model disruption and technology disruption coming out of this.

Watch the full interview

*Quotes have been lightly edited for clarity and length.

Want to watch more interviews and catch up on all of the announcements from Cloudflare during Birthday Week? Visit Cloudflare Birthday Week 2020

Looking Ahead: Five Opportunities on The Horizon According to Tech Leaders

Post Syndicated from Jason Kincaid original https://blog.cloudflare.com/looking-ahead-five-opportunities-on-the-horizon-according-to-tech-leaders/

Looking Ahead: Five Opportunities on The Horizon According to Tech Leaders

Dozens of top leaders and thinkers from the tech industry and beyond recently joined us for a series of fireside chats commemorating Cloudflare’s 10th birthday. Over the course of 24 hours of conversation, these leaders shared their thoughts on everything from entrepreneurship to mental health — and how the Internet will continue to play a vital role.

Here are some of the highlights.

On the global opportunity for entrepreneurs

Anu Hariharan
Partner, Y Combinator’s Continuity Fund

Looking Ahead: Five Opportunities on The Horizon According to Tech Leaders

Fast forwarding ten years from now, I think entrepreneurship is global, and you’re already seeing signs of that. 27% of YC startups are headquartered outside the US. And I’m willing to bet that in a decade, at least 50% of YC startups will be headquartered outside the US. And so I think the sheer nature of the Internet democratizing information, more companies being global, like Facebook, Google, Uber — talent is everywhere. I think you will see multi-billion dollar companies coming out of other regions.

People have this perception that everything is a zero sum game, or that we are already at peak Internet penetration. Absolutely not. The global market cap is ~$85 trillion. Less than 10% is e-commerce. Internet enabled businesses is $8 trillion. So even if you play this out for another twenty years, Internet enabled businesses should be at least $66 trillion. So we have a lot more to go. And I think the zero sum game that investors tend to think of, what we’ve gotten wrong is — most of these Internet enabled businesses are expanding TAM.

Watch the full interview

On democratizing and normalizing mental health

Karan Singh
Co-founder and COO of Ginger

Looking Ahead: Five Opportunities on The Horizon According to Tech Leaders

Our vision is a world where mental health is never an obstacle, and that’s a never-ending vision. I don’t think that will be done in 10 years, but I am hopeful that in 10 years or even well before that, this whole new virtual-first sort of care paradigm can really start to take shape, where you start digitally and then progress to an in-person should you need it.

And for some people who are more acute, or in specific situations, they absolutely do need to see an in-person provider. But for many people, starting virtual — virtual being the default — feels like a more democratic and equitable experience in the world.

Watch the full interview

On leveling the playing field

Jennifer Hyman,
CEO and co-founder of Rent the Runway

Looking Ahead: Five Opportunities on The Horizon According to Tech Leaders

Where I’m optimistic is that I think that in a life post-vaccine, when kids are back in school, when things are a little bit more normal, businesses are no longer going to require their employees to come to work five days a week in the same way and in the same structure that existed in the past. We realize that because of technology, we can work more flexibly, we can work more virtually.

And I think that that is going to have unlocks for everyone, don’t get me wrong, but it’ll have huge unlocks for women who are often the ones making the sacrifice to spend more time with the kids, be at home, do all of the house-related leadership, so I think that this will be a great equalizer in many ways.

Watch the full interview

On expecting the unexpected

Eric Schmidt
Former CEO & Executive Chairman, Google
Co-Founder, Schmidt Futures

Looking Ahead: Five Opportunities on The Horizon According to Tech Leaders

It seems to me that the gains in machine learning and the investment that everyone, including Cloudflare, Google, et cetera, is putting in it — are going to yield a whole new set of applications.

We should expect more of the unexpected because of the level of investment. And so the people who sit there and say, oh, you know, it’s Apple and Google and Amazon and Microsoft and so forth, and it’s all done. They’re missing the narrative. The narrative is that there’s a new platform emerging which the big guys and the new guys, the new little guys are going to compete over. And that competition will generally be incredibly helpful. It will produce very significant large companies as they figure out a way to monetize. But more importantly, it’ll have an impact on society, both in terms of entertainment, as we saw with TikTok and its predecessors, but also in terms of information and productivity.

Watch the full interview

On the future of video conferencing

Eric Yuan
Founder and CEO of Zoom

Looking Ahead: Five Opportunities on The Horizon According to Tech Leaders

For now, if we all work from home, from a productivity perspective there’s no productive loss. However, social interaction is a problem. Mental health is another problem. The reason why, no matter how good we think it is now — it cannot deliver a better experience than a face-to-face meeting.

If I didn’t see you for a while, and I wanted to give you a big hug — you cannot feel my intimacy over Zoom, right? And if you are getting a cup of coffee, I can not enjoy the smell, not like when you and I are in a Starbucks.

I think that technology-wise, in the future with those cutting edge technologies, we should believe that videoconferencing like Zoom can deliver a better experience than a face-to-face meeting. I shake hands with you remotely, you can feel my hand-shaking. And even if you speak a different language, with AI, with real-time language translation — I think those technologies can truly help make sure that online communication is better than face to face meeting. In the next 10 or 15 years, I think we will get there with some technology.

Watch the full interview

Quotes have been lightly edited for clarity and length.