Zero Trust application security means that every request to an application is denied unless it passes a specific set of defined security policies. Most Zero Trust solutions allow the use of a user’s identity, device, and location as variables to define these security policies.
We heard from customers that they wanted more control and more customizability in defining their Zero Trust policies.
Starting today, we’re excited that Access policies can consider anythingbefore allowing a user access to an application. And by anything, we really do mean absolutely anything. You can now build infinitely customizable policies through the External Evaluation rule option, which allows you to call any API during the evaluation of an Access policy.
Why we built external evaluation rules
Over the past few years we added the ability to check location and device posture information in Access. However, there are always additional signals that can be considered depending on the application and specific requirements of an organization. We set out to give customers the ability to check whatever signal they require without any direct support in Access policies.
The Cloudflare security team, as an example, needed the ability to verify a user’s mTLS certificate against a registry to ensure applications can only be accessed by the right user from a corporate device. Originally, they considered using a Worker to check the user’s certificate after Access evaluated the request. However, this was going to take custom software development and maintenance over time. With External Evaluation rules, an API call can be made to verify whether a user is presenting the correct certificate for their device. The API call is made to a Worker that stores the mapping of mTLS certificates and user devices. The Worker executes the custom logic and then returns a true or false to Access.
How it works
Cloudflare Access is a reverse proxy in front of any web application. If a user has not yet authenticated, they will be presented with a login screen to authenticate. The user must meet the criteria defined in your Access policy. A typical policy would look something like:
The user’s email ends in @example.com
The user authenticated with a hardware based token
The user logged in from the United States
If the user passes the policy, they are granted a cookie that will give them access to the application until their session expires.
To evaluate the user on other custom criteria, you can add an external evaluation rule to the Access policy. The external evaluation rule requires two values: an API endpoint to call and a key to verify that any request response is coming from a trusted source.
After the user authenticates with your identity provider, all information about the user, device and location is passed to your external API. The API returns a pass or fail response to Access which will then either allow or deny access to the user.
Example logic for the API would look like this:
/**
* Where your business logic should go
* @param {*} claims
* @returns boolean
*/
async function externalEvaluation(claims) {
return claims.identity.email === '[email protected]'
}
Where the claims object contains all the information about the user, device and network making the request. This externalEvaluation function can be extended to perform any desired business logic. We have made an open-source repository available with example code for consuming the Access claims and verifying the signing keys from Access.
This is really powerful! Any Access policy can now be infinitely extended to consider any information before allowing a user access. Potential examples include:
Integrating with endpoint protection tools we don’t yet integrate with by building a middleware that checks the endpoint protection tool’s API.
Checking IP addresses against external threat feeds
Calling industry-specific user registries
And much more!
We’re just getting started with extending Access policies. In the future we’ll make it easier to programmatically decide how a user should be treated before accessing an application, not just allow or deny access.
This feature is available in the Cloudflare Zero Trust dashboard today. Follow this guide to get started!
Today, we’re excited to announce Cloudflare One Observability. Cloudflare One Observability will help customers work across Cloudflare One applications to troubleshoot network connectivity, security policies, and performance issues to ensure a consistent experience for employees everywhere. Cloudflare One, our comprehensive SASE platform, already includes visibility for individual products; Cloudflare One Observability is the next step in bringing data together across the Cloudflare One platform.
Network taps and legacy enterprise networks
Traditional enterprise networks operated like a castle protected by a moat. Employees working from a physical office location authenticated themselves at the beginning of their session, they were protected by an extensive office firewall, and the majority of the applications they accessed were on-premise.
Many enterprise networks had a strictly defined number of “entrances” for employees at office locations. Network taps (devices used to measure and report events on a local network) monitored each entrance point, and these devices gave network administrators and engineers complete visibility into their operations.
Incomplete observability in today’s enterprise network
Today’s enterprise networks have expanded beyond the traditional on-premise model and have become extremely fragmented. Now, employees can work from anywhere. People access enterprise networks from across the Internet, and the applications they use every day are a mix of on-premise and SaaS cloud instances.
SaaS applications are hosted outside the enterprise network, leaving your security teams with limited observability into how users access those applications and move data through them. Without observability on the applications your employees are using, you can’t control how sensitive data is stored, shared, or exposed to third parties.
Now that enterprise networks have become more fragmented, it is increasingly difficult to understand how the various fragments are operating. To even gain limited observability, you have to implement a disorganized combination of network taps, flow data, synthetic probes, and dashboards that fail to share data across one another.
Total observability across an enterprise & cloud network built on Cloudflare One
Cloudflare One Observability is built to solve today’s issue of network fragmentation in a zero trust world. Instead of having data spread across multiple network tools, Cloudflare One Observability will combine data from different Cloudflare One functions into a single experience. Customers will be able to go to one place to troubleshoot any issues they’re experiencing with their enterprise applications and networks.
In today’s world of fragmented enterprise networks, there are some questions that can be difficult to answer. Let’s break down a couple of customer examples and walk through how Cloudflare One Observability will simplify the troubleshooting process.
Troubleshooting bandwidth issues across branch locations
A customer may want to know, “What applications are using up the majority of my bandwidth across multiple office locations?” In a typical enterprise network, a network engineer would need to install a network tap or collect flow data at each office location, aggregate the information across separate networks, then build a custom tool to visualize the bandwidth data.
Instead, for Cloudflare One customers, Cloudflare will automatically do all the upfront data collection and aggregation. Customers will be able to skip straight to troubleshooting and solving their bandwidth problem by using Cloudflare One Observability to visualize bandwidth usage across office locations.
Identifying network vulnerabilities
Another challenging question that customers face is, “What attack trends are popular, and is my network vulnerable?” Assessing a network’s vulnerability is time-consuming as administrators dive into separate applications for VPNs, firewalls, user policies, and endpoints to understand their network’s security posture.
Cloudflare One is built from the ground up to simplify this problem. Observability is straightforward when your network on-ramps, firewalls, user policies, and endpoint protection are all managed within the same platform. Customers will be able to go to the Cloudflare One Observability experience to see security patches that are automatically applied by Cloudflare so that customers don’t have to worry. Cloudflare One lets you know whether you’ve been targeted by an attack and gives you confidence that you’re protected.
Troubleshooting slow network performance
Many people have experienced logging into a slow enterprise network. The general problem of “my network is slow when I access an on-premise or SaaS application” can be tough to solve. If employees are working remotely, a network engineer would need to dig through different applications to troubleshoot latency and jitter between VPNs, firewalls, user policies, and endpoint connections.
Cloudflare One Observability simplifies this time-consuming troubleshooting process. When your on-ramps, firewalls, user policies, and endpoint monitoring are all configured on the same platform, you only need to go to one place to troubleshoot these network functions. Cloudflare One’s architecture is built on the concept of single pass inspection. When a request lands on a Cloudflare server, that request passes through instances of Cloudflare One services all on that same single server. This makes it easy to visualize end-to-end network request handling, so customers can seamlessly analyze traffic and identify a network bottleneck or misconfiguration.
Observability powered by Cloudflare’s network
Cloudflare One Observability is built on Cloudflare’s best-in-class network. We have data centers in 270+ cities and over 100 countries. Since every Cloudflare One product runs on every server, we can provide an unparalleled fast and consistent experience to customers everywhere. Cloudflare built its network and security applications from the ground up on the same infrastructure. Unlike our competitors that have strung together a zero trust platform by building siloed applications or through acquisitions, Cloudflare One applications are seamlessly integrated and designed from day one to share data between one another.
As our applications are all built on the same infrastructure, so are our data pipelines and logging services. When you use Cloudflare One, you get the full benefits of our advanced data tools, like Instant Logs for delivering live network data as it arrives and ABR for analyzing network data at scale.
Delivering the Zero Trust observability customers need today
Since 2009, Cloudflare has built one of the fastest, most reliable, and most secure networks in the world. We’ve built Cloudflare One and Cloudflare One Observability on top of this network, and we’re extending its power to meet the challenges of any company. The move to Zero Trust is a paradigm shift, and we believe the security benefits of this new paradigm will make it inevitable for every company. We’re proud of how we have helped and continue to help existing and new customers reinvent their corporate networks.
Construction of Cloudflare One Observability is still in progress. If you’re excited about this new product, you can sign up for our wait list now!
In an ideal world, intrusion detection would apply across your entire network – data centers, cloud properties, and branch locations. It wouldn’t impact the performance of your traffic. And there’d be no capacity constraints. Today, we’re excited to bring this one step closer to reality by announcing the private beta of Cloudflare’s intrusion detection capabilities: live monitoring for threats across all of your network traffic, delivered as-a-service — with none of the constraints of legacy hardware approaches.
Cloudflare’s Network Services, part of Cloudflare One, help you connect and secure your entire corporate network — data center, cloud, or hybrid — from DDoS attacks and other malicious traffic. You can apply Firewall rules to keep unwanted traffic out or enforce a positive security model, and integrate custom or managed IP lists into your firewall policies to block traffic associated with known malware, bots, or anonymizers. Our new Intrusion Detection System (IDS) capabilities expand on these critical security controls by actively monitoring for a wide range of known threat signatures in your traffic.
What is an IDS?
Intrusion Detection Systems are traditionally deployed as standalone appliances but often incorporated as features in more modern or higher end firewalls. They expand the security coverage of traditional firewalls – which focus on blocking traffic you know you don’t want in your network – to analyze traffic against a broader threat database, detecting a variety of sophisticated attacks such as ransomware, data exfiltration, and network scanning based on signatures or “fingerprints” in network traffic. Many IDSs also incorporate anomaly detection, monitoring activity against a baseline to identify unexpected traffic patterns that could indicate malicious activity. (If you’re interested in the evolution of network firewall capabilities, we recommend this where we’ve dived deeper on the topic).
What problems have users encountered with existing IDS solutions?
We’ve interviewed tons of customers about their experiences deploying IDS and the pain points they’re hoping we can solve. Customers have mentioned the full list of historical problems we hear frequently with other hardware-based security solutions, including capacity planning, location planning and back hauling traffic through a central location for monitoring, downtime for installation, maintenance, and upgrades, and vulnerability to congestion or failure with large volumes of traffic (e.g. DDoS attacks).
Customers we talked to also consistently cited challenges making trade off decisions between security and performance for their network traffic. One network engineer explained:
“I know my security team hates me for this, but I can’t let them enable the IDS function on our on-prem firewalls – in the tests my team ran, it cut my throughput by almost a third. I know we have this gap in our security now, and we’re looking for an alternative way to get IDS coverage for our traffic, but I can’t justify slowing down the network for everyone in order to catch some theoretical bad traffic.”
Finally, customers who did choose to take the performance hit and invest in an IDS appliance reported that they often mute or ignore the feed of alerts coming into their SOC after turning it on. With the amount of noise on the Internet and the potential risk of missing an important signal, IDSs can end up generating a lot of false positives or non-actionable notifications. This volume can lead busy SOC teams to get alert fatigue and end up silencing potentially important signals buried in the noise.
How is Cloudflare tackling these problems?
We believe there’s a more elegant, efficient, and effective way to monitor all of your network traffic for threats without introducing performance bottlenecks or burning your team out with non-actionable alerts. Over the past year and a half, we’ve learned from your feedback, experimented with different technology approaches, and developed a solution to take those tough trade off decisions out of the picture.
One interface across all your traffic
Cloudflare’s IDS capabilities operate across all of your network traffic – any IP port or protocol — whether it flows to your IPs that we advertise on your behalf, IPs we lease to you, or soon, traffic within your private network. You can enforce consistent monitoring and security control across your entire network in one place.
No more hardware headaches
Like all of our security functions, we built our IDS from scratch in software, and it is deployed across every server on Cloudflare’s global Anycast network. This means:
No more capacity planning: Cloudflare’s entire global network capacity is now the capacity of your IDS – currently 142 Tbps and counting.
No more location planning: No more picking regions, backhauling traffic to central locations, or deploying primary and backup appliances – because every server runs our IDS software and traffic is automatically attracted to the closest network location to its source, redundancy and failover are built in.
No maintenance downtime: Improvements to Cloudflare’s IDS capabilities, like all of our products, are deployed continuously across our global network.
Threat intelligence from across our interconnected global network
The attack landscape is constantly evolving, and you need an IDS that stays ahead of it. Because Cloudflare’s IDS is delivered in software we wrote from the ground up and maintain, we’re able to continuously feed threat intelligence from the 20+ million Internet properties on Cloudflare back into our policies, keeping you protected from both known and new attack patterns.
Our threat intelligence combines open-source feeds that are maintained and trusted by the security community – like Suricata threat signatures – with information collected from our unique vantage point as an incredibly interconnected network carrying a significant percentage of all Internet traffic. Not only do we share these insights publicly through tools like Cloudflare Radar; we also feed them back into our security tools including IDS so that our customers are protected as quickly as possible from emerging threats. Cloudflare’s newly announced Threat Intel team will augment these capabilities even further, applying additional expertise to understanding and deriving insights from our network data.
Excited to get started?
If you’re an Advanced Magic Firewall customer, you can get access to these features in private beta starting now. You can reach out to your account team to learn more or get started now – we can’t wait to hear your feedback as we continue to develop these capabilities!
Meet our new threat operations and research team: Cloudforce One. While this team will publish research, that’s not its reason for being. Its primary objective: track and disrupt threat actors.
The security teams we speak with tell us the same thing: they’re inundated with reports from threat intelligence and security product vendors that do little to improve their actual security. The stories are indeed interesting, but they want deeper insights into the techniques and actors targeting their industry—but even more than that, they want to be protected against these threats with minimal to no involvement. That is the mission on which Cloudforce One will deliver.
This team is led by me, Blake Darché, Area 1’s co-founder and former head of Threat Intelligence. Before starting Area 1, which was acquired by Cloudflare earlier this year, I was a founding member of CrowdStrike’s services organization, and before that a Computer Network Exploitation Analyst at the National Security Agency (NSA). My career has focused on identifying and disrupting sophisticated nation-state sponsored cyber threats before they compromise enterprises and governments, and I’m excited to accelerate that work at Cloudflare.
The Cloudforce One team comprises analysts assigned to Threat Research, Malware and Vulnerability Research, and Threat Operations (i.e., disrupting actors once identified). Collectively, members of the team have tracked many of the most sophisticated cyber criminals on the Internet while at the National Security Agency and Area 1 Security, and have worked closely with similar organizations and governments to disrupt these threat actors. They’ve also been prolific in publishing “finished intel” reports on security topics of significant geopolitical importance, such as targeted attacks against governments, technology companies, the energy sector, and law firms, and have regularly briefed top organizations around the world on their efforts. Oh, and we’re growing the team, so please reach out if you’re interested.
How will Cloudforce One work?
First and foremost, the team will help protect all Cloudflare customers by working closely with our existing product, engineering, and security teams to improve our products based on tactics, techniques, and procedures (TTPs) observed in the wild. Customers will get better protection without having to take any action, and will be able to read a subset of research published on our blog and within the Cloudflare Security Center.
Additionally, enterprise customers who wish to receive one-on-one live briefings from the team, submit periodic inquiries for follow-up, and obtain early access to threat research, will soon be able to sign up for our new Threat Intelligence subscription. All other enterprise customers will be invited to join periodic group briefings.
Lastly, new capabilities within Security Center, such as access to historical threat data via API and threat pivoting features, will also be introduced by the dedicated threat intel engineering team paired with Cloudforce One.
Getting started
If you’re looking to benefit from the insights uncovered by Cloudforce One, there is nothing you need to do. But if you’re interested in receiving regular briefings from Cloudforce One tailored to your industry, contact your Customer Success manager today or fill out this form and someone will be in touch. Finally, if you’re interested in joining the team, check out our open job postings here.
Today, June 21, 2022, Cloudflare suffered an outage that affected traffic in 19 of our data centers. Unfortunately, these 19 locations handle a significant proportion of our global traffic. This outage was caused by a change that was part of a long-running project to increase resilience in our busiest locations. A change to the network configuration in those locations caused an outage which started at 06:27 UTC. At 06:58 UTC the first data center was brought back online and by 07:42 UTC all data centers were online and working correctly.
Depending on your location in the world you may have been unable to access websites and services that rely on Cloudflare. In other locations, Cloudflare continued to operate normally.
We are very sorry for this outage. This was our error and not the result of an attack or malicious activity.
Background
Over the last 18 months, Cloudflare has been working to convert all of our busiest locations to a more flexible and resilient architecture. In this time, we’ve converted 19 of our data centers to this architecture, internally called Multi-Colo PoP (MCP): Amsterdam, Atlanta, Ashburn, Chicago, Frankfurt, London, Los Angeles, Madrid, Manchester, Miami, Milan, Mumbai, Newark, Osaka, São Paulo, San Jose, Singapore, Sydney, Tokyo.
A critical part of this new architecture, which is designed as a Clos network, is an added layer of routing that creates a mesh of connections. This mesh allows us to easily disable and enable parts of the internal network in a data center for maintenance or to deal with a problem. This layer is represented by the spines in the following diagram.
This new architecture has provided us with significant reliability improvements, as well as allowing us to run maintenance in these locations without disrupting customer traffic. As these locations also carry a significant proportion of the Cloudflare traffic, any problem here can have a very wide impact, and unfortunately, that’s what happened today.
Incident timeline and impact
In order to be reachable on the Internet, networks like Cloudflare make use of a protocol called BGP. As part of this protocol, operators define policies which decide which prefixes (a collection of adjacent IP addresses) are advertised to peers (the other networks they connect to), or accepted from peers.
These policies have individual components, which are evaluated sequentially. The end result is that any given prefixes will either be advertised or not advertised. A change in policy can mean a previously advertised prefix is no longer advertised, known as being “withdrawn”, and those IP addresses will no longer be reachable on the Internet.
While deploying a change to our prefix advertisement policies, a re-ordering of terms caused us to withdraw a critical subset of prefixes.
Due to this withdrawal, Cloudflare engineers experienced added difficulty in reaching the affected locations to revert the problematic change. We have backup procedures for handling such an event and used them to take control of the affected locations.
03:56 UTC: We deploy the change to our first location. None of our locations are impacted by the change, as these are using our older architecture. 06:17: The change is deployed to our busiest locations, but not the locations with the MCP architecture. 06:27: The rollout reached the MCP-enabled locations, and the change is deployed to our spines. This is when the incident started, as this swiftly took these 19 locations offline. 06:32: Internal Cloudflare incident declared. 06:51: First change made on a router to verify the root cause. 06:58: Root cause found and understood. Work begins to revert the problematic change. 07:42: The last of the reverts has been completed. This was delayed as network engineers walked over each other’s changes, reverting the previous reverts, causing the problem to re-appear sporadically. 09:00: Incident closed.
The criticality of these data centers can clearly be seen in the volume of successful HTTP requests we handled globally:
Even though these locations are only 4% of our total network, the outage impacted 50% of total requests. The same can be seen in our egress bandwidth:
Technical description of the error and how it happened
As part of our continued effort to standardize our infrastructure configuration, we were rolling out a change to standardize the BGP communities we attach to a subset of the prefixes we advertise. Specifically, we were adding informational communities to our site-local prefixes.
These prefixes allow our metals to communicate with each other, as well as connect to customer origins. As part of the change procedure at Cloudflare, a Change Request ticket was created, which includes a dry-run of the change, as well as a stepped rollout procedure. Before it was allowed to go out, it was also peer reviewed by multiple engineers. Unfortunately, in this case, the steps weren’t small enough to catch the error before it hit all of our spines.
The change looked like this on one of the routers:
[edit policy-options policy-statement 4-COGENT-TRANSIT-OUT term ADV-SITELOCAL then]
+ community add STATIC-ROUTE;
+ community add SITE-LOCAL-ROUTE;
+ community add TLL01;
+ community add EUROPE;
[edit policy-options policy-statement 4-PUBLIC-PEER-ANYCAST-OUT term ADV-SITELOCAL then]
+ community add STATIC-ROUTE;
+ community add SITE-LOCAL-ROUTE;
+ community add TLL01;
+ community add EUROPE;
[edit policy-options policy-statement 6-COGENT-TRANSIT-OUT term ADV-SITELOCAL then]
+ community add STATIC-ROUTE;
+ community add SITE-LOCAL-ROUTE;
+ community add TLL01;
+ community add EUROPE;
[edit policy-options policy-statement 6-PUBLIC-PEER-ANYCAST-OUT term ADV-SITELOCAL then]
+ community add STATIC-ROUTE;
+ community add SITE-LOCAL-ROUTE;
+ community add TLL01;
+ community add EUROPE;
This was harmless, and just added some additional information to these prefix advertisements. The change on the spines was the following:
[edit policy-options policy-statement AGGREGATES-OUT]
term 6-DISABLED_PREFIXES { ... }
! term 6-ADV-TRAFFIC-PREDICTOR { ... }
! term 4-ADV-TRAFFIC-PREDICTOR { ... }
! term ADV-FREE { ... }
! term ADV-PRO { ... }
! term ADV-BIZ { ... }
! term ADV-ENT { ... }
! term ADV-DNS { ... }
! term REJECT-THE-REST { ... }
! term 4-ADV-SITE-LOCALS { ... }
! term 6-ADV-SITE-LOCALS { ... }
[edit policy-options policy-statement AGGREGATES-OUT term 4-ADV-SITE-LOCALS then]
community delete NO-EXPORT { ... }
+ community add STATIC-ROUTE;
+ community add SITE-LOCAL-ROUTE;
+ community add AMS07;
+ community add EUROPE;
[edit policy-options policy-statement AGGREGATES-OUT term 6-ADV-SITE-LOCALS then]
community delete NO-EXPORT { ... }
+ community add STATIC-ROUTE;
+ community add SITE-LOCAL-ROUTE;
+ community add AMS07;
+ community add EUROPE;
An initial glance at this diff might give the impression that this change is identical, but unfortunately, that’s not the case. If we focus on one part of the diff, it might become clear why:
! term REJECT-THE-REST { ... }
! term 4-ADV-SITE-LOCALS { ... }
! term 6-ADV-SITE-LOCALS { ... }
In this diff format, the exclamation marks in front of the terms indicate a re-ordering of the terms. In this case, multiple terms moved up, and two terms were added to the bottom. Specifically, the 4-ADV-SITE-LOCALS and 6-ADV-SITE-LOCALS terms moved from the top to the bottom. These terms were now behind the REJECT-THE-REST term, and as might be clear from the name, this term is an explicit reject:
term REJECT-THE-REST {
then reject;
}
As this term is now before the site-local terms, we immediately stopped advertising our site-local prefixes, removing our direct access to all the impacted locations, as well as removing the ability of our servers to reach origin servers.
On top of the inability to contact origins, the removal of these site-local prefixes also caused our internal load balancing system Multimog (a variation of our Unimog load-balancer) to stop working, as it could no longer forward requests between the servers in our MCPs. This meant that our smaller compute clusters in an MCP received the same amount of traffic as our largest clusters, causing the smaller ones to overload.
Remediation and follow-up steps
This incident had widespread impact, and we take availability very seriously. We have identified several areas of improvement and will continue to work on uncovering any other gaps that could cause a recurrence.
Here is what we are working on immediately:
Process: While the MCP program was designed to improve availability, a procedural gap in how we updated these data centers ultimately caused a broader impact in MCP locations specifically. While we did use a stagger procedure for this change, the stagger policy did not include an MCP data center until the final step. Change procedures and automation need to include MCP-specific test and deploy procedures to ensure there are no unintended consequences.
Architecture: The incorrect router configuration prevented the proper routes from being announced, preventing traffic from flowing properly to our infrastructure. Ultimately the policy statement that caused the incorrect routing advertisement will be redesigned to prevent an unintentional incorrect ordering.
Automation: There are several opportunities in our automation suite that would mitigate some or all of the impact seen from this event. Primarily, we will be concentrating on automation improvements that enforce an improved stagger policy for rollouts of network configuration and provide an automated “commit-confirm” rollback. The former enhancement would have significantly lessened the overall impact, and the latter would have greatly reduced the Time-to-Resolve during the incident.
Conclusion
Although Cloudflare has invested significantly in our MCP design to improve service availability, we clearly fell short of our customer expectations with this very painful incident. We are deeply sorry for the disruption to our customers and to all the users who were unable to access Internet properties during the outage. We have already started working on the changes outlined above and will continue our diligence to ensure this cannot happen again.
Two bills attempting to reduce the power of Internet monopolies are currently being debated in Congress: S. 2992, the American Innovation and Choice Online Act; and S. 2710, the Open App Markets Act. Reducing the power to tech monopolies would do more to “fix” the Internet than any other single action, and I am generally in favor of them both. (The Center for American Progress wrote a good summary and evaluation of them. I have written in support of the bill that would force Google and Apple to give up their monopolies on their phone app stores.)
There is a significant problem, though. Both bills have provisions that could be used to break end-to-end encryption.
Let’s start with S. 2992. Sec. 3(c)(7)(A)(iii) would allow a company to deny access to apps installed by users, where those app makers “have been identified [by the Federal Government] as national security, intelligence, or law enforcement risks.” That language is far too broad. It would allow Apple to deny access to an encryption service provider that provides encrypted cloud backups to the cloud (which Apple does not currently offer). All Apple would need to do is point to any number of FBI materials decrying the security risks with “warrant proof encryption.”
Sec. 3(c)(7)(A)(vi) states that there shall be no liability for a platform “solely” because it offers “end-to-end encryption.” This language is too narrow. The word “solely” suggests that offering end-to-end encryption could be a factor in determining liability, provided that it is not the only reason. This is very similar to one of the problems with the encryption carve-out in the EARN IT Act. The section also doesn’t mention any other important privacy-protective features and policies, which also shouldn’t be the basis for creating liability for a covered platform under Sec. 3(a).
In Sec. 2(a)(2), the definition of business user excludes any person who “is a clear national security risk.” This term is undefined, and as such far too broad. It can easily be interpreted to cover any company that offers an end-to-end encrypted alternative, or a service offered in a country whose privacy laws forbid disclosing data in response to US court-ordered surveillance. Again, the FBI’s repeated statements about end-to-end encryption could serve as support.
Finally, under Sec. 3(b)(2)(B), platforms have an affirmative defense for conduct that would otherwise violate the Act if they do so in order to “protect safety, user privacy, the security of nonpublic data, or the security of the covered platform.” This language is too vague, and could be used to deny users the ability to use competing services that offer better security/privacy than the incumbent platform—particularly where the platform offers subpar security in the name of “public safety.” For example, today Apple only offers unencrypted iCloud backups, which it can then turn over governments who claim this is necessary for “public safety.” Apple can raise this defense to justify its blocking third-party services from offering competing, end-to-end encrypted backups of iMessage and other sensitive data stored on an iPhone.
S. 2710 has similar problems. Sec 7. (6)(B) contains language specifying that the bill does not “require a covered company to interoperate or share data with persons or business users that…have been identified by the Federal Government as national security, intelligence, or law enforcement risks.” This would mean that Apple could ignore the prohibition against private APIs, and deny access to otherwise private APIs, for developers of encryption products that have been publicly identified by the FBI. That is, end-to-end encryption products.
I want those bills to pass, but I want those provisions cleared up so we don’t lose strong end-to-end encryption in our attempt to reign in the tech monopolies.
EDITED TO ADD (6/23): A few DC insiders have responded to me about this post. Their basic point is this: “Your threat model is wrong. The big tech companies can already break end-to-end encryption if they want. They don’t need any help, and this bill doesn’t give the FBI any new leverage they don’t already have. This bill doesn’t make anything any worse than it is today.” That’s a reasonable response. These bills are definitely a net positive for humanity.
Някакъв си Ами отговаря или е отговарял за тази граница. Ами, Ами (Размиг Чакърян – Ами – бел. ред.) в момента не отговаря за тази граница. Това заяви министър-председателят Кирил…
This
Meta blog post by Johannes Weiner and Dan Schatzberg describes a set of
memory-management changes used there that they call “transparent memory
offloading”.
Transparent Memory Offloading (TMO) is Meta’s solution for
heterogeneous data center environments. It introduces a new Linux
kernel mechanism that measures the lost work due to resource
shortage across CPU, memory, and I/O in real time. Guided by this
information and without any prior application knowledge, TMO
automatically adjusts the amount of memory to offload to a
heterogeneous device, such as compressed memory or an SSD.
The article doesn’t say where to find the relevant code, not all of which
is in the mainline kernel (and some of which runs in user space).
I recently had cause to reflect on the changes to the NFS (Network File
System) protocol over the years and found that it was a story worth
telling. It would be easy for such a story to become swamped by the
details, as there are many of those, but one idea does stand out from
the rest. The earliest version of NFS has been described as a
“stateless” protocol, a term I still hear used occasionally. Much of
the story of NFS follows the growth in the acknowledgment of, and
support for, state. This article looks at the evolution of NFS (and its
handling of state) during the
early part of its life; a second installment will bring the story up to the
present.
If you haven’t heard about Amazon Connect, it is an easy-to-use cloud contact center service that helps companies of any size deliver superior customer service at lower cost. You can read the original blog post Jeff wrote at launch in 2017, with amazing Lego art 🙂
Contact centers not only receive calls and communications, but they also send outbound communications to customers. There are a variety of reasons to send outbound communication: appointment reminders, telemarketing, subscription renewals, and billing reminders. The vast majority of these communications are phone calls, and in many contact centers, agents make the calls manually using customer contact lists in external systems. Since customers only answer about ten percent of calls, these agents can spend nearly half of their time dialing and waiting. This can result in millions of dollars in lost productivity each year for a contact center with as few as 200 agents.
To help you to address this challenge, today we are adding to Amazon Connect outbound campaigns a set of high-volume outbound communication capabilities that allows you to proactively reach more of your customers across voice, SMS, and email. When using this capability, you will have a scalable way for proactive outreach for hundreds to millions of your customers, and you will increase your agents’ productivity and lower your operational costs.
Amazon Connect outbound campaigns delivers a predictive phone dialer. The dialer includes an answering machine detection system powered by machine learning. It allows the automatic detection of answering machines for voice calls and passes calls to agents only when the call is answered by a human. The dialer also adjusts the call rate depending on factors such as percentage of human-answered the calls, call duration, and agent availability. There is no integration required to get the benefit of existing Amazon Connect features, such as automated workflows, routing, and machine learning capabilities like Contact Lens. You now have a single system for inbound and outbound communications.
To further refine the customer experience or use multiple channels in your campaigns, for example, to send an SMS or email message to your customers when they do not answer calls, you have the option to use Amazon Pinpoint. Amazon Pinpoint is a flexible and scalable outbound and inbound marketing communications service. It allows you to define customer segments, define the customer journey, define the contact strategy, and more. Amazon Pinpoint is the system handling high-volume SMS and email campaigns.
Let’s show you how it works Imagine I am a contact center manager, and I want to create an outbound call campaign to target a selected list of customers.
I first import my customer contact list from a spreadsheet on Amazon S3. I may also import it from popular customer relationship management (CRM) and marketing automation applications, such as Marketo, Salesforce, Twilio’s Segment, ServiceNow, Shopify, Zendesk, and Amazon Pinpoint itself.
Then I create a campaign and define some journey parameters: the communication channel, the start time, and the corresponding content, such as a call script, email template, or SMS message. At the scheduled start time, the journey is executed using Amazon Connect for calls or Amazon Pinpoint for SMS or emails, as specified.
When I configure the campaign to run in Predictive dial mode, as I mentioned before, the dialer automatically adjusts the dial rate based on the duration of calls and the real-time availability of agents. Once a call is answered, Amazon Connect distinguishes whether it is a live voice or a recorded message and routes the live customer to an available agent in the Amazon Connect agent application, where the agent can see the call script that I specified during setup, along with relevant customer information.
As explained earlier, I may use Amazon Pinpoint to define the customer journey. By doing so, I can combine voice, email, and SMS channels in the same outbound communication campaign to improve the efficiency of my agents and my customer’s experience. For example, a financial institution can use Amazon Connect to send an SMS notification to remind a customer of a missed payment and include a link to request a call back from an agent. When a call is requested, Amazon Connect automatically queues the call, dials the customer’s number, detects their voice, and connects an available agent to the customer.
Amazon Pinpoint allows you to define the details of the customer journey.
As usual with AWS services, I can analyze contact events sent via Amazon EventBridge. EventBridge is a serverless event bus that makes it easier to build event-driven applications at scale using events generated from your applications, integrated software-as-a-service (SaaS) applications, and AWS service. When filtering or analyzing events posted to EventBridge, I can create metrics such as time to connect to an agent, duration of the contact, and call abandonment rate
These metrics help me understand the status of my campaign and ensure compliance with applicable regulations, such as maximum call abandonment rates. I also can use historical reports of these metrics to understand the effectiveness of all my communications campaigns over time.
Speaking of compliance, we do not want anyone to abuse the system, intentionally or not, or to break any local compliance rules.
Access and Compliance Using automated services to drive outbound communication campaigns is strictly regulated in several countries and territories. For example, the US adopted the Telephone Consumer Protection Act (TCPA) in 1991, and the United Kingdom’s Office of Communications has similar rules.
Amazon Connect outbound campaigns gives you the tools to stay compliant with these regulations and many others. However, just like with traditional IT security, it is a shared responsibility. It is your responsibility to use the service in a compliant manner. We are happy to assist you in addressing specific use cases.
Let’s share two examples to illustrate how Amazon Connect outbound campaigns can help you meet your compliance status: respect quiet time and monitor call abandonment rate.
The use of quiet times allows contact center managers to configure a schedule for channel communications based on the day of the week and the hours of the day. More precise delivery times means your customers are most likely to engage with the communication and increase metrics such as open rates for SMS and email, as well as pick-up rates for voice calls. It also allows contact center managers to follow country and state-level voice dialing legislation. The following screenshot shows how you can configure quiet times using Amazon Pinpoint.
According to TCPA, call abandonment rate is the percentage of calls picked up by a live customer but not connected to a live agent within two seconds after the customer greeting. I found it interesting that in the UK, the time is measured from the start of your customer greetings, while in the US, it is measured from the end of the greeting. Amazon Connect outbound campaigns provides you with metrics, such as customerGreetingStart, customerGreetingStop, andconnectedToAgent for each outbound communication. Contact center managers can use these to compute the abandonment rate and dial up or down the outgoing communication channel accordingly.
Other metrics, configuration parameters, and AWS Lambda API integration allow contact center managers to consult a Do-Not-Call (DNC) registry or list scrubbing and verify your customer’s local time zone or bank holiday calendars, just to name a few.
Pricing and Availability Amazon Connect outbound campaigns is available in US East (N. Virginia), US West (Oregon), Asia Pacific (Sydney), and Europe (London) AWS Regions. This allows you to start your outbound campaigns for customers in the USA, UK, Australia, and New Zealand.
As usual, pricing is based on your usage; you only pay for what you use with no upfront or minimum engagement. The key metrics we are using for pricing are the minutes of outbound calls. The pricing page has all the details.
Amazon Web Services (AWS) is excited to announce that AWS Wickr has achieved Federal Risk and Authorization Management Program (FedRAMP) authorization at the Moderate impact level from the FedRAMP Joint Authorization Board (JAB).
FedRAMP is a U.S. government–wide program that promotes the adoption of secure cloud services by providing a standardized approach to security and risk assessment for cloud technologies and federal agencies.
Customers find security and control in Wickr
AWS Wickr is an end-to-end encrypted messaging and collaboration service with features designed to help keep your communications secure, private, and compliant. Wickr protects one-to-one and group messaging, voice and video calling, file sharing, screen sharing, and location sharing with 256-bit encryption, and provides data retention capabilities.
Administrative controls allow your AWS Wickr administrators to add, remove, and invite users, and organize them into security groups to manage messaging, calling, security, and federation settings. You can reset passwords and delete profiles remotely, helping you reduce the risk of data exposure stemming from a lost or stolen device.
You can log internal and external communications—including conversations with guest users, contractors, and other partner networks—in a private data store that you manage. This allows you to retain messages and files that are sent to and from your organization, to help meet requirements such as those that fall under the Federal Records Act (FRA) and the National Archives and Records Administration (NARA).
The FedRAMP milestone
In obtaining a FedRAMP Moderate authorization, AWS Wickr has been measured against a set of security controls, procedures, and policies established by the U.S. Federal Government, based on National Institute of Standards and Technology (NIST) standards.
“For many federal agencies and organizations, having the ability to securely communicate and share information—whether in an office or out in the field—is key to helping achieve their critical missions. AWS Wickr helps our government customers collaborate securely through messaging, calling, file and screen sharing with end-to-end encryption. The FedRAMP Moderate authorization for Wickr demonstrates our commitment to delivering solutions that give government customers the control and confidence they need to support their sensitive and regulated workloads.” – Christian Hoff, Director, US Federal Civilian & Health at AWS
FedRAMP on AWS
AWS is continually expanding the scope of our compliance programs to help you use authorized services for sensitive and regulated workloads. We now offer148 services authorized in the AWS US East/West Regions under FedRAMP Moderate authorization, and 128 services authorized in the AWS GovCloud (US) Regions under FedRAMP High authorization.
The FedRAMP Moderate authorization of AWS Wickr further validates our commitment at AWS to public-sector customers. With AWS Wickr, you can combine the security of end-to-end encryption with the administrative flexibility you need to secure mission-critical communications, and keep up with recordkeeping requirements. AWS Wickr is available under FedRAMP Moderate in the AWS US East (N. Virginia) Region.
Last Week’s Launches It’s been a quiet week on the AWS News Blog, however a glance at What’s New page shows the various service teams have been busy as usual. Here’s a round-up of announcements that caught my attention this past week.
VPC endpoint support is now available in Amazon SageMaker Canvas – SageMaker Canvas is a visual point-and-click service enabling business analysts to generate accurate machine-learning (ML) models without requiring ML experience or needing to write code. The new VPC endpoint support, available in all Regions where SageMaker Canvas is suppported, eliminates the need for an internet gateway, NAT instance, or a VPN connection when connecting from your SageMaker Canvas environment to services such as Amazon Simple Storage Service (Amazon S3), Amazon Redshift, and more.
Additional data sources for Amazon AppFlow – Facebook Ads, Google Ads, and Mixpanel are now supported as data sources, providing the ability to ingest marketing and product analytics for downstream analysis in AppFlow-connected software-as-a-service (SaaS) applications such as Marketo and Salesforce Marketing Cloud.
Some of the breakout session recordings from our .NET Enterprise Developer Day event in EMEA during April are now available on the .NET on AWS YouTube playlist.
Upcoming AWS Events The AWS New York Summit is approaching quickly, on July 12. Registration is also now open for the AWS Summit Canberra, an in-person event scheduled for August 31.
Amazon re:MARS is taking place this week in Las Vegas. I’ll be there as a host of the AWS on Air show, along with special guests highlighting their latest news from the conference. I also have some On Air sessions on using our AI services from .NET lined up! As usual, we’ll be streaming live from the expo hall, so if you’re at the conference, give us a wave. You can watch the show live on Twitch.tv/aws, Twitter.com/AWSOnAir, and LinkedIn Live.
No doubt there’ll be a whole new batch of releases and announcements from re:MARS, so be sure to check back next Monday for a summary of the announcements that caught our attention!
At Cloudflare, we strive to make our customers’ lives simpler by building products that solve their problems, are extremely easy to use, and integrate well with their existing tech stack. Another element of ensuring that we fit well with existing deployments is integrating seamlessly with additional solutions that customers subscribe to, and making sure those solutions work collaboratively together to solve a pain point.
Today, we are announcing new integrations that enable our customers to integrate third-party threat intel data with the rich threat intelligence from Cloudflare One products — all within the Cloudflare dashboard. We are releasing this feature in partnership with Mandiant, Recorded Future, and VirusTotal, and will be adding new partners in the coming months.
Customers of these threat intel partners can upload their API keys to the Cloudflare Security Center to enable the use of additional threat data to create rules within Cloudflare One products such as Gateway and Magic Firewall, and infrastructure security products including the Web Application Firewall and API Gateway. Additionally, search results from Security Center’s threat investigations portal will also be automatically enriched with licensed data.
Entering your API keys
Customers will be able to enter their keys by navigating to Security Center → Reference Data, and clicking on the ellipsis next to desired rows and selecting “Edit API key”. Once a valid key has been added, the status listed on the row should change from “No key provided” to “Active key”.
Mandiant
Mandiant Advantage customers with a Threat Intelligence subscription can enter their API keys and leverage Mandiant’s most popular feeds of FQDN and IP address indicators of security threats and their related context throughout Cloudflare One products.
These include lists organized by threat category and aggregations of most active malicious infrastructure. By curating the most recent data and data relevant to your infrastructure on the Cloudflare network, Cloudflare will make it easy to take advantage of active and relevant indicators of malicious activity from Mandiant’s extensive threat intelligence data. Cloudflare takes care of importing the data and refreshing it regularly to help protect you from the latest threats Mandiant sees on the frontlines. Cloudflare products such as Gateway, Magic Firewall, and Web Application Firewall (WAF) will have access to the threat intelligence data and make it easy to operationalize using the same rule builder you use today.
“As cyber threats continue to rapidly evolve, organizations require up-to-date and relevant intelligence integrated with their preferred technology solutions to comprehensively protect their environments. Together, Mandiant and Cloudflare are enabling our mutual customers to better protect themselves from malicious actors that are active on the front lines right now”. – Robert Wallace, Senior Director, Strategy, Mandiant
Recorded Future
Recorded Future customers can upload their API key to unlock use of Security Control Feeds. Once you have set up your API key, Recorded Future intelligence will also be available in the rule builder of Cloudflare Gateway and Magic Firewall. Cloudflare will present the intelligence that is relevant to and actionable by the product being configured. Intelligence will be regularly updated for you, freeing you to focus on the security policies and actions that are relevant for your organization.
For example, customers will be able to create a rule that blocks connections where the source or destination IP is in the Security Control feed “Command and Control – IP Addresses [Prevent]”. This list will be automatically updated daily for each customer who has a valid API key.
As threats accelerate and converge in the world around us, Recorded Future and Cloudflare are working together to empower customers with the right intelligence at the right time, to keep our people and infrastructure safe. – Craig Adams, Chief Product & Engineering officer, Recorded Future
VirusTotal
Virus Total Premium customers can upload their API key to augment and enrich Security Center search results for IPs, domains, and URLs. In the future we plan to add additional object types such as binary files.
Results will be automatically populated within a new card in the ‘Investigate’ tab. When searching an IP address, you will see a summary of the IP address information from VirusTotal including the overall results of the last analysis (e.g., harmless, suspicious, malicious, etc.), reputation score, tags, community votes, and the top files (if any) associated with that IP address by communications.
“Cybersecurity teams face a challenging environment as attackers become more sophisticated. They need complete visibility and real-time threat intelligence from multiple sources to combat malicious threats. We are partnering with Cloudflare to help our mutual customers outsmart adversaries.” – Emiliano Martinez Contreras, Head of Product for VirusTotal — Google
Want to get started?
If you are interested in gaining access during our beta testing phase, please complete this form. And if there are additional data vendors you would like to see us integrate with, including your own sources, click here.
Data Loss Prevention (DLP) enables you to protect your data based on its characteristics — or what it is. Today, we are very excited to announce that Data Loss Prevention is arriving as a native part of the Cloudflare One platform. If you’re interested in early access, please see the bottom of this post!
In the process of building Cloudflare One’s DLP solution, we talked to customers of all sizes and across dozens of industries. We focused on learning about their experiences, what products they are using, and what solutions they lack. The answers revealed significant customer challenges and frustrations. We are excited to deliver a product to put those problems in the past — and to do so as part of a comprehensive Zero Trust solution.
Customers are struggling to understand their data flow
Some customers have been using DLP solutions in their organizations for many years. They have deployed endpoint agents, crafted custom rulesets, and created incident response pipelines. Some built homemade tools to trace credit card numbers on the corporate network or rulesets to track hundreds of thousands of exact data match hashes.
Meanwhile, other customers are brand new to the space. They have small, scrappy teams supporting many IT and security functions. They do not have readily available resources to allocate to DLP and do not want to deprioritize other work to get started.
Still, many told the same story: the meteoric rise of SaaS tools left them unsure of where their data is moving and living. The migration of data off of corporate servers and into the cloud resulted in a loss of visibility and control. Even teams with established data protection programs strive for better visibility on the network. They are all asking the same types of questions:
Where is the data going?
Are uploads and downloads moving to and from corporate or personal SaaS instances?
What applications are storing sensitive data?
Who has access to those applications?
Can we see and block large downloads from file repositories?
Many customers seem to feel as though they have fallen behind because they haven’t solved these problems — and yet many customers are reporting the exact same story. However, these struggles do not mean anyone is behind — just that a better solution is needed. This told us that building a DLP product was the right choice, but why build it within Cloudflare One?
How Data Loss Prevention ties in to Zero Trust
A Zero Trust network architecture is fundamentally designed to secure your data. By checking every attempt to access a protected app, machine, or remote desktop, your data is protected on the basis of identity and device posture. With DNS and HTTP filtering, your data is protected based on content category and reputation. By adding an API-driven CASB, your data is protected based on your applications’ configurations, too.
With each piece of the architecture, your data is protected based on a new identifier. The identifiers above help you understand: who accessed the data, who owned the device that accessed it, where the data went, and how the destination was configured. However, what was the data that was moved?
Data Loss Prevention enables you to protect your data based on its characteristics, or what it is. For example, sensitive or confidential data can be identified a number of ways, such as keywords, patterns, or file types. These indicators help you understand the information being transmitted across or out of the network.
With DLP embedded in Cloudflare One, you can combine these identifiers to create rules catered to your organization. You get to specify the who, how, where, and what that meets your needs. We aim to deliver a comprehensive, detailed understanding of your network and your data, as well as allow you to easily implement protection.
How It Works
First: Identify the Data
DLP Profiles are being added to the Zero Trust dashboard. These profiles are where you define what data you want to protect. You will be able to add keywords and craft regexes to identify the presence of sensitive data. Profiles for common detections, such as credit card numbers, will be provided by Cloudflare.
Next: Create an HTTP Policy
After configuring a DLP Profile, you can then create a Cloudflare Gateway HTTP policy to allow or block the sensitive data from leaving your organization. Gateway will parse and scan your HTTP traffic for strings matching the keywords or regexes specified in the DLP profile.
Why Cloudflare
We know DLP is a big challenge to do comprehensively, and at scale. Those are the types of problems we excel at. Our network securely delivers traffic to 95% of the world’s Internet connected population within 50ms. It also supports our market leading products that send and protect customer traffic at unimaginable speed and scale. We are using that powerful network and our experience solving problems like this to take on Data Loss Prevention, and we’re very excited by our results
Join the waitlist
We are launching a closed beta of our Data Loss Prevention product. If you’re interested in early access, you can join the waitlist today by filling out this form.
What’s next?
We’re just getting started with DLP! We already have many plans for growth and integration with other Cloudflare One products, such as Remote Browser Isolation.
През 1945 г. започва чудовищното насилствено налагане на “македонизма” над част от българския народ в Югославия и България. Това е ставало под натиска и директивите на Коминтерна и Кремъл, а…
Over the last several years, both Area 1 and Cloudflare built pipelines for ingesting threat indicator data, for use within our products. During the acquisition process we compared notes, and we discovered that the overlap of indicators between our two respective systems was smaller than we expected. This presented us with an opportunity: as one of our first tasks in bringing the two companies together, we have started bringing Area 1’s threat indicator data into the Cloudflare suite of products. This means that all the products today that use indicator data from Cloudflare’s own pipeline now get the benefit of Area 1’s data, too.
Area 1 built a data pipeline focused on identifying new and active phishing threats, which now supplements the Phishing category available today in Gateway. If you have a policy that references this category, you’re already benefiting from this additional threat coverage.
How Cloudflare identifies potential phishing threats
Cloudflare is able to combine the data, procedures and techniques developed independently by both the Cloudflare team and the Area 1 team prior to acquisition. Customers are able to benefit from the work of both teams across the suite of Cloudflare products.
Cloudflare curates a set of data feeds both from our own network traffic, OSINT sources, and numerous partnerships, and applies custom false positive control. Customers who rely on Cloudflare are spared the software development effort as well as the operational workload to distribute and update these feeds. Cloudflare handles this automatically, with updates happening as often as every minute.
Cloudflare is able to go beyond this and work to proactively identify phishing infrastructure in multiple ways. With the Area 1 acquisition, Cloudflare is now able to apply the adversary-focused threat research approach of Area1 across our network. A team of threat researchers track state-sponsored and financially motivated threat actors, newly disclosed CVEs, and current phishing trends.
Cloudflare now operates mail exchange servers for hundreds of organizations around the world, in addition to its DNS resolvers, Zero Trust suite, and network services. Each of these products generates data that is used to enhance the security of all of Cloudflare’s products. For example, as part of mail delivery, the mail engine performs domain lookups, scores potential phishing indicators via machine learning, and fetches URLs. Data which can now be used through Cloudflare’s offerings.
How Cloudflare Area 1 identifies potential phishing threats
The Cloudflare Area 1 team operates a suite of web crawling tools designed to identify phishing pages, capture phishing kits, and highlight attacker infrastructure. In addition, Cloudflare Area 1 threat models assess campaigns based on signals gathered from threat actor campaigns; and the associated IOCs of these campaign messages are further used to enrich Cloudflare Area 1 threat data for future campaign discovery. Together these techniques give Cloudflare Area 1 a leg up on identifying the indicators of compromise for an attacker prior to their attacks against our customers. As part of this proactive approach, Cloudflare Area 1 also houses a team of threat researchers that track state-sponsored and financially motivated threat actors, newly disclosed CVEs, and current phishing trends. Through this research, analysts regularly insert phishing indicators into an extensive indicator management system that may be used for our email product or any other product that may query it.
Cloudflare Area 1 also collects information about phishing threats during our normal operation as the mail exchange server for hundreds of organizations across the world. As part of that role, the mail engine performs domain lookups, scores potential phishing indicators via machine learning, and fetches URLs. For those emails found to be malicious, the indicators associated with the email are inserted into our indicator management system as part of a feedback loop for subsequent message evaluation.
How Cloudflare data will be used to improve phishing detection
In order to support Cloudflare products, including Gateway and Page Shield, Cloudflare has a data pipeline that ingests data from partnerships, OSINT sources, as well as threat intelligence generated in-house at Cloudflare. We are always working to curate a threat intelligence data set that is relevant to our customers and actionable in the products Cloudflare supports. This is our North star: what data can we provide that enhances our customer’s security without requiring our customers to manage the complexity of data, relationships, and configuration. We offer a variety of security threat categories, but some major focus areas include:
Malware distribution
Malware and Botnet Command & Control
Phishing,
New and newly seen domains
Phishing is a threat regardless of how the potential phishing link gets entry into an organization, whether via email, SMS, calendar invite or shared document, or other means. As such, detecting and blocking phishing domains has been an area of active development for Cloudflare’s threat data team since almost its inception.
Looking forward, we will be able to incorporate that work into Cloudflare Area 1’s phishing email detection process. Cloudflare’s list of phishing domains can help identify malicious email when those domains appear in the sender, delivery headers, message body or links of an email.
1+1 = 3: Greater dataset sharing between Cloudflare and Area 1
Threat actors have long had an unfair advantage — and that advantage is rooted in the knowledge of their target, and the time they have to set up specific campaigns against their targets. That dimension of time allows threat actors to set up the right infrastructure, perform reconnaissance, stage campaigns, perform test probes, observe their results, iterate, improve and then launch their ‘production’ campaigns. This precise element of time gives us the opportunity to discover, assess and proactively filter out campaign infrastructure prior to campaigns reaching critical mass. But to do that effectively, we need visibility and knowledge of threat activity across the public IP space.
With Cloudflare’s extensive network and global insight into the origins of DNS, email or web traffic, combined with Cloudflare Area 1’s datasets of campaign tactics, techniques, and procedures (TTPs), seed infrastructure and threat models — we are now better positioned than ever to help organizations secure themselves against sophisticated threat actor activity, and regain the advantage that for so long has been heavily weighted towards the bad guys.
If you’d like to extend Zero Trust to your email security to block advanced threats, contact your Customer Success manager, or request a Phishing Risk Assessment here.
Leaders and practitioners responsible for email security are faced with a few truths every day. It’s likely true that their email is cloud-delivered and comes with some built-in protection that does an OK job of stopping spam and commodity malware. It’s likely true that they have spent considerable time, money, and staffing on their Secure Email Gateway (SEG) to stop phishing, malware, and other email-borne threats. Despite this, it’s also true that email continues to be the most frequent source of Internet threats, with Deloitte research finding that 91% of all cyber attacks begin with phishing.
If anti-phishing and SEG services have both been around for so long, why do so many phish still get through? If you’re sympathetic to Occam’s razor, it’s because the SEG was not designed to protect the email environments of today, nor is it effective at reliably stopping today’s phishing attacks.
But if you need a stronger case than Occam delivers — then keep on reading.
Why the world has moved past the SEG
The most prominent change within the email market is also what makes a traditional SEG redundant – the move to cloud-native email services. More than 85% of organizations are expected to embrace a “cloud-first” strategy by 2025, according to Gartner®. Organizations that expect cloud-native scale, resiliency, and flexibility from their security controls are not going to get it from legacy devices such as SEGs.
When it comes to email specifically, Gartner® notes that, “Advanced email security capabilities are increasingly being deployed as integrated cloud email security solutions rather than as a gateway” – with at least 40% of organizations using built-in protection capabilities from cloud email providers instead of a SEG, by 2023. Today, email comes from everywhere and goes everywhere – putting a SEG in front of your Exchange server is anachronistic; and putting a SEG in front of cloud inboxes in a mobile and remote-first world is intractable. Email security today should follow your user, should be close to your inbox, and should “be everywhere”.
Apart from being architecturally out of time, a SEG also falls short at detecting advanced phishing and socially engineered attacks. This is because a SEG was originally designed to stop spam – a high-volume problem that needs large attack samples to detect and nullify. But today’s phishing attacks are more sniper than scattergun. They are low volume, highly targeted, and exploit our implicit trust in email communications to steal money and data. Detecting modern phishing attacks requires compute-intensive advanced email analysis and threat detection algorithms that a SEG cannot perform at scale.
Nowhere is a SEG’s outdated detection philosophy more laid bare than when admins are confronted with a mountain of email threat policies to create and tune. Unlike most other cyber attacks, email phishing and Business Email Compromise (BEC) have too many “fuzzy” signals and cannot solely be detected by deterministic if-then statements. Moreover, attackers don’t stand still while you create email threat policies – they adapt fast and modify techniques to bypass the rules you just created. Relying on SEG tuning to stop phishing is like playing a game of Whack-A-Mole rigged in the attacker’s favor.
To stop phishing, look ahead
Traditional email security defenses rely on knowledge of yesterday’s active attack characteristics, such as reputation data and threat signatures, to detect the next attack, and therefore can’t reliably defend against modern phishing attacks that continually evolve.
What’s needed is forward-looking security technology that is aware not only of yesterday’s active phishing payloads, websites, and techniques — but also has insight into the threat actors’ next moves. Which sites and accounts are they compromising or establishing for use in tomorrow’s attacks? What payloads and techniques are they preparing to use in those attacks? Where are they prodding and probing before an attack?
Cloudflare Area 1 proactively scans the Internet for attacker infrastructure and phishing campaigns that are under construction. Area 1’s threat-focused web crawlers dynamically analyze suspicious web pages and payloads, and continuously update detection models as attacker tactics evolve – all to stop phishing attacks days before they reach the inbox.
When combined with the 1T+ daily DNS requests observed by Cloudflare Gateway, this corpus of threat intelligence enables customers to stop phishing threats at the earliest stage of the attack cycle. In addition, the use of deep contextual analytics to understand message sentiment, tone, tenor and thread variations allows Area 1 to understand and distinguish between valid business process messages and sophisticated impersonation campaigns.
While we are big believers in layering security, the layers should not be redundant. A SEG duplicates a lot of capabilities that customers now get bundled in with their cloud email offering. Area 1 is built to enhance – not duplicate – native email security and stop phishing attacks that get past initial layers of defense.
Planning for your SEG replacement project
The best way to get started with your SEG replacement project is deciding whether it’s a straight replacement or an eventual replacement that starts with augmentation. While Cloudflare Area 1 has plenty of customers that have replaced their SEG (more on that later), we have also seen scenarios where customers prefer to run Cloudflare Area 1 downstream of their SEG initially, assess the efficacy of both services, and then make a more final determination. We make the process straightforward either way!
As you start the project, it’s important to involve the right stakeholders. At a minimum, you should involve an IT admin to ensure email delivery and productivity isn’t impacted and a security admin to monitor detection efficacy. Other stakeholders might include your channel partner if that’s your preferred procurement process and someone from the privacy and compliance team to verify proper handling of data.
Next, you should decide your preferred Cloudflare Area 1 deployment architecture. Cloudflare Area 1 can be deployed as the MX record, over APIs, and can even run in multi-mode deployment. We recommend deploying Cloudflare Area 1 as the MX record for the most effective protection against external threats, but the service fits into your world based on your business logic and specific needs.
The final piece of preparation involves mapping out your email flow. If you have multiple domains, identify where emails from each of your domains route to. Check your different routing layers (e.g. are there MTAs that relay inbound messages?). Having a good understanding of the logical and physical SMTP layers within the organization will ensure proper routing of messages. Discuss what email traffic Cloudflare Area 1 should scan (north/south, east/west, both) and where it fits with your existing email policies.
Executing the transition plan
Step 1: Implement email protection Here are the broad steps you should follow if Cloudflare Area 1 is configured as the MX record (time estimate: ~30 minutes):
Configure the downstream service to accept mail from Cloudflare Area 1.
Ensure that Cloudflare Area 1’s egress IPs are not rate limited or blocked as this would affect delivery of messages.
If the email server is on-premises, update firewall rules to allow Cloudflare Area 1 to deliver to these systems.
Configure remediation rules (e.g. quarantine, add subject or message body prefix, etc.).
Test the message flow by injecting messages into Cloudflare Area 1 to confirm proper delivery. (our team can assist with this step.)
Update MX records to point to Cloudflare Area 1.
Here are the steps if Cloudflare Area 1 is deployed downstream from an existing email security solution (time estimate: ~30 minutes):
Configure the proper look back hops on Cloudflare Area 1, so that Cloudflare Area 1 can detect the original sender IP address.
If your email server is on-premises, update firewall rules to allow Cloudflare Area 1 to deliver to the email server.
Configure remediation rules (e.g. quarantine, add subject or message body prefix, etc.).
Test the message flow by injecting messages into Cloudflare Area 1 to confirm proper delivery. (our team can assist with this step.)
Update the delivery routes on your SEG to deliver all mail to Cloudflare Area 1, instead of the email servers.
Step 2: Integrate DNS One of the most common post-email steps customers follow is to integrate Cloudflare Area 1 with their DNS service. If you’re a Cloudflare Gateway customer, good news – Cloudflare Area 1 now uses Cloudflare Gateway as its recursive DNS to protect end users from accessing phishing and malicious sites through email links or web browsing.
Step 3: Integrate with downstream security monitoring and remediation services Cloudflare Area 1’s detailed and customizable reporting allows for at-a-glance visibility into threats. By integrating with SIEMs through our robust APIs, you can easily correlate Cloudflare Area 1 detections with events from network, endpoint and other security tools for simplified incident management.
While Cloudflare Area 1 provides built-in remediation and message retraction to allow customers to respond to threats directly within the Cloudflare Area 1 dashboard, many organizations also choose to integrate with orchestration tools for custom response playbooks. Many customers leverage our API hooks to integrate with SOAR services to manage response processes across their organization.
Metrics to measure success
How will you know your SEG replacement project has been successful and had the desired impact? We recommend measuring metrics relevant to both detection efficacy and operational simplicity.
On the detection front, the obvious metric to measure is the number and nature of phishing attacks blocked before and after the project. Are you seeing new types of phishing attacks being blocked that you weren’t seeing before? Are you getting visibility into campaigns that hit multiple mailboxes? The other detection-based metric to keep in mind is the number of false positives.
On the operational front, it’s critical that email productivity isn’t impacted. A good proxy for this is measuring the number of IT tickets related to email delivery. The availability and uptime of the email security service is another key lever to keep an eye on.
Finally, and perhaps most importantly, measure how much time your security team is spending on email security. Hopefully it’s much less than before! A SEG is known to be a heavy-lift service deployment to ongoing maintenance. If Cloudflare Area 1 can free up your team’s time to work on other pressing security concerns, that’s as meaningful as stopping the phish themselves.
You have lots of company
The reason we are articulating a SEG replacement plan here is because many of our customers have done it already and are happy with the outcomes.
For example, a Fortune 50 global insurance provider that serves 90 million customers in over 60 countries found their SEG to be insufficient in stopping phishing attacks. Specifically, it was an onerous process to search for “missed phish” once they got past the SEG and reached the inbox. They needed an email security service that could catch these phishing attacks and support a hybrid architecture with both cloud and on-premises mailboxes.
After deploying Cloudflare Area 1 downstream of their Microsoft 365 and SEG layers, our customer was protected against more than 14,000 phishing threats within the first month; none of those phishing messages reached a user’s inbox. A one-step integration with existing email infrastructure meant that maintenance and operational issues were next to none. Cloudflare Area 1’s automated message retraction and post-delivery protection also enabled the insurance provider to easily search and remediate any missed phish as well.
If you are interested in speaking with any of our customers that have augmented or replaced their SEG with Cloudflare Area 1, please reach out to your account team to learn more! If you’d like to see Cloudflare Area 1 in action, sign up for a Phishing Risk Assessment here.
Replacing a SEG is a great project to fit into your overall Zero Trust roadmap. For a full summary of Cloudflare One Week and what’s new, tune in to our recap webinar.
–
1Gartner Press Release, “Gartner Says Cloud Will Be the Centerpiece of New Digital Experiences”, 11 November 2021 2Gartner, “Market Guide for Email Security,” 7 October 2021, Mark Harris, Peter Firstbrook, Ravisha Chugh, Mario de Boer
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
The collective thoughts of the interwebz
By continuing to use the site, you agree to the use of cookies. more information
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.
