Post Syndicated from Sam Marsh original https://blog.cloudflare.com/future-of-page-rules/

Page Rules is one of our most well-used products. Adopted by millions of users, it is used for configuring everything from cache to security levels. It is the ‘If This Then That’ of Cloudflare. Where the ‘If…’ is a URL, and the ‘Then That’ is changing how we handle traffic to specific parts of a ‘zone’. But it’s not without its limitations.

Page Rules can only trigger on a URL or URL pattern. There is a maximum of 125 Page Rules per zone. Page Rules are also tricky to debug. Even the idea of a “Page” sounds rather old-fashioned now.

We’re replacing Page Rules with four new dedicated products, offering increased rules quota, more functionality, and better granularity. These products are available immediately for testing. Page Rules is not going away yet, but we do anticipate being able to formally begin the end-of-life process soon.

Why change?

In the 10 years since it launched, Page Rules has become a well established product, and a very well adopted one. One million Page Rules have been deployed in the past three months alone.

Page Rules are used to tune how long files should be cached. Page Rules are used to override zone-wide settings for certain URLs. Page Rules are used to create simple URL redirects. Page Rules are used to selectively add/remove HTTP headers. Page Rules is a multitool of a product.

Like multitools and other generalist products, Page Rules does a good job at many things, but is not best-of-breed at anything. This is the trade-off of generalism. As we have grown as a company our customers have rightfully demanded more. Filtering on URL-alone is no longer sufficient; users are demanding more – and today we are delivering.

Over the last two years we have been working on the future of Page Rules and distilling hundreds of pieces of feedback into common themes, such as:

1. I need more than 125 Page Rules
2. I need to be able to trigger Page Rules on more than just the URL
3. I need to be able to use regular expressions in my Page Rules
4. It’s hard for me to understand how different Page Rules interact one another
5. Page Rules is hard to debug
6. I want more actions in Page Rules

Analyzing these themes we came to the conclusion that the best thing for Page Rules was to disassemble it and create new discrete products, each of which could be best-of-breed for their relevant areas. This dissolution would also provide better clarity regarding interoperation (cache vs configuration vs …), and make debugging simpler.

Today, we announce those new products:

1. Cache Rules: A dedicated product for setting and tuning ‘everything caching’.

2. Configuration Rules: A dedicated product for setting and selectively enabling, disabling and overriding zone-wide settings.

3. Dynamic Redirects: Like ‘Forwarding URL’ but turned up to 11. Redirect based on the visitors country, their preferred language, their device type, use regular expressions (plan level dependant) and more.

4. Origin Rules: A dedicated product for ‘where does this traffic go where it leaves Cloudflare’. Not only have we added host header and resolve override into this new product (ENT only), we’ve also productized another common Workers use case by enabling customers to selectively override the destination port. We’ve also added the ability to override the Server Name Indication (SNI).

All four of these products are available for use now via the dashboard, API and Terraform – and sitting alongside Transform Rules provide the replacement suite of products that will eventually enable an Page Rules end-of-life announcement.

We have dedicated blogs for each of these product launches with more information on what they offer and problems they address.

Order of execution

One of the main benefits of this new product suite is clarity.

Page Rules is a black box, where traffic goes in, ‘things happen’, and traffic comes out. It’s hard to debug the interplay between cache, configuration, header modification etc and it can vary from zone to zone as it’s entirely user defined.

By having discrete, separate areas per ‘function’, it makes visualizing the flow of a HTTP request much easier:

Rather than a single lozenge of Page Rules, we now have visibility that Origin Rules will run first, then Cache Rules, then Configuration Rules, and finally Dynamic Redirects. This means we will modify host headers first, before tuning cache settings. And we will tune the cache parameters before we modify which settings are enabled for the specific traffic.

We have integrated these new products into the Traffic Sequence dashboard element also.

(For zones using both Page Rules and this new suite of products: The new products will take precedence over Page Rules. This means that Page Rules will be overridden if a conflict occurs).

I need more than 125 Page Rules

One of the limitations of Page Rules was how each Page Rule was stored and executed on our backend architecture. We are unable to offer more than 125 Page Rules per zone before we begin to see a performance hit – latency on HTTP requests begins to increase as evaluating them vs the Page Rules takes longer and longer. To combat this limitation users moved simple workloads to Workers, or split the zone into multiple sub domains, each with a 125 Page Rules quota. Neither of these are ideal for the customer.

To combat this limitation we have built all of the replacement products atop our lightning-fast Rulesets Engine, which also runs products such as Transform Rules, Custom Rules (WAF), Bulk Redirects and API Shield.

This allows us to offer much more quota to customers, as this engine is built to scale well beyond 125 rules per product. The table below summarizes the before and after impact of these new products, showing the default rules quota per plan:

Plan	Page Rules	Origin Rules	Cache Rules	Config Rules	Dynamic Redirects
Enterprise	125	125+	125+	125+	125+
Business	50	50	50	50	50
Pro	20	25	25	25	25
Free	3	10	10	10	10

Additional rules cannot be purchased for these new products.

This means zone’s on the Enterprise plan now have a minimum of 500 rules to use where before they had 125 via Page Rules. For Enterprises the quota for the new products is negotiable. Pro plan zones go from 20 Page Rules to 100.  Combined with the fine-grained control that the ruleset engine offers, these changes allow customers to customize their zone’s traffic to the finest of margins.

The other benefit from building all of these products on the Rulesets Engine is extensibility. Currently there are over 30 products that are built and running on the Rulesets Engine. Each of these products is essentially a logical bucket called a ‘phase’ which contains a single ruleset scoped to that product. Each phase is restricted to specific actions and fields, for example the field cf.bot_management.score is unavailable in http_request_transform as we have not calculated the bot score at the time we perform URL rewrites. Also, only the rewrite action is permitted. Whereas in Origin Rules (http_request_origin) we only allow the action route

When we create new capabilities for a product that is built atop the Rulesets Engine it is trivially simple for us to extend that new capability to other products at a later date.

For example, we added a new ‘field’, http.request.accepted_languages to Transform Rules earlier in the year. Until recently this was only available in Transform Rules. However, as both products are built on the Rulesets Engine it was trivial to enable this feature for Dynamic Redirects. This allows customers to perform URL redirects based on the visitor’s language preference, and the cost to us from an engineering perspective is negligible as the field is already implemented.

This also means in the future should a new field be created for Cache Rules due to a customer request, e.g. http.request.super_cool_field, in a matter of clicks we can enable this field for any of the other 30 products rather than have to duplicate effort across multiple platforms.

Simply put, the more products we build on top of the Rulesets Engine, the faster we can move and the more functionality we can put into users hands.

A single user experience

The most important benefit of all is consistency. Each of these products has a consistent and predictable API. A consistent and predictable Terraform configuration, and a consistent and predictable user experience in the dashboard. The ruleset engine allows us to keep everything the same except for the ‘action’. The filtering stays the same, the API stays the same, the UI stays (largely) the same, the only change is the ‘…then’, the action section of the rule.

This ensures that as a user when you are clicking around the dashboard setting up a new zone you aren’t having to learn each individual product’s page and how to navigate it. The only part you need to learn is what makes that product unique, its actions:

Finally, when we add a new product, extending the Terraform provider to support it is trivial. That consistent experience has been a north star for us during this project and will continue to be in the future.

Try it them now

We are replacing Page Rules with a new suite of products, each built to be best-of-breed and put more power into the hands of our users.

Read more about the new products in each of their dedicated blogs. Then, try them for yourself!

Post Syndicated from Sam Marsh original https://blog.cloudflare.com/dynamic-redirect-rules/

The Internet is a dynamic place. Websites are constantly changing as technologies and business practices evolve. What was front-page news is quickly moved into a sub-directory. To ensure website visitors continue to see the correct webpage even if it has been moved, administrators often implement URL redirects.

A URL redirect is a mapping from one location on the Internet to another, effectively telling the visitor’s browser that the location of the page has changed, and where they can now find it. This is achieved by providing a virtual ‘link’ between the content’s original and new location.

URL Redirects have typically been implemented as Page Rules within Cloudflare, however Page Rules only match on the URL, rather than other elements such as the visitor’s source country or preferred language. This limitation meant customers with a need for more dynamic URL redirects had to implement alternative solutions such Cloudflare Workers to achieve their goals.

To simplify the management of these more complex use cases we have created Dynamic Redirects. With Dynamic Redirects, users can redirect visitors to another webpage or website based upon hundreds of options such as the visitor’s country of origin or language, without having to write a single line of code.

More than a URL

For nine years users were limited to 125 URL redirects per zone. This limitation meant those with a need for more URL redirects had to implement alternative solutions such Cloudflare Workers to achieve their goals.

In December 2021, we launched Bulk Redirects, allowing up to 100,000 URL redirects per account at the time. In April 2022 we increased this maximum number to over six million URL redirects per account. However, there is still a gap in the ‘URL redirect’ product unfulfilled. Until now.

Bulk Redirects, much like the ‘Forwarding URL’ Page Rule, are prescriptive URL redirects. You tell us what URL to look for, and where to redirect the user to when they visit it. We can support this use case at a huge scale.

If a visitor asks for..	Redirect them to…
https://www.cloudflare.com/r2-storage	https://www.cloudflare.com/products/r2
https://www.cloudflare.com/apishield	https://www.cloudflare.com/products/api-gateway
https://www.cloudflare.com/welcome-center	https://developers.cloudflare.com/fundamentals/get-started/

That’s a simple concept to understand, however user needs have evolved. What if a user wanted to redirect visitors to a localized version of the requested page based on their preferred language? What if a user wanted to redirect visitors to their local subsidiary on the website? Or direct them to an optimized site when they visit from a mobile device? Suddenly, this well understood concept doesn’t work – and they have to deploy code in Workers to solve what is actually a common problem. And common problems deserve to be productized.

This is where Dynamic Redirects can help. The new product provides the same consistent user interface as Transform Rules, Custom Rules, Bulk Redirects, etc. and provides a new action allowing for the target URL to be dynamically created, much like the dynamic rewrite action offered in Transform Rules.

This dynamic action frees the user from having to define explicitly what the target URL should look like, and instead provides them with a full gamut of fields and functions to custom generate the target URL based upon the parameters of the request. For example, rather than redirecting all traffic for www.example.com/shop to www.example.com/en/shop, users can conceptually redirect the traffic to www.example.com/{PREFERRED_LANGUAGE}/shop (not actual syntax!). With this, traffic from a browser with a preferred language of French will be redirected to www.example.com/fr/shop, likewise those with a preferred language of German will be redirected to www.example.com/de/shop.

The other big difference between Dynamic Redirects and  Page Rules is in the filtering. Page Rules are limited to filtering on a URL, or a URL with asterisks as wildcards. Dynamic Redirects is built atop our lightning-fast Rulesets Engine, which also runs products such as Transform Rules, Custom Rules (WAF), Bulk Redirects and API Shield.

Due to this, Dynamic Redirects offers almost the entire suite of Ruleset Engine fields for use in filtering; from http.request.full_uri for the whole URL, to ip.geoip.country (where is the visitor located) and http.request.accepted_languages[] (the language preferred by the visitor). The possibilities are endless.

Users can also now use logical operators such as ‘OR’. Where previously, if a user wanted to redirect five distinct URLs to the same URL they would need to deploy five Page Rules. Today, they can simply use an ‘OR’ to consolidate this use case into just one Dynamic Redirect rule:

#	URL	Destination URL
1	https:/www.cloudflare.com/partners/integrations/	www.cloudflare.com/partners/
2	https:/www.cloudflare.com/partners/become-a-partner/	www.cloudflare.com/partners/
3	https:/www.cloudflare.com/partners/digital-agency/	www.cloudflare.com/partners/
4	https:/www.cloudflare.com/partners/technology-integrator/	www.cloudflare.com/partners/
5	https:/www.cloudflare.com/partners/view-partners/	www.cloudflare.com/partners/

#	Expression	Destination URL
1	(http.request.full_uri eq “https:/www.cloudflare.com/partners/integrations/”) or (http.request.full_uri eq “https:/www.cloudflare.com/partners/become-a-partner/”) or (http.request.full_uri eq “https:/www.cloudflare.com/partners/digital-agency/”) or (http.request.full_uri eq “https:/www.cloudflare.com/partners/technology-integrator/”) or (http.request.full_uri eq “https:/www.cloudflare.com/partners/view-partners/”)	www.cloudflare.com/partners/

We can further simplify this use case in the future by adding hostname lists, allowing users to add URLs to a list and reference it from within the rule expression, similar to IP Lists. This allows an expression like (http.request.full_uri in $vanity_urls), for example.

A dedicated quota, just for U(RL)

Page Rules are used to set everything from configuration and caching behaviour to header modification and also URL redirection (Forwarding URL). This means that users tend to run out of available rules quickly.

To address this, we’re matching the Page Rule quota in each of the four new products that are being announced today. This means where in Page Rules an Enterprise customer would get 125 Page Rules to share amongst the aforementioned functions, in Dynamic Redirects they have 125 rules just for redirects. This number can be increased for Enterprise customers, also.

We’re also increasing the quota for free plans; where today free plans get three Page Rules, they will now get 10 rules for dynamic redirects, along with each of the other three products (cache, origin, config rules). That’s a net increase of 37 additional rules!

Plan	Page Rules	Dynamic Redirects
Enterprise	125	125+
Business	50	50
Pro	20	25
Free	3	10

Users can now get more out of their Cloudflare setup, being more specific about when traffic is redirected, optimizing cache and adjusting which settings are and aren’t applied – without having to trade off between these areas due to a limit in rules quota.

Localized redirects

One of the examples covered earlier is being able to redirect visitors to localized content depending on their preferred language.

This can be done by analyzing the ‘Accept-Language’ header sent by the browser, which is stored as an array in the field http.request.accepted_languages[]. This field is an array of the values received by Cloudflare within the Accept-Language HTTP request header, sorted in descending weight order. This header is calculated based on the preferences set by the visitor in the ‘Language’ section of their web browser.

For example, if the visitors browser sends an ‘Accept-Language’ header containing “Accept-Language: fr-CH, fr;q=0.8, en;q=0.9, de;q=0.7, *;q=0.5”, then the field http.request.accepted_languages[0] would contain “en”, with http.request.accepted_languages[1] containing “fr” and so forth.

With this information, we can create a dynamic redirect using the action:

With this rule in place, traffic from visitors with a preferred language of English (en) will be redirected to www.example.com/en/shop. The rule can be duplicated for other languages also, ensuring those with a preferred language of French (fr) will be redirected to www.example.com/fr/shop.

Mobile redirects, cookie redirects, …

There are so many use cases for Dynamic Redirects we couldn’t fit them all in this blog.

Other possible use cases include mobile redirects, redirects based on cookies, redirects to different endpoints based on request headers for live testing. The potential list is huge, and we can’t wait to hear what you come up with. Try it today!