Post Syndicated from Sam Marsh original https://blog.cloudflare.com/future-of-page-rules/
Page Rules is one of our most well-used products. Adopted by millions of users, it is used for configuring everything from cache to security levels. It is the ‘If This Then That’ of Cloudflare. Where the ‘If…’ is a URL, and the ‘Then That’ is changing how we handle traffic to specific parts of a ‘zone’. But it’s not without its limitations.
Page Rules can only trigger on a URL or URL pattern. There is a maximum of 125 Page Rules per zone. Page Rules are also tricky to debug. Even the idea of a “Page” sounds rather old-fashioned now.
We’re replacing Page Rules with four new dedicated products, offering increased rules quota, more functionality, and better granularity. These products are available immediately for testing. Page Rules is not going away yet, but we do anticipate being able to formally begin the end-of-life process soon.
Why change?
In the 10 years since it launched, Page Rules has become a well established product, and a very well adopted one. One million Page Rules have been deployed in the past three months alone.
Page Rules are used to tune how long files should be cached. Page Rules are used to override zone-wide settings for certain URLs. Page Rules are used to create simple URL redirects. Page Rules are used to selectively add/remove HTTP headers. Page Rules is a multitool of a product.
Like multitools and other generalist products, Page Rules does a good job at many things, but is not best-of-breed at anything. This is the trade-off of generalism. As we have grown as a company our customers have rightfully demanded more. Filtering on URL-alone is no longer sufficient; users are demanding more – and today we are delivering.
Over the last two years we have been working on the future of Page Rules and distilling hundreds of pieces of feedback into common themes, such as:
- I need more than 125 Page Rules
- I need to be able to trigger Page Rules on more than just the URL
- I need to be able to use regular expressions in my Page Rules
- It’s hard for me to understand how different Page Rules interact one another
- Page Rules is hard to debug
- I want more actions in Page Rules
Analyzing these themes we came to the conclusion that the best thing for Page Rules was to disassemble it and create new discrete products, each of which could be best-of-breed for their relevant areas. This dissolution would also provide better clarity regarding interoperation (cache vs configuration vs …), and make debugging simpler.
Today, we announce those new products:
1. Cache Rules: A dedicated product for setting and tuning ‘everything caching’.
2. Configuration Rules: A dedicated product for setting and selectively enabling, disabling and overriding zone-wide settings.
3. Dynamic Redirects: Like ‘Forwarding URL’ but turned up to 11. Redirect based on the visitors country, their preferred language, their device type, use regular expressions (plan level dependant) and more.
4. Origin Rules: A dedicated product for ‘where does this traffic go where it leaves Cloudflare’. Not only have we added host header and resolve override into this new product (ENT only), we’ve also productized another common Workers use case by enabling customers to selectively override the destination port. We’ve also added the ability to override the Server Name Indication (SNI).
All four of these products are available for use now via the dashboard, API and Terraform – and sitting alongside Transform Rules provide the replacement suite of products that will eventually enable an Page Rules end-of-life announcement.
We have dedicated blogs for each of these product launches with more information on what they offer and problems they address.
Order of execution
One of the main benefits of this new product suite is clarity.
Page Rules is a black box, where traffic goes in, ‘things happen’, and traffic comes out. It’s hard to debug the interplay between cache, configuration, header modification etc and it can vary from zone to zone as it’s entirely user defined.
By having discrete, separate areas per ‘function’, it makes visualizing the flow of a HTTP request much easier:
Rather than a single lozenge of Page Rules, we now have visibility that Origin Rules will run first, then Cache Rules, then Configuration Rules, and finally Dynamic Redirects. This means we will modify host headers first, before tuning cache settings. And we will tune the cache parameters before we modify which settings are enabled for the specific traffic.
We have integrated these new products into the Traffic Sequence dashboard element also.
(For zones using both Page Rules and this new suite of products: The new products will take precedence over Page Rules. This means that Page Rules will be overridden if a conflict occurs).
I need more than 125 Page Rules
One of the limitations of Page Rules was how each Page Rule was stored and executed on our backend architecture. We are unable to offer more than 125 Page Rules per zone before we begin to see a performance hit – latency on HTTP requests begins to increase as evaluating them vs the Page Rules takes longer and longer. To combat this limitation users moved simple workloads to Workers, or split the zone into multiple sub domains, each with a 125 Page Rules quota. Neither of these are ideal for the customer.
To combat this limitation we have built all of the replacement products atop our lightning-fast Rulesets Engine, which also runs products such as Transform Rules, Custom Rules (WAF), Bulk Redirects and API Shield.
This allows us to offer much more quota to customers, as this engine is built to scale well beyond 125 rules per product. The table below summarizes the before and after impact of these new products, showing the default rules quota per plan:
| Plan | Page Rules | Origin Rules | Cache Rules | Config Rules | Dynamic Redirects |
|---|---|---|---|---|---|
| Enterprise | 125 | 125+ | 125+ | 125+ | 125+ |
| Business | 50 | 50 | 50 | 50 | 50 |
| Pro | 20 | 25 | 25 | 25 | 25 |
| Free | 3 | 10 | 10 | 10 | 10 |
Additional rules cannot be purchased for these new products.
This means zone’s on the Enterprise plan now have a minimum of 500 rules to use where before they had 125 via Page Rules. For Enterprises the quota for the new products is negotiable. Pro plan zones go from 20 Page Rules to 100. Combined with the fine-grained control that the ruleset engine offers, these changes allow customers to customize their zone’s traffic to the finest of margins.
The other benefit from building all of these products on the Rulesets Engine is extensibility. Currently there are over 30 products that are built and running on the Rulesets Engine. Each of these products is essentially a logical bucket called a ‘phase’ which contains a single ruleset scoped to that product. Each phase is restricted to specific actions and fields, for example the field cf.bot_management.score is unavailable in http_request_transform as we have not calculated the bot score at the time we perform URL rewrites. Also, only the rewrite action is permitted. Whereas in Origin Rules (http_request_origin) we only allow the action route
When we create new capabilities for a product that is built atop the Rulesets Engine it is trivially simple for us to extend that new capability to other products at a later date.
For example, we added a new ‘field’, http.request.accepted_languages to Transform Rules earlier in the year. Until recently this was only available in Transform Rules. However, as both products are built on the Rulesets Engine it was trivial to enable this feature for Dynamic Redirects. This allows customers to perform URL redirects based on the visitor’s language preference, and the cost to us from an engineering perspective is negligible as the field is already implemented.
This also means in the future should a new field be created for Cache Rules due to a customer request, e.g. http.request.super_cool_field, in a matter of clicks we can enable this field for any of the other 30 products rather than have to duplicate effort across multiple platforms.
Simply put, the more products we build on top of the Rulesets Engine, the faster we can move and the more functionality we can put into users hands.
A single user experience
The most important benefit of all is consistency. Each of these products has a consistent and predictable API. A consistent and predictable Terraform configuration, and a consistent and predictable user experience in the dashboard. The ruleset engine allows us to keep everything the same except for the ‘action’. The filtering stays the same, the API stays the same, the UI stays (largely) the same, the only change is the ‘…then’, the action section of the rule.
This ensures that as a user when you are clicking around the dashboard setting up a new zone you aren’t having to learn each individual product’s page and how to navigate it. The only part you need to learn is what makes that product unique, its actions:
Finally, when we add a new product, extending the Terraform provider to support it is trivial. That consistent experience has been a north star for us during this project and will continue to be in the future.
Try it them now
We are replacing Page Rules with a new suite of products, each built to be best-of-breed and put more power into the hands of our users.
Read more about the new products in each of their dedicated blogs. Then, try them for yourself!
