Post Syndicated from Stefan Gromoll original https://aws.amazon.com/blogs/big-data/amazon-redshift-rg-faster-and-lower-cost-graviton-powered/
Amazon Redshift recently announced the general availability of a new Graviton-powered instance called RG. Built on Amazon’s own Graviton processors, RG delivers:
- Up to 2.2x faster performance for data warehouse workloads compared to RA3.
- Up to 2.4x faster for Iceberg queries and 1.5x faster for Parquet queries through an integrated vectorized data lake engine.
- No per-TB scan charges for data lake queries, eliminating the Amazon Redshift Spectrum cost applied on RA3 clusters.
- 30 percent lower cost per vCPU compared to RA3.
RG is both faster and cheaper. While cloud vendors typically charge more for faster performance or newer generation hardware, Amazon Redshift delivers better performance at lower cost.
In this post, we describe the innovations that make RG instances so much faster. We also share benchmark results showing that RG delivers up to 4.2x better price-performance than other leading data warehouses.
What makes RG so fast
The new RG instances are built from the ground up to take advantage of Graviton processors. The vectorized engine of Amazon Redshift is optimized with Graviton-based single instruction, multiple data (SIMD) kernels to deliver accelerated, parallelized execution for analytics workloads. Operations like predicate evaluations over Parquet encodings use Graviton vector comparison, table lookup, and vector manipulation intrinsics. To support these increased processing speeds, RG instances use custom-built Nitro SSDs. This lets RG use faster local storage as a caching layer for Amazon Redshift Managed Storage (RMS), data lake scans, and intermediate result sets for computations that can’t fit in memory. RG’s JIT (Just-In-Time) Analyze feature also collects statistics from data lake files automatically as queries run, so the optimizer can produce significantly better query plans. Together, these represent innovation across the entire stack: hardware acceleration with Graviton, vectorized execution with SIMD kernels, high-speed storage with Nitro SSDs, and intelligent query planning with JIT Analyze.
These optimizations, coupled with RG’s purpose-built high-performance vectorized data lake engine, combine to make Amazon Redshift’s new RG instances up to 2.2x faster than RA3 for analytics workloads at 30 percent lower cost.
Purpose-built high-performance vectorized data lake engine
With RA3, data lake queries offloaded scans to a separate compute fleet known as Amazon Redshift Spectrum. Because data lake queries ran on this separate compute, additional overhead was introduced to transfer query metadata and results between RA3 clusters and the Spectrum fleet. Amazon Redshift RG instances include a completely new built-in scan layer designed from the ground up for data lakes. This new scan layer includes a purpose-built I/O subsystem that incorporates smart prefetch capabilities to reduce data latency. The new scan layer is also optimized to process Apache Parquet files, the most commonly used file format for Iceberg, through fast vectorized scans that use SIMD kernels optimized for Graviton. The scan layer includes sophisticated data pruning mechanisms that operate at both partition and file levels, which significantly reduces the volume of data that needs to be scanned. This pruning capability works with the smart prefetch system to create a coordinated approach that maximizes efficiency throughout the entire data retrieval process.
The new purpose-built vectorized data lake engine is up to 2.4x faster than RA3 for Iceberg queries and 1.5x faster than RA3 for Parquet queries.
Because this new vectorized data lake engine integrates directly with the core execution engine of Amazon Redshift, new performance optimizations are possible compared to RA3. With this architecture, data lake queries on RG now benefit from fast local data caching, improved bloom filters, vectorized Parquet scans, and advanced filtering and pruning.
RG also solves a common problem customers face when querying data in the lake: open-format files like Iceberg in Amazon Simple Storage Service (Amazon S3) often lack useful metadata and statistics, which makes it difficult to run a SQL query optimally.
Statistics are metadata about your data, such as distinct value counts, min/max values, distribution patterns, and row counts. The query optimizer uses this information to choose the most efficient way to run a query. For example, when joining two tables, the optimizer needs to know how many unique values each side produces to pick the right join strategy. Without statistics, it has to guess, which often leads to slower joins and unnecessary data movement across nodes. This is where Amazon Redshift’s new feature called JIT (Just-In-Time) Analyze comes in. RG instances automatically fetch and store statistics of your Iceberg files as queries run, so Amazon Redshift can choose query execution strategies that are far more optimized than it could without these statistics.
These improvements make scans of Iceberg and Parquet data much faster than RA3. Removing Amazon Redshift Spectrum compute also means RG instances remove the $5/TB cost for data lake queries, which makes data lake queries cheaper and costs predictable. This is a triple win for data lake price-performance: faster performance, lower compute cost, and no per-TB scan cost.
Faster insights from faster data loads
Amazon Redshift RG’s fast I/O and Graviton-optimized engine result in faster data loads compared to RA3. To measure this improved performance, we ran the data ingestion step of 10TB TPC-DS and TPC-H on equivalently sized RA3 and RG clusters. RG ingested the TPC-DS dataset 2x faster and the TPC-H dataset 1.4x faster, as shown in the following figure.
The new Graviton-based RG instances are up to 2.0x faster for data loads compared to RA3 instances. This means workloads can see the latest data sooner, and users and agents can get up-to-date insights faster. This faster ingestion on RG comes at 30 percent lower cost compared to RA3, resulting in up to 2.9x better price-performance for data loads compared to RA3 instances.
What customers are saying
Amazon Redshift customers are already seeing performance and cost benefits of switching to RG. Southwest Airlines and tombola tested their business-critical workloads, and found they could get better performance and save on cost:
|
Southwest Airlines
“Amazon Redshift RG instances have the potential to deliver meaningful business impact for Southwest Airlines. Based on initial testing in our development environment, our data warehouse workloads run 50–60% faster, and data lake analytics are 45% faster—enabling teams to get insights sooner, respond to operational conditions faster, and make data‑driven decisions with less latency. These early results are encouraging, and we are excited to validate and scale these improvements in production. All of this comes without per‑terabyte Spectrum scanning charges, delivering 30% lower cost than RA3 at a time when fuel prices continue to pressure industry margins!!” — Sean Lynch, Vice President, Data and Architecture, Southwest Airlines |
|
tombola
“The new Graviton-based Amazon Redshift RG instances delivered 1.8x–2x faster write throughput and up to 2.2x faster read speeds compared to RA3 across a diverse set of batch and analytical jobs — enabling us to process 40% more within the same window. Compressed ETL cycles, accelerated time-to-insight, and decision-making no longer bottlenecked by the pipeline — together, these translated directly into fresher data reaching our analysts and business teams sooner. What made this even more compelling was a concurrent 30% reduction in compute spend alongside the gains — delivering more for less is a rare outcome, and one worth highlighting. In a volume-heavy gaming industry at tombola, where query latency and cost compound at scale, this has been one of the more impactful platform decisions we’ve made this year.” — Akshay Srinivasan, Data Engineer, tombola |
|
Qoala
“After migrating our Amazon Redshift cluster from RA3 to the new Graviton-based RG instances, we saw 60–70% faster query processing times across our BI and analytics workloads. As a growing insurtech platform handling millions of policy transactions, faster time-to-insight means our data team can deliver dashboards and reports to the business sooner. We moved to a larger node configuration to accommodate future growth, and the performance gains far exceeded the incremental investment – making this one of the most impactful infrastructure decisions we’ve made this year.” — Umar Abdul Aziz, VP of Data, Qoala |
Performance results
To see how RG stacks up, we ran benchmarks derived from the industry-standard TPC-DS and TPC-H benchmarks at 10TB scale on the new Amazon Redshift RG instances and on leading alternative data warehouses. These benchmarks are designed to run queries of various operational requirements and complexities, such as ad hoc, reporting, iterative online analytical processing (OLAP), and data mining. We sized each data warehouse at approximately the same on-demand cost ($32/hr) and ran three power runs of each benchmark out of the box, with no special tuning or manual customization. The results are shown in the following charts.
The new RG instance leads, and by a large margin. Better price-performance means better performance and lower cost.
Conclusion
Amazon Redshift RG instances are the next generation of analytics engine, delivering high performance for data warehouse and data lake workloads. Because RG supports all the same workloads and features as RA3, getting started is straightforward. See our migration guide for how to upgrade and start getting better performance at lower cost.
Find the best price-performance for your workloads
The benchmarks used in this post are derived from the industry-standard TPC-DS and TPC-H benchmarks, and have the following characteristics:
- We use the schema and data unmodified from TPC-DS and TPC-H.
- The queries are generated using the official TPC-DS and TPC-H kits with query parameters generated using the default random seed of the kits. TPC-approved query variants are used for a warehouse if the warehouse doesn’t support the SQL dialect of the default queries.
- The test includes the 99 TPC-DS SELECT queries and 22 TPC-H SELECT queries. It doesn’t include maintenance and throughput steps.
- Three power runs were run, and the best run is taken for each data warehouse.
- Price-performance is calculated as the cost per hour (USD) divided by 3,600 seconds/hour times the benchmark geomean in seconds, which is equivalent to the geomean cost per query. The latest published on-demand pricing is used for all data warehouses.
We call this the Cloud Data Warehouse benchmark, and you can reproduce the preceding benchmark results using the scripts, queries, and data available in our GitHub repository. It’s derived from the TPC-DS benchmarks as described in this post, and as such isn’t comparable to published TPC-DS results, because the results of our tests don’t comply with the official specification.