Maximize Amazon EC2 Capacity Reservations with Capacity Manager data exports

Post Syndicated from Venu Geddam original https://aws.amazon.com/blogs/compute/maximize-amazon-ec2-capacity-reservations-with-capacity-manager-data-exports/

In our previous post, we introduced Amazon EC2 Capacity Manager and its data export capability. Amazon EC2 Capacity Manager provides centralized visibility into your Amazon Elastic Compute Cloud (Amazon EC2) capacity usage across all accounts and Regions in your organization. It tracks capacity usage for three types of EC2 capacity: On-Demand instances, Spot instances, and On-Demand Capacity Reservations (ODCR). On the AWS Management Console, it provides 90 days of historical capacity data. With data exports to Amazon Simple Storage Service (Amazon S3), you can retain and analyze capacity trends beyond this period using your preferred analytics tools.

In this post, we demonstrate how to configure EC2 Capacity Manager data exports to Amazon S3 and query historical capacity data using Amazon Athena. This approach helps you identify long-term usage patterns, plan capacity needs, and optimize resource allocation across your organization.

Solution overview

The following diagram illustrates the solution architecture. EC2 Capacity Manager exports capacity data to Amazon S3, where Amazon Athena queries it using SQL with automatic partition discovery.

Architecture diagram showing EC2 Capacity Manager exporting capacity data to Amazon S3 on a scheduled basis, and Amazon Athena querying the data directly from S3 using SQL with partition projection for automatic partition discovery

The solution involves the following steps:

  1. Set up an S3 bucket for capacity data export.
  2. Configure EC2 Capacity Manager data export.
  3. Set up Amazon Athena to query the exported data.
  4. Run queries to analyze capacity patterns.

Prerequisites:

  • An AWS account with permissions to create S3 buckets and configure EC2 Capacity Manager.
  • AWS Command Line Interface (AWS CLI) installed and configured (optional, for CLI-based setup).
  • Familiarity with SQL for querying data in Athena.

Setting up data export to Amazon S3

EC2 Capacity Manager can export capacity data in compressed CSV (Gzip) or compressed Parquet (Snappy) format. Use Parquet format for query performance in Athena (Parquet’s columnar format is designed to optimize analytical queries).

Configure the data export

You can configure data export through the EC2 Capacity Manager console or AWS CLI.

To configure data export using the console:

  1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
  2. In the navigation pane, choose Capacity Manager.
  3. Choose the Data exports tab.
  4. Choose Create data export.
  5. For Output format, select Parquet.
  6. For S3 bucket, choose Create bucket for me to create a new bucket with the required permissions, or select an existing bucket from the list.
  7. If you selected an existing bucket, add the bucket policy shown in the following section to grant EC2 Capacity Manager write access.
  8. (Optional) For S3 prefix, enter a prefix to organize your exported files (for example, capacity-data/).
  9. For Schedule, select Hourly.
  10. Choose Create.

Screenshot of EC2 Capacity Manager Create data export page in AWS Console showing export properties: Data export name, output format set to Parquet, S3 location for export delivery and Tags

After you create the data export, EC2 Capacity Manager displays the export details.

Screenshot showing EC2 Capacity Manager data export status with ‘Latest Delivery’ column displaying ‘delivered’ status, indicating the export is ready for Athena setup

Update the S3 bucket policy (existing buckets only)

If you use an existing S3 bucket, you must add the bucket policy shown in the following example to grant EC2 Capacity Manager write access.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "ec2.capacitymanager.amazonaws.com"
            },
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::<BUCKET_NAME>",
                "arn:aws:s3:::<BUCKET_NAME>/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "<AWS_ACCOUNT_NUMBER>"
                },
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:ec2:<AWS_REGION>:<AWS_ACCOUNT_NUMBER>:capacity-manager-data-export/*"
                }
            }
        }
    ]
}

Replace <AWS_ACCOUNT_NUMBER> with your AWS account number, <AWS_REGION> with your AWS Region (for example, us-west-2), and <BUCKET_NAME> with your bucket name.

To configure data export using the AWS CLI :

aws ec2 create-capacity-manager-data-export \
    --s3-bucket-name <BUCKET_NAME> \
    --s3-bucket-prefix <BUCKET_PREFIX>/ \
    --schedule hourly \
    --output-format <FORMAT> \
    --region <AWS_REGION>

# Replace:
# <BUCKET_NAME> with your bucket name,
# <BUCKET_PREFIX> with your bucket prefix (for example, capacity-data/),
# <FORMAT> with parquet or csv, and
# <AWS_REGION> with your AWS Region (for example, us-west-2)

# The output of the above command would give you a Data Export ID

{ "CapacityManagerDataExportId": "cmde-00a7d0e64e43889f1" }

After creating the data export, wait for the first export to complete before proceeding to set up Athena. You can check the export status using the following command:

aws ec2 describe-capacity-manager-data-exports --region <AWS_REGION>

The command returns details about your data export configuration, including the delivery status:

{
    "CapacityManagerDataExports": [
        {
            "CapacityManagerDataExportId": "cmde-00a7d0e64e43889f1",
            "S3BucketName": "capacity-manager-exports-123456789012",
            "S3BucketPrefix": "capacity-data/",
            "Schedule": "hourly",
            "OutputFormat": "parquet",
            "CreateTime": "2026-04-10T19:04:36.824000+00:00",
            "LatestDeliveryStatus": "delivered",
            "LatestDeliveryStatusMessage": "Successfully delivered to s3://capacity-manager-exports-123456789012/y=2026/m=04/d=10/h=15/...",
            "LatestDeliveryTime": "2026-04-10T19:05:39.176000+00:00"
        }
    ]
}

Wait until LatestDeliveryStatus shows "delivered" before proceeding to the next section. The first export typically appears in your S3 bucket in a couple of hours. Subsequent exports follow your configured schedule.

Setting up Amazon Athena to query capacity data

After EC2 Capacity Manager exports data to your S3 bucket, you can use Amazon Athena to query the data using standard SQL. Athena uses AWS Glue as its metadata store. Specifically, it relies on the AWS Glue Data Catalog, which contains table definitions that tell Athena where you have stored your data in S3 and how you have structured it. When you create tables in Athena, you’re actually creating metadata entries in the Data Catalog that Athena references when running queries.

Create an Athena database and table

You can create the table using AWS Glue crawler or manually with SQL. AWS Glue crawler automatically discovers the complete schema from your exported Parquet files, including optional fields like resource tags if enabled. It helps minimize manual schema definition efforts. If the export format changes in the future, you can re-run the crawler to update the table definition. For detailed instructions on creating a Glue Crawler, see Use a crawler to add a table in the Amazon Athena User Guide.

In this post, we create the table manually using a SQL statement. We also use partition projection for automatic partition discovery. We do this because EC2 Capacity Manager continuously adds new partitions to the S3 bucket according to your configured schedule. As new partitions arrive in S3, Athena doesn’t know about them until you run MSCK REPAIR TABLE or ALTER TABLE ADD PARTITION to update the AWS Glue Data Catalog. This becomes an overhead when data arrives frequently.

With partition projection, you define the partition scheme and specify its range/rules in the table properties. Athena then computes the partitions at query time instead of looking them up in the Glue Data Catalog. So partition projection automatically makes new partitions visible as soon as EC2 Capacity Manager exports the data to S3, eliminating the need for you to update metadata. The CREATE TABLE statement that follows defines the schema for EC2 Capacity Manager exports. If your capacity reservations are already tagged, add the corresponding tag columns (for example, tag_environment string or tag_costcenter string). Alternatively, use an AWS Glue crawler to automatically discover your complete schema, including tag columns.

  1. Open the Athena console at https://console.aws.amazon.com/athena/
  2. If prompted, configure a query result location in S3. This is where Athena writes query output. It is separate from the S3 bucket that stores your capacity data.
  3. Run the following query to create a database:
CREATE DATABASE IF NOT EXISTS capacity_manager_db;
  1. Create a table for Parquet format data:
CREATE EXTERNAL TABLE IF NOT EXISTS capacity_manager_db.capacity_data (
    metricgroupname string,
    periodstarttimestamp string,
    periodendtimestamp string,
    orgid string,
    accountid string,
    region string,
    `az-id` string,
    instancefamily string,
    instancetype string,
    platform string,
    tenancy string,
    reservationid string,
    `reservation arn` string,
    unusedfinancialowner string,
    reservationtype string,
    instancematchcriteria string,
    reservationcreatetimestamp string,
    reservationstarttimestamp string,
    reservationendtimestamp string,
    reservationenddatetype string,
    reservationstate string,
    reservationtotalcapacityhrsvcpu string,
    reservationtotalcapacityhrsinst string,
    reservationtotalestimatedcost string,
    reservationmaxsizevcpu string,
    reservationmaxsizeinst string,
    reservationminsizevcpu string,
    reservationminsizeinst string,
    reservationunusedtotalcapacityhrsvcpu string,
    reservationunusedtotalcapacityhrsinst string,
    reservationunusedtotalestimatedcost string,
    reservationmaxunusedsizevcpu string,
    reservationmaxunusedsizeinst string,
    reservationminunusedsizevcpu string,
    reservationminunusedsizeinst string,
    reservationmaxutilization string,
    reservationminutilization string,
    reservationavgutilizationvcpu string,
    reservationavgutilizationinst string,
    reservationavgfuturesizevcpu string,
    reservationavgfuturesizeinst string,
    reservationmaxfuturesizevcpu string,
    reservationmaxfuturesizeinst string,
    reservationminfuturesizevcpu string,
    reservationminfuturesizeinst string,
    reservationavgcommittedsizevcpu string,
    reservationavgcommittedsizeinst string,
    reservationmaxcommittedsizevcpu string,
    reservationmaxcommittedsizeinst string,
    reservationmincommittedsizevcpu string,
    reservationmincommittedsizeinst string,
    reservedtotalusagehrsvcpu string,
    reservedtotalusagehrsinst string,
    unreservedtotalusagehrsvcpu string,
    unreservedtotalusagehrsinst string,
    reservedtotalestimatedcost string,
    unreservedtotalestimatedcost string,
    spottotalusagehrsvcpu string,
    spottotalusagehrsinst string,
    spottotalestimatedcost string,
    spotavgruntimebeforeinterruptioninst string,
    spotmaxruntimebeforeinterruptioninst string,
    spotminruntimebeforeinterruptioninst string,
    spottotalinterruptionsinst string,
    spottotalinterruptionsvcpu string,
    spottotalcountinst string,
    spottotalcountvcpu string,
    spotinterruptionrateinst string,
    spotinterruptionratevcpu string
)
PARTITIONED BY (
    y string,
    m string,
    d string,
    h string
)
ROW FORMAT SERDE
    'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT
    'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
OUTPUTFORMAT
    'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
    's3://<BUCKET_NAME>/'
TBLPROPERTIES (
    'projection.enabled' = 'true',
    'projection.y.type' = 'integer',
    'projection.y.range' = '2024,2030',
    'projection.y.digits' = '4',
    'projection.m.type' = 'integer',
    'projection.m.range' = '01,12',
    'projection.m.digits' = '2',
    'projection.d.type' = 'integer',
    'projection.d.range' = '01,31',
    'projection.d.digits' = '2',
    'projection.h.type' = 'integer',
    'projection.h.range' = '00,23',
    'projection.h.digits' = '2',
    'storage.location.template' = 's3://<BUCKET_NAME>/y=${y}/m=${m}/d=${d}/h=${h}/'
);

Replace <BUCKET_NAME> in the LOCATION clause and storage.location.template property with your bucket name, and capacity-data/ with table name of your choice.

Now that your table is set up, you can explore how to query the exported data.

Example queries for common use cases

The following queries demonstrate how to analyze your EC2 capacity data for cost optimization and capacity planning. The queries use date values (year=‘2026’, month=‘04’) and table name capacity_data. Adjust the partition values to match your actual data’s time period and table name to match your value. When querying EC2 Capacity Manager export data:

  1. Metric group filtering: EC2 Capacity Manager exports contain two types of data that the metricgroupname column identifies: Reservation Usage for analyzing ODCR utilization and optimization opportunities, and Instance Usage for analyzing overall capacity consumption across reserved, unreserved, and Spot instances. Always filter by the appropriate metric group for your analysis needs.
  2. Partition filtering: Always include partition filters (y, m, d, h) to improve query performance.
  3. Numeric operations: Use CAST to convert string columns to numeric types for proper comparison and sorting (for example, CAST(reservationavgutilizationinst AS double)).
  4. NULL handling: Use COALESCE to handle NULL values in calculations (for example, COALESCE(CAST(column AS double), 0)) to prevent NULL results in totals. Without COALESCE, when you add a column value NULL to a non-NULL value, the result is NULL.

Use case 1: Identify underutilized ODCRs

Discover On-Demand Capacity Reservations with low utilization that are generating cost waste. Identify specific reservations to downsize, cancel, or share with other teams to reduce unnecessary spending.

SELECT
    reservationid,
    instancetype,
    region,
    "az-id",
    reservationstate,
    ROUND(CAST(reservationavgutilizationinst AS double) * 100, 2) AS utilization_pct,
    CAST(reservationtotalcapacityhrsinst AS double) AS total_capacity_hrs,
    CAST(reservationunusedtotalcapacityhrsinst AS double) AS unused_capacity_hrs,
    ROUND(CAST(reservationunusedtotalestimatedcost AS double), 4) AS wasted_cost_usd
FROM capacity_data
WHERE metricgroupname = 'Reservation Usage'
    AND CAST(reservationavgutilizationinst AS double) < 0.5
    AND reservationavgutilizationinst IS NOT NULL
    AND y = '2026'
    AND m = '04'
ORDER BY wasted_cost_usd DESC
LIMIT 3;

Sample output:

reservationid instancetype region az-id reservationstate utilization_pct total_capacity_hrs unused_capacity_hrs wasted_cost_usd
cr-041aedfba865106c1 m5.8xlarge us-west-2 usw2-az2 active 0 1 1 1.536
cr-089f17dc178e993a8 m5.8xlarge us-west-2 usw2-az1 active 0 1 1 1.536
cr-089f17dc178e993a8 m5.8xlarge us-west-2 usw2-az1 active 0 1 1 1.536

Use case 2: ODCR utilization summary by instance type

Get a comprehensive view of ODCR utilization across instance types to identify which instance families have the worst utilization rates. This helps prioritize optimization efforts on the reservations with the highest cost impact.

SELECT
    instancetype,
    COUNT(DISTINCT accountid) AS account_count,
    COUNT(DISTINCT reservationid) AS reservation_count,
    ROUND(SUM(COALESCE(CAST(reservationtotalcapacityhrsinst AS double), 0)), 2) AS total_odcr_capacity,
    ROUND(SUM(COALESCE(CAST(reservationunusedtotalcapacityhrsinst AS double), 0)), 2) AS total_unused_capacity,
    ROUND(AVG(COALESCE(CAST(reservationavgutilizationinst AS double), 0)) * 100, 2) AS avg_utilization_pct,
    ROUND(SUM(COALESCE(CAST(reservationunusedtotalestimatedcost AS double), 0)), 2) AS total_unused_cost_usd
FROM capacity_data
WHERE metricgroupname = 'Reservation Usage'
    AND y = '2026'
    AND m = '04'
GROUP BY instancetype
ORDER BY total_unused_cost_usd DESC
LIMIT 3;

Sample output:

instancetype account_count reservation_count total_odcr_capacity total_unused_capacity avg_utilization_pct total_unused_cost_usd
m5.8xlarge 1 2 311.99 311.99 0.0 479.22
c5.9xlarge 1 1 211.0 211.0 0.0 322.83
t3.micro 1 1 1055.0 1055.0 0.0 10.97

Use case 3: Identify peak usage patterns

Analyze average hourly usage patterns across reserved, unreserved, and Spot capacity to identify when your workloads typically hit peak demand. This breakdown helps you understand your capacity mix, plan for peak periods, and optimize your purchasing strategy.

SELECT
    h AS hour,
    ROUND(AVG(COALESCE(CAST(reservedtotalusagehrsinst AS double), 0)), 2) AS avg_reserved_usage_hours,
    ROUND(AVG(COALESCE(CAST(unreservedtotalusagehrsinst AS double), 0)), 2) AS avg_unreserved_usage_hours,
    ROUND(AVG(COALESCE(CAST(spottotalusagehrsinst AS double), 0)), 2) AS avg_spot_usage_hours,
    ROUND(AVG(COALESCE(CAST(reservedtotalusagehrsinst AS double), 0) + COALESCE(CAST(unreservedtotalusagehrsinst AS double), 0) + COALESCE(CAST(spottotalusagehrsinst AS double), 0)), 2) AS avg_total_usage_hours
FROM capacity_data
WHERE metricgroupname = 'Instance Usage'
    AND y = '2026'
    AND m = '04'
GROUP BY h
ORDER BY avg_total_usage_hours DESC
LIMIT 3;

Sample output:

hour avg_reserved_usage_hours avg_unreserved_usage_hours avg_spot_usage_hours avg_total_usage_hours
09 0.75 0.5 0 1.25
10 0.75 0.5 0 1.25
15 0.75 0.5 0 1.25

Use case 4: Regional capacity distribution

Understand how your ODCR capacity is distributed across AWS Regions and instance types. This geographic view helps you identify Regions with excess capacity that could be redistributed or consolidated to improve utilization and reduce costs.

SELECT
    region,
    instancetype,
    ROUND(SUM(COALESCE(CAST(reservationtotalcapacityhrsinst AS double), 0)), 2) AS total_reserved_capacity,
    ROUND(SUM(COALESCE(CAST(reservationunusedtotalcapacityhrsinst AS double), 0)), 2) AS unused_reserved_capacity,
    ROUND(AVG(COALESCE(CAST(reservationavgutilizationinst AS double), 0)) * 100, 2) AS avg_utilization_pct
FROM capacity_data
WHERE metricgroupname = 'Reservation Usage'
    AND y = '2026'
    AND m = '04'
GROUP BY region, instancetype
ORDER BY region, total_reserved_capacity DESC
LIMIT 3;

Sample output:

region instancetype total_reserved_capacity unused_reserved_capacity avg_utilization_pct
us-west-2 t2.nano 1484.0 1060.0 33.34
us-west-2 t3.micro 1060.0 1060.0 0.0
us-west-2 t2.micro 848.0 636.0 25.0

Use case 5: Unused capacity reservations by Region and Availability Zone

Pinpoint exactly where you have unused ODCR capacity at the Availability Zone level. This granular view enables you to share unused capacity with other teams in the same AZ or modify reservations to better match actual usage patterns.

SELECT
    region,
    "az-id",
    instancetype,
    ROUND(SUM(COALESCE(CAST(reservationunusedtotalcapacityhrsinst AS double), 0)), 2) AS unused_capacity_instances,
    ROUND(AVG(COALESCE(CAST(reservationavgutilizationinst AS double), 0)) * 100, 2) AS avg_utilization_pct,
    ROUND(SUM(COALESCE(CAST(reservationunusedtotalestimatedcost AS double), 0)), 2) AS unused_cost_usd
FROM capacity_data
WHERE metricgroupname = 'Reservation Usage'
    AND y = '2026'
    AND m = '04'
    AND CAST(reservationunusedtotalcapacityhrsinst AS double) > 0
GROUP BY region, "az-id", instancetype
ORDER BY unused_cost_usd DESC
LIMIT 3;

Sample output:

region az-id instancetype unused_capacity_instances avg_utilization_pct unused_cost_usd
us-west-2 usw2-az1 c5.9xlarge 212.0 0.0 324.36
us-west-2 usw2-az2 m5.8xlarge 157.0 0.0 241.15
us-west-2 usw2-az1 m5.8xlarge 157.0 0.0 241.15

Clean up

To avoid incurring future charges, delete the resources you created:

Warning: This permanently deletes the table definition. Verify that you no longer need to query this data before proceeding.

  1. Delete the Athena table by running the following query: DROP TABLE IF EXISTS capacity_manager_db.capacity_data;
  2. Delete the database by running the following query: DROP DATABASE IF EXISTS capacity_manager_db;
  3. Navigate to the Athena console settings.
  4. Note the query result location S3 bucket.
  5. If you created this bucket specifically for this tutorial:
    1. Empty the S3 bucket by running: aws s3 rm s3://<QUERY_RESULT_BUCKET_NAME> --recursive
    2. Delete the S3 bucket by running: aws s3 rb s3://<QUERY_RESULT_BUCKET_NAME>
  6. Delete the data export configuration by using AWS Management Console or AWS CLI.
    1. If using AWS Console, select the “Delete data export” option in the Actions menu.

Screenshot of AWS Management Console showing the Delete option in the Actions menu for an EC2 Capacity Manager data export configuration

  1. To delete the configuration using AWS CLI: 
    # List your data export configurations to find the export ID.
aws ec2 describe-capacity-manager-data-exports --region <AWS_REGION>

# The output shows details for export configuration which has the Data Export ID
# Sample output is shown in the configure data export section above.

# Then delete the export configuration using the ID from the output
aws ec2 delete-capacity-manager-data-export \
    --data-export-id <EXPORT_ID> \
    --region <AWS_REGION>

    Replace <EXPORT_ID> with your data export ID and <AWS_REGION> with your AWS Region.

    Warning: Deleting the S3 bucket permanently removes all exported capacity data. Verify that you have backed up any data you need before proceeding.

  1. (Optional) Delete the S3 bucket and data: If you no longer need the exported data, complete the following steps:
    1. Empty the S3 bucket by running: aws s3 rm s3://<BUCKET_NAME> --recursive
    2. Delete the S3 bucket by running: aws s3 rb s3://<BUCKET_NAME>

Conclusion

In this post, we demonstrated how to configure EC2 Capacity Manager data exports to Amazon S3 and query historical capacity data using Amazon Athena. This approach enables you to retain capacity data beyond the 90-day console limit.

As you scale your capacity management practices, consider integrating these exports with your existing analytics and monitoring workflows. By combining EC2 Capacity Manager data with your broader infrastructure metrics, you can make data-driven decisions about capacity allocation and optimization across your organization.

To deepen your understanding, explore the EC2 Capacity Manager documentation for additional features, learn more about Amazon Athena for advanced query capabilities, and review EC2 capacity optimization best practices. Share your feedback and tell us how you’re using EC2 Capacity Manager data exports to optimize your capacity planning in the comments.

[$] Hardening the kernel with allocation tokens and bootpatch-SLR

Post Syndicated from corbet original https://lwn.net/Articles/1078699/

There is a lot of work going into eliminating exploitable bugs from the
kernel and preventing the addition of new ones. Even if this work is
maximally successful, though, there is no chance that the kernel will be
free of these bugs anytime soon. Thus, there is also ongoing interest in
hardening the kernel to make the existing bugs more difficult to exploit.
The upcoming 7.2 kernel release will include a change to how dynamically
allocated structures are placed in memory to make them harder to overwrite,
while a project to randomize structure layout at boot time has a rather
longer timeline.

Как Русия унищожава българската идентичност в окупираните територии на Украйна

Post Syndicated from original https://www.toest.bg/kak-rusiya-unishtozhava-bulgarskata-identichnost-v-okupiranite-teritorii-na-ukrayna/

Как Русия унищожава българската идентичност в окупираните територии на Украйна

Украинските българи са най-голямата историческа българска общност извън границите на държавата. Общият им брой е над 200 000, а по-голямата част от тях живеят в Одеска област. Около 30 000 етнически българи населяват Запорожка, Херсонска и Донецка област и след февруари 2022 г. се намират под временна руска военна окупация. За съдбата на тези 30 000 души през последните четири години се знае твърде малко, а българското общество и политиците у нас очевидно смятат темата за доста неудобна заради нуждата от пряка конфронтация с руската държава. 

Крайно време е това да се промени. 

Украинските българи са успели да запазят народните си обичаи, езика и културата си още от средата на XIX век, когато на вълни се преместват от Балканите на север към дивата степ, каквато е представлявала тогава Южна Украйна. Техните общности са преминали през изключителни исторически изпитания, включително сталинския терор в СССР, Гладомор и целенасочена съветска политика за изличаване на българската им идентичност. България като свободна страна, членка на ЕС и НАТО е длъжна да защити украинските българи под руска окупация чрез максимален дипломатически натиск върху Русия и чрез допълнителна военна помощ за Киев и да не допусне унищожаване на националната им идентичност. Това е въпрос на национален интерес, но преди всичко на морал.

Руската доктрина за изличаване на идентичност

За да се разбере в дълбочина трагедията на българското малцинство, живеещо в момента под вражеска окупация в Украйна, случващото се трябва да бъде разгледано през призмата на общата стратегия на Кремъл. Руската политика във временно окупираните области на Украйна не се изчерпва с военен контрол на територии и заграбване на ресурси. Целта ѝ е пълно асимилиране на местното население с целенасочена, системна и брутална кампания.

В анализ на Atlantic Council тези действия от страна на Москва се дефинират като опит за „изличаване на идентичността“ на окупираното население. Крайната цел на Владимир Путин е изграждането на нова изкуствена демографска реалност, в която няма място за национално самоопределяне, различно от официалния имперски наратив на Москва.

В тази доктрина на културен геноцид образованието и администрацията са превърнати в оръжие. Веднага след установяването на военен контрол в Запорожка, Херсонска и Донецка област окупационните власти задействат насилствена русификация. Местните училищни програми са заменени с руските държавни стандарти. Руският език е наложен като единствен език на администрацията, образованието и публичните институции, а използването на украински е системно ограничавано и репресирано.

Този жесток подход не е нов. Исторически е познат от германизацията на окупираните полски територии през Втората световна война, русификацията на Полша, Литва и Украйна в Руската империя през XIX век, съветизацията на балтийските държави след 1940 г., дори и отвъд океана, в САЩ и Канада през XIX и XX век, когато деца от коренните народи са извеждани насилствено от семействата им и са настанявани в интернати, където им се забранява да говорят на родния си език и да практикуват традиционната си култура в опит да им се наложи нова национална идентичност. В тези интернати много от тях загиват.

Руската окупационна политика е насочена както срещу украинците като по-голяма част от населението на окупираните територии, така и срещу десетки хиляди хора, които не са етнически украинци, но представляват неразделна част от културната мозайка на региона. Такива са местните българи, гърци, татари и др. Именно в капана на тази безмилостна машина за заличаване на идентичност и език днес са уловени и 30-те хиляди таврийски българи, чиито вековни традиции са на път да бъдат унищожени от Русия.

Българската реакция – твърде скромна и много закъсняла

Преди руската инвазия през 2022 г. таврийските българи в Украйна имат пълни граждански права и свободно се възползват от образование на български език и от широко развити български културни дейности. Местните общности с подкрепата на властите в Киев активно работят за запазване на българското народно самосъзнание в Запорожка област, координирайки откриването на неделни училища и културни центрове с подкрепата на София.

Сред най-важните места на тези общности са училище „Васил Левски“ в Бердянск, неделното българско училище в Мелитопол и украинско-българският лицей в град Приморск. През 2023 г. образователният център в Приморск е ограбен от руски военни, които изнасят от сградата компютърна техника и ценно имущество, както свидетелстват очевидци. Учебното заведение има 27-годишна история, като близо половината от неговите над 1000 завършили ученици са продължили образованието си в университети в България.

След разпадането на СССР и обявяването на украинската независимост таврийските българи успяват да изградят мемориална мрежа, посветена на националните ни герои, а тези паметници стават центрове на ежегодни събори, фестивали и чествания на Деня на бесарабските българи. До началото на войната регионът се развива като свободна и сигурна среда, в която българската култура и език съжителстват в пълен синхрон с украинската гражданска идентичност. 

Населените с българи украински територии в Запорожка област са окупирани от руската армия още в първите дни и седмици на пълномащабната инвазия след 24 февруари 2022 г. През юли 2022 г. разследване на „Свободна Европа“ разкрива руската окупационна политика в Южна Украйна, под чиито удари попадат всички български училища и културни центрове в региона. Москва прехвърля пропагандни методи, които десетилетия наред са прилагани в самата Русия.

Според документирани свидетелства на Human Rights Watch и други международни организации окупационните власти са налагали руските учебни програми чрез натиск върху родители и учители. В отделни случаи родители са били заплашвани с глоби, задържане или отнемане на родителски права, ако откажат да запишат децата си в училища под руски контрол или ако продължат обучението им по украинската програма.

Тази принудителна русификация и заличаване на украинския, българския и други езици в областта протича в почти пълен информационен вакуум поради блокирани комуникации с външния свят в условията на война. През септември 2022 г. в интервю по Нова телевизия проф. Владимир Милчев, етнически българин и декан на Историческия факултет в Запорожкия университет, за първи път официално алармира, че в окупираните територии руските власти са наложили пълна забрана на българския език.

Всички неделни училища, културни центрове и дружества в Мелитополски и Бердянски район – сърцето на таврийските българи, са затворени. Учебните програми по български език, история и традиции са ликвидирани, а на местните преподаватели е поставен ултиматум: да преминат изцяло към руските държавни стандарти или да напуснат. 

Тази политика на етническо заличаване на българите от страна на руските нашественици в Украйна получава своето институционално потвърждение в България няколко месеца по-късно. През декември 2022 г. Агенцията за българите в чужбина официално обобщава мащаба на образователната катастрофа под руска окупация. Данните сочат, че руската окупационна администрация е прекратила дейността на 13 български неделни училища в Запорожка област, в които дотогава са се обучавали над 1000 деца.

Според руската посланичка у нас Елеонора Митрофанова забрана за изучаване на български език в училищата в Запорожка област, Мариупол и Бердянск няма. Има специфична организация.

Нямаме никакви забрани за изучаване на езици, но имаме специфична система за организиране на този процес. По-конкретно, това зависи от броя на децата, които искат да изучават български, литовски, грузински или друг език. Ако се достигне необходимият брой деца, училището винаги ще се съобрази с техните желания. Но тъй като българската страна повдигна този въпрос, аз, разбира се, ще изясня ситуацията с ръководството и хората, отговорни за тези региони, за да се разбере реалната ситуация,

казва Митрофанова в интервю за ТАСС, цитирано от OffNews.bg.

Физически терор, репресии и заплахи за сексуално насилие

Зад фасадата на административните забрани и затварянето на училища обаче се крие далеч по-мрачна реалност – физически терор, изтезания и страх за живота, които принуждават местните българи масово да напускат домовете си. Специален доклад на украинския омбудсман за правата на националните малцинства в условията на руска агресия разкрива шокиращи лични свидетелства на етнически българи, преминали през ада на окупацията в Запорожка и Херсонска област.

Един от основните мотиви за бягство на жените от българската общност е постоянният страх от сексуално насилие от страна на руските окупатори. За армията на Путин изнасилванията на жени в Украйна са просто още едно оръжие във войната, средство за унижение и мъчение на нападната страна. Цитирана в доклада представителка на българското малцинство от окупираната част на Запорожието разказва пред разследващите за системния натиск, на който са подложени жените на публични места в Мелитопол от страна на руските военни формирования и в частност от т.нар. кадировци:

Пътувах от селото до пазара в Мелитопол, за да продавам мляко и да изкарам пари. Често вземах дъщеря си с мен, защото училището в селото вече не работеше – окупаторите го превърнаха във военна база, а ме беше страх да я оставя сама. В един момент в града дойдоха много кадировци. Постоянно се заканваха и тормозеха младите жени на пазара, по улиците, по спирките. Пияни, с оръжие в ръце, те се държаха с нас като с робини и демонстрираха пълната си власт.

Българите в окупираните територии са обект на политически репресии, филтрация и затваряне в тайни центрове за изтезания (т.нар. мъчилища). В доклада е документиран тежкият случай на мъж с български корени от Херсон, задържан от руските сили заради участие в проукраински протести, доброволческа дейност и изразяване на позиции в социалните мрежи. Той отказва да напусне града преди окупацията, защото се грижи за 77-годишния си баща, болен от рак. Свидетелството му за момента на ареста показва абсолютната безмилостност на окупационния режим:

Към 5 сутринта се събудих от ярка светлина в прозореца и крясъци в двора, последвани от силно блъскане по вратата. Петима въоръжени окупатори с маски нахлуха в къщата, докато други петима чакаха отвън. Наредиха на мен и баща ми да седнем на дивана, докато обискират. Когато ме натикаха в колата, видях как баща ми изскочи на улицата – викаше и плачеше. Той е на 77 години, с онкологично заболяване, много слаб. Един от войниците го удари с автомат в гърдите. Видях как падна на земята. Това беше последният път, в който го видях.

Тези и други свидетелства, събрани от кабинета на украинския омбудсман, категорично доказват, че руската окупационна политика спрямо таврийските българи е част от кампания на терор, при която отстояването на човешкото достойнство, свободната воля или каквато и да е идентичност, различна от наложената от Кремъл, се наказват с насилие, отвличания и масов страх.

Срещу това унищожаване на българската идентичност държавата ни реагира едва през 2025 г., когато в отговор на депутатски въпрос Георг Георгиев, министър на външните работи, потвърждава, че над 30 000 таврийски българи в окупираните Запорожка, Херсонска и Донецка област са подложени на системно и грубо погазване на основните човешки права от страна на Москва.

България официално отчита, че окупационните власти целенасочено унищожават възможностите за изучаване на майчиния език и затварят българските неделни училища и центрове. От външното ни министерство подчертават, че тези действия на Кремъл представляват грубо нарушение на международното хуманитарно право и са директен опит за насилствено заличаване на етническата и културна идентичност на българската общност в Украйна.

България на Радев като съучастник във войната срещу българите

В контекста на доказаното унищожаване на част от най-старата българска общност, опазила българския дух от XIX век насам, позицията на държавата изглежда, меко казано, неадекватна и клони към национално предателство. Румен Радев практически от началото на политическата си кариера повтаря опорните точки на руската пропаганда и се доказва като един от най-близките до Путин европейски политици.

Руският терор срещу българите под окупация е тема, която властта активно прикрива с мълчанието си, помагайки по този начин за унищожаването на общността на таврийските българи. Това престъпно мълчание превръща българските управляващи в директни съучастници в етническото прочистване на нашите сънародници под руска окупация. Докато таврийските българи биват подложени на терор и насилствена русификация, София позорно си затваря очите заради зависимостта на Румен Радев от Москва. Подобно абдикиране от националния интерес е исторически срам и унижение, което с всеки изминал ден заличава вековната история на българите в Украйна и обрича сънародниците ни на забвение.

Security updates for Thursday

Post Syndicated from jzb original https://lwn.net/Articles/1079551/

Security updates have been issued by AlmaLinux (libpng, libsolv, libtasn1, libxml2, libxslt, python3.14, tigervnc, and vim), Debian (cloud-init, postgresql-13, and yelp), Mageia (nats-server), Oracle (.NET 10.0, .NET 8.0, .NET 9.0, bind9.18, cockpit, compat-openssl11, dnsmasq, dovecot, evince, expat, flatpak, freerdp, gimp, golang, grafana, grafana-pcp, httpd, jmc, jq, kernel, libsndfile, libsoup, libtiff, mod_http2, mysql:8.0, nginx, nginx:1.24, openexr, php:8.2, poppler, pyOpenSSL, python-markdown, redis:7, samba, thunderbird, tigervnc, unbound, and vim), Red Hat (libpng, libpng12, and libpng15), SUSE (apptainer, bind, crun, freeipmi, ghc-crypton-x509-store, ghc-crypton-x509-system, google-guest-agent, google-osconfig-agent, GraphicsMagick, gstreamer-plugins-bad, hamlib, iproute2, java-1_8_0-openjdk, kubevirt1, libarchive, libheif, libpng15, mbedtls, mbedtls-2, openssl-1_1, python-biopython, python-PyJWT, tar, webkit2gtk3, and xen), and Ubuntu (ffmpeg, libdbi-perl, and perl).

How we built saga rollbacks for Cloudflare Workflows

Post Syndicated from Vaishnav Kavitha original https://blog.cloudflare.com/rollbacks-for-workflows/

Cloudflare Workflows allows you to build durable, multi-step applications with built-in retries and state persistence across long-running processes. When a Workflow executes, each step can call external systems, retry failures, and persist state across restarts. But if one step fails, it may leave earlier work from completed steps in an inconsistent or partial state.

Today we’re shipping saga rollbacks for Workflows, allowing you to declare rollback logic within the step itself, in case of failure.

For example, consider a workflow for transferring funds between accounts at two different banks:

  1. Debit from account at Bank A

  2. Credit to account at Bank B

  3. Send email confirmation to both account owners

What happens if Step 2, the credit to account at Bank B, fails? Once the debit succeeds at Bank A, the transaction is committed and the money has left its system. As the orchestrator of the transaction, you cannot simply “undo” the operation in Bank A’s system. Instead, the money must be credited back to the account at Bank A through a new operation that semantically reverses the first one.


This pairing of an operation and its compensation logic is called the saga pattern.

Before today, developers had to implement their own compensation logic to track what succeeded, what failed, and what actions should be taken upon failure, outside of the steps’ direct definitions. Now, you can define compensation logic for each step.do() as an argument within the steps themselves, maintaining your workflow’s durability for the rollback as well.

// track what completed so we know what to undo
let debitA;
let creditB;
try {
  debitA = await step.do("debit-bank-a", () => bankA.debit(from, amount));
  creditB = await step.do("credit-bank-b", () => bankB.credit(to, amount));
  await step.do("notify", () => notifyBoth(from, to, amount));
} catch (error) {
  // unwind in reverse. each undo is its own durable step,
  // must be idempotent, and must keep going if one fails.
  if (creditB) {
    try {
      await step.do("reverse-credit-b", () => bankB.debit(to, amount, creditB.id));
    } catch (e) {
      await alertOnCall("reverse-credit-b failed", e);
    }
  }
  if (debitA) {
    try {
      await step.do("refund-debit-a", () => bankA.credit(from, amount, debitA.id));
    } catch (e) {
      await alertOnCall("refund-debit-a failed", e);
    }
  }
  throw error;
}

Without rollbacks

// each step ships with its own undo. add a step,
// add its rollback right here. no growing catch
// block, no manual ordering, no replay logic.
await step.do("debit-bank-a", () => bankA.debit(from, amount), {
  rollback: async ({ output }) => bankA.credit(from, amount, output.id),
});
await step.do("credit-bank-b", () => bankB.credit(to, amount), {
  rollback: async ({ output }) => bankB.debit(to, amount, output.id),
});
await step.do("notify", () => notifyBoth(from, to, amount));

With rollbacks

Try it out

To use rollbacks, just pass an options object containing a rollback function as the last argument to step.do().

const debit = await step.do(
  "debit-account-a",
  async () => {
    return await bankA.debit({
      accountId: fromAccountId,
      amount,
      idempotencyKey: `${transferId}:debit-account-a`,
    });
  },
  {
    rollback: async () => {
      await bankA.credit({
        accountId: fromAccountId,
        amount,
        idempotencyKey: `${transferId}:rollback-debit-account-a`,
      });
    },
  }
);

// The idempotency keys make both the forward operations and rollback operations safe to retry without duplicating the transfer

const credit = await step.do(
  "credit-account-b",
  async () => {
    return await bankB.credit({
      accountId: toAccountId,
      amount,
      idempotencyKey: `${transferId}:credit-account-b`,
    });
  },
  {
    rollback: async ({ output }) => {
      if (output === undefined) {
        return;
      }

      await bankB.debit({
        accountId: toAccountId,
        amount,
        idempotencyKey: `${transferId}:rollback-credit-account-b`,
      });
    },
  }
);


// If we fail here, we may want to revert all previous payments. Users should not have to wrap their code in complex try-catch logic just to revert two small payments (see below)

await step.do("send-confirmation", async () => {
  await sendTransferConfirmation({ ... });
});

Rollback functions should be idempotent, just like regular Workflow steps. If you refund a charge, use the payment provider’s idempotency key. If you release inventory, make the release safe to call more than once.

If any step fails, the rollback handlers will execute in reverse step-start order. It sounds simple: run the undo steps when something fails. In practice, there are a few details that make the API and execution model important.

1. The failed step may still need rollback. A failed step.do() can still be rollback-eligible if it registered a rollback handler.

The rollback will not start if user code catches an error and the Workflow continues, but if a step error is caught and the Workflow later fails for another reason, rollback can still run for previously registered handlers, which execute in reverse step-start order.

Why? The step may have partially interacted with an external system before failing. For example, a payment provider may capture a charge, but the step may fail before returning the chargeId to Workflows. That is why rollback handlers receive output, but must handle output === undefined.

2. Rollback only starts when the Workflow fails. Adding a rollback handler does not mean every step error triggers rollback. If user code catches an error and continues, the Workflow continues. Rollback starts when the Workflow itself is about to fail terminally.

When rollback starts, Workflows finds eligible step.do() calls, runs their rollback handlers, then records the final Workflow failure.

3. Ordering has to be predictable. For sequential Workflows, rollback order feels obvious:

  1. Reserve inventory.

  2. Charge card.

  3. Create shipment.

  4. If shipment fails, refund the card and release the inventory.

Parallel steps make this more subtle. Completion order can differ from start order, so Workflows uses reverse step-start order instead of reverse completion order.

The practical rules are:

  1. Any started or completed steps with rollback handlers are eligible.

  2. The failing step.do() is also eligible if it registered a rollback handler.

  3. Handlers run in reverse step-start order, not completion order.

How we designed the API

Once we had the expected behavior in mind, we had to add this new pattern into the Workflows API. Rollbacks went through a few iterations before we landed on rollback options. 

Why not a fluent or builder API?

The first approach was a fluent form: step.do(...).rollback(...) It reads well. The forward action and the compensation sit next to each other, and the call site looks like ordinary JavaScript chaining.

The problem is that step.do() already has an important meaning: it starts a durable step and returns a Promise for the step output. In Workers, promise-like values are especially meaningful because Workers RPC supports promise pipelining, a pattern inherited from systems like Cap’n Proto.

Promise pipelining lets code call a method on a future value before that value has fully returned to the caller. For example:

const session = api.authenticate(apiKey);
const name = await session.whoami();

Here, session is not the real session object yet. It is more like a handle to the session that will exist soon. When you call session.whoami(), Workers can send that call to the remote side early and say: “once authentication creates the session, call whoami() on it.”


That saves a round trip. The caller does not need to wait for authenticate() to fully finish before asking for whoami().

We considered a fluent API:

step.do("charge-card", chargeCard).rollback(refundCharge);

To a reader, that can look like “call .rollback() on the result of charge-card.”   But rollback is not part of the step’s output. It is part of the step.do() options, registered before the step starts, so Workflows knows how to compensate the step if a later step fails.

A fluent API also makes step timing harder to reason about. Today, step.do() starts the step when it is called, so developers can start a step, do other work, and await the first step later:

const first = step.do("first", () => serviceA.call());

await step.do("second", () => serviceB.call());

await first;

With today’s execution model, first starts immediately, before second. A fluent API would complicate that. Workflows would need to wait and see whether .rollback() gets attached before it knows the full step definition. That could delay when the step is sent to the engine.

In the earlier example, first could start at await first instead of at step.do("first", ...), after second has already completed.

That makes concurrent Workflows harder to reason about: step timing would depend on when the returned Promise is consumed, not just where step.do() is called.

We also considered a builder-style API:

const charge = await step
	.saga("charge")
	.do(() => chargeCard())
	.rollback(() => refundCharge())
	.run();

A builder API avoids the Promise ambiguity. It also gives us an obvious place for future step-level options, and makes it clear that the forward action and rollback action belong to the same saga step.

But it adds ceremony. Every step needs a final .run(), forgetting .run() would be easy and hard to spot without tooling, and simple one-step cases start to look like configuration chains. It also introduces a new step.saga() builder, breaking from the existing step.<action> pattern. Most importantly, it makes step.do() feel like an older API rather than the primary Workflows primitive. The goal of rollback was to extend step.do(), not replace it.

Rollback as step metadata

step.do(..., { rollback })

Ultimately, we chose the explicit form where rollback is metadata on the step.

This way, each rollback is defined within the forward step itself. Each handler receives the error that caused the rollback to start, the step context, and the output, which is either the persisted value returned by the forward step (which can be undefined) or undefined if the step failed before persisting a value.

Rollbacks emit lifecycle events, so you can tell whether compensation started, which rollback handler failed, and whether rollback completed successfully.

Crucially, the original Workflow failure remains separate: rollback is what Workflows does after the failure, not the reason the Workflow failed.

Just as you can define custom retry and timeout behavior in the step configuration via WorkflowStepConfig, you add rollback-specific values in rollbackConfig.

{
  rollback: async ({ output }) => {
    await bankA.credit({ accountId: fromAccountId, amount, transferId: `${transferId}-reversal` });
  },
  rollbackConfig: {
    retries: { limit: 10, delay: '30 seconds', backoff: 'exponential' },
    timeout: '2 minutes',
  },
}

This matches the lifecycle-event mental model we wanted. A step.do() already describes a durable unit of work that Workflows records, retries, and later shows in logs. Rollback is another lifecycle behavior for that same unit of work. It should travel with the step definition, not live in a separate wrapper or builder.

  • The step still starts when step.do() normally starts.

  • The returned promise still represents the step output.

  • Concurrent Workflow code keeps the same execution model.

  • Retry and timeout options for rollback live next to the rollback handler.

  • Existing step.do() calls keep working exactly as they do today.

This shape is slightly more explicit than the fluent API, but that explicitness is useful. The operation and its compensation are still in one place, and the API does not introduce a new step builder or a new kind of promise. Developers who already understand step.do() only need to learn one additional options object.

This is less magical, but it is simpler to adopt, and clearer to understand.

How it works under the hood

Rollback feels like a small API addition, but it changes what Workflows needs to record about each step.

A regular step.do() already has a durable record. Workflows records that the step started, whether it completed, what it returned, and whether it should be skipped instead of repeated if the Workflow resumes later.

Rollbacks add one more thing to that record: whether the step registered compensation logic.

This means Workflows has two pieces of information to bring together if the Workflow fails.

The first is durable step history. The Workflow engine stores data to know what ran, what completed, what output was saved, and whether rollback was registered.

The second is the rollback handler itself, which is the function written to compensate for that step. Workflows does not save the text of that function as data. Instead, it keeps a callable reference to the handler while the Workflow is running.

In Workers RPC, this kind of callable reference is called a stub. A stub lets one part of the system call code that is running somewhere else. Stubs also have lifetimes such that they can be disposed when a call or execution context ends. If you need to keep a stub past that point, Workers RPC provides a dup() method, which creates another handle to the same target.

For rollback, that model is useful. The durable step history records what needs compensation. The rollback stub gives Workflows a way to invoke the compensation code. And because rollback handlers may need to outlive the immediate step.do() call that registered them, Workflows keeps its own callable reference to the handler for the rollback phase.

In the common case, when a Workflow enters rollback in the same engine lifetime, Workflows already has the rollback stubs it needs. It can use the durable step history to find eligible steps, then invoke the rollback stubs that were registered during forward execution.

This gets more subtle when Workflows has to recover after a restart.

If the engine is evicted, crashes, or restarts while rollback is needed, Workflows still has the durable step history, but it may no longer have the in-memory rollback stubs. To recover, Workflows uses replay: a recovery mode where it can re-run the Workflow code without re-executing completed forward step bodies.

When replay reaches a completed step.do(), Workflows reads the persisted result instead of running the step body again. For rollback recovery, Workflows only needs to rebuild handlers for steps that had rollback attached and are eligible for rollback. As those step.do() calls are encountered, their rollback options can register the callable stubs again

That lets Workflows recover the rollback handlers it needs without duplicating the original external side effects.


With those pieces in place, rollback can work whether the handler is still available in memory or has to be rebuilt during recovery.

When the workflow is about to fail, Workflows does not ask your application to reconstruct what happened. It already has the step history. It can look at the persisted record and answer the important questions:

  • Which steps started?

  • Which steps finished?

  • Which failed step may still need cleanup?

  • Which steps registered rollback handlers?

  • What output should each rollback handler receive?

  • What order should compensation run in?

Then Workflows invokes each rollback stub with a rollback context: the original error, the step context, and the step output, if one was persisted.

The ordering detail matters. In normal JavaScript, especially with Promise.all(), completion order is not always the same as start order. If step A starts first and step B starts second, step B might finish first. For rollback, Workflows uses the persisted start order as the stable source of truth, then unwinds it in reverse.

Rollback handlers also run through Workflows’ normal step machinery. That means compensation gets the same operational properties you expect from Workflows: retries, timeouts, lifecycle events, logs, and a final recorded outcome. If a rollback handler keeps failing after its configured retries, Workflows records the rollback outcome as failed, stops running the remaining rollback handlers, and the Workflow instance ultimately ends in the Errored state.

This is the main difference between saga rollbacks and a catch block. A catch block only knows what is still in memory at its exact point in your JavaScript execution. Workflows rollback uses persisted step history to decide what already happened, invokes the stubs it already has in the common case, and safely rebuilds missing stubs during recovery when it needs to.

That is also why the API puts rollback on step.do() itself. Rollback is not a separate global error handler — it is metadata attached to the durable unit of work Workflows already understands.

What’s next

Our first iteration of rollbacks includes: 

  • Explicit per-step rollback handlers for step.do()

  • Sequential rollback execution

  • Retry and timeout configuration for compensation

Next, we want to explore:

When a multi-step application fails halfway through, the hardest part is often not knowing that it failed. It is knowing what already happened, and what needs to happen next.

Saga rollbacks let you put that answer directly beside each step. If you are building multi-step applications with Workflows, try saga rollbacks and tell us what compensation patterns you want next. Get started with the Workflows documentation and share feedback in the Cloudflare Community.

Experts on Experts: Why AI and Compliance Are Forcing A New Security Operating Model

Post Syndicated from Corey Thomas original https://www.rapid7.com/blog/post/it-experts-video-series-ai-compliance-force-new-security-operating-models

This week on Experts on Experts, I sat down with Sabeen Malik, Rapid7’s VP of Global Government Affairs and Public Policy, to discuss a shift security leaders can’t afford to treat as separate threads: frontier AI, vulnerability discovery, cybersecurity compliance, and operational resilience.

AI is changing how quickly vulnerabilities can be found, validated, and potentially exploited. At the same time, regulators, boards, and customers are asking for stronger proof that controls are working and risk is being reduced. Security leaders are being pushed to move at machine speed while proving the business is resilient.

AI vulnerability discovery is moving faster than security standards

Sabeen and I started with the policy question. Many of the systems security teams rely on today were designed for a slower era of human-led discovery. Vulnerability disclosure processes, scoring systems, prioritization frameworks, and regulatory expectations all assume organizations have time to assess, verify, and respond.

Frontier AI challenges that assumption. If models can help find and chain vulnerabilities faster, the industry needs stronger standards around verification, access, disclosure, and accountability. Access to powerful models matters, but access alone does not solve the governance problem. The bigger question is whether the ecosystem can responsibly validate, prioritize, and act on what these systems produce.

AI in cybersecurity must move from discovery to risk reduction

For defenders, faster discovery is only useful if it leads to faster action. Finding more vulnerabilities does not automatically make organizations safer. In many cases, it creates more noise for teams already under pressure.

The real challenge is exploitability. Security teams need to understand which risks are actually reachable, which issues matter most in their environment, and where action will reduce exposure fastest. That is where the shift from reactive security to preemptive security becomes critical. The goal is to use data, context, AI, and expertise to act earlier, not simply respond faster after something happens.

Cybersecurity compliance is becoming continuous

We also discussed how the compliance environment is changing. Organizations are no longer being asked to prove readiness once a year. Increasingly, they need to provide detailed evidence on shorter timelines across a growing set of regulatory and assurance requirements.

That creates a real challenge when evidence is collected manually or disconnected from live security operations. Leaders need to show what changed, what was fixed, who owns the response, and what risk remains. Static snapshots are no longer enough.

Cyber GRC connects security operations, risk, and compliance

One of the clearest themes from the conversation is that the future of security operations will be AI-driven, but human-led. AI can help teams move faster, surface what matters, and respond with greater scale and consistency. But governance, accountability, and judgment still matter.

That same principle applies to compliance. Security and compliance teams need live operational context, not disconnected reports. They need to connect what they detect, what they fix, and what they can prove.

Watch the full episode to hear our conversation on what this moment means for AI in cybersecurity, cybersecurity compliance, and resilient security operations:

⠀

Interesting Paper Exploring Prompt Injection

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/06/interesting-paper-exploring-prompt-injection.html

This is a fascinating explotation of how LLMs fall for prompt injection attacks. It turns out that they learn to recognize the style of text in different role/instruction blocks, and not just the tags.

Their conclusion:

Role tags were a formatting trick that became the security architecture and the cognitive scaffolding of modern LLMs. We’ve shown that this architecture doesn’t survive into the model’s actual representations, and that such role confusion is linked to prompt injection.

Unless LLMs achieve genuine role perception, we think injection defense will remain a perpetual whack-a-mole game. And the continuous nature of role boundaries opens the threat of injections designed to subtly shift LLM states through seemingly innocuous text, legally and at scale.

More generally, roles are quietly one of the most important abstractions in the LLM stack, providing the boundaries meant to separate self from other, thought from communication, instruction from data. They’re human-controlled switches in an otherwise continuous system. We think they deserve a lot more study than they’ve gotten.

Full paper: “Prompt Injection as Role Confusion.” Simon Willison comments.

IBM Outlines Sub-1nm Nanostack Transistor Technology: Building the Next Gen By Going Up

Post Syndicated from Ryan Smith original https://www.servethehome.com/ibm-outlines-sub-1nm-nanostack-transistor-technology/

Today IBM is unveiling their nanostack transistor architecture. Meant to drive chip construction in the sub-1nm era in the 2030s, nanostack aims for building better and smaller chips by building them taller via wafer stacking

The post IBM Outlines Sub-1nm Nanostack Transistor Technology: Building the Next Gen By Going Up appeared first on ServeTheHome.

Celebrating over 15,000 young creators at the Coolest Projects 2026 online showcase

Post Syndicated from Helen Gardner original https://www.raspberrypi.org/blog/celebrating-over-15000-young-creators-at-the-coolest-projects-2026-online-showcase/

From first-time coders to seasoned makers, this year every single Coolest Projects creator brought all their creativity to their tech projects and made something to be proud of. Over 15,000 young people showcased more than 4,500 creations on a global stage. With participants from 40 countries and 47% girls, this year’s online showcase is a true reflection of what the next generation of tech creators looks like.

Yesterday our global livestream brought together the whole Coolest Projects community to celebrate the young people’s creations with some very special guests.

Meet our 2026 VIP judges and their favourite projects

Every year, we invite new special VIP judges to choose their favourite projects from each of the seven Coolest Projects categories. Meet our 2026 judges and find out about the projects they picked.

Ronit Levavi Morad, Chief of Staff at Google Research 

Ronit’s role involves reimagining the future of learning and she is a passionate advocate for technology in service of pedagogy, leading Google Research’s global Al literacy initiatives. She champions a human-centered vision for technology, leading programmes like Al Quests, a gamified experience designed to teach teens how Al can be applied to humanity’s greatest challenges.

Ronit’s favourite projects are:

Ben Powley, Senior Developer at Jagex

Ben is a senior developer at British video game company Jagex, working on RuneScape to deliver immersive experiences for both new and existing players. He previously worked at Ubisoft on the Assassin’s Creed series and the Avatar Frontiers of Pandora game. Ben learned to code at 12 through making mods for Minecraft, and he has been passionate about game development ever since.

Ben’s favourite projects are:

Akari Kawaguchi, youth mentor, CoderDojo Japan

Akari is a 16-year-old CoderDojo member from Japan who has showcased her own projects at Coolest Projects seven times. She knows exactly what it takes to make something special for the showcase, and we’re so excited to have a young creator’s perspective on the judging panel!

Akari’s favourite projects are:

Sebin Sunny, CEO, EIC IIITM-K

Sebin is CEO of the Entrepreneurship and Innovation Center (EIC) and the Centre of Excellence in loT Sensors at the Indian Institute of Information Technology and Management – Kerala (IIITM-K). A passionate advocate for innovation, he is committed to empowering entrepreneurs, creating sustainable solutions, and shaping the future of technology-driven industries.

Sebin’s favourite projects are:

Broadcom Coding with Commitment® award

The Broadcom Coding With Commitment® award shines a light on creators who use coding to support and strengthen their communities with a project that aligns with 17 sustainable development goals of the United Nations.

A screenshot of a young person's Scratch project showcased at Coolest Projects.
The start screen of Viridia – Back to the World

This year’s Broadcom Coding with Commitment® recipient for the online showcase is Egehan from Türkiye, with their Scratch project Viridia – Back to the World, an educational game designed to teach players about the importance of water and how to use it responsibly.

Get inspired and keep creating!

Browse the Coolest Projects 2026 online gallery to discover thousands of incredible projects from young people all over the world. 

Inspired to make your own project? Or encourage a young person you know? To get you started, we offer over 200 free coding projects, in English and many other languages.

Want to know more about next year’s showcase?

Coolest Projects will be back online in 2027. Sign up to the newsletter to be the first to hear about dates, deadlines, and exciting updates.

Coolest Projects logo.

And did you know there are in-person Coolest Projects events around the globe? There is still time to take part in Coolest Projects India and other partner events this year. Find out more.

Thank you to the Coolest Projects sponsors

We want to say a big thank you to Broadcom Foundation, Allianz, Amazon Future Engineer, Qube-RT, Avnet, and GoTo for sponsoring Coolest Projects 2026 and helping to celebrate young tech creators around the world.

The post Celebrating over 15,000 young creators at the Coolest Projects 2026 online showcase appeared first on Raspberry Pi Foundation.

Restrict AWS Management Console access to expected networks with sign-in resource-based policies and RCPs

Post Syndicated from Swara Gandhi original https://aws.amazon.com/blogs/security/restrict-aws-management-console-access-to-expected-networks-with-sign-in-resource-based-policies-and-rcps/

Amazon Web Services (AWS) recently announced support for resource-based policies and resource control policies (RCPs) for AWS Sign-In. By using resource-based policies and RCPs, you can restrict access to the AWS Management Console sign-in and aws login CLI sessions to requests from your expected networks, your on-premises data center networks, and your Amazon Virtual Private Cloud (Amazon VPC) VPCs.

Sign-in resource-based policies and RCPs support several security objectives: restricting console sign-in to corporate networks, limiting which principals can sign-in to the console, and applying consistent network perimeter controls across an entire AWS Organizations organization.

In this post, we walk through a common use case: a financial services company restricting console access to its corporate network for regulatory compliance. We show you how to implement this using a sign-in resource-based policy for a single account, verify the controls with AWS CloudTrail, and explain how these policies integrate with AWS Management Console Private Access and the broader AWS data perimeter framework.

Restricting console sign-in access to a corporate network

Consider a financial services company that requires access to AWS Management Console sign-in to originate from the corporate network. The company has the following requirements:

  • Users sign in to the console only from the corporate VPN, office network, or customer VPC.
  • Sign-in attempts from personal networks, public Wi-Fi, or other unexpected locations must be denied.
  • A designated principal should retain access from any network to prevent lockout.
  • All sign-in attempts (allowed and denied) must be logged to CloudTrail for compliance evidence.

In the steps that follow, we show you how to create a resource-based policy to enforce these requirements on a single account.

Prerequisites

  • AWS Command Line Interface (AWS CLI) installed and configured with the latest version.
  • Permission to manage Sign-in resource policies. Attach the AWS managed policy AWSSignInResourcePolicyManagement or grant permissions to the following actions to respective principals:
    • Manage resource permission statements: signin:PutResourcePermissionStatement, signin:DeleteResourcePermissionStatement, signin:ListResourcePermissionStatements, signin:GetResourcePolicy.
    • Manage console authorization: signin:PutConsoleAuthorizationConfiguration, signin:GetConsoleAuthorizationConfiguration, signin:DeleteConsoleAuthorizationConfiguration
  • Identified corporate network: IP CIDR range or VPC ID.
  • Designated principal Amazon Resource Name (ARN) to exclude, so it retains access if network conditions change.

Note: For the complete list of AWS Sign-In actions see Actions, resources, and condition keys for AWS Sign-In in the Service Authorization Reference.

Step 1: Create resource permission statements

Most resource-based policies require the author to input the full policy document (JSON statements). A Sign-in resource permission statement is different: you provide parameters, and AWS Sign-In generates the policy for you.

The following command provides your corporate IP range, your VPC, and an excluded principal as parameters. AWS Sign-In uses these parameters to generate a policy that restricts console sign-in to those networks, while letting the excluded principal sign in from any network. You control the parameter values, not the policy structure. You can review the generated policy at any time with the get-resource-policy command.

Note: Creating resource permission statements has no effect until console authorization is enabled in Step 2, so you can review the complete policy before it takes effect. Write operations must target us-east-1.

To create resource permission statements

1. Open your terminal and ensure you have the latest AWS CLI installed.
2. Run the following command, replacing the placeholder values <my-vpc>, <my-vpc-region>, <my-corporate-cidr>, and <excluded-IAM-principal-arn> with your specific configuration:

aws signin put-resource-permission-statement \
  --source-vpc <my-vpc> \
  --requested-region <my-vpc-region> \
  --source-ip <my-corporate-cidr> \
  --excluded-principal <excluded-IAM-principal-arn> \
  --region us-east-1

3. Verify the command succeeded by checking for a statementId in the output.

Example output:
{
“statementId":"b2HfHli9qCF1P4eGNll13CrZtusXlcPxxVBqz2aYLjlAcWtWQHP6Hg0"
}

4. Review the complete resource-based policy by running get-resource-policy command.

aws signin get-resource-policy

Example output:

{
  "signinResourceBasedPolicy": {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "DENY",
        "Principal": {"AWS": "*"},
        "Action": ["signin:Authenticate"],
        "Resource": "*",
        "Condition": {
          "ArnNotEquals": {"signin:PrincipalArn": ["<excluded-IAM-principal-arn>"]},
          "NotIpAddress": {"aws:SourceIp": ["<my-corporate-cidr>"]},
          "StringEquals": {"aws:ResourceAccount": ["<account-id>"]},
          "StringNotEquals": {"aws:SourceVpc": ["<my-vpc>"]}
        }
      },
      {
        "Effect": "DENY",
        "Principal": {"AWS": "*"},
        "Action": ["signin:CreateOAuth2Token", "signin:AuthorizeOAuth2Access"],
        "Resource": "*",
        "Condition": {
          "ArnNotEquals": {"aws:PrincipalArn": ["<excluded-IAM-principal-arn>"]},
          "NotIpAddress": {"aws:SourceIp": ["<my-corporate-cidr>"]},
          "StringEquals": {"aws:ResourceAccount": ["<account-id>"]},
          "StringNotEquals": {"aws:SourceVpc": ["<my-vpc>"]}
        }
      },
      {
        "Effect": "DENY",
        "Principal": {"AWS": "*"},
        "Action": ["signin:Authenticate"],
        "Resource": "*",
        "Condition": {
          "ArnNotEquals": {"signin:PrincipalArn": ["<excluded-IAM-principal-arn>"]},
          "StringEquals": {"aws:SourceVpc": ["<my-vpc>"]},
          "StringNotEquals": {"aws:RequestedRegion": ["<my-vpc-region>"]}
        }
      },
      {
        "Effect": "DENY",
        "Principal": {"AWS": "*"},
        "Action": ["signin:CreateOAuth2Token", "signin:AuthorizeOAuth2Access"],
        "Resource": "*",
        "Condition": {
          "ArnNotEquals": {"aws:PrincipalArn": ["<excluded-IAM-principal-arn>"]},
          "StringEquals": {"aws:SourceVpc": ["<my-vpc>"]},
          "StringNotEquals": {"aws:RequestedRegion": ["<my-vpc-region>"]}
        }
      }
    ]
  }
}

The generated policy contains four statements, grouped into two pairs. The first pair restricts access by network source—it denies any principal making a request from outside your corporate IP range (<my-corporate-cidr>) or your VPC (<my-vpc>). The second pair restricts which AWS Region your VPC can target—it denies requests originating from <my-vpc> unless they are directed at <my-vpc-region>. This Region binding is necessary because VPC IDs are only unique within a single Region.

AWS Sign-In evaluates these policies in two phases: before authentication and after authentication. The post-authentication evaluation repeats each time the console session requests new credentials. Within each pair, one statement covers the pre-authentication phase and one covers the post-authentication phase.

The pre-authentication statement evaluates the signin:Authenticate action. Since the principal is not yet authenticated in this phase, the statement uses the signin:PrincipalArn condition key to exempt your excluded principal. This key supports all principal types: root user, AWS Identity and Access Management (IAM) user, federated user, and role.

The post-authentication statement evaluates the signin:AuthorizeOAuth2Access and signin:CreateOAuth2Token actions. AWS Sign-In evaluates these actions after authentication, when it issues the tokens that establish the console session. These actions do not support the signin:PrincipalArn key. Instead, they use aws:PrincipalArn, which resolves to the authenticated principal.

The aws:ResourceAccount value is the recipient account ID. AWS Sign-In pulls it automatically from your caller credentials, so you do not set it yourself. For the full list of supported actions and condition keys, including which keys apply at each phase and to each principal type, see Controlling console access with resource-based policies and resource control policies and AWS Sign-In condition keys reference.

Step 2: Turn on sign-in policy enforcement for your account

This step turns on enforcement of the policy you created in Step 1. Until you run this step, the resource permission statements you created in Step 1 have no effect.

5. Turn on enforcement of sign-in policies using the following command:

aws signin put-console-authorization-configuration \
  --target-id <account-id> \
  --region us-east-1

6. Verify the command succeeded by checking for a “consoleAuthorizationEnabled": true in the output.

Example output:

{
“Output": {
“consoleAuthorizationEnabled": true,
“scope": “ACCOUNT”,
“targetId": "<account-id>"
}
}

7. You can also verify the configuration by executing the get-console-authorization-configuration command as shown below:

aws signin get-console-authorization-configuration \
  --target-id <account-id> \
  --region us-east-1

8. To disable enforcement or remove individual statements, use delete-console-authorization-configuration or delete-resource-permission-statement. For more details, see Controlling console access with resource-based policies and resource control policies in the AWS Sign-In User Guide.

Verifying the implementation

Now that enforcement is active, sign-in attempts are evaluated against your resource-based policy. Verify the behavior by testing sign-in from different network conditions.

Scenario 1: Allowed sign-in from the corporate network

A principal signing in from the allowed corporate IP range or VPC succeeds normally. The CloudTrail event shows ConsoleLogin:Success

Example CloudTrail event details for successful console sign-in:

{
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "AROAEXAMPLEID:Dev1",
        "arn": "arn:aws:sts::123456789123:assumed-role/Developer/Dev1",
        "accountId": "123456789123"
    },
    "eventTime": "2026-06-09T19:20:38Z",
    "eventSource": "signin.amazonaws.com",
    "eventName": "ConsoleLogin",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "192.0.2.100",
    "responseElements": {
        "ConsoleLogin": "Success"
    },
    "eventID": "dd004e78-6447-4f56-8d2d-a795da66f598",
    "readOnly": false,
    "eventType": "AwsConsoleSignIn",
    "managementEvent": true,
    "recipientAccountId": "123456789123",
    "eventCategory": "Management"
}

Scenario 2: Denied sign-in from an unexpected network

A principal signing in from a network other than the allowed IP address range or a VPC endpoint attached to the source VPC, is blocked. The CloudTrail event shows ConsoleLogin: Failure with an error message identifying the policy that caused the denial:

Example CloudTrail event details for failed console sign-in:

{    
"userIdentity": {
    "type": "IAMUser",
    "accountId": "123456789123",
    "accessKeyId": "",
    "userName": "Dev1"
    },
    "eventTime": "2026-06-09T19:20:38Z",
    "eventSource": "signin.amazonaws.com",
    "eventName": "ConsoleLogin",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "198.51.100.76",
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36",
    "errorCode": "AccessDenied",
    "errorMessage": "Authorization denied because of a resource-based policy",
    "requestParameters": null,
    "responseElements": {
        "ConsoleLogin": "Failure"
    },
"eventID": "d88a7543-ae89-4186-b1b6-d3116413f2ee",
"readOnly": false,
"eventType": "AwsConsoleSignIn",
"managementEvent": true,
"recipientAccountId": "123456789123",
"eventCategory": "Management"
}

The error message field shows the policy type that caused the denial: “Authorization denied because of a resource-based policy”.

Scaling with RCPs

The steps above apply a Sign-in resource-based policy to a single account. For organizations managing many accounts, RCPs offer a better path: they can be attached at the organization, OU, or account level in AWS Organizations and apply automatically to every account in scope. To view an RCP example, see here .

When a sign-in to the console is denied because of an RCP, the error message field shows the denial as “Authorization denied because of a resource control policy”.

Extending with Console Private Access and data perimeters

The sign-in resource-based policy you created controls which networks can reach your account’s sign-in flow. AWS Management Console Private Access adds a complementary control: from within your network, it limits console access to a known set of AWS accounts, preventing sign-in to unexpected AWS accounts.

Together, these capabilities contribute to a data perimeter for console access:

  • Network perimeter: Sign-in resource-based policies and RCPs restrict console sign-in to expected networks (corporate IP ranges, VPCs).
  • Identity perimeter: Sign-in resource-based policy and RCP ensure only trusted identities can sign in to the console. Console VPC endpoint policy and Sign-in VPC endpoint policy ensure only trusted identities can use the console from your VPC.
  • Resource perimeter: Sign-in VPC endpoint policy and Console VPC endpoint policy restrict which AWS accounts are reachable from your network.

The controls in this post focus on console access. To extend these perimeters to other AWS services and broader implementation scenarios, see the Data perimeter policy examples repository and the Data Perimeters Blog Post Series.

Conclusion

By using sign-in resource-based policies and RCPs, you can restrict AWS Management Console access to expected networks. These controls are available at no additional cost in all AWS commercial Regions.

To get started, see the AWS Sign-in User Guide. For organization-wide enforcement, see Resource control policies in the AWS Organizations User Guide.

If you have feedback about this post, submit comments in the Comments section below.

Swara Gandhi

Swara Gandhi is a Senior Solutions Architect on the AWS Identity Solutions team. She works on building secure and scalable end-to-end identity solutions. She is passionate about everything identity, security, and cloud.

Rishi Tripathy

Rishi Tripathy

Rishi is a Principal Product Manager on the AWS Identity and Access Management (IAM) team. He focuses on access control mechanisms that help enterprises secure their AWS environments at scale. He is passionate about building security primitives that are straightforward to adopt and hard to misconfigure.

Qualcomm Investor Day 2026 Data Center Announcements CPUs, AI Accelerators, and More

Post Syndicated from Patrick Kennedy original https://www.servethehome.com/qualcomm-investor-day-2026-data-center-announcements-cpus-ai-accelerators-and-more/

We are at Qualcomm Investor Day 2026 where the company is talking High Bandwidth Compute, its HBM alternative, Its Dragonfly C1000 CPU, and AI accelerators

The post Qualcomm Investor Day 2026 Data Center Announcements CPUs, AI Accelerators, and More appeared first on ServeTheHome.

Implement multi-tenant search with Amazon OpenSearch Serverless next generation

Post Syndicated from Jon Handler original https://aws.amazon.com/blogs/big-data/implement-multi-tenant-search-with-amazon-opensearch-serverless-next-generation/

Learn how to implement cost-effective multi-tenant search using Amazon OpenSearch Serverless next-generation architecture with scale-to-zero compute and simplified routing through per-account, regional endpoints.

Building multi-tenant search architectures requires balancing data isolation with operational cost and complexity. In this post, we provide code examples for an implementation of multi-tenant search using a collection-per-tenant model with Amazon OpenSearch Serverless per-account, regional endpoints. Collection-per-tenant provides data and workload isolation. The regional endpoint simplifies routing requests for indexing and searching data.

Amazon OpenSearch Serverless is a serverless deployment option for Amazon OpenSearch Service that simplifies infrastructure management, index tuning, and data lifecycle management. OpenSearch Serverless automatically provisions and scales resources to provide consistently fast data ingestion rates and millisecond query response times during changing usage patterns and application demand.

The multi-tenant search problem

In search workloads, a tenant is a logical unit of data and the queries against that data. An eCommerce site has product categories. Each category is a tenant. A blog-hosting platform has blogs. Each blog is a tenant. Tenants map to resources in different ways. In the siloed model, each tenant gets its own container: a domain, collection, or index. In the pooled model, tenants share a container. The hybrid model silos large tenants and pools smaller ones together. Regardless of model, you need a mapping between tenant identifiers and the containers that hold their data, so your application routes requests correctly.

OpenSearch Serverless classic offered a collection-per-tenant strategy that simplified, but did not remove, the need for maintaining a tenant-container mapping. In addition, the cost structure of maintaining collection-per-tenant in classic was not ideal. Classic shared hardware across collections with the same AWS Key Management Service (AWS KMS) key. Tenants with different keys could not share hardware. The cost of the solution was the minimum monthly collection cost multiplied by the tenant count. Building for hundreds or thousands of tenants was cost-prohibitive. Collection groups improved this by allowing hardware sharing across AWS KMS keys, but compute costs were still driven by your indexed data, even during idle periods.

With the next-generation architecture, collection groups scale compute to zero. You pay for compute only when a tenant is actively indexing or searching (storage charges still apply). The addition of the regional endpoint further simplifies multi-tenant workloads by routing traffic to any collection through a single hostname. Together, scale-to-zero compute and the regional endpoint make the collection-per-tenant model both economically viable and operationally straightforward.

The OpenSearch Serverless per-account endpoint

OpenSearch Serverless next generation introduces a per-account, regional endpoint that serves all collections through a single hostname:

https://<account-id>.aoss.<region>.on.aws

The x-amz-aoss-collection-name or x-amz-aoss-collection-id header identifies the target collection on each request. This means one connection pool, one TLS session, and one endpoint to manage regardless of how many collections you have.

From a client perspective, you create a single OpenSearch client pointed at the regional endpoint and route requests by setting a header:

def get_opensearch_client(account_id: str, region: str) -> OpenSearch:
    """Create an OpenSearch client using the regional endpoint."""
    host = f"{account_id}.aoss.{region}.on.aws"
    auth = get_aws4auth(region)

    return OpenSearch(
        hosts=[{"host": host, "port": 443}],
        http_auth=auth,
        use_ssl=True,
        verify_certs=True,
        connection_class=RequestsHttpConnection,
        timeout=60,
    )

Every subsequent request includes the routing header to target a specific collection:

headers = {"x-amz-aoss-collection-name": collection_name}

This is a significant improvement over the classic architecture, where each collection had its own endpoint and you needed to manage separate connections for each.

Collection per tenant with query routing

The architecture is straightforward: one collection group holds all tenant collections, and the regional endpoint handles routing.

Create a collection group with scale-to-zero

client.create_collection_group(
    name="amazon-pqa-cg",
    generation="NEXTGEN",
    standbyReplicas="ENABLED",
    capacityLimits={
        "minIndexingCapacityInOCU": 0,
        "maxIndexingCapacityInOCU": 8,
        "minSearchCapacityInOCU": 0,
        "maxSearchCapacityInOCU": 8,
    },
)

When you set minIndexingCapacityInOCU and minSearchCapacityInOCU to 0, OpenSearch Serverless scales down your compute to 0 OpenSearch Compute Units (OCUs) when they are idle for 10 minutes. You pay only for the storage for your indices. If you want to maintain compute and avoid cold starts, set minIndexingCapacityInOCU or minSearchCapacityInOCU to a value greater than 0.

Create one collection per tenant

Each product category maps to its own collection within the group:

client.create_collection(
    name=name,
    type="SEARCH",
    collectionGroupName=COLLECTION_GROUP_NAME,
)

When choosing a collection name for your tenants, consider privacy, name length, and future ease of upgrading your application. You can use a hash function to map tenant identifiers to collection names.

import hashlib

def collection_name_for_tenant(tenant_id: str) -> str:
    """Generate an opaque collection name from a tenant identifier."""
    return hashlib.sha256(tenant_id.encode()).hexdigest()[:16]

Collection names are visible in API calls and logs. If your tenant ID contains personally identifiable information (PII), that information is also visible in logs. Hashing the tenant ID obfuscates the sensitive information.

OpenSearch Serverless has a 64-character limit on collection names. Your tenant ID can be longer than that. Hashing helps stay within this limit.

You might also want to add a prefix to collection names so that you can use wildcard patterns in access policies. For example, naming collections pqa-a1b2c3d4 lets you write a single data access policy matching collection/pqa-*. Including a version component in the name (such as pqa-v2-a1b2c3d4) makes it straightforward to create new collections during schema migrations without disrupting existing tenants.

Index data using the regional endpoint

A single OpenSearch client handles all collections. The x-amz-aoss-collection-name header routes each request to the correct collection:

headers = {"x-amz-aoss-collection-name": collection_name}

# Build bulk request
action = {"index": {"_index": index_name, "_id": doc["question_id"]}}
batch.append(json.dumps(action))
batch.append(json.dumps(doc))

# Send bulk request routed to the target collection
body = "\n".join(batch) + "\n"
resp = os_client.bulk(body=body, headers=headers)

Query a specific tenant’s data

Searching works the same way. Set the header to target the tenant’s collection:

os_client = get_opensearch_client(account_id, region)
headers = {"x-amz-aoss-collection-name": collection_name}

query = {
    "size": 3,
    "query": {
        "match": {
            "question_text": "4k resolution hdmi"
        }
    },
}

resp = os_client.search(index="questions", body=query, headers=headers)

The application layer maps a tenant ID (in this case, a product category) to a collection name, and the regional endpoint handles the rest. No connection pool management, no endpoint lookups, no per-tenant client instances.

Limitations

There are practical constraints to consider when adopting this pattern.

Cold start latency. When a collection group has scaled to zero compute, the first request takes approximately 10 seconds while capacity provisions. For latency-sensitive tenants, you can send a lightweight warmup query (such as a match_all with size=1) before production traffic arrives.

Collection group limits. There are account-level limits on the number of collections and collection groups. Check the Amazon OpenSearch Serverless quotas for current numbers if you are planning thousands of tenants.

Security policy size. Encryption, network, and data access policies list collection resource patterns. Because tenant count grows, these policy documents grow linearly. Use wildcard patterns to stay within OpenSearch Serverless policy size limits.

No cross-collection queries. Each search request targets exactly one collection. If you need to query across tenants for analytics or global search, you need an aggregation layer or a separate shared collection.

Conclusion

In this post, we showed how the next-generation OpenSearch Serverless architecture makes the collection-per-tenant model practical for multi-tenant search. Scale-to-zero reduces the minimum cost for inactive tenants, fitting the compute resources to the demands of tenants. The regional endpoint eliminates the operational complexity of managing per-tenant connections. You get full data isolation between tenants, independent scaling for each tenant’s workload, and a single endpoint to manage in your application code.

For more information, see the Amazon OpenSearch Serverless documentation.

About the author

Jon Handler

Jon Handler

Jon is a Senior Principal Solutions Architect for Search Services at Amazon Web Services. Jon works closely with OpenSearch and Amazon OpenSearch Service, providing help and guidance to a broad range of customers who have search and log analytics workloads. Prior to joining AWS, Jon’s career as a software developer included four years of coding a large-scale, eCommerce search engine.

Global SMS strategy for gaming: Lessons from migrating 2FA across 100+ countries

Post Syndicated from Jose Velazquez Hernandez original https://aws.amazon.com/blogs/messaging-and-targeting/global-sms-strategy-for-gaming-lessons-from-migrating-2fa-across-100-countries/

When a player in Turkey, Brazil, or India tries to log in, they expect a verification code on their phone within seconds. If it doesn’t arrive, that’s a lost session, a support ticket, or worse, a player who doesn’t come back. For gaming studios operating across 100+ countries, delivering those SMS messages reliably is a complex operational challenge involving country-specific regulations, carrier registrations, and routing rules that can change overnight.

This post shares practical lessons from a real-world migration of a global gaming SMS platform from Amazon Simple Notification Service (Amazon SNS) to AWS End User Messaging. You’ll learn how to use resource sharing to avoid months of re-registration, how to build a country inventory that survives regulatory changes, and how to avoid the operational pitfalls that surface only when you migrate production traffic across accounts. Although the examples here are drawn from gaming, the operational challenges are universal. Any organization delivering SMS at international scale will face the same complexities, whether your users are players, customers, or patients.

Why gaming studios migrate SMS accounts

Before diving into the how, it’s worth understanding the why. Gaming studios commonly need to move SMS workloads between AWS accounts for several reasons:

  • Security and scope isolation — Separating messaging infrastructure from game servers and backend services into a dedicated account reduces the impact of a security incident in either environment.
  • Organizational changes — Studio acquisitions, team restructuring, or moving from a shared corporate account to a studio-owned account.
  • Service evolution — Migrating from Amazon SNS (which supports SMS as one of many pub/sub capabilities) to AWS End User Messaging (purpose-built for SMS/MMS/voice with features like phone pools, protect configurations, and event destinations)
  • Compliance boundaries — Isolating messaging resources to meet data residency or audit requirements.

Whatever the reason, the challenge is the same: you have a working global SMS setup with phone numbers, Sender IDs, short codes, and carrier registrations spread across dozens of countries, and you need to move it without disrupting millions of players’ 2FA flows.

Resource sharing: A faster migration path

The traditional approach would be to re-register every origination identity (that is, phone numbers, Sender IDs, short codes, see Choosing an origination identity) in the consuming account from scratch. For a global gaming platform, this is impractical:

Multiply these timelines across every country where you operate, and a “clean” re-registration migration could take months.

AWS Resource Access Manager (RAM) offers a better path. Instead of re-registering, you share your existing origination identities from the sharing account to the consuming account, which can immediately begin sending. For the step-by-step setup including CLI commands, AWS Identity and Access Management (IAM) policies, and architecture diagrams, see A Complete Guide to Resource Sharing for AWS End User Messaging. This post focuses on the operational challenges that surface at global scale.

Know your country landscape before you start

The single most important preparation step is building a complete inventory of your SMS footprint, country by country. AWS publishes a comprehensive reference table listing supported countries, origination identity types, and registration requirements.

For a ready-made planning template, download the Global SMS Planning Sheet from How to Manage Global Sending of SMS with AWS End User Messaging. Your inventory should go beyond what’s published: document exact registered values (case matters), fallback behavior, carrier/aggregator, and why you chose each approach.

This inventory becomes invaluable for future account transitions, incident response, and onboarding new team members.

Plan for regulatory change, not just today’s rules

The global SMS regulatory landscape is not static. It’s tightening. Countries that previously had relaxed requirements are increasingly introducing mandatory Sender ID registration, content filtering, and message format restrictions. A migration is the ideal time to future-proof your setup, not just replicate what worked yesterday.

For help navigating the registration process for specific countries, see Registration support.

Example: Turkey’s URL blocking regulation (April 2026)

In April 2026, Turkey’s Information and Communications Technologies Authority (BTK) began enforcing a regulation that blocks all international A2P SMS messages containing URLs, hyperlinks, or shortened links. For gaming studios that include account recovery links, promotional URLs, or support links in their SMS messages, this type of regulatory change can mean immediate delivery failure to an entire market.

This type of regulatory shift is becoming more common:

  • Optional to mandatory Sender ID registration. Australia is mandating Sender ID registration with ACMA by mid-2026. AWS now supports Australia sender ID registration directly through the EUM console.
  • Content template registration. India requires DLT registration through TRAI, and China requires message template pre-registration through AWS Support.
  • Content filtering and URL restrictions. Carriers in multiple markets are implementing automated filtering that rejects messages containing URLs or specific patterns, even from registered senders. Your country inventory shouldn’t just document what works today. It should also track regulatory change dates, upcoming requirements, content restrictions, and your fallback plan for each market. For each country, ask: “If the rules change tomorrow, how quickly can we adapt?” If the answer is weeks, that country should have a fallback channel (email, push notification) ready to activate immediately.

Diagram showing the global SMS migration workflow from a sharing account to a consuming account using AWS RAM

Lessons learned: What can go wrong at scale

Even with careful planning, migrating global SMS infrastructure surfaces unexpected issues. The following sections cover the most common pitfalls we encountered.

Sender ID case sensitivity

As documented by AWS, Sender ID values are case-sensitive at the carrier and aggregator level. In countries with mandatory registration (Turkey, UK, India, Australia), carriers might perform exact-match validation. If your application sends STUDIONAME but the carrier has StudioName on file, the message can be rejected.

The AWS End User Messaging console displays all Sender IDs in uppercase regardless of registration casing, which is something to watch out for. Always verify casing against the Registrations section of the console, which preserves the original value. Countries can also begin enforcing stricter matching overnight as regulations tighten, turning previously-working messages into failures.

What to watch for:

  • Test delivery to countries with mandatory Sender ID registration before rolling out any casing changes globally.
  • Check the country capabilities table for countries marked “Registration required.”

Phone pool quotas for global operations

Phone pools are the recommended way to manage origination identities (that is, phone numbers, Sender IDs, short codes, see Choosing an origination identity) at scale with AWS End User Messaging. A pool automatically selects the appropriate identity based on the destination country code, handling failover between identities for the same country.

You can find the default quota for origination identities per phone pool in Quotas for AWS End User Messaging SMS. You can request increases through the Service Quotas console, or contact AWS Support for larger increases.

Plan ahead: Inventory your origination identities before creating your pool, and request the quota increase before you start associating resources. Hitting the quota mid-migration creates unnecessary delays.

Batch your RAM sharing operations

When sharing a large number of origination identities through RAM (50+), the RAM API creates resource-based policies on each shared resource. The order of operations matters: add the consuming account (principal) to the resource share first, then add resources incrementally. If you create a resource share with hundreds of resources and then add the consuming account afterward, RAM attempts to set resource-based policies on all resources simultaneously. This is the most common trigger for throttling at scale. In severe cases, this throttling can result in resources with empty or corrupted resource policies, making them inaccessible to the consuming account until manually remediated.

Best practice:

  • Add the consuming account (principal) to the resource share before adding any resources. Batch resource associations in groups of 10 or fewer.
  • After each batch, verify that all resources have reached ASSOCIATED status.
  • From the consuming account, confirm that shared resources are visible using the describe-* CLI commands with the --owner SHARED flag.
  • Retain AWS CloudTrail logs for troubleshooting if any associations don’t complete as expected.
  • If you suspect corrupted policies, verify the resource policies on affected resources and re-associate them individually.

Not everything transfers through RAM

RAM sharing covers the “what you send with”: phone numbers, Sender IDs, pools, and opt-out lists. But several critical components are account-specific and must be recreated in the consuming account:

Note on Protect Configurations: Protect Configurations apply at three levels: account default, configuration set, and per-message. A common mistake when setting up a consuming account is checking only the account-level default and missing a restriction at the configuration set level, resulting in unexpectedly blocked destination countries. When recreating your configuration in the consuming account, verify protect settings at all three levels.

Production mode requirement: Both the sharing account and the consuming account must be in Production mode. If the consuming account is still in the SMS sandbox, it can only send messages to verified destination phone numbers, which means shared production-ready resources cannot be used for real-world traffic. Verify production status in the consuming account before beginning your migration.

One tip worth highlighting: The registration process for certain countries’ message templates (notably China) is handled through AWS Support cases. After the registration is approved, retain a copy of the approved template details, the case correspondence, and the registration parameters. Support case history might not be accessible indefinitely. If you need to re-register in a new account later, having your own records of what was approved and the exact template content saves significant time.

Countries without Sender ID support

Not every country supports Sender IDs. If your phone pool is configured with Sender IDs and a message is routed to a country that doesn’t support them (for example, Belgium or Puerto Rico), the delivery will fail, and the error might not clearly indicate that the issue is Sender ID incompatibility.

Build an exclusion list: For countries that don’t support Sender IDs you have two options:

  • Route those messages through a different origination identity (long code or short code) if available.
  • Exclude those countries from Sender ID routing and allow them to fall back to shared routes.

The country capabilities table is your definitive reference for which countries support which origination identity types.

The “hidden fallback” discovery

A common surprise during migration is that your existing setup might have been silently falling back to shared routes in countries where you assumed you had branded Sender ID delivery. When SMS is sent through Amazon SNS with automatic origination identity selection, the service quietly falls back to shared routes (often displaying “NOTICE” as the Sender ID) if no registered identity is available for the destination country.

This fallback is invisible in normal operations. Messages still deliver, and unless you’re checking the Sender ID displayed on the recipient’s device, there’s no visible indication. But when you migrate to a phone pool with explicit Sender ID routing, these gaps become visible as delivery failures.

Before migrating, audit your actual delivery patterns, not just your configuration. Check delivery reports for countries where you expect branded Sender ID delivery and verify that the Sender ID displayed matches your brand, not “NOTICE” or a generic shared route.

Operational recommendations

The following practices help you avoid disruptions during migration and maintain reliable delivery afterward.

Have a fallback channel ready during migration

Before you begin migrating traffic, ensure you have an alternative authentication channel (typically email-based 2FA) ready to activate on short notice per-country. When an issue is detected, disable SMS for that country and fall back to email while you investigate, rather than leaving players locked out.

Build your country registry

Maintain a living document that tracks what you have (identity type, registration status, exact casing, account location), what you chose and why, what gaps exist (countries relying on shared routes), and registration details (approval dates, case references, template content). This registry is your migration playbook.

Set up delivery monitoring from day one

In the consuming account, configure Configuration Sets with Event Destinations before sending your first message. Consider using the Message Feedback API to track actual OTP conversion, not just carrier delivery. See Track OTP Success with AWS End User Messaging SMS Feedback for implementation details.

Test country by country, not just “it works”

Before scaling traffic, test delivery to representative countries from each category: mandatory Sender ID registration, no Sender ID support, optional Sender IDs, message template registration, and carrier-specific content restrictions. A successful test to the US doesn’t validate your Turkey configuration.

Conclusion

Global SMS for gaming works reliably until something needs to change. Migrations, account restructuring, and service upgrades expose the complexity that was previously hidden behind automatic fallbacks and “it just works” behavior.

In this post, we covered how to use RAM sharing to avoid months of re-registration, how to build a country inventory that accounts for regulatory change, and how to navigate the operational pitfalls of migrating production SMS traffic across accounts. The key takeaways:

  1. Use RAM sharing to avoid months of re-registration. See A Complete Guide to Resource Sharing for AWS End User Messaging for the step-by-step setup.
  2. Know your country landscape before you start. The country capabilities table is your starting point, but build your own registry with registration details, exact Sender ID values, and decision rationale.
  3. Respect the details. Case sensitivity, phone pool quotas, RAM batching limits, non-shareable resources, and record retention can each independently block your migration.
  4. Monitor from day one and plan for regulatory change. Set up delivery tracking before sending production traffic, and track what’s changing in each market.

What matters to players is that their verification code arrives in seconds, every time, regardless of location. The operational discipline described in this post helps make that happen and keeps it working through whatever infrastructure changes come next. Although this post is framed around gaming, the lessons apply broadly. Whether your users are players, shoppers, or account holders, reliable global SMS delivery requires the same operational rigor.

References

About the author

Multi-Region identity-based access to Amazon Redshift and S3 Tables

Post Syndicated from Maneesh Sharma original https://aws.amazon.com/blogs/big-data/multi-region-identity-based-access-to-amazon-redshift-and-s3-tables/

Organizations with lines of business operating across multiple AWS Regions increasingly run analytics workloads on globally distributed data. These organizations want to manage users and groups centrally, typically in the AWS Organizations management account and in a single Region, while still letting each line of business access data from the Region where its workloads run. Organizations should govern access based on the actual workforce user and their group memberships in the corporate directory.

With multi-Region support for AWS IAM Identity Center, organizations can federate workforce identities into a single organization instance in their primary Region. After you replicate this instance to additional Regions, member accounts running services such as Amazon Redshift or Amazon Athena in those Regions can integrate with IAM Identity Center locally, to resolve the same centrally managed users and groups.

This solution uses Trusted Identity Propagation (TIP), a capability that passes a user’s Identity Center identity and group memberships through a chain of AWS services. With TIP, when a user authenticates through Identity Center, that identity context flows to downstream services like AWS Lake Formation and Amazon S3 Access Grants. With this approach, you get consistent, identity-based access control without additional AWS Identity and Access Management (IAM) role configurations.

In Part 1 of this series, we showed how to simplify enterprise data access using the Amazon Redshift integration with Amazon S3 Access Grants. We demonstrated how to grant Amazon Simple Storage Service (Amazon S3) permissions to AWS IAM Identity Center users and groups using S3 Access Grants, and tested the integration using a federated user to unload and load data between Amazon Redshift and Amazon S3 within a single AWS Region.

In this post, we extend that solution across AWS Regions. We introduce a fictional company, AnyCompany Global, to illustrate how organizations with global operations can use AWS IAM Identity Center Multi-Region to set up consistent, identity-based access to Amazon Redshift and Amazon S3 Tables across Regions.

Specifically, we demonstrate:

  • How IAM Identity Center Multi-Region replicates identity data so that the same users and groups are available in each enabled Region.
  • How AWS Lake Formation grants fine-grained table-level and column-level access to S3 Tables based on group membership.
  • How S3 Access Grants controls UNLOAD/COPY operations to Amazon S3 based on the same identity.

We also show how to connect with your preferred SQL client.

Fictional scenario: AnyCompany Global

AnyCompany Global is a retail analytics company with a centralized IT team and distributed analytics teams. They use the following personas:

  • Alice — IT administrator (manages IAM Identity Center and AWS accounts).
  • Bob — platform engineer (sets up data infrastructure in us-west-2).
  • Ethan — data analyst (member of the awssso-sales group, queries data).

AnyCompany Global has two AWS accounts:

  • Account A (us-east-1) — management account with IAM Identity Center.
  • Account B (us-west-2) — analytics account with Amazon Redshift, Amazon S3, and the AWS Glue Data Catalog.

The same IAM Identity Center user (Ethan) authenticates once and accesses data in Account B (us-west-2) using the same credentials and group memberships — you don’t need additional user provisioning because IAM Identity Center replicates identities to the secondary Region.

Solution overview

The following diagram illustrates the multi-account, multi-Region architecture. Account A (us-east-1) hosts IAM Identity Center, which replicates identities to us-west-2 where Account B runs the analytics workloads.

Multi-account, multi-Region architecture diagram showing IAM Identity Center in us-east-1 replicating to us-west-2, where Amazon Redshift queries S3 Tables through Lake Formation and writes to Amazon S3 through S3 Access Grants

Figure 1: Multi-account, multi-Region architecture with S3 Access Grants, AWS Lake Formation, and IAM Identity Center.

This solution demonstrates two complementary data access patterns, both controlled by the end user identity:

Pattern Access method Permission controlled by
Pattern A SELECT on S3 table bucket through Amazon Redshift Spectrum Lake Formation
Pattern B UNLOAD/COPY to and from Amazon S3 S3 Access Grants

The solution workflow includes the following steps:

  • Ethan connects from Amazon Redshift Query Editor v2 in us-west-2 and authenticates via the IAM Identity Center endpoint (replicated to us-west-2) using his corporate IdP credentials.
  • For Pattern A (SELECT): Amazon Redshift queries the Amazon S3 Tables catalog (s3tablescatalog). Lake Formation evaluates Ethan’s IAM Identity Center group membership and grants access to the cataloged data.
  • For Pattern B (UNLOAD/COPY): Amazon Redshift requests temporary credentials from S3 Access Grants in us-west-2. S3 Access Grants evaluates the request, matches Ethan’s identity and group membership, and vends scoped temporary credentials for the authorized S3 location.
  • Ethan runs SELECT to query data through Lake Formation, and UNLOAD to write data to Amazon S3 through S3 Access Grants. You don’t need an IAM role ARN in the commands.

Walkthrough

The following sections walk you through enabling IAM Identity Center Multi-Region, configuring Amazon S3 Tables with Lake Formation in the secondary Region, testing both access patterns, and verifying the result with AWS CloudTrail. Start with the prerequisites, then complete each step in order.

Prerequisites

You should have the following prerequisites already set up:

  • AWS Organizations enabled with at least two AWS accounts – Centralized Account(Region 1) and Member Account(Region2)
  • IAM Identity Center enabled in the management account (Account A, us-east-1) with a delegated administration account
  • Corporate IdP integrated with IAM Identity Center (users and groups synced, for example, awssso-sales and awssso-finance groups).
  • Resource sharing enabled in your organization with AWS Resource Access Manager (AWS RAM)
  • Complete solution from Part 1 replicated in us-west-2 (Account B), including:
    • Amazon Redshift cluster (in us-west-2) with IAM Identity Center integration enabled (using the replicated Identity Center endpoint in us-west-2).
    • S3 Access Grants instance configured with IAM Identity Center association
    • Amazon S3 bucket (for example, amzn-s3-demo-bucket-west) with folders for each group (for example, awssso-sales/, awssso-finance/).
    • IAM role for S3 Access Grants (for example, iamidcs3accessgrant) with trust policy and permissions policy.
    • S3 Access Grants location registered and grant created for the awssso-sales group.
    • S3 Access Grants enabled on the Amazon Redshift managed application under Trusted identity propagation
    • Cross-account resource sharing via AWS RAM (if Amazon Redshift and S3 Access Grants are in different accounts)
    • Lake Formation enabled on the Amazon Redshift managed application under Trusted identity propagation
    • Lake Formation and Glue permissions added to the IAM role used in the Amazon Redshift managed application (for example, IAMIDCRedshiftRole). For the required permissions, see Querying data through AWS Lake Formation.
  • An AWS account with an IAM role that has administrative access (e.g., Admin role) configured as a Data Lake Admin in Lake Formation

Note: Creating and using AWS resources in this tutorial incurs charges, including AWS Key Management Service (AWS KMS) keys, S3 table buckets, Amazon Redshift clusters, and Amazon S3 storage. See the cleanup section at the end of this post to avoid ongoing charges.

Step 1: Set up IAM Identity Center Multi-Region

Alice performs this step in the management account (Account A, us-east-1). IAM Identity Center uses encryption at rest for identity data. To enable multi-Region, you must first create a multi-Region customer-managed AWS Key Management Service (AWS KMS) key and replicate it to the additional Region.

Create a multi-Region AWS KMS key

  1. On the AWS KMS console in us-east-1, choose Create key.
  2. For Key type, select Symmetric.
  3. For Key usage, select Encrypt and decrypt.
  4. Under Advanced options, select Multi-Region key.
  5. Provide an alias (for example, idc-multi-region-key).
  6. Apply the AWS KMS key policy as documented in Baseline KMS key policy.

Replicate the key to us-west-2

  1. On the AWS KMS console in us-east-1, select the key you created.
  2. Choose the Regionality tab.
  3. Choose Create new replica keys.
  4. Select US West (Oregon) us-west-2.
  5. Choose Replicate key.

For detailed instructions, see Creating multi-Region replica keys.

AWS KMS console Regionality tab showing the multi-Region replica key configured for an additional Region

Figure 2: Replica key configured for the additional Region.

Add us-west-2 to IAM Identity Center

  1. On the IAM Identity Center console in us-east-1, in the navigation pane, choose Settings.
  2. Choose Add Region.
  3. From the Region list, select US West (Oregon) us-west-2. The list shows Regions where you replicated the customer-managed AWS KMS key.
  4. Choose Add Region.

A blue banner indicates that Identity Center is replicating your workforce identities, configuration, and metadata to the new Region. After the initial replication, the Replication Status column changes to Replicated. Your Identity Center endpoints in us-west-2 are now active.

For detailed instructions, see Add the Region in IAM Identity Center.

IAM Identity Center Settings page with the multi-Region replica key added for us-west-2 and replication status set to Replicated

Figure 3: IAM Identity Center settings showing the multi-Region replica key added for us-west-2.

Update your IdP configuration for the additional Region

You’ve successfully replicated your Identity Center instance to the Oregon (us-west-2) Region. Your workforce identities are now available in that additional Region and can use the new AWS access portal endpoint.

To make sure AWS managed application (service provider-initiated) authentication redirect user to respective application, add the ACS URL for the additional Region so that the app contains both Regional ACS URLs.

In the following section highlighted in red, you can view all ACS URL information:

IAM Identity Center settings page with the View ACS URLs section highlighted in red

Figure 4: IAM Identity Center settings showing the View ACS URLs option.

Copy the respective ACS URL as shown in the following figure:

IAM Identity Center settings page listing the ACS URLs for both Regions

Figure 5: IAM Identity Center settings showing the ACS URLs for both Regions.

Use the following instructions to add the ACS URL for the additional Region in your Identity Center application in Okta:

  1. Log in to the Okta portal as an Admin.
  2. Expand the Applications drop-down in the left pane, then choose Applications
  3. Choose your Identity Center Application
  4. Select the Sign-on tab and choose Edit in the Settings windows.
  5. In the AWS SSO ACS URL1 box under Advanced Sign-on Settings – add the additional ACS URL
  6. Choose Save.

Okta application Sign-on tab with the AWS SSO ACS URL1 box configured for the IAM Identity Center application

Figure 6: Okta application for IAM Identity Center Sign-on tab to add ACS URLs.

Create a permission set for the secondary Region

Create a permission set in the management account to grant federated users console access to Amazon Redshift Query Editor V2 in the secondary Region (us-west-2). For more information about permission sets, see Permission sets.

  1. In the management account, open the IAM Identity Center console.
  2. In the navigation pane, under Multi-Account permissions, choose Permission sets → Create permission set.
  3. Choose Custom permission set, then choose Next.
  4. Under AWS managed policies, select AmazonRedshiftQueryEditorV2ReadSharing.
  5. Under Inline policy, add the following policy: 
    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "redshift:DescribeQev2IdcApplications",
        "redshift-serverless:ListNamespaces",
        "redshift-serverless:ListWorkgroups",
        "redshift-serverless:GetWorkgroup"
      ],
      "Resource": "*"
    }
  ]
}

  6. Choose Next. Enter a permission set name (for example, Redshift-QEV2-West).
  7. Under Relay state, set the default to the Query Editor V2 URL for the secondary Region: https://us-west-2.console.aws.amazon.com/sqlworkbench/home.
  8. Choose Next, then Create.

After creation, assign this permission set to the relevant IAM Identity Center group (for example, awssso-sales) for Account B (us-west-2).

Step 2: Set up Amazon S3 Tables integration with AWS Glue Data Catalog and Lake Formation in Account B (us-west-2)

In this step, the data lake administrator (Bob) sets up Amazon S3 Tables with Lake Formation for fine-grained access control. He completes the following tasks:

  1. Create an S3 tables bucket.
  2. Enable S3 Tables integration with AWS Glue Data Catalog and Lake Formation.
  3. Register the table bucket with Lake Formation (removes default IAM-based access).
  4. Grant Lake Formation permissions to an IAM Identity Center group (awssso-sales) so that only authorized users can query data through Trusted Identity Propagation.

Step 2.1: Remove default Lake Formation permissions

Before creating S3 Tables resources, disable the default IAMAllowedPrincipals grants that Lake Formation applies to new databases and tables. By default, Lake Formation grants IAMAllowedPrincipals access to new resources, which means that standard IAM policies (rather than Lake Formation permissions) control access. For identity-based access through Trusted Identity Propagation, you need Lake Formation to be the sole arbiter of access.

The order matters. If you remove these defaults before registering the S3 Tables resource, Lake Formation will not apply IAMAllowedPrincipals to your S3 Tables catalog or its children. If you register the resource first, you need to manually revoke the IAMAllowedPrincipals grants from each resource.

From the console

  1. Open the Lake Formation console in your target Region (for example, us-west-2).
  2. In the left navigation, choose Administration → Data Catalog settings.
  3. Uncheck both options:
    • Use only IAM access control for new databases
    • Use only IAM access control for new tables in new databases
  4. Choose Save.

Lake Formation Data Catalog settings page with both default IAM access control options cleared

Figure 7: Lake Formation Data Catalog settings with default IAM access control disabled.

Optional: Verify Lake Formation default permissions through the AWS CLI

aws lakeformation get-data-lake-settings --region <REGION>

Confirm both CreateDatabaseDefaultPermissions and CreateTableDefaultPermissions are empty arrays ([]).

Add AWSServiceRoleForRedshift as a read-only admin

If you plan to query S3 Tables from Amazon Redshift Query Editor V2, you must add the Amazon Redshift service-linked role as a Read-Only Admin in Lake Formation. Complete the following steps:

  • In the Lake Formation console, go to Administration → Administrative roles and tasks.
  • Under Data lake administrators, choose Add. Choose Read only administrator.
  • From the menu, choose AWSServiceRoleForRedshift.
  • Choose Confirm.

Important: Without this, Amazon Redshift Query Editor V2 doesn’t display external databases from s3tablescatalog. The Amazon Redshift service-linked role needs read-only admin access to browse the Data Catalog on behalf of users.

Step 2.2: Create the Lake Formation data access role for S3 Tables

Create an IAM role that Lake Formation assumes to generate temporary, scoped credentials on behalf of users requesting access to S3 Tables data. Lake Formation uses this role (instead of its service-linked role) because Trusted Identity Propagation requires sts:SetContext in the trust policy, which is not available on the service-linked role. Without a custom role with this permission, Lake Formation cannot propagate the user’s IAM Identity Center identity when accessing S3 Tables.

Create the role with the trust policy

aws iam create-role \
    --role-name LFAccessRole-S3Tables \
    --assume-role-policy-document '{
        "Version": "2012-10-17",
        "Statement": [{
            "Effect": "Allow",
            "Principal": {
                "Service": "lakeformation.amazonaws.com"
            },
            "Action": [
                "sts:AssumeRole",
                "sts:SetSourceIdentity",
                "sts:SetContext"
            ]
        }]
    }'

Attach the S3 Tables permissions policy

aws iam put-role-policy \
    --role-name LFAccessRole-S3Tables \
    --policy-name S3TablesDataAccess \
    --policy-document '{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "LakeFormationPermissionsForS3ListTableBucket",
                "Effect": "Allow",
                "Action": ["s3tables:ListTableBuckets"],
                "Resource": ["*"]
            },
            {
                "Sid": "LakeFormationDataAccessPermissionsForS3TableBucket",
                "Effect": "Allow",
                "Action": [
                    "s3tables:CreateTableBucket",
                    "s3tables:GetTableBucket",
                    "s3tables:CreateNamespace",
                    "s3tables:GetNamespace",
                    "s3tables:ListNamespaces",
                    "s3tables:DeleteNamespace",
                    "s3tables:DeleteTableBucket",
                    "s3tables:CreateTable",
                    "s3tables:DeleteTable",
                    "s3tables:GetTable",
                    "s3tables:ListTables",
                    "s3tables:RenameTable",
                    "s3tables:UpdateTableMetadataLocation",
                    "s3tables:GetTableMetadataLocation",
                    "s3tables:GetTableData",
                    "s3tables:PutTableData"
                ],
                "Resource": ["arn:aws:s3tables:<REGION>:<ACCOUNT_ID>:bucket/*"]
            }
        ]
    }'

Step 2.3: Register S3 Tables with Lake Formation

Register the S3 Tables resource with Lake Formation using the data access role. This step lets Lake Formation manage access to S3 Tables through the Data Catalog and creates the s3tablescatalog federated catalog automatically.

Open the Lake Formation console and complete the following steps:

  1. Choose Catalogs in the navigation pane and choose Enable S3 Table integration.

Lake Formation Catalogs page with the Enable S3 Table integration option highlighted

Figure 8: Lake Formation Catalogs page with the Enable S3 Table integration option.

  1. Select the IAM role and select Allow external engines to access data in Amazon S3 locations with full table access. Choose Enable.

Enable S3 Table integration dialog with the IAM role selected and the Allow external engines option enabled

Figure 9: Enable S3 Table integration dialog with the IAM role and external-engine access configured.

Alternative: Register through the AWS CLI

aws lakeformation register-resource \
    --resource-arn "arn:aws:s3tables:<REGION>:<ACCOUNT_ID>:bucket/*" \
    --role-arn "arn:aws:iam::<ACCOUNT_ID>:role/LFAccessRole-S3Tables" \
    --with-federation \
    --region <REGION>

Important: Verify that the --role-arn matches the exact ARN of the role created in Step 2.2 (including the path). A mismatch (e.g., role/service-role/LFAccessRole-S3Tables vs role/LFAccessRole-S3Tables) will cause credential vending failures later.

Optional: Verify the registration

aws lakeformation list-resources --region <REGION>

Confirm the S3 Tables entry shows WithFederation: true and the correct role ARN.

Step 2.4: Create the S3 table bucket and namespace

Create an S3 table bucket and a namespace. Complete the following steps on the Amazon S3 console:

  1. In the navigation pane, choose Table buckets.
  2. Choose Create table bucket.
  3. On the next page, enter the bucket name as <TABLE_BUCKET_NAME>.
  4. Keep the other options as default and choose Create table bucket.
  5. After you create it, the AWS Management Console redirects you to the list of table buckets. Choose the table bucket <TABLE_BUCKET_NAME>.
  6. Choose Create table with Athena.
  7. Create a namespace in S3 Tables (equivalent to a database in AWS Glue Data Catalog). Enter the namespace (database) name as <NAMESPACE_NAME> and choose Create namespace.

You can also perform these steps using the AWS Command Line Interface (AWS CLI). Refer to Creating a table bucket using the AWS CLI for equivalent commands.

Step 2.5: Grant admin role access

After you remove default permissions, you need to give your Admin role explicit Lake Formation permissions to create tables. Because your Admin role is a Data Lake Admin, you can already see s3tablescatalog in the Amazon Athena console, but creating tables requires an explicit grant.

From the console

  • Open the Lake Formation console in your Region.
  • Choose Data permissions → Grant.
  • Under Principals, select IAM users and roles and choose your Admin role.
  • Under LF-Tags or catalog resources, select Named Data Catalog resources.
  • For Catalogs, choose <Account ID>:s3tablescatalog/<Table_Bucket_Name>.
  • For Databases, select your database (for example, customer_ns_db).
  • Select Super for Database permissions and Grantable permissions.
  • Choose Grant.

After this grant, you can create and insert data into tables from the Athena console.

Note: Your Admin role must be a Data Lake Admin (configured in Step 2.1) to browse s3tablescatalog in Athena. You need the explicit database grant for write operations (CREATE TABLE, INSERT).

Step 2.6: Create a table from the Athena console

  1. Open the Amazon Athena console in your Region.
  2. In the Data source menu, select AwsDataCatalog.
  3. For Catalog, choose s3tablescatalog/<Table_Bucket_Name>.
  4. For Database, choose your namespace.
  5. Run a CREATE TABLE statement. For example:
CREATE TABLE <NAMESPACE_NAME>.<TABLE_NAME> (
    customer_id int,
    first_name string,
    last_name string,
    region string,
    membership_tier string
)
TBLPROPERTIES ('table_type' = 'ICEBERG');

INSERT INTO <NAMESPACE_NAME>.<TABLE_NAME> VALUES
  (1, 'Joyce', 'Deaton', 'West', 'Gold'),
  (2, 'Daniel', 'Dow', 'East', 'Silver'),
  (3, 'Marie', 'Lange', 'West', 'Gold'),
  (4, 'Wesley', 'Harris', 'East', 'Bronze'),
  (5, 'Jerry', 'Tracy', 'West', 'Silver');

Step 2.7: Grant permissions to the IAM Identity Center group

Give your IAM Identity Center group access to query tables. This step enables Trusted Identity Propagation (TIP) for this group. When users in the group access data through TIP-integrated services like Amazon Redshift, Lake Formation evaluates their IAM Identity Center group membership and enforces table-level and column-level permissions accordingly.

From the console

Grant DESCRIBE on the database:

  1. Open the Lake Formation console in your Region.
  2. Choose Data permissions → Grant.
  3. Under Principals, select IAM Identity Center and choose your IAM Identity Center group (for example, awssso-sales).
  4. Under LF-Tags or catalog resources, select Named Data Catalog resources.
  5. For Catalogs, choose <Account ID>:s3tablescatalog/<Table_Bucket_Name>.
  6. For Databases, select your database (for example, customer_ns_db).
  7. For Database permissions, select Describe.
  8. Choose Grant.

Grant SELECT and DESCRIBE on tables:

  1. Choose Data permissions → Grant.
  2. Under Principals, select IAM Identity Center and choose your IAM Identity Center group (for example, awssso-sales).
  3. Under LF-Tags or catalog resources, select Named Data Catalog resources.
  4. For Catalogs, choose <Account ID>:s3tablescatalog/<Table_Bucket_Name>.
  5. For Databases, select your database (for example, customer_ns_db).
  6. For Tables, select All tables (or a specific table).
  7. For Table permissions, select Select and Describe.
  8. Choose Grant.

Tip: You can also configure column-level or row-level permissions for fine-grained access control. When granting on a specific table, additional options for Column permissions and Data filters become available.

Step 2.8: Optional: Verify the Lake Formation permissions

Confirm database-level permissions

aws lakeformation list-permissions \
    --resource '{"Database": {"CatalogId": "<ACCOUNT_ID>:s3tablescatalog/<TABLE_BUCKET_NAME>", "Name": "<NAMESPACE_NAME>"}}' \
    --region <REGION>

Confirm table-level permissions

aws lakeformation list-permissions \
    --resource '{"Table": {"CatalogId": "<ACCOUNT_ID>:s3tablescatalog/<TABLE_BUCKET_NAME>", "DatabaseName": "<NAMESPACE_NAME>", "TableWildcard": {}}}' \
    --region <REGION>

You should see:

  • Your Admin role with ALL permissions at the database level.
  • Your IAM Identity Center group with DESCRIBE permissions at the database level.
  • Your IAM Identity Center group with DESCRIBE on ALL_TABLES and SELECT on ALL_TABLES (with ColumnWildcard) at the table level.
  • No IAM_ALLOWED_PRINCIPALS entries.

Step 2.9: Create Amazon Redshift tables and grant permissions

Connect to the Amazon Redshift cluster in us-west-2 as an admin user and create Redshift local tables. Grant permissions on those local resources to IAM Identity Center groups.

Create a schema and table

CREATE SCHEMA IF NOT EXISTS sales_schema;

CREATE TABLE IF NOT EXISTS
sales_schema.store_sales (
  customer_id INTEGER ENCODE az64,
  product VARCHAR(50),
  sales_amount INTEGER ENCODE az64
)
DISTSTYLE AUTO;

-- Insert sample data
INSERT INTO sales_schema.store_sales VALUES
  (1, 'Laptop', 1200),
  (2, 'Phone', 800),
  (3, 'Tablet', 450),
  (4, 'Monitor', 350),
  (5, 'Keyboard', 120);

Grant permissions to the IAM Identity Center group

GRANT USAGE ON SCHEMA sales_schema TO ROLE "awsidc:awssso-sales";
GRANT SELECT, INSERT FOR TABLES IN SCHEMA sales_schema TO ROLE "awsidc:awssso-sales";

-- Grant access to the S3 Tables external database in Redshift (for Lake Formation queries on customer profiles)
GRANT USAGE ON DATABASE "customers3tables@s3tablescatalog" TO ROLE "awsidc:awssso-sales";

Step 3: Test the solution

In the management account, navigate to the IAM Identity Center console and copy the AWS access portal URL (for example, https://d-1234560789.awsapps.com/start) from the dashboard.

  • Log out from the management account and paste the AWS access portal URL in a new browser window.
  • A pop-up redirects you to your IdP login page. Enter Ethan’s IdP credentials.
  • After successful authentication, you’re logged into the AWS console as a federated user. Select the QEV2 permission set for the secondary Region (us-west-2).
  • In Query Editor V2, open the context (right-click) menu on your Amazon Redshift instance, choose Create connection, and for Authentication, select IAM Identity Center.
  • Because your IdP credentials are already cached, the browser reuses them automatically. You’re now connected to Amazon Redshift.

Pattern A: Query the S3 table catalog using Lake Formation permissions

Query the customer profile data through s3tablescatalog. Lake Formation enforces access based on Ethan’s IAM Identity Center group membership:

SELECT *
FROM "customers3tables@s3tablescatalog"."customer_ns_db"."customer_profiles";

Amazon Redshift Query Editor V2 results pane displaying customer profile rows returned from the s3tablescatalog through Lake Formation

Figure 10: Query results from s3tablescatalog returned through Lake Formation in Amazon Redshift Query Editor V2.

This query reads customer profile data from Amazon S3 through Amazon Redshift Spectrum, with Lake Formation controlling who can access which tables and columns.

Pattern B: Unload data to Amazon S3 using S3 Access Grants

Run the UNLOAD command to write data from Amazon Redshift to the S3 bucket:

UNLOAD ('SELECT * FROM "dev"."sales_schema"."store_sales"')
TO 's3://west-idc-amzn-s3-demo-bucket/awssso-sales/';

You don’t need an IAM role ARN in the command. S3 Access Grants handles authorization based on Ethan’s IAM Identity Center identity and group membership, propagated across Regions using IAM Identity Center Multi-Region support.

Verify the data in Amazon S3

On the Amazon S3 console, navigate to s3://west-idc-amzn-s3-demo-bucket/awssso-sales/ and verify that the unloaded data files are present.

Join Lake Formation data with locally loaded Amazon Redshift data

Combine customer profile data (queried via Lake Formation) with sales data (loaded via S3 Access Grants) using the shared customer_id column:

SELECT c.first_name, c.last_name, c.membership_tier,
  s.product, s.sales_amount
FROM "customers3tables@s3tablescatalog"."customer_ns_db"."customer_profiles" c
JOIN  dev.sales_schema.store_sales s ON c.customer_id = s.customer_id
ORDER BY s.sales_amount DESC;

Amazon Redshift Query Editor V2 results joining S3 Tables customer profiles with the local store_sales table

Figure 11: Joined results from S3 Tables and Amazon Redshift local data, ordered by sales amount.

This shows that you can join S3 Tables data with Amazon Redshift using the same IAM Identity Center identity.

Verify access control

To confirm that S3 Access Grants is enforcing access, try accessing a folder Ethan does not have a grant for:

UNLOAD ('SELECT * FROM "dev"."sales_schema"."store_sales"')
TO 's3://west-idc-amzn-s3-demo-bucket/awssso-finance/';

This should return an access denied error, confirming that S3 Access Grants is controlling access based on the user’s identity and group membership.

Step 4: Verify with AWS CloudTrail

You can verify that Amazon Redshift used both S3 Access Grants and Lake Formation for authorization by checking AWS CloudTrail:

  • On the CloudTrail console, choose Event history.
  • Filter by Event source: s3.amazonaws.com. Look for GetDataAccess events (S3 Access Grants).
  • Filter by Event source: lakeformation.amazonaws.com. Look for GetDataAccess events (Lake Formation).

Both event types show Ethan’s IAM Identity Center user identity, confirming trusted identity propagation works end-to-end for both access patterns.

The following table lists related blog posts and integration guides covering additional identity-based access patterns with Amazon Redshift. Although many of these were written for single-Region deployments, you can extend them to multi-Region environments by first enabling IAM Identity Center Multi-Region as described in Step 1 of this post. Use the table to find the guide that matches your identity provider and tooling:

Integration / use case Identity provider What it covers Blog link
Amazon Redshift federated permissions Any Centralize permission management across multiple Amazon Redshift clusters within a Region using IAM Identity Center-linked database roles. Simplify multi-warehouse data governance with Amazon Redshift federated permissions
Amazon Redshift Query Editor V2, DbVisualizer, DBeaver Any Foundational Amazon Redshift and IAM Identity Center setup, role-based access control (RBAC), JDBC single sign-on (SSO) with PKCE. Integrate IdP with Query Editor V2 and SQL client
Amazon Redshift and S3 Access Grants (single Region and cross-account) Any Amazon S3 data access through UNLOAD/LOAD with identity-based permissions. Simplify data access with S3 Access Grants
Amazon SageMaker Unified Studio with Athena and Amazon Redshift Any SQL analytics with Lake Formation governance. Configure SSO with SageMaker Unified Studio
Amazon QuickSight with Lake Formation Any Cross-account Glue Data Catalog, business intelligence dashboards. Cross-account Glue and Lake Formation
Tableau (Desktop, Server, Prep) Okta TTI plus OIDC setup, Tableau OAuth XML configuration. Integrate Tableau with Okta
Tableau (Desktop, Server, Prep) PingFederate TTI plus OIDC setup, JWT access token manager. Integrate Tableau with PingFederate
Tableau (Desktop, Server, Prep) Microsoft Entra ID TTI plus OIDC setup, Entra app registration. Integrate Tableau with Entra ID
ThoughtSpot Okta / Microsoft Entra ID Native OIDC integration, supports both IdPs. Integrate ThoughtSpot

Key considerations

When implementing this multi-Region architecture, keep the following operational and configuration considerations in mind. These reflect common challenges and design decisions encountered during deployment:

  • IAM Identity Center Multi-Region requires a customer-managed multi-Region AWS KMS key replicated to each additional Region before you can add the Region to Identity Center.
  • S3 Access Grants instances are regional. You need a separate instance in each Region where your users access data. A bucket must be in the same Region as the Access Grants instance that manages it.
  • IAM Identity Center Multi-Region provides the same user and group identities across Regions, so you can use the same group IDs in grants across Regions.
  • You must register Lake Formation data locations with a customer-managed role that includes sts:SetContext in its trust policy. For S3 Tables, use aws lakeformation register-resource with the --with-federation flag and the resource ARN format arn:aws:s3tables:<REGION>:<ACCOUNT_ID>:bucket/*. Using the service-linked role causes the error: Cannot vend credentials from service-linked role to Identity Center principal.
  • SELECT and UNLOAD use different permission models. Lake Formation controls query-time access to cataloged data (SELECT through Spectrum). S3 Access Grants controls direct Amazon S3 access (COPY/UNLOAD). Both use the same IAM Identity Center identity.
  • The Amazon Redshift managed application IAM role must include sts:SetContext in its trust policy and have both Lake Formation/Glue and S3 Access Grants permissions.
  • Cross-account setup requires AWS RAM resource sharing for S3 Access Grants and proper IAM Identity Center application configuration in the analytics account.
  • Scoped vs object-level permissions in Amazon Redshift. When granting permissions with GRANT ... FOR TABLES IN SCHEMA, use REVOKE ... FOR TABLES IN SCHEMA to remove them. The REVOKE ... ON ALL TABLES IN SCHEMA syntax only removes object-level permissions, not scoped permissions.
  • The Lake Formation data access role for S3 Tables requires sts:SetContext in its trust policy (for TIP) and s3tables:* permissions on the table bucket resources.
  • AWSServiceRoleForRedshift must be a Read-Only Admin in Lake Formation for Amazon Redshift Query Editor V2 to display external databases from s3tablescatalog.
  • Federated catalog CatalogId format. When using CLI commands for S3 Tables resources in Lake Formation, use the full path format: <ACCOUNT_ID>:s3tablescatalog/<TABLE_BUCKET_NAME>. Using the account ID alone returns empty results.

Clean up

To avoid ongoing charges, clean up the resources created in this post:

  • Delete the S3 table bucket (delete tables → namespaces → bucket using aws s3tables CLI commands).
  • Deregister the S3 Tables resource from Lake Formation (aws lakeformation deregister-resource --resource-arn "arn:aws:s3tables:<REGION>:<ACCOUNT_ID>:bucket/*").
  • Delete s3tablescatalog from Glue (aws glue delete-catalog --catalog-id "s3tablescatalog").
  • Delete the LFAccessRole-S3Tables IAM role and associated policies.
  • Delete the S3 Access Grants instance and grants in us-west-2.
  • Delete the S3 bucket used for UNLOAD/COPY in us-west-2.
  • Delete the iamidcs3accessgrant IAM role and associated policies.
  • Deregister the S3 data location from Lake Formation.
  • Delete the Lake Formation IAM Identity Center integration.
  • Delete the Amazon Redshift cluster in us-west-2 if you created one for testing.
  • Remove us-west-2 from IAM Identity Center Multi-Region (if no longer needed).
  • Schedule deletion of the AWS KMS replica key in us-west-2 (minimum 7-day waiting period).

Conclusion

In this post, we extended the Amazon Redshift and S3 Access Grants integration to a multi-Region setup using IAM Identity Center Multi-Region replication. We demonstrated two complementary data access patterns: SELECT through Lake Formation for fine-grained access control on S3 Tables data, and UNLOAD/COPY through S3 Access Grants for direct Amazon S3 access. Both patterns use the same IAM Identity Center identity for access control. We also showed how to set up a customer-managed multi-Region AWS KMS key, enable IAM Identity Center in an additional Region, configure Amazon S3 Tables with Lake Formation for identity-based access control using Trusted Identity Propagation, and replicate the complete S3 Access Grants setup in a different Region and account.

With this approach, AnyCompany Global’s analysts authenticate once and access data in any enabled Region while Lake Formation and S3 Access Grants enforce per-user, per-group access policies.

For additional guidance, refer to the following resources:

About the authors

Maneesh Sharma

Maneesh Sharma

Maneesh is a Sr. Specialist Solutions Architect in Analytics at AWS, bringing more than 15 years of hands-on experience in designing and implementing large-scale data warehouse and analytics solutions. He collaborates closely with customers to help them build scalable, high-performance analytical data platforms.

Rohit Vashishtha

Rohit Vashishtha

Rohit is a Senior Analytics Specialist Solutions Architect at AWS based in Dallas, Texas. He has two decades of experience architecting, building, leading, and maintaining big data platforms. Rohit helps customers modernize their analytic workloads using the breadth of AWS services and ensures that customers get the best price/performance with utmost security and data governance.

Srividya Parthasarathy

Srividya Parthasarathy

Srividya is a Senior Big Data Architect with Amazon SageMaker Lakehouse. She works with the product team and customers to build robust features and solutions for their analytical data platform. She enjoys building data mesh solutions and sharing them with the community.

Sandeep Adwankar

Sandeep Adwankar

Sandeep is a Senior Product Manager with Amazon SageMaker Lakehouse. Based in the California Bay Area, he works with customers around the globe to translate business and technical requirements into products that help customers improve how they manage, secure, and access data.

„Къде е шейтанът тук?“ Аллах и тежка музика (продължение)

Post Syndicated from Атанас Шиников original https://www.toest.bg/kude-e-sheytanut-tuk-allah-i-tezhka-muzika-produlzhenie/

<< Към първа част

… ако питате ходжата, рогатият е навсякъде, особено като стане дума за хевиметъл.

„Къде е шейтанът тук?“ Аллах и тежка музика (продължение)

Стига да имаш ищах (от арабското иштияк, „желание“) да го търсиш, изборът е голям, направо безкраен. Така казва и Умберто Еко в „Името на розата“ в диалозите между Уилям от Баскервил и инквизитора Бернар Ги. Може да откриеш Иблиса и в турски банди като „Курбан“ („Жертва“, да не си мислите, че става въпрос за мазна овнешка чорба), „Кърмъзъ“ („Червено“, ако ви се прииска феминистки трашметъл), „маНга“, „Мор ве Отеси“ и един бюлюк други.

А че търсенето на шейтана в метъла може да има тежки последици, усеща на гърба си първата иракска трашбанда със звучното име „Акрасикауда“ (Acrassicauda, „Черен скорпион“). Нейната история въобще не е толкова гладка като рок сцената в Турция. И пак има късмет, защото попада в полезрението на медийната платформа „Вайс“ (VICE, „Порок“!) след падането на Саддам Хюсеин през 2003 г. и гражданската война. Тогава за малко да се озова там като „лингвист, способстващ за администрацията на следвоенен Ирак“, така поне вървеше офертата на военните за арабисти като мен. Добре че не заминах, за разлика от други мои колеги, които после пострадаха тежко. Но през 2006 г. Суруш Алви, съосновател на „Вайс“, и продуцентът Еди Морети заминават за Багдад през Ербил, Кюрдистан, за да снимат филм за траша в следвоенен Ирак. Неслучаен избор, предвид статията на „Вайс“ за „Акрасикауда“ през 2003 г. А 2006-та е лош избор за време за пътуване до Двуречието. Но пък благодатно за улавяне на новите културни феномени. Нови, ала не безпроблемни. Резултатът е документалният филм „Хевиметълът в Багдад“, който излиза през 2007 г.

Вижте ги. Момчета с кози брадички в хаоса на следвоенен Ирак, които свирят с китарите си в мазе и се самоопределят като мюсюлмани, ама не твърде посветени. Мазето прилича малко на мазето на пазарджишката гимназия, където тамошните групари свиреха, докато аз драсках с маркери графити по стените. Но обстановката не прилича на Пазарджик, пък било и през 90-те. Не може да ходиш по улиците на Багдад току-така. Може да те застрелят, защото седиш сам на едно място. Може да те застрелят, защото се движиш подозрително. Може да те застрелят, защото се събираш с хора. Може да те застрелят, защото говориш на английски – едно от най-опасните неща. Може да те застрелят, защото пътуваш. Може да те застрелят, защото не пътуваш. Може да те застрелят, защото не спазваш вечерния час. Може да те застрелят, защото си религиозен. Или защото не си религиозен. И да, защото правиш шейтанска музика. Може да си чужд агент. Не може да си пуснеш дълга коса или да носиш тежки тениски, защото веднага се набиваш на очи.

Да, правят компромис с режима на Саддам преди това. Пишат песен за иракския диктатор. Нещо като да споменеш Енгелс, когато издаваш превод на средновековния философ Пиер Абелар в България¹. „За да избегнеш дявола, пей за него“, гласи една арабска поговорка. Не знам каква точно е тя, но според мен е фриволна интерпретация на мюсюлманското схващане, че дяволът може да бъде неутрализиран чрез подходящите молитви и напевно споменаване (зикр) на Аллах.

Но истинските обвинения в сатанизъм идват после. Най-малкото защото „главотръскането“ приличало на религиозен ритуал, особено на еврейската молитва. Тъй де, има си йерархия на шейтаните. И ако приличаш на юдейския шейтан, си най за осъждане. 

Но през 2005 г. групата успява да изнесе концерт в багдадски хотел. Това все пак е оптимистична история. Всички членове на „Черен скорпион“, да не се лъжем, са представители на образованата младежка висока средна класа, имат собствени инструменти и оборудване, говорят отличен английски. В крайна сметка бягат през Сирия (където изнасят концерт в Дамаск), после до Турция и стигат до САЩ като тяхна обетована земя, подпомогнати от „Вайс“. През 2009 г. даже лично идолът им Джеймс Хетфилд от „Металика“ им подарява китара, а през 2015 г. официално излиза дебютният им албум „Гилгамеш“. Името отново е препратка към славното минало, този път на Месопотамия. Разбира се, че няма да е „Халифат“, нито „Абасид“, дори и „Харун ар-Рашид“ не е. Никакви асоциации със средновековното мюсюлманско наследство на Ирак. И без това „поклонниците на дявола“ вече са натрупали достатъчно негативи…

Но да мръднем още пò на юг и да сгъстим тъмните краски.

„Черният скорпион“ все пак отървава кожата. Историята на групата прилича на рисково приключение с щастлив край. Но не може да кажем същото за феновете на тежката музика в Египет малко по-назад в миналото.

През 1997 г., докато ние минаваме през Виденовата зима, в Египет избухва първата публична „сатанинска паника“ от хевиметъла.

Описва я Марк Ливайн в най-интересната монография по въпроса, на която съм попадал². Не само за Египет, но и за Мароко („Когато музиката бъде забранена, започва истинският сатанизъм“!), Израел, Палестина, Ливан, Иран, Пакистан.

А хевиметъл сцена има в Египет още от 90-те въпреки традиционния религиозен пласт, който съставлява огромното мнозинство от обществото. Да не забравяме също, че Египет е един от големите центрове за продукция на филми, музика и телевизия в арабския свят. Затова и египетският диалект впрочем е един от най-използваните. Само че борбата на метълите за признание и място в обществото минава през сложна политическа ситуация по времето на Хосни Мубарак и влиянието на ислямистки движения като „Мюсюлмански братя“. Същите тези, на които Саид Кутб с „Под сенките на Корана“ е идеолог.

Та сенките на Корана не се оказват благодатна шарена сянка за младежите, които гледат навън през призмата на рок културата. На 22 януари 1997 г. правителството подема кампания срещу метъл общността, след като във вестниците излизат снимки на концерти, на които уж се показва как се носят обърнати кръстове. И мюсюлмански, и християнски религиозни служители се скандализират. Макар че мен ако питате, мюсюлманските активисти би трябвало да са доволни – нали християнският кръст като цяло е пълен харам в исляма? Но да речем, че обърнатият кръст говори за шейтанопоклонничество.

Стотина фенове на тежката музика са окошарени. Десетки са прибрани от старата изоставена къща на белгийския барон генерал от XIX век в Кайро, където младежи се събират да слушат музика, да пръскат графити и да купонясват. Това пък води и до описанието на мястото в египетските медии като пълно с татуирани поклонници на дявола, които правят оргии, дерат котки и пишат имената си с кръв от плъхове по стените. Че даже и египетският мюфтия Наср Фарид Уасил призовава към покаяние на задържаните или смъртна казън, полагаща се за отстъпничество от исляма. Отгоре на всичко „сатанинската афера“ се случва в същата година, в която терористи удрят туристически места в Луксор и Кайро.

Събитието е с такава значимост за сцената, че някои банди дори изгарят или изхвърлят инструментите си, за да избегнат арести и обвинения, а историята на тежката музика в Египет се разделя на преди и след 1997 г. Едва десетина години по-късно, през 2008-ма, „Ройтерс“ публикува статия за хевиметъла, който предпазливо се завръща в Египет. Завръща се, ала не много решително, тъй като демонизацията продължава. През 2010 г. дори бившият съветник на главния мюфтия Тухами Мунтасир твърди, че метъл сцената в Египет е финансирана от „ционистите“. Че то кое лошо нещо в арабския свят не е, да сипя и аз сол в раната?! В крайна сметка и „Протоколите на ционските мъдреци“, този доказан фалшификат, все още е популярно четиво там.

През 2016 г. отново се надига брожение с цел да се възбрани музиката на „поклонниците на дявола“. „Таймс“ отразява полицейско нахлуване по време на хевиметъл концерт в Кайро с обвинения към организаторите, че участват в „сатанински ритуали“ и се подготвя „сатанинско парти“. „Сепултура“ отменят участието си в събитието.

А обвързването на метъла с шейтана става стереотипна част от сцената не само в Египет. В Мароко изплуват обвинения в сатанизъм и арести по време на фестивала в Сиди Касем, да не говорим за Иран, където хеви- и блекметълът се инкриминират по най-различни линии, включително защото вървят със сатанински ритуали и притежаване на наркотици. И ако на музиката в средите на мюсюлманите по принцип се гледа като нещо греховно и възбранено, както твърди една фетва тук, нейният тежък пласт направо потъва и удря дъното в обиталището на самия Иблис. Ето как тече разговорът с богословите от популярния и авторитетен сайт за фетви на катарското Министерство на религиозните дарения и дела.

Какво означава метълист (миталджи)?

(Метълджия. Харесва ми като словообразуване на арабски. Западна дума с турска наставка за професия, подобно на чорбаджия, бозаджия, ваксаджия, скъпчия.)

В какво вярват метълите? –

продължава запитването от анонимен вярващ, свързани ли са с кланянето на шейтана, как трябва да се отнасяме с тях и какво правят по време на събиранията си? Тъй де, звучи като ударен курс относно „що е метъл и какво да правим с него“.

Отговорът върви така. Думата миталджи означава някой, който слуша или изпълнява с музикални инструменти вид музика, наречена „метъл“, отличаваща се с голяма извратеност. Откъде мюфтията знае това? Много лесно – „научихме го от прочита на някои техни форуми и сайтове в интернет, които не подобава да слагаме тук като линкове, защото би било насърчаване на порицаеми действия“. Обаче не е задължително всеки, който слуша тази греховна, възбранена музика, да бъде поклонник на дявола. Само че поклонниците на дявола слушат тази музика. А тя, особено във вариацията хевиметъл, насърчава подобни поведенчески модели, сатанински действия, ритуали и начин на живот по време на упадъчните им събирания. Песните изразяват вярванията им, които се основават на освещаването, възвеличаването и поклонението пред дявола, призиви към разврат, убийство и самоунищожение. За да могат накрая, по техните собствени твърдения, да бъдат удостоени с наградата да влязат в Ада през някоя от неговите седем двери. Други видове музика, които слушат по време на своите сбирки, са блекметълът (ал-блак митал) и хардрокът (ал-хард рук). И именно тази музика се свързва със странните им ритуали, които са противни на естеството, че и на всички религии, дори и на извратените. Сред ритуалите им са вземане на наркотици, пиене на алкохол, сексуална перверзия, оскверняване на мъртвите и пиене на човешка кръв. Добави към това и „истерично танцуване почти до степен на припадък и притъпяване на човешките сетива“ и става ясно, че миталджиите не са поели по добър път.

В катарския сайт цитират даже статия относно „Историческите корени на сатанизма“, публикувана в журнала на Ислямския университет в Газа. А за повече информация как да се отнасяме с метълите се препраща към фетва, която отговаря на въпроса как да се противопоставим на „интелектуалната инвазия“ (газуу фикри), тоест на външните атаки, с цел отслабване на силите на мюсюлманската общност и подриване на нейните морални устои. Не стига моралната и верска развала, ами и метълите са натоварени с основна роля в офанзивата срещу здравите основи на самата умма и разгръщането на мисията ѝ в хода на историята.

Очевидно такава гледна точка не отчита разликите между музикалните жанрове. Тъй де, все едно вие да правите разлика между правните школи на шафиити, ханбалити, маликити и ханифити в исляма. Някакви ходжи там се карат, но отвън на нас всичко ни изглежда едно и също. Та и тук така. Да, знаем за любимия жест на Рони Джеймс Дио, който напомня на рогатия. Или пък за адския електронен амбианс на „Бурзум“ (вокално-инструментален състав „Тъмнина“, да спазим правилото за превод на имената). За дет- и блекметъла в Скандинавието, че дори и в арабския свят. А и за турците от „Пентаграм“ – най-малкото заради символа. Те може и да пасват на фетвата от Катар. Ала очакваме ли същото ниво на „шейтанство“ за всички групи и концерти? Еднакъв призив към пиянство, разврат, „седемте порти на Ада“, „оскверняване на мъртвите“ и „пиене на човешка кръв“? Е да, концертите може и да вървят с порочно пиене на бира и съмнителни общувания между двата пола. Ама то е по акциденция, не е същностно, както би отсъдил кадията Ибн Рушд от Кордоба, известен ни като Авероес. А какво да кажем за възприемащия се за християнски „бял метъл“?

И тук е време да пристъпим в една по-оптимистична, макар и ничия територия, където тежката музика и ислямът се припокриват.

Обитателите на тази територия се опитват да изградят положителен разказ за успеха, който удържа жицата в едно с правоверието. Шейховете може и да не харесват това. И то наистина звучи малко като философски разказ за невъзможно смесване на същностите. Русалка. Кентавър. Химера. Соева наденица. Халален алкохол. Позволено прасе.

Но има музиканти, които упорито настояват, че е възможно да бъдеш добър мюсюлманин и миталджи едновременно.

През 2003 г. американският автор Майкъл Мухаммад Найт, приел исляма, измисля термина такуакор, който става заглавие на едноименната му книга, рекламирана като „Спасителя в ръжта за млади мюсюлмани“. Много е просто – такуа на арабски е „благочестие“, „богобоязливост“, а кор идва от „хардкор“. Книгата разказва за измислени герои, мюсюлмански бунтари – суфи с пънк гребени, скинхедс шиити, ъндърграунд феминистки с бурки. Измислени, но вдъхновяват реална музикална сцена. Пънк с мюсюлмански привкус. През 2009 г. излиза и документалният филм за американското движение под заглавие „Такуакор: раждането на пънк исляма“.

В рамките на това екзотично движение се пръкват и американски групи със забавни имена като „Дъ Коминас“ („Келешите“ на урду), „Ас-Саура“ („Революция“ на арабски), „Воут Хезболла“ („Гласувайте за Хизбулла“). И добре че този творчески фюжън е в САЩ, защото в Египет, Ирак или Пакистан няма как да изкласи. Само ще припомня, че втората книга на Майкъл Мухаммад Найт е озаглавена „Осама Ван Хейлън“.

А на другия край на света, в Индонезия, сме свидетели на друг феномен – тежък метъл и забрадки.

Момичетата от „Войс ъв бачепрот“ (V.O.B., „Гласът на шума“ на местния им език) не съзират противоречие в това да правят кавъри на „Металика“, докато носят консервативно изглеждащи хиджаби. През 2014 г. се събират в Гарут, провинция Западна Ява в Индонезия. (А Индонезия е най-голямата страна с мюсюлманско мнозинство в света – над 270 млн. жители, около 90% от които са мюсюлмани.) Ерза Сатия, техният учител по музика, който открива метъл таланта им и поема ролята на пръв мениджър, опровергава разпространените мнения, че музиката била сатанинска. Тя е всъщност начин учениците да избегнат пороци като наркотиците и непозволения от исляма предбрачен секс.

Все пак една забрадка халал не прави, ако трябва да перифразирам нашенската поговорка за едната птичка и пролетта. И тях постоянно ги заплашват със смърт. Така де, един от най-лесните начини да си докараш смъртна присъда по Свещения закон на исляма е да те обвинят в богохулство или отстъпничество. Механизмът е много ясен, отработен, вековен и води обикновено до един-единствен резултат. Не че това пречи на миловидните момичета с хиджаби да правят турнета и да ги канят на фестивала във Вакен, тази германска Мека на метъла.

От Мароко до Пакистан, от Турция до Субсахарска Африка и Саудитска Арабия, от САЩ до Индонезия: всяка банда има своята история като част от местния контекст. Дали ще е следвоенен Ирак, кемалистка Турция, Арабската пролет – очевидно е, че тежката музика няма да си отиде въпреки благочестивото ѝ очерняне. Като казахме очерняне, препрочитам един разказ на Орхан Памук, в който говори за рисунките на Сиях Калем – „Черния писец“. Легендарен илюстратор, група от художници или стил на едни много разпознаваеми картинки от XV век. Вижте този сюжет от колекцията на музея в Кливланд.

„Къде е шейтанът тук?“ Аллах и тежка музика (продължение)
Окован демон от колекцията на музея в Кливланд

Не е ли това демонът на тежката музика в „дома на исляма“?

Със зачервените си очи, увиснали клепачи и тъжни зъбати бърни изглежда хем грозен, хем нещастен. Хем опасен, хем безпомощен. От едната му страна стои мъжка фигура, която го налага. Уж това е фигурата на добрия, който се бори със злото. От другата страна е женски силует, който пристяга верига около шията му. Само че колкото повече ги гледаш, толкова повече се чудиш кой е по-зъбат, по-страшен и по-злочест – окованият или тези, които са го оковали. Както казва и Памук, хем се боим от ония дяволи, хем осъзнаваме, че с тях сме замесени от едно и също тесто³. Пък и съгласете се, Сиях Калем звучи като чудесен избор за име на метъл банда. С дебютен албум „Гул“ или „Див“ („Демон“) и тази картинка за обложка. Напълно съответства на традициите в жанра.

1 Абелар, Пиер. Избрани съчинения. София: Наука и изкуство, 1986, с. 21 – 22.

2 LeVine, Mark. Heavy Metal Islam. Rock, Resistance, and the Struggle for the Soul of Islam. Oakland: University of California Press, 2022.

3 Памук, Орхан. Други цветове. София: Еднорог, с. 363.

В рубриката „Ориент кафе“ Атанас Шиников поднася любопитни теми, свързани не толкова с горещата политика, колкото с историята и културата на Близкия изток. А той, древен и днешен, е по-близко до нас и съвремието ни, отколкото си представяме.

The collective thoughts of the interwebz