Using Amazon EMR DeltaStreamer to stream data to multiple Apache Hudi tables

2026-01-15 Gautam Bhaghavatula

Post Syndicated from Gautam Bhaghavatula original https://aws.amazon.com/blogs/big-data/using-amazon-emr-deltastreamer-to-stream-data-to-multiple-apache-hudi-tables/

In this post, we show you how to implement real-time data ingestion from multiple Kafka topics to Apache Hudi tables using Amazon EMR. This solution streamlines data ingestion by processing multiple Amazon Managed Streaming for Apache Kafka (Amazon MSK) topics in parallel while providing data quality and scalability through change data capture (CDC) and Apache Hudi.

Organizations processing real-time data changes across multiple sources often struggle with maintaining data consistency and managing resource costs. Traditional batch processing requires reprocessing entire datasets, leading to high resource usage and delayed analytics. By implementing CDC with Apache Hudi’s MultiTable DeltaStreamer, you can achieve real-time updates; efficient incremental processing with atomicity, consistency, isolation, durability (ACID) guarantees; and seamless schema evolution while minimizing storage and compute costs.

Using Amazon Simple Storage Service (Amazon S3), Amazon CloudWatch, Amazon EMR, Amazon MSK and AWS Glue Data Catalog, you’ll build a production-ready data pipeline that processes changes from multiple data sources simultaneously. Through this tutorial, you’ll learn to configure CDC pipelines, manage table-specific configurations, implement 15-minute sync intervals, and maintain your streaming pipeline. The result is a robust system that maintains data consistency while enabling real-time analytics and efficient resource utilization.

What is CDC?

Imagine a constantly evolving data stream, a river of information where updates flow continuously. CDC acts like a sophisticated net, capturing only the modifications—the inserts, updates, and deletes—happening within that data stream. Through this targeted approach, you can focus on the new and changed data, significantly improving the efficiency of your data pipelines.There are numerous advantages to embracing CDC:

Reduced processing time – Why reprocess the entire dataset when you can focus only on the updates? CDC minimizes processing overhead, saving valuable time and resources.
Real-time insights – With CDC, your data pipelines become more responsive. You can react to changes almost instantaneously, enabling real-time analytics and decision-making.
Simplified data pipelines – Traditional batch processing can lead to complex pipelines. CDC streamlines the process, making data pipelines more manageable and easier to maintain.

Why Apache Hudi?

Hudi simplifies incremental data processing and data pipeline development. This framework efficiently manages business requirements such as data lifecycle and improves data quality. You can use Hudi to manage data at the record-level in Amazon S3 data lakes to simplify CDC and streaming data ingestion and handle data privacy use cases requiring record-level updates and deletes. Datasets managed by Hudi are stored in Amazon S3 using open storage formats, while integrations with Presto, Apache Hive, Apache Spark, and Data Catalog give you near real time access to updated data. Apache Hudi facilitates incremental data processing for Amazon S3 by:

Managing record-level changes – Ideal for update and delete use cases
Open formats – Integrates with Presto, Hive, Spark, and Data Catalog
Schema evolution – Supports dynamic schema changes
HoodieMultiTableDeltaStreamer – Simplifies ingestion into multiple tables using centralized configurations

Hudi MultiTable Delta Streamer

The HoodieMultiTableStreamer offers a streamlined approach to data ingestion from multiple sources into Hudi tables. By processing multiple sources simultaneously through a single DeltaStreamer job, it eliminates the need for separate pipelines while reducing operational complexity. The framework provides flexible configuration options, and you can tailor settings for diverse formats and schemas across different data sources.

One of its key strengths lies in unified data delivery, organizing information in respective Hudi tables for seamless access. The system’s intelligent upsert capabilities efficiently handle both inserts and updates, maintaining data consistency across your pipeline. Additionally, its robust schema evolution support enables your data pipeline to adapt to changing business requirements without disruption, making it an ideal solution for dynamic data environments.

Solution overview

In this section, we show how to stream data to Apache Hudi Table using Amazon MSK. For this example scenario, there are data streams from three distinct sources residing in separate Kafka topics. We aim to implement a streaming pipeline that uses the Hudi DeltaStreamer with multitable support to ingest and process this data at 15-minute intervals.

Mechanism

Using MSK Connect, data from multiple sources flows into MSK topics. These topics are then ingested into Hudi tables using the Hudi MultiTable DeltaStreamer. In this sample implementation, we create three Amazon MSK topics and configure the pipeline to process data in JSON format using JsonKafkaSource, with the flexibility to handle Avro format when needed through the appropriate deserializer configuration

The following diagram illustrates how our solution processes data from multiple source databases through Amazon MSK and Apache Hudi to enable analytics in Amazon Athena. Source databases send their data changes—including inserts, updates, and deletes—to dedicated topics in Amazon MSK, where each data source maintains its own Kafka topic for change events. An Amazon EMR cluster runs the Apache Hudi MultiTable DeltaStreamer, which processes these multiple Kafka topics in parallel, transforming the data and writing it to Apache Hudi tables stored in Amazon S3. Data Catalog maintains the metadata for these tables, enabling seamless integration with analytics tools. Finally, Amazon Athena provides SQL query capabilities on the Hudi tables, allowing analysts to run both snapshot and incremental queries on the latest data. This architecture scales horizontally as new data sources are added, with each source getting its dedicated Kafka topic and Hudi table configuration, while maintaining data consistency and ACID guarantees across the entire pipeline.

To set up the solution, you need to complete the following high-level steps:

Set up Amazon MSK and create Kafka topics
Create the Kafka topics
Create table-specific configurations
Launch Amazon EMR cluster
Invoke the Hudi MultiTable DeltaStreamer
Verify and query data

Prerequisites

To perform the solution, you need to have the following prerequisites. For AWS services and permissions, you need:

AWS account:
- Amazon MSK. Refer to Getting started with Amazon MSK.
- Amazon EMR (version 6.15.0 or later recommended).
- Amazon S3.
- Data Catalog.
- IAM roles and policies.
IAM roles:
- Amazon EMR service role (EMR_DefaultRole) with permissions for Amazon S3, AWS Glue and CloudWatch.
- Amazon EC2 instance profile (EMR_EC2_DefaultRole) with S3 read/write access.
- Amazon MSK access role with appropriate permissions.
S3 buckets:
- Configuration bucket for storing properties files and schemas.
- Output bucket for Hudi tables.
- Logging bucket (optional but recommended).
Network configuration:
- Amazon Virtual Private Cloud (Amazon VPC) with appropriate subnets.
- Security groups allowing traffic between Amazon EMR and Amazon MSK.
- NAT gateway or VPC endpoints for Amazon S3 access.
Development tools:
- AWS CLI configured.
- Access to Amazon EMR console or AWS CLI.
- Kafka client tools for testing.

Set up Amazon MSK and create Kafka topics

In this step, you’ll create an MSK cluster and configure the required Kafka topics for your data streams.

To create an MSK cluster:

aws kafka create-cluster \
    --cluster-name hudi-msk-cluster \
    --broker-node-group-info file://broker-nodes.json \
    --kafka-version "2.8.1" \
    --number-of-broker-nodes 3 \
    --encryption-info file://encryption-info.json \
    --client-authentication file://client-authentication.json

Verify the cluster status:

aws kafka describe-cluster --cluster-arn $CLUSTER_ARN | jq '.ClusterInfo.State'

The command should return ACTIVE when the cluster is ready.

Schema setup

To set up the schema, complete the following steps:

Create your schema files.

input_schema.avsc:

{
    "type": "record",
    "name": "CustomerSales",
    "fields": [
        {"name": "Id", "type": "string"},
        {"name": "ts", "type": "long"},
        {"name": "amount", "type": "double"},
        {"name": "customer_id", "type": "string"},
        {"name": "transaction_date", "type": "string"}
    ]
}

output_schema.avsc:

{
    "type": "record",
    "name": "CustomerSalesProcessed",
    "fields": [
        {"name": "Id", "type": "string"},
        {"name": "ts", "type": "long"},
        {"name": "amount", "type": "double"},
        {"name": "customer_id", "type": "string"},
        {"name": "transaction_date", "type": "string"},
        {"name": "processing_timestamp", "type": "string"}
    ]
}

Create and upload schemas to your S3 bucket:

# Create the schema directory
aws s3 mb s3://hudi-config-bucket-$AWS_ACCOUNT_ID
aws s3api put-object --bucket hudi-config-bucket-$AWS_ACCOUNT_ID --key HudiProperties/
# Upload schema files
aws s3 cp input_schema.avsc s3://hudi-config-bucket-$AWS_ACCOUNT_ID/HudiProperties/
aws s3 cp output_schema.avsc s3://hudi-config-bucket-$AWS_ACCOUNT_ID/HudiProperties/

Create the Kafka topics

To create the Kafka topics, complete the following steps:

Get the bootstrap broker string:

# Get bootstrap brokers
BOOTSTRAP_BROKERS=$(aws kafka get-bootstrap-brokers --cluster-arn $CLUSTER_ARN --query 'BootstrapBrokerString' --output text)

Create the required topics:

kafka-topics.sh --create \
    --bootstrap-server $BOOTSTRAP_BROKERS \
    --replication-factor 3 \
    --partitions 3 \
    --topic cust_sales_details
kafka-topics.sh --create \
    --bootstrap-server $BOOTSTRAP_BROKERS \
    --replication-factor 3 \
    --partitions 3 \
    --topic cust_sales_appointment
kafka-topics.sh --create \
    --bootstrap-server $BOOTSTRAP_BROKERS \
    --replication-factor 3 \
    --partitions 3 \
    --topic cust_info

Configure Apache Hudi

The Hudi MultiTable DeltaStreamer configuration is divided into two major components to streamline and standardize data ingestion:

Common configurations – These settings apply across all tables and define the shared properties for ingestion. They include details such as shuffle parallelism, Kafka brokers, and common ingestion configurations for all topics.
Table-specific configurations – Each table has unique requirements, such as the record key, schema file paths, and topic names. These configurations tailor each table’s ingestion process to its schema and data structure.

Create common configuration file

Common Config: kafka-hudi config file where we specify kafka broker and common configuration for all topics as below

Create the kafka-hudi-deltastreamer.properties file with the following properties:

# Common parallelism settings
hoodie.upsert.shuffle.parallelism=2
hoodie.insert.shuffle.parallelism=2
hoodie.delete.shuffle.parallelism=2
hoodie.bulkinsert.shuffle.parallelism=2
# Table ingestion configuration
hoodie.deltastreamer.ingestion.tablesToBeIngested=hudi_sales_tables.cust_sales_details,hudi_sales_tables.cust_sales_appointment,hudi_sales_tables.cust_info
# Table-specific config files
hoodie.deltastreamer.ingestion.hudi_sales_tables.cust_sales_details.configFile=s3://hudi-config-bucket-$AWS_ACCOUNT_ID/HudiProperties/tableProperties/cust_sales_details.properties
hoodie.deltastreamer.ingestion.hudi_sales_tables.cust_sales_appointment.configFile=s3://hudi-config-bucket-$AWS_ACCOUNT_ID/HudiProperties/tableProperties/cust_sales_appointment.properties
hoodie.deltastreamer.ingestion.hudi_sales_tables.cust_info.configFile=s3://hudi-config-bucket-$AWS_ACCOUNT_ID/HudiProperties/tableProperties/cust_info.properties
# Source configuration
hoodie.deltastreamer.source.dfs.root=s3://hudi-config-bucket-$AWS_ACCOUNT_ID/HudiProperties/
# MSK configuration
bootstrap.servers=BOOTSTRAP_BROKERS_PLACEHOLDER
auto.offset.reset=earliest
group.id=hudi_delta_streamer
# Security configuration
hoodie.sensitive.config.keys=ssl,tls,sasl,auth,credentials
sasl.mechanism=PLAIN
security.protocol=SASL_SSL
ssl.endpoint.identification.algorithm=
# Deserializer
hoodie.deltastreamer.source.kafka.value.deserializer.class=io.confluent.kafka.serializers.KafkaAvroDeserializer

Create table-specific configurations

For each topic, create its own configuration with a topic name and primary key details. Complete the following steps:

cust_sales_details.properties:

# Table: cust sales
hoodie.datasource.write.recordkey.field=Id
hoodie.deltastreamer.source.kafka.topic=cust_sales_details
hoodie.deltastreamer.keygen.timebased.timestamp.type=UNIX_TIMESTAMP
hoodie.deltastreamer.keygen.timebased.input.dateformat=yyyy-MM-dd HH:mm:ss.S
hoodie.streamer.schemaprovider.registry.schemaconverter=
hoodie.datasource.write.precombine.field=ts

cust_sales_appointment.properties:

# Table: cust sales appointment
hoodie.datasource.write.recordkey.field=Id
hoodie.deltastreamer.source.kafka.topic=cust_sales_appointment
hoodie.deltastreamer.keygen.timebased.timestamp.type=UNIX_TIMESTAMP
hoodie.deltastreamer.keygen.timebased.input.dateformat=yyyy-MM-dd HH:mm:ss.S hoodie.streamer.schemaprovider.registry.schemaconverter=
hoodie.datasource.write.precombine.field=ts

cust_info.properties:

# Table: cust info
hoodie.datasource.write.recordkey.field=Id
hoodie.deltastreamer.source.kafka.topic=cust_info
hoodie.deltastreamer.keygen.timebased.timestamp.type=UNIX_TIMESTAMP
hoodie.deltastreamer.keygen.timebased.input.dateformat= yyyy-MM-dd HH:mm:ss.S
hoodie.streamer.schemaprovider.registry.schemaconverter=
hoodie.datasource.write.precombine.field=ts
hoodie.deltastreamer.schemaprovider.source.schema.file=-$AWS_ACCOUNT_ID/HudiProperties/input_schema.avsc
hoodie.deltastreamer.schemaprovider.target.schema.file=-$AWS_ACCOUNT_ID/HudiProperties/output_schema.avsc

These configurations form the backbone of Hudi’s ingestion pipeline, enabling efficient data handling and maintaining real-time consistency. Schema configurations define the structure of both source and target data, maintaining seamless data transformation and ingestion. Operational settings control how data is uniquely identified, updated, and processed incrementally.

The following are critical details for setting up Hudi ingestion pipelines:

hoodie.deltastreamer.schemaprovider.source.schema.file – The schema of the source record
hoodie.deltastreamer.schemaprovider.target.schema.file – The schema for the target record
hoodie.deltastreamer.source.kafka.topic – The source MSK topic name
bootstap.servers – The Amazon MSK bootstrap server’s private endpoint
auto.offset.reset – The consumer’s behavior when there is no committed position or when an offset is out of range

Key operational fields to achieve in-place updates for the generated schema include:

hoodie.datasource.write.recordkey.field – The record key field. This is the unique identifier of a record in Hudi.
hoodie.datasource.write.precombine.field – When two records have the same record key value, Apache Hudi picks the one with the largest value for the pre-combined field.
hoodie.datasource.write.operation – The operation on the Hudi dataset. Possible values include UPSERT, INSERT, and BULK_INSERT.

Launch Amazon EMR cluster

This step creates an EMR cluster with Apache Hudi installed. The cluster will run the MultiTable DeltaStreamer to process data from your Kafka topics. To create the EMR cluster, enter the following:

# Create EMR cluster with Hudi installed
aws emr create-cluster \
    --name "Hudi-CDC-Cluster" \
    --release-label emr-6.15.0 \
    --applications Name=Hadoop Name=Spark Name=Hive Name=Livy \
    --ec2-attributes KeyName=myKey,SubnetId=$SUBNET_ID,InstanceProfile=EMR_EC2_InstanceProfile \
    --service-role EMR_ServiceRole \
    --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=m5.xlarge InstanceGroupType=CORE,InstanceCount=2,InstanceType=m5.xlarge \
    --configurations file://emr-configurations.json \
    --bootstrap-actions Name="Install Hudi",Path="s3://hudi-config-bucket-$AWS_ACCOUNT_ID/bootstrap-hudi.sh"

Invoke the Hudi MultiTable DeltaStreamer

This step configures and starts the DeltaStreamer job that will continuously process data from your Kafka topics into Hudi tables. Complete the following steps:

Connect to the Amazon EMR master node:

# Get master node public DNS
MASTER_DNS=$(aws emr describe-cluster --cluster-id $CLUSTER_ID --query 'Cluster.MasterPublicDnsName' --output text)

# SSH to master node
ssh -i myKey.pem hadoop@$MASTER_DNS

Execute the DeltaStreamer job:

# 
spark-submit --deploy-mode client \
  --conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" \
  --conf "spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog" \
  --conf "spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension" \
  --jars "/usr/lib/hudi/hudi-utilities-bundle_2.12-0.14.0-amzn-0.jar,/usr/lib/hudi/hudi-spark-bundle.jar" \
  --class "org.apache.hudi.utilities.deltastreamer.HoodieMultiTableDeltaStreamer" \
  /usr/lib/hudi/hudi-utilities-bundle_2.12-0.14.0-amzn-0.jar \
  --props s3://hudi-config-bucket-$AWS_ACCOUNT_ID/HudiProperties/kafka-hudi-deltastreamer.properties \
  --config-folder s3://hudi-config-bucket-$AWS_ACCOUNT_ID/HudiProperties/tableProperties/ \
  --table-type MERGE_ON_READ \
  --base-path-prefix s3://hudi-data-bucket-$AWS_ACCOUNT_ID/hudi/ \
  --source-class org.apache.hudi.utilities.sources.JsonKafkaSource \
  --schemaprovider-class org.apache.hudi.utilities.schema.FilebasedSchemaProvider \
  --op UPSERT

For continuous mode, you need to add the following property:


--continuous \
--min-sync-interval-seconds 900

With the job configured and running on Amazon EMR, the Hudi MultiTable DeltaStreamer efficiently manages real-time data ingestion into your Amazon S3 data lake.

Verify and query data

To verify and query the data, complete the following steps:

# Start Spark shell
spark-shell --conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" \
  --conf "spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog" \
  --conf "spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension" \
  --jars "/usr/lib/hudi/hudi-spark-bundle.jar"

# In Spark shell
spark.sql("CREATE DATABASE IF NOT EXISTS hudi_sales_tables")

spark.sql("""
CREATE TABLE hudi_sales_tables.cust_sales_details
USING hudi
LOCATION 's3://hudi-data-bucket-$AWS_ACCOUNT_ID/hudi/hudi_sales_tables.cust_sales_details'
""")

# Repeat for other tables

Query with Athena:

-- Sample query
SELECT * FROM hudi_sales_tables.cust_sales_details LIMIT 10;

You can use Amazon CloudWatch alarms to alert you of issues with the EMR job or data processing. To create a CloudWatch alarm to monitor EMR job failures, enter the following:

aws cloudwatch put-metric-alarm \
    --alarm-name EMR-Hudi-Job-Failure \
    --metric-name JobsFailed \
    --namespace AWS/ElasticMapReduce \
    --statistic Sum \
    --period 300 \
    --threshold 1 \
    --comparison-operator GreaterThanOrEqualToThreshold \
    --dimensions Name=JobFlowId,Value=$CLUSTER_ID \
    --evaluation-periods 1 \
    --alarm-actions $SNS_TOPIC_ARN

Real-world impact of Hudi CDC pipelines

With the pipeline configured and running, you can achieve real-time updates to your data lake, enabling faster analytics and decision-making. For instance:

Analytics – Up-to-date inventory data maintains accurate dashboards for ecommerce platforms.
Monitoring – CloudWatch metrics confirm the pipeline’s health and efficiency.
Flexibility – The seamless handling of schema evolution minimizes downtime and data inconsistencies.

Cleanup

To avoid incurring future charges, follow these steps to clean up resources:

Conclusion

In this post, we showed how you can build a scalable data ingestion pipeline using Apache Hudi’s MultiTable DeltaStreamer on Amazon EMR to process data from multiple Amazon MSK topics. You learned how to configure CDC with Apache Hudi, set up real-time data processing with 15-minute sync intervals, and maintain data consistency across multiple sources in your Amazon S3 data lake.

To learn more, explore these resources:

By combining CDC with Apache Hudi, you can build efficient, real-time data pipelines. The streamlined ingestion processes simplify management, enhance scalability, and maintain data quality, making this approach a cornerstone of modern data architectures.

About the authors

When protections outlive their purpose: A lesson on managing defense systems at scale

2026-01-15 Thomas Kjær Aabo

Post Syndicated from Thomas Kjær Aabo original https://github.blog/engineering/infrastructure/when-protections-outlive-their-purpose-a-lesson-on-managing-defense-systems-at-scale/

To keep a platform like GitHub available and responsive, it’s critical to build defense mechanisms. A whole lot of them. Rate limits, traffic controls, and protective measures spread across multiple layers of infrastructure. These all play a role in keeping the service healthy during abuse or attacks.

We recently ran into a challenge: Those same protections can quietly outlive their usefulness and start blocking legitimate users. This is especially true for protections added as emergency responses during incidents, when responding quickly means accepting broader controls that aren’t necessarily meant to be long-term. User feedback led us to clean up outdated mitigations and reinforced that observability is just as critical for defenses as it is for features.

We apologize for the disruption. We should have caught and removed these protections sooner. Here’s what happened.

What users reported

We saw reports on social media from people getting “too many requests” errors during normal, low-volume browsing, such as when following a GitHub link from another service or app, or just browsing around with no obvious pattern of abuse.

Screenshot of a 'Too many requests' screen encountered by users. — *Users encountered a “Too many requests” error during normal browsing.*

These were users making a handful of normal requests hitting rate limits that shouldn’t have applied to them.

What we found

Investigating these reports, we discovered the root cause: Protection rules added during past abuse incidents had been left in place. These rules were based on patterns that had been strongly associated with abusive traffic when they were created. The problem is that those same patterns were also matching some logged-out requests from legitimate clients.

These patterns are combinations of industry-standard fingerprinting techniques alongside platform-specific business logic — composite signals that help us distinguish legitimate usage from abuse. But, unfortunately, composite signals can occasionally produce false positives.

The composite approach did provide filtering. Among requests that matched the suspicious fingerprints, only about 0.5–0.9% were actually blocked; specifically, those that also triggered the business-logic rules. Requests that matched both criteria were blocked 100% of the time.

Chart showing percentage of fingerprint matches that were blocked by also triggering business-logic rules, fluctuating between 0.5-0.9% over 60 minutes — *Not all fingerprint matches resulted in blocks — only those also matching business logic patterns.*

The overall impact was small but consistent; however, for the customers who were affected, we recognize that any incorrect blocking is unacceptable and can be disruptive. To put all of this in perspective, the following shows the false-positive rate relative to total traffic.

Chart showing false positives as approximately 0.003-0.004% of total traffic, with a reference line at 100% — *False positives represented roughly 0.003-0.004% of total traffic.*

Although the percentage was low, it still meant that real users were incorrectly blocked during normal browsing, which is not acceptable. The chart below zooms in specifically on this false-positive pattern over time.

Chart showing false positive rate over 60 minutes, hovering around 0.003-0.004% — *In the hour before cleanup, approximately 3-4 requests per 100,000 (0.003-0.004%) were incorrectly blocked.*

This is a common challenge when defending platforms at scale. During active incidents, you need to respond quickly, and you accept some tradeoffs to keep the service available. The mitigations are correct and necessary at that moment. Those emergency controls don’t age well as threat patterns evolve and legitimate tools and usage change.

Without active maintenance, temporary mitigations become permanent, and their side effects compound quietly.

Tracing through the stack

The investigation itself highlighted why these issues can persist. When users reported errors, we traced requests across multiple layers of infrastructure to identify where the blocks occurred.

To understand why this tracing is necessary, it helps to see how protection mechanisms are applied throughout our infrastructure. We’ve built a custom, multi-layered protection infrastructure tailored to GitHub’s unique operational requirements and scale, building upon the flexibility and extensibility of open-source projects like HAProxy. Here’s a simplified view of how requests flow through these defense layers (simplified to avoid disclosing specific defense mechanisms and to keep the concepts broadly applicable):

Diagram showing user requests flowing through multiple infrastructure layers (Edge, Application, Service, Backend), with protection mechanisms at each layer including DDoS protection, rate limits, authentication, and access controls.

Each layer has legitimate reasons to rate-limit or block requests. During an incident, a protection might be added at any of these layers depending on where the abuse is best mitigated and what controls are fastest to deploy.

The challenge: When a request gets blocked, tracing which layer made that decision requires correlating logs across multiple systems, each with different schemas.

In this case, we started with user reports and worked backward:

User reports provided timestamps and approximate behavior patterns.
Edge tier logs showed the requests reaching our infrastructure.
Application tier logs revealed 429 “Too Many Requests” responses.
Protection rule analysis ultimately identified which rules matched these requests.

The investigation took us from external reports to distributed logs to rule configurations, demonstrating that maintaining comprehensive visibility into what’s actually blocking requests and where is essential.

The lifecycle of incident mitigations

Here’s how these protections outlived their purpose:

Diagram showing incident mitigation lifecycle: control added during incident, works initially, remains active over time without review, eventually blocks legitimate traffic.

Each mitigation was necessary when added. But the controls where we didn’t consistently apply lifecycle management (setting expiration dates, conducting post-incident rule reviews, or monitoring impact) became technical debt that accumulated until users noticed.

What we did

We reviewed these mitigations, analyzing what each one was blocking today versus what it was meant to block when created. We removed the rules that were no longer serving their purpose, and kept protections against ongoing threats.

What we’re building

Beyond the immediate fix, we’re improving the lifecycle management of protective controls:

Better visibility across all protection layers to trace the source of rate limits and blocks.
Treating incident mitigations as temporary by default. Making them permanent should require an intentional, documented decision.
Post-incident practices that evaluate emergency controls and evolve them into sustainable, targeted solutions.

Defense mechanisms – even those deployed quickly during incidents – need the same care as the systems they protect. They need observability, documentation, and active maintenance. When protections are added during incidents and left in place, they become technical debt that quietly accumulates.

Thanks to everyone who reported issues publicly! Your feedback directly led to these improvements. And thanks to the teams across GitHub who worked on the investigation and are building better lifecycle management into how we operate. Our platform, team, and community are better together!

The post When protections outlive their purpose: A lesson on managing defense systems at scale appeared first on The GitHub Blog.

Running Debian on the OpenWrt One (Collabora Blog)

2026-01-15 jzb

Post Syndicated from jzb original https://lwn.net/Articles/1054519/

Sjoerd Simons has published
a blog post about running Debian on the OpenWrt One
router hardware:

With openwrt-one-debian, you can now install and run a full Debian
system leveraging the OpenWrt One’s NVMe storage, enabling everything
from custom services and containers to development tools and
lightweight server workloads, all on open hardware.

This project provides a rust-based flasher to install Debian on the
OpenWrt One, opening the door to standard Debian tooling, packages,
and workflows. For developers and power users, it transforms the
OpenWrt One from a network appliance into a compact, general-purpose
Linux system.

See the GitHub
repository for the code and latest build. LWN reviewed the device in
November 2024, and covered Denver
Gingerich’s talk at SCALE 22x about
the making of the router in March 2025.

From AI agent prototype to product: Lessons from building AWS DevOps Agent

2026-01-15 Efe Karakus

Post Syndicated from Efe Karakus original https://aws.amazon.com/blogs/devops/from-ai-agent-prototype-to-product-lessons-from-building-aws-devops-agent/

At re:Invent 2025, Matt Garman announced AWS DevOps Agent, a frontier agent that resolves and proactively prevents incidents, continuously improving reliability and performance. As a member of the DevOps Agent team, we’ve focused heavily on making sure that the “incident response” capability of the DevOps Agent generates useful findings and observations. In particular, we’ve been working on making root cause analysis for native AWS applications accurate and performant. Under the hood, DevOps Agent has a multi-agent architecture where a lead agent acts as an incident commander: it understands the symptom, creates an investigation plan, and delegates individual tasks to specialized sub-agents when those tasks benefit from context compression. A sub-agent executes its task with a pristine context window and reports compressed results back to the lead agent. For example, when examining high-volume log records, a sub-agent filters through the noise to surface only relevant messages to the lead agent.

In this blog post, we want to focus on the mechanisms one needs to develop to build an agentic product that works. Building a prototype with large language models (LLMs) has a low barrier to entry – you can showcase something that works fairly quickly. However, graduating that prototype into a product that performs reliably across diverse customer environments is a different challenge entirely, and one that is frequently underestimated. This post shares what we learned building AWS DevOps Agent so you can apply these lessons to your own agent development.

In our experience, there are five mechanisms necessary to continuously improve agent quality and bridge the gap from prototype to production. First, you need evaluations (evals) to identify where your agent fails and where it can improve, while establishing a quality baseline for the types of scenarios your agent handles well. Second, you need a visualization tool to debug agent trajectories and understand where exactly the agent went wrong. Third, you need a fast feedback loop with the ability to rerun those failing scenarios locally to iterate. Fourth, you need to make intentional changes: establishing success criteria before modifying your system to avoid confirmation bias. Finally, you need to read production samples regularly to understand actual customer experience and discover new scenarios your evals don’t yet cover.

Evaluations

Evals are the machine learning equivalent of a test suite in traditional software engineering. Just like building any other software product, a collection of good test cases builds confidence in quality. Iterating on agent quality is similar to test-driven development (TDD): you have an eval scenario that the agent fails (the test is red), you make changes until the agent passes (the test is green). A passing eval means the agent arrived at an accurate, useful output through correct reasoning.

For AWS DevOps Agent, the size of an individual eval scenario is similar to an end-to-end test in the traditional software engineering testing pyramid. Looking through the lens of “Given-When-Then” style tests:

Given – The test setup portion tends to be the most time-consuming to author. For the AWS DevOps Agent, an example eval scenario includes an application running on Amazon Elastic Kubernetes Service composed of several microservices fronted by Application Load Balancers, reading and writing from data stores such as Amazon Relational Database Service databases and Amazon Simple Storage Service buckets, with AWS Lambda functions doing data transformations. We inject a fault by deploying a code change that accidentally removes a key AWS Identity and Access Management (IAM) permission to write to S3 deep in the dependency chain.
When – Once the fault is injected, an alarm fires, and this triggers the AWS DevOps Agent to start its investigation. The eval framework polls the records that the Agent generates, just like how the DevOps Agent web application renders them. This section isn’t fundamentally different from defining the action in an integration or end-to-end test.
Then – This asserts and reports on multiple metrics. Fundamentally, there’s a single “PASS” (1) or “FAIL” (0) metric for quality. For the DevOps Agent’s incident response capability, a “PASS” means the right root cause surfaced to the customer – in our example, this means identifying the faulty deployment as the root cause and tracing the dependency chain to surface the impacted resources and observations that reveal the missing S3 write permission; otherwise “FAIL”. We define this as a rubric: not just “did the agent find the root cause?” but “did the agent arrive at the root cause through the correct reasoning with the right supporting evidence?”The ground truth (the “expected” or “wanted” in software testing parlance) is compared to the system response (the “actual”) via an LLM Judge – an LLM that receives both the ground truth and the agent’s actual output, then emits its reasoning and a verdict on whether they match. We use an LLM for comparison because the agent’s output is non-deterministic: the agent follows an overall output format but generates the actual text freely, so each run may use different words or sentence structures while conveying the same semantic meaning. We don’t want to strictly search for keywords in the final root cause analysis report but rather evaluate whether the essence of the rubric is met.

The evaluation report is structured with scenarios as rows and metrics as columns. Key metrics that we keep track of are capability (pass@k – whether the agent passed at least once in k attempts), reliability (pass^k – how many times the agent passed across k attempts, e.g., 0.33 means passed 1 out of 3 times for k=3), latency, and token usage.

Evaluation results table with two scenario rows. Headers: Scenario, Pass@3, Pass^3, Avg. E2E Latency, Avg. Time-To-First-Observation, and Avg. Total tokens. Lambda throttle scenario shows Pass@3 of 1 and Pass^3 of 1 (highlighted green). SQS permission removal scenario shows Pass@3 of 1 and Pass^3 of 0.33 (highlighted red), indicating it passed only 1 of 3 attempts.

Why are evals important?

There are several benefits to having evals:

Red scenarios provide obvious investigation points for the agent development team to increase product quality.
Over time, green scenarios act as regression tests, notifying us when changes to the system degrade the existing customer experience.
Once pass rates are green, we can improve customer experience along additional metrics. For example, reducing end-to-end latency and/or optimizing cost (proxied by token usage) while maintaining the quality bar.

What makes evals challenging?

Fast feedback loops help developers know whether code works (is it correct, performant, secure) and whether ideas are good (do they improve key business metrics). This may seem obvious, but far too often, teams and organizations tolerate slow feedback loops […] — Nicole Forsgren and Abi Noda, Frictionless: 7 Steps to Remove Barriers, Unlock Value, and Outpace Your Competition in the AI Era

There are several challenges with evals. In decreasing order of difficulty:

Realistic and diverse scenarios are hard to author. Coming up with realistic applications and fault scenarios is difficult. Authoring high fidelity microservice applications and faults is significant work that requires prior industry experience. What we’ve found effective: we author a few “environments” (based on real application architectures) but create many failure scenarios on top of them. The environment is the expensive portion of the evaluation setup, so we maximize reuse across multiple scenarios.
Slow feedback loops. If the “Given” takes 20 minutes to deploy for an eval scenario and then the “When” takes another 10-20 minutes for complex investigations to complete, agent developers won’t thoroughly test their changes. Instead, they’ll be satisfied with a single passing eval, then release to production, potentially introducing regressions until the comprehensive eval report is generated. Additionally, slow feedback loops encourages batching multiple changes together rather than small incremental experiments, making it harder to understand which change actually moved the needle. We’ve found three mechanisms effective for speeding up feedback loops:
1. Long-running environments for eval scenarios. The application and its healthy state are created once and kept running. Fault injection happens periodically (e.g., over weekends), and developers point their agent credentials at the faulty environment, completely skipping the “Given” portion of the test.
2. Isolated testing of only the agent surface area that matters. In our multi-agent system, developers can trigger a specific sub-agent directly with a prompt from a past eval run rather than running the entire end-to-end flow. Additionally, we built a “fork” feature: developers can initialize any agent with the conversation history from a failing run up to a specific checkpoint message, then iterate only on the remaining trajectory. Both of these approaches significantly lowers the wait time of the “When” portion.
3. Local development of the agentic system. If developers must merge changes and release to a cloud environment before testing, the loop is too slow. Running locally enables rapid iteration.

Visualize trajectories

When an agent fails an eval or a production run, where do you start investigating? The most productive method is error analysis. Visualize the agent’s complete trajectory, every user-assistant message exchange including sub-agent trajectories, and annotate each step as “PASS” or “FAIL” with notes on what went wrong. This process is tedious but effective.

For AWS DevOps Agent, agent trajectories map to OpenTelemetry traces and you can use tools like Jaeger to visualize them. Software development kits like Strands provide tracing integration with minimal setup.

Figure 1 – A sample trace from AWS DevOps Agent.

Each span contains user-assistant message pairs. We annotate each span’s quality in a table such as the following:

This low-level analysis consistently surfaces multiple improvements, not just one. For a single failing eval, one will typically identify many concrete changes spanning accuracy, performance, and cost.

Intentional changes

I had learned from my dad the importance of intentionality — knowing what it is you’re trying to do, and making sure everything you do is in service of that goal. — Will Guidara, Unreasonable Hospitality: The Remarkable Power of Giving People More Than They Expect

You’ve identified failing scenarios and diagnosed the issues through trajectory analysis. Now it’s time to modify the system.

The biggest fallacy we’ve observed at this stage: confirmation bias leading to overfitting. Given the eval challenges mentioned earlier (slow feedback loops and the impracticality of comprehensive test suites) developers typically test only the few specific failing scenarios locally until they pass. One modifies the context (system prompt, tool specifications, tool implementations, etc.) until one or two scenarios pass, without considering broader impact. When changes don’t follow context engineering best practices, they likely have negative effects that we can’t capture through limited evals.

You need both diligence and judgment: establish success criteria through available evals and reusable past production scenarios, but also educate yourself on context engineering best practices to guide your changes. We’ve found Anthropic’s prompting best practices and engineering blog, Drew Breunig’s how long contexts fail, and lessons from building Manus particularly helpful resources.

Establish success criteria first

Before making any change, define what success looks like:

Baseline. Fix specific git commit IDs for the current system. Think deliberately about which metrics would improve both the agent’s experience and the customer’s experience, then gather those metrics for the baseline.
Test scenarios. Which evals will measure your change’s impact? How many times will you rerun these evals? Convince yourself this set represents broader customer patterns, not just the one failure you’re investigating.
Comparison. Measure your changes against the baseline using the same metrics.

This intentional framing protects against confirmation bias (interpreting results favorably) and sunk cost fallacy (accepting changes simply because you invested time). If your modifications don’t move the metrics as expected, reject them.

For example, when optimizing a sub-agent within AWS DevOps Agent, we establish a baseline by fixing git commit IDs and running the same scenario seven times. This reveals both typical performance and variance.

Baseline metrics table comparing multiple runs of a sub-agent. Headers: Run, Correct observations, Irrelevant observations, Latency, Sub-agent Tokens, and Lead-agent Tokens. Run 1 shows 4 out of 6 correct observations with 1 irrelevant observation. Run 7 shows 5 out of 6 correct observations with 0 irrelevant observations, demonstrating variance across repeated runs of the same scenario.

Each metric measures a different dimension:

Correct observations – How many relevant signals (log records, metric data, code snippets, etc.) that are directly related to the incident did the sub-agent surface?
Irrelevant observations – How much noise did the sub-agent introduce to the lead agent? This counts signals that are unrelated to the incident and could distract the agent’s investigation.
Latency – How long did the sub-agent take (measured in minutes and seconds)?
Sub-agent tokens – How many tokens did the sub-agent to accomplish its task? This serves as a proxy for the cost of running the sub-agent.
Lead-agent tokens – How much of the lead agent’s context window is the sub-agent’s input and output consuming? This gives us a tangible way to identify optimization opportunities for the sub-agent tool: can we compress the instructions to the sub-agent or the results it returns?

After establishing the baseline, we compare these metrics against the same measurements with our proposed changes. This makes it clear whether the change is an actual improvement.

Read production samples

We’ve been fortunate to have several Amazon teams adopt AWS DevOps Agent early. A DevOps agent team member on rotation regularly samples real production runs using our trajectory visualization tool (similar to the OpenTelemetry-based visualization discussed earlier, but customized to render DevOps Agent-specific artifacts like root cause analysis reports and observations), marking whether the agent’s output was accurate and identifying failure points. Production samples are irreplaceable; they reveal the actual customer experience. Additionally, reviewing samples continuously refines your intuition of what the agent is good and bad at. When production runs aren’t satisfactory, you have real-world scenarios to iterate against: modify your agent locally, then rerun it against the same production environment until the desired outcome is reached. Establishing rapport with a few critical early adopter teams willing to partner in this way is invaluable. They provide ground truth for rapid iteration and create opportunities to identify new eval scenarios. This tight feedback loop with production data works in conjunction with eval-driven development to form a comprehensive test suite.

Closing thoughts

Building an agent prototype that demonstrates the feasibility of solving a real business problem is an exciting first step. The harder work is graduating that prototype into a product that performs reliably across diverse customer environments and tasks. In this post, we’ve shared five mechanisms that form the foundation for systematically improving agent quality: evals with realistic and diverse scenarios, fast feedback loops, trajectory visualization, intentional changes, and production sampling.

If you’re building an agentic application, start building your eval suite today. Even starting with a handful critical scenarios will establish the quality baseline needed to measure and improve systematically. To see how AWS DevOps Agent applies these principles to incident response, check out our getting started guide.

Unlock granular resource control with queue-based QMR in Amazon Redshift Serverless

2026-01-15 Srini Ponnada

Post Syndicated from Srini Ponnada original https://aws.amazon.com/blogs/big-data/unlock-granular-resource-control-with-queue-based-qmr-in-amazon-redshift-serverless/

Amazon Redshift Serverless removes infrastructure management and manual scaling requirements from data warehousing operations. Amazon Redshift Serverless queue-based query resource management, helps you protect critical workloads and control costs by isolating queries into dedicated queues with automated rules that prevent runaway queries from impacting other users. You can create dedicated query queues with customized monitoring rules for different workloads, providing granular control over resource usage. Queues let you define metrics-based predicates and automated responses, such as automatically aborting queries that exceed time limits or consume excessive resources.

Different analytical workloads have distinct requirements. Marketing dashboards need consistent, fast response times. Data science workloads might run complex, resource-intensive queries. Extract, transform, and load (ETL) processes might execute lengthy transformations during off-hours.

As organizations scale analytics usage across more users, teams, and workloads, ensuring consistent performance and cost control becomes increasingly challenging in a shared environment. A single poorly optimized query can consume disproportionate resources, degrading performance for business-critical dashboards, ETL jobs, and executive reporting. With Amazon Redshift Serverless queue-based Query Monitoring Rules (QMR), administrators can define workload-aware thresholds and automated actions at the queue level—a significant improvement over previous workgroup-level monitoring. You can create dedicated queues for distinct workloads such as BI reporting, ad hoc analysis, or data engineering, then apply queue-specific rules to automatically abort, log, or restrict queries that exceed execution-time or resource-consumption limits. By isolating workloads and enforcing targeted controls, this approach protects mission-critical queries, improves performance predictability, and prevents resource monopolization—all while maintaining the flexibility of a serverless experience.

In this post, we discuss how you can implement your workloads with query queues in Redshift Serverless.

Queue-based vs. workgroup-level monitoring

Before query queues, Redshift Serverless offered query monitoring rules (QMRs) only at the workgroup level. This meant the queries, regardless of purpose or user, were subject to the same monitoring rules.

Queue-based monitoring represents a significant advancement:

Granular control – You can create dedicated queues for different workload types
Role-based assignment – You can direct queries to specific queues based on user roles and query groups
Independent operation – Each queue maintains its own monitoring rules

Solution overview

In the following sections, we examine how a typical organization might implement query queues in Redshift Serverless.

Architecture Components

Workgroup Configuration

The foundational unit where query queues are defined
Contains the queue definitions, user role mappings, and monitoring rules

Queue Structure

Multiple independent queues operating within a single workgroup
Each queue has its own resource allocation parameters and monitoring rules

User/Role Mapping

Directs queries to appropriate queues based on:
User roles (e.g., analyst, etl_role, admin)
Query groups (e.g., reporting, group_etl_inbound)
Query group wildcards for flexible matching

Query Monitoring Rules (QMRs)

Define thresholds for metrics like execution time and resource usage
Specify automated actions (abort, log) when thresholds are exceeded

Prerequisites

To implement query queues in Amazon Redshift Serverless, you need to have the following prerequisites:

Redshift Serverless environment:

Active Amazon Redshift Serverless workgroup
Associated namespace

Access requirements:

AWS Management Console access with Redshift Serverless permissions
AWS CLI access (optional for command-line implementation)
Administrative database credentials for your workgroup

Required permissions:

IAM permissions for Redshift Serverless operations (CreateWorkgroup, UpdateWorkgroup)
Ability to create and manage database users and roles

Identify workload types

Begin by categorizing your workloads. Common patterns include:

Interactive analytics – Dashboards and reports requiring fast response times
Data science – Complex, resource-intensive exploratory analysis
ETL/ELT – Batch processing with longer runtimes
Administrative – Maintenance operations requiring special privileges

Define queue configuration

For each workload type, define appropriate parameters and rules. For a practical example, let’s assume we want to implement three queues:

Dashboard queue – Used by analyst and viewer user roles, with a strict runtime limit set to stop queries longer than 60 seconds
ETL queue – Used by etl_role user roles, with a limit of 100,000 blocks on disk spilling (query_temp_blocks_to_disk) to control resource usage during data processing operations
Admin queue – Used by admin user roles, without a query monitoring limit enforced

To implement this using the AWS Management Console, complete the following steps:

On the Redshift Serverless console, go to your workgroup.
On the Limits tab, under Query queues, choose Enable queues.
Configure each queue with appropriate parameters, as shown in the following screenshot.

Each queue (dashboard, ETL, admin_queue) is mapped to specific user roles and query groups, creating clear boundaries between query rules. The query monitoring rules implement automated resource governance—for example, the dashboard queue automatically stops queries exceeding 60 seconds (short_timeout) while allowing ETL processes longer runtimes with different thresholds. This configuration helps prevent resource monopolization by establishing separate processing lanes with appropriate guardrails, so critical business processes can maintain necessary computational resources while limiting the impact of resource-intensive operations.

Alternatively, you can implement the solution using the AWS Command Line Interface (AWS CLI).

In the following example, we create a new workgroup named test-workgroup within an existing namespace called test-namespace. This makes it possible to create queues and establish associated monitoring rules for each queue using the following command:

aws redshift-serverless create-workgroup \
  --workgroup-name test-workgroup \
  --namespace-name test-namespace \
  --config-parameters '[{"parameterKey": "wlm_json_configuration", "parameterValue": "[{\"name\":\"dashboard\",\"user_role\":[\"analyst\",\"viewer\"],\"query_group\":[\"reporting\"],\"query_group_wild_card\":1,\"rules\":[{\"rule_name\":\"short_timeout\",\"predicate\":[{\"metric_name\":\"query_execution_time\",\"operator\":\">\",\"value\":60}],\"action\":\"abort\"}]},{\"name\":\"ETL\",\"user_role\":[\"etl_role\"],\"query_group\":[\"group_etl_inbound\",\"group_etl_outbound\"],\"rules\":[{\"rule_name\":\"long_timeout\",\"predicate\":[{\"metric_name\":\"query_execution_time\",\"operator\":\">\",\"value\":3600}],\"action\":\"log\"},{\"rule_name\":\"memory_limit\",\"predicate\":[{\"metric_name\":\"query_temp_blocks_to_disk\",\"operator\":\">\",\"value\":100000}],\"action\":\"abort\"}]},{\"name\":\"admin_queue\",\"user_role\":[\"admin\"],\"query_group\":[\"admin\"]}]"}]'

You can also modify an existing workgroup using update-workgroup using the following command:

aws redshift-serverless update-workgroup \
  --workgroup-name test-workgroup \
  --config-parameters '[{"parameterKey": "wlm_json_configuration", "parameterValue": "[{\"name\":\"dashboard\",\"user_role\":[\"analyst\",\"viewer\"],\"query_group\":[\"reporting\"],\"query_group_wild_card\":1,\"rules\":[{\"rule_name\":\"short_timeout\",\"predicate\":[{\"metric_name\":\"query_execution_time\",\"operator\":\">\",\"value\":60}],\"action\":\"abort\"}]},{\"name\":\"ETL\",\"user_role\":[\"etl_role\"],\"query_group\":[\"group_etl_load\",\"group_etl_replication\"],\"rules\":[{\"rule_name\":\"long_timeout\",\"predicate\":[{\"metric_name\":\"query_execution_time\",\"operator\":\">\",\"value\":3600}],\"action\":\"log\"},{\"rule_name\":\"memory_limit\",\"predicate\":[{\"metric_name\":\"query_temp_blocks_to_disk\",\"operator\":\">\",\"value\":100000}],\"action\":\"abort\"}]},{\"name\":\"admin_queue\",\"user_role\":[\"admin\"],\"query_group\":[\"admin\"]}]"}]'

Best practices for queue management

Consider the following best practices:

Start simple – Begin with a minimal set of queues and rules
Align with business priorities – Configure queues to reflect critical business processes
Monitor and adjust – Regularly review queue performance and adjust thresholds
Test before production – Validate query metrics behavior in a test environment before applying to production

Clean up

To clean up your resources, delete the Amazon Redshift Serverless workgroups and namespaces. For instructions, see Deleting a workgroup.

Conclusion

Query queues in Amazon Redshift Serverless bridge the gap between serverless simplicity and fine-grained workload control by enabling queue-specific Query Monitoring Rules tailored to different analytical workloads. By isolating workloads and enforcing targeted resource thresholds, you can protect business-critical queries, improve performance predictability, and limit runaway queries, helping minimize unexpected resource consumption and better control costs, while still benefiting from the automatic scaling and operational simplicity of Redshift Serverless.

Get started with Amazon Redshift Serverless today.

About the authors

Establishing finops management: Integrating AWS Budgets with WhatsApp using AWS End User Messaging

2026-01-15 Ruchikka Chaudhary

Post Syndicated from Ruchikka Chaudhary original https://aws.amazon.com/blogs/messaging-and-targeting/establishing-finops-management-integrating-aws-budgets-with-whatsapp-using-aws-end-user-messaging/

Managing cloud costs effectively is a critical concern for organizations of all sizes. While AWS Budgets provides powerful tools to set spending thresholds and receive notifications, these alerts traditionally arrive through email or through AWS Management Console notifications. These traditional notification methods face several challenges when managing cloud costs:

Email notifications might not be seen immediately
Important budget alerts can get lost in crowded inboxes
Team members might not have immediate access to their email or the console
Global teams need accessible alerting mechanisms that work across time zones

Today, we’re sharing a solution that brings AWS Budgets alerts directly to your WhatsApp using AWS End User Messaging—enabling real-time cost awareness and faster response to budget thresholds wherever you are.

Overview of solution

Our solution integrates AWS Budgets with WhatsApp messaging using AWS End User Messaging, AWS Lambda, and Amazon Simple Notification Service (Amazon SNS). When a budget threshold is crossed, the alert is processed and delivered as a formatted WhatsApp message to designated recipients.

The architecture, shown in the following figure, consists of four main AWS services to deliver budget alerts. AWS Budgets tracks expenses against your defined thresholds. When expenses exceed these thresholds, Amazon SNS receives an alert. An AWS Lambda function processes this alert and sends it through AWS End User Messaging to WhatsApp. Users then receive actionable budget notifications directly on their WhatsApp.

Billing and Cost Management data, which AWS Budgets uses to monitor resources, is updated at least once per day. Keep in mind that budget information and associated alerts are updated and sent according to this data refresh cadence. In a budget period,

notifications are triggered every time the notification state goes from OK to Exceeded (when the threshold is exceeded). If the budget stays in Exceeded state in the same budget period, AWS Budgets doesn’t send an additional alert.

Prerequisites

Implementation requires an AWS account with appropriate permissions for AWS CloudFormation, Lambda, Amazon SNS, and AWS Budgets. You must also have a WhatsApp Business Account integrated with AWS End User Messaging Social and the WhatsApp phone number ID from the AWS End User Messaging console. For instructions to locate this information, see View a phone number’s ID in AWS End User Messaging Social.

For more information about how to set up WhatsApp using AWS End User Messaging Social, see Automate workflows with WhatsApp using AWS End User Messaging Social.

Before you deploy this solution, create an approved utility template in your Meta account named aws_budgets_notification_template(as shown in the following screenshot). Alternatively, use your preferred template name and modify the Lambda function code accordingly.

The preceding figure shows variable samples that can be used while creating a message template. You can also use the following AWS Command Line Interface (AWS CLI) command to create the messaging template-

aws socialmessaging create-whatsapp-message-template \
  --region <region> \
  --id <waba-id> \
  --template-definition "$(echo '{
    "name": "aws_budgets_notification_template",
    "language": "en",
    "category": "UTILITY",
    "parameter_format": "named",
    "components": [
      {
        "type": "HEADER",
        "format": "TEXT",
        "text": "{{emoji}} Budget Alert",
        "example": {
          "header_text_named_params": [
            {"param_name": "emoji", "example": "💰"}
          ]
        }
      },
      {
        "type": "BODY",
        "text": "Subject: {{subject}}\\nDetails: {{notification_title}}\\nAWS Account {{account_info}}\\n\\n{{notification_msg}}\\n\\nBudget Name: {{budget_name}}\\nBudget Type: {{budget_type}}\\nBudgeted Amount: {{budgeted_amount}}\\nAlert Type: {{alert_type}}\\nAlert Threshold: {{alert_threshold}}\\nFORECASTED Amount: {{forecasted_amount}}\\n\\nAWS Console: {{console_link}}\\n\\nTime: {{time}}\\n\\nTip: Check your AWS Billing Dashboard for detailed cost breakdown",
        "example": {
          "body_text_named_params": [
            {"param_name": "subject", "example": "AWS Budgets: Budget-Cloudwatch-budget-dev has exceeded your alert threshold"},
            {"param_name": "notification_title", "example": "AWS Budget Notification Oct 19, 2025"},
            {"param_name": "account_info", "example": "AWS Account 12345"},
            {"param_name": "notification_msg", "example": "You requested that we alert you when the FORECASTED Cost associated with your Budget-Cloudwatch-dev Budget is greater"},
            {"param_name": "budget_name", "example": "Budget-Cloudwatch-budget-dev"},
            {"param_name": "budget_type", "example": "Cost"},
            {"param_name": "budgeted_amount", "example": "$1"},
            {"param_name": "alert_type", "example": "Cost"},
            {"param_name": "alert_threshold", "example": "> 1"},
            {"param_name": "forecasted_amount", "example": "$2"},
            {"param_name": "console_link", "example": "https://console.aws.amazon.com/billing/home#/budgets"},
            {"param_name": "time", "example": "2025-10-19 12:38:52 UTC"}
          ]
        }
      }
    ]
  }' | base64)"

You can confirm the template approval status and type in the Meta portal.

Solution walkthrough

The core component consists of a Python-based Lambda function that processes Budget alerts and formats them for WhatsApp delivery. The function receives SNS events containing budget alerts data, extracts relevant information, formats contextual messages, and delivers notifications through AWS End User Messaging Social.The following function shows an example to parse the SNS message content:

 def process_notification(record):
"""Process SNS notification and send WhatsApp message"""
sns_message = record['Sns']
subject = sns_message.get('Subject', 'AWS Notification')
message_body = sns_message.get('Message', '')

logger.info(f"Processing notification - Subject: {subject}")
process_budget_notification(subject, message_body)

The following example demonstrates how to format alert information—including subject, details, and timestamp—and deliver the message to WhatsApp.

#Create template message with parsed values
template_name = "aws_budgets_notification_template"
template_message = {
    "name": template_name,
    "language": {
        "code": "en"
    },
    "components": [
        {
            "type": "header",
            "parameters": [{
                "type": "text",
                "parameter_name": "emoji",
                "text": "💰"
            }]
        },
        {
            "type": "body",
            "parameters": [
                {
                    "type": "text",
                    "parameter_name": "subject",
                    "text": subject
                },
                {
                    "type": "text",
                    "parameter_name": "notification_title",
                    "text": notification_title
                },
                {
                    "type": "text",
                    "parameter_name": "account_info",
                    "text": f"AWS Account {account_number}"
                },
                {
                    "type": "text",
                    "parameter_name": "notification_msg",
                    "text": notification_msg + "."
                },
                {
                    "type": "text",
                    "parameter_name": "budget_name",
                    "text": budget_details.get('Budget Name', '')
                },
                {
                    "type": "text",
                    "parameter_name": "budget_type",
                    "text": budget_details.get('Budget Type', '')
                },
                {
                    "type": "text",
                    "parameter_name": "budgeted_amount",
                    "text": budget_details.get('Budgeted Amount', '')
                },
                {
                    "type": "text",
                    "parameter_name": "alert_type",
                    "text": budget_details.get('Alert Type', '')
                },
                {
                    "type": "text",
                    "parameter_name": "alert_threshold",
                    "text": budget_details.get('Alert Threshold', '')
                },
                {
                    "type": "text",
                    "parameter_name": "forecasted_amount",
                    "text": budget_details.get('FORECASTED Amount', '')
                },
                {
                    "type": "text",
                    "parameter_name": "console_link",
                    "text": "https://console.aws.amazon.com/billing/home#/budgets"
                },
                {
                    "type": "text",
                    "parameter_name": "time",
                    "text": datetime.now().strftime('%Y-%m-%d %H:%M:%S UTC')
                }
            ]
        }
    ]
}
send_whatsapp_message(template_message)

The send_whatsapp_message function uses AWS End User Messaging Social to deliver formatted messages through the socialmessaging client, as shown in the following example:

def send_whatsapp_message(message):
  client = boto3.client('socialmessaging')
  # Get environment variables
  phone_number_id = os.environ.get('WHATSAPP_PHONE_NUMBER_ID')
  recipient = os.environ.get('ALERT_RECIPIENT')
                    
# Prepare message object
  message_object = {
"messaging_product": "whatsapp",
"recipient_type": "individual",
"to": recipient,
"type": "template",
"template": message
}
  # Send message
response = client.send_whatsapp_message(
originationPhoneNumberId=phone_number_id,
metaApiVersion="v20.0",
message=bytes(json.dumps(message_object), "utf-8")  
)

Deploying the solution

The solution uses AWS CloudFormation for infrastructure as code (IaC) deployment. The main template creates an SNS topic for alert notifications, a Lambda function for message processing, and required AWS Identity and Access Management (IAM) roles with least-privilege permissions.

The CloudFormation template requires a recipient number with an active WhatsApp account to receive alert notifications as messages. The template also requires the WhatsApp phone number ID retrieved from the AWS End User Messaging Social console, as noted in the prerequisites. The template must be deployed in the same AWS Region as AWS End User Messaging Social. See the following code:

aws cloudformation deploy \
  --template-file <> \
  --stack-name budget-eum-whatsapp-alerts \
  --parameter-overrides \
    ActualSpendThreshold= \
    AlertRecipient=<+1234567890> \
    BudgetAmount=<Budget Amount e.g. 3000> \
    Environment= \
    ForecastedSpendThreshold= \
    WhatsAppPhoneNumberId= \
  --capabilities CAPABILITY_NAMED_IAM \
  --region <EUM-region>

Testing the solution

The solution can be tested and validated using the SNS topic created by the CloudFormation stack. Use the following AWS CLI command to publish a test message to the SNS topic:

aws sns publish \
  --topic-arn arn:aws:sns:<region>:<account>:<topic-name> \
  --subject "[Test] AWS Budget Alert: Budget-Alert-EUM has exceeded 80% of your budgeted amount" \
  --message "AWS Budget Notification - Your budget has exceeded the alert threshold
AWS Account 123456789012
Budget Name: Budget-Alert-EUM
Budget Type: Cost
Budgeted Amount: $100.00
Alert Type: ACTUAL
Alert Threshold: 80%
FORECASTED Amount: $85.00
You have exceeded 80% of your budget for this period" \
  --region <region>

The SNS message triggers the Lambda function, which sends the alert to your configured WhatsApp recipient, as shown in the following screenshot.

Clean up

Use the following steps to clean up your resources when you no longer need this solution:

Delete the CloudFormation stack deployed in this solution: budget-eum-whatsapp-alerts.
Delete the template in your Meta account.

Conclusion

Integrating AWS Budgets alerts with WhatsApp notifications represents a significant step forward in modern cost monitoring. By using AWS End User Messaging Social, you can send teams critical alerts through their preferred communication channel while maintaining the reliability and scalability of AWS services. The solution’s modular architecture, basic yet effective security model, and cost-effective design make it suitable for organizations of various sizes.

About the authors

Адски сме толерантни!

2026-01-15 Светла Енчева

Post Syndicated from Светла Енчева original https://www.toest.bg/adski-sme-tolerantni/

Адски сме толерантни!

Социолозите, които се занимават с изследване на общественото мнение, знаят (или поне би трябвало да знаят), че понякога респондентите дават т.нар. социално желателни отговори. Тоест казват това, което смятат, че е престижно или че се очаква от тях. Това нерядко води до грешки в предизборните проучвания и в екзитполовете – ако хората мислят да гласуват или тъкмо са пуснали бюлетина за партия, за която смятат, че не е престижно да дадат гласа си, е доста вероятно да поизлъжат. Понякога проучванията включват контролни въпроси, чиято цел е да се установи дали респондентите отговарят искрено, или по-скоро се опитват да се харесат. При кратки анкети, каквито са екзитполовете например, това не е подходящ вариант.

Колко толерантно е населението на България към ЛГБТИ хората?

На този въпрос търси отговор национално представително изследване, проведено от „Алфа Рисърч“ сред 1000 респонденти по поръчка на Фондация GLAS. На директното питане дали са съгласни с твърдението „Аз съм толерантен към ЛГБТИ хората“, близо половината (49,7%) отговарят утвърдително. Не се възприемат като толерантни малко над една трета – 35,6%, а 14,7% се затрудняват да преценят.

Аз съм толерантен към ЛГБТИ хората

49,7%

35,6%

14,7%

Чувствам се комфортно при вида на еднополова двойка, която се държи за ръце

30,5%

59,2%

10,3%

Не би ме притеснило, ако знаех, че преподавател в училището на детето ми е ЛГБТИ

27,8%

54,2%

18,0%

Ограничаването на правата на ЛГБТ хората води до изтичане на квалифицирани млади кадри в чужбина

21,7%

54,1%

24,2%

Бих позволил(а) на детето ми да си играе в дома на свой приятел, чиито родители са еднополова двойка

27,4%

52,7%

19,9%

Еднополовите двойки трябва да имат правото да легализират партньорските си отношения чрез фактическо съжителство

30,5%

47,9%

21,5%

Транссексуалните хора трябва да могат да променят своите документи, така че да съответстват на тяхната полова идентичност

26,5%

46,3%

27,2%

Бих се чувствал(а) комфортно, ако работя с ЛГБТИ човек

35,0%

44,8%

20,2%

Публичното подбуждане към омраза във връзка с ЛГБТИ хората трябва да се наказва като престъпление

34,3%

40,7%

25,0%

Отношението ми към близък човек (дете, брат, сестра, роднина) не би се променило, ако разбера, че той/тя/те са ЛГБТИ

47,0%

32,7%

20,3%

Равният достъп на ЛГБТ хората до социални, здравни и образователни услуги е важен за общественото благосъстояние

56,6%

21,4%

22,0%

Поправките в ЗПУО по-скоро изостриха конфронтацията между хората в обществото или по-скоро допринесоха за подобряване на средата и живота на хората в България?

45,1%

15,6%

39,4%

Средни стойности на толерантност и нетолерантност

34,8%

42,7%

22,5%

Някои от оригиналните въпроси са леко преформулирани с цел отговорите да се подредят в единна скала и с цел съкращаване.
Източник: Alpha Research, GLAS Foundation.

Такива общи твърдения са особено благодатни за контролни въпроси – хората могат да разбират под „толерантност“ много различни неща. Изследването предлага на респондентите да се позиционират спрямо повече от 10 твърдения, за да се получи по-ясна представа за действителната им толерантност към ЛГБТИ хората. Очаквано, резултатите доста се разминават с декларираната толерантност.

Можете ли да предположите какво кара близо 60% (59,2%) от населението да се чувства некомфортно?

Двама души от един и същи пол, които… се държат за ръце.

По тази тема анкетираните показват най-високо неодобрение, а тези, които не са сигурни, са най-малко – едва 10,3% (при средно за контролните въпроси 22,5%). Тук следва да се отбележи, че все пак не всички са посочили с еднаква категоричност, че се чувстват некомфортно – крайно некомфортно им е на 30,9%, докато 28,3% изпитват умерен дискомфорт. В проучването липсва въпрос за целуване, но ако от едното държане на ръце хората реагират така, може би при проява на по-голяма еднополова интимност някои от най-негативно реагиращите биха имали нужда от медицинска помощ.

На второ място по степен на неприемане се подреждат ЛГБТИ учителите – 54,2% биха се притеснявали, ако детето им има такъв учител. Почти толкова (54,1%) не са съгласни с твърдението, че ограничаването на правата на ЛГБТ хората води до изтичане на квалифицирани млади кадри в чужбина. Макар че 44,8% биха се чувствали некомфортно да работят с „такива хора“.

Независимо какво смята мнозинството от населението в България обаче, много ЛГБТ хора напускат страната тъкмо по тази причина. Един от тях – Борислав Герасимов – преди няколко години направи за несъществуващия вече сайт out.bg поредицата „Немили-недраги“ – 12 интервюта с ЛГБТ емигранти и 7 лични разказа, от които става ясно как дискриминацията прогонва млади хора. Пред „Тоест“ той пое ангажимента да ги публикува отново онлайн.

52,7% пък не биха пуснали детето си да си играе в дома на приятелче, чиито родители са от един и същи пол.

Близо половината (47,9%) не са съгласни, че фактическото съжителство на еднополовите двойки трябва да се легализира, срещу малко над една четвърт, които смятат, че трябва. За сравнение – 37,5% биха искали правно признаване на съжителството на хетеросексуални двойки, а почти толкова (36,7%) са против. Същевременно 51,7% смятат, че имуществените и неимуществените отношения на хората в съжителство, които не са сключили брак, трябва да бъдат регулирани от закона както на двойките в брак. Но изглежда, за много хора това не включва легализиране на съжителството, още по-малко пък на еднополовите двойки.

След кампанията срещу Истанбулската конвенция, в която взеха дейно участие Конституционният съд и Върховният касационен съд, 46,3% не са съгласни транссексуалните да могат да сменят юридическия си пол. А според 40,7% публичното подбуждане към омраза към ЛГБТИ хората не трябва да се наказва като престъпление (спрямо 34,3%, според които трябва, и една четвърт, които нямат мнение).

Едва една трета от хората в България казват, че биха променили отношението си към близък човек, ако научат, че е ЛГБТИ.

Като имаме предвид обаче, че близо 60% се скандализират при вида на хора от един и същи пол, държащи се за ръце, можем да предположим, че често пъти запазването на доброто отношение към близкия ЛГБТИ човек си има цена – да не „парадира“, да не му личи, да не научат хората, че какво ще си кажат… Има и родители и близки, които реагират в стил „не ме интересува, не искам да знам, това си е твоя работа“, смятайки, че по този начин не са променили отношението си към човека.

Цели 56% смятат, че равният достъп на ЛГБТ хората до социални, здравни и образователни услуги е важен за общественото благосъстояние, срещу 21,4%, които не мислят така. Едва 15,6% смятат, че законодателната промяна, забраняваща т.нар. ЛГБТ пропаганда в училище, е довела до подобрение, а според 45,1% тя е изострила конфронтацията между хората. Близо 40% обаче не са сигурни, което е най-високият дял на нямащи мнение за цялото проучване.

Като теглим чертата, средното равнище на толерантност при отговорите на контролните въпроси е 34,8%, а на нетолерантност – 42,7% – картина, доста по-различна от декларираната.

Добрите новини

Макар данните от проучването да изглеждат обезсърчаващи за отношението на българското население към ЛГБТИ хората, съвсем не всичко е толкова черно. „Алфа Рисърч“ провежда това изследване за трета поредна година, което позволява да се забележи известно положително развитие.

Например подкрепата за легализиране на съжителството и регулиране на имуществените и неимуществените отношения между партньори от един пол, макар и недостатъчна, нараства с около 5 процентни пункта, а в сравнение с 2023 г. – с почти една трета.

Така че държавата има все по-малко основание да не изпълнява решенията на европейски съдилища,

според които трябва да се намери форма на правно признаване на еднополовите бракове, сключени в чужбина.

Изследването отчита също, че за младите е далеч по-лесно да общуват, учат и работят с ЛГБТИ хора, отколкото за по-възрастните. Все пак голяма част от поколението Z не помни времето, когато е нямало „София прайд“, и за него видимостта на тази група от населението е част от пейзажа.

Също така очевидно пропагандата си има граници. Промяната в Закона за предучилищното и училищното образование не консолидира обществото срещу ЛГБТИ хората, а по-скоро му показа, че както би казал Хамлет, „има нещо гнило в Дания“. Макар близо 40% да се затрудняват да оценят поправките в закона, по-малко от 16% ги одобряват.

Най-добрата новина всъщност е, че толерантността продължава да е ценност.

В противен случай хората не биха се изкарвали по-толерантни, отколкото са всъщност. А като се има предвид дългогодишната пропаганда против „толерастията“, както и в какво се превръща светът в последните години, никак не се разбира от само себе си, че толерантността все още се възприема като нещо добро.

Тук немалко ЛГБТИ хора и експерти в областта на човешките права биха казали, че толерантността не е достатъчна дори когато е действителна. Защото тя означава търпимост и нищо повече. Истинското приемане отива отвъд толерантността – едно е просто да търпиш някого, друго е да го смяташ за толкова ценна част от обществото, колкото си и ти.

Но и едната толерантност е нещо в среда като българската и в свят, в който емпатията все повече се презира, а грубата сила е на все по-голяма почит. Въпреки че нашенската толерантност, както става ясно от изследването, прилича повече на пътя към ада, който, казват, бил постлан с много добри намерения.

Forgejo 14.0 released

2026-01-15 jzb

Post Syndicated from jzb original https://lwn.net/Articles/1054461/

Version
14.0 of the Forgejo software forge has been released. Notable
changes in this release include several database
improvements, new options for approving
actions execution from pull requests, a new
file editor, and progress toward making
Forgejo’s web UI work without JavaScript.

Lawn & Pool Robots Got LIDAR at CES 2026!

2026-01-15 The Hook Up

Post Syndicated from The Hook Up original https://www.youtube.com/watch?v=kPanB3ApITI

[$] Removing a pointer dereference from slab allocations

2026-01-15 corbet

Post Syndicated from corbet original https://lwn.net/Articles/1053870/

Al Viro does not often stray outside of the core virtual filesystem area;
when he does, it is usually worthy of note. Recently, he wandered into
memory management with this patch
series to the slab allocator and some of its users. Kernel developers
will often put considerable effort into small optimizations, but it is
still interesting to look at just how much effort has gone toward the purpose of
avoiding a single pointer dereference in some memory-allocation hot paths.

A note for MXroute users

2026-01-15 jzb

Post Syndicated from jzb original https://lwn.net/Articles/1054410/

We have recently noticed that email from LWN.net seems to be
blocked by MXroute. Unfortunately, the company also does not seem to
have a way for non-customers to report problems in mail delivery, so
we have no good way to get ourselves unblocked.

As a result, readers who have subscribed to an LWN mailing list
from a domain hosted with MXroute will probably not receive our
mailings. We have not yet unsubscribed addresses that are being
blocked by MXroute, but will soon if the problem persists. Please
accept our apologies for the inconvenience; it is unfortunate that it
is becoming so difficult to send legitimate email as a small
business.

SiFive To Adopt NVLInk Fusion For Future Data Center RISC-V CPU Designs

2026-01-15 Ryan Smith

Post Syndicated from Ryan Smith original https://www.servethehome.com/sifive-to-adopt-nvlink-fusion-for-future-data-center-risc-v-cpu-designs/

For a while now, NVIDIA has been working to turn its NVLink high speed interconnect technology into more than just a bus used by NVIDIA, aiming to fashion an entire ecosystem around it. Those efforts kicked into high gear last year with the announcement of NVLInk Fusion, a program to allow third parties to connect […]

The post SiFive To Adopt NVLInk Fusion For Future Data Center RISC-V CPU Designs appeared first on ServeTheHome.

Security updates for Thursday

2026-01-15 jzb

Post Syndicated from jzb original https://lwn.net/Articles/1054408/

Security updates have been issued by Debian (chromium, gnupg2, and mongo-c-driver), Fedora (firefox, gpsd, linux-firmware, and seamonkey), Mageia (net-snmp), Oracle (kernel, podman, postgresql16, postgresql:13, postgresql:15, postgresql:16, and uek-kernel), Red Hat (libpq, net-snmp, and transfig), Slackware (libpng and mozilla), SUSE (avahi, bluez, capstone, curl, dpdk, firefox, firefox-esr, fluidsynth, glib2, kernel, kernel-devel, libmicrohttpd, libpcap, libpng16, libsoup, libsoup-3_0-0, libtasn1, libvirt, mcphost, openvswitch, ovmf, podman, poppler, python-tornado6, python311, qemu, rsync, and valkey), and Ubuntu (erlang, klibc, libpng1.6, and ruby-rack).

Human Native is joining Cloudflare

2026-01-15 Will Allen

Post Syndicated from Will Allen original https://blog.cloudflare.com/human-native-joins-cloudflare/

Today, we’re excited to share that Cloudflare has acquired Human Native, a UK-based AI data marketplace specializing in transforming multimedia content into searchable and useful data.

Human Native x Cloudflare

The Human Native team has spent the past few years focused on helping AI developers create better AI through licensed data. Their technology helps publishers and developers turn messy, unstructured content into something that can be understood, licensed and ultimately valued. They have approached data not as something to be scraped, but as an asset class that deserves structure, transparency and respect.

Access to high-quality data can lead to better technical performance. One of Human Native’s customers, a prominent UK video AI company, threw away their existing training data after achieving superior results with data sourced through Human Native. Going forward they are only training on fully licensed, reputably sourced, high-quality content.

This gives a preview of what the economic model of the Internet can be in the age of generative AI: better AI built on better data, with fair control, compensation and credit for creators.

The Internet needs new economic models

For the last 30 years, the open Internet has been based on a fundamental value exchange: creators create content, aggregators (such as search engines or social media) send traffic. Creators can monetize that traffic through advertisements, subscriptions or direct support. This is the economic loop that has powered the explosive growth of the Internet.

But it’s under real strain.

Crawl-to-referral ratios are skyrocketing, with 10s of thousands of AI and bot crawls per real human visitor, and it’s unclear how multipurpose crawlers are using the content they access.

The community of creators who publish on the Internet is a diverse group: news publishers, content creators, financial professionals, technology companies, aggregators and more. But they have one thing in common: They want to decide how their content is used by AI systems.

Cloudflare’s work in building AI Crawl Control and Pay Per Crawl is predicated on a simple philosophy: Content owners should get to decide how and when their content is accessed by others. Many of our customers want to optimize their brand and content to make sure it is in every training data set and shows up in every new search; others want to have more control and only allow access if there is direct compensation.

Our tools like AI Search, AI Crawl Control and Pay Per Crawl can help, wherever you land in that equation. The important thing is that the content owner gets to decide.

New tools for AI developers

With the Human Native team joining Cloudflare, we are accelerating our work in helping customers transform their content to be easily accessed and understood by AI bots and agents in addition to their traditional human audiences.

Crawling is complex, expensive in terms of engineering and compute to process the content, and has no guarantees of quality control. A crawled index can contain duplicates, spam, illegal material and many more headaches. Developers are left with messy, unstructured data.

We recently announced our work in building the AI Index, a powerful new way for both foundation model companies and agents to access content at scale.

Instead of sending crawlers blindly and repeatedly across the open Internet, AI developers will be able to connect via a pub/sub model: participating websites will expose structured updates whenever their content changes, and developers will be able to subscribe to receive those updates in real time.

This opens up new avenues for content creators to experiment with new business models.

Building the foundation for these new business models

Cloudflare is investing heavily in creating the foundations for these new business models, starting with x402.

We recently announced that we are creating the x402 Foundation, in partnership with Coinbase, to enable machine-to-machine transactions for digital resources.

Payments on the web have historically been designed for humans. We browse a merchant’s website, show intent by adding items to a cart, and confirm our intent to purchase by putting in our credit card information and clicking “Pay.” But what if you want to enable direct transactions between automated systems? We need protocols to allow machine-to-machine transactions.

Together, Human Native and Cloudflare will accelerate our work in building the basis of these new economic models for the Internet.

What’s next

The Internet works best when it is open, fair, and independently sustainable. We’re excited to welcome the Human Native team to Cloudflare, and even more excited about what we will build together to improve the foundations of the Internet in the age of AI.

Onwards.

New Vulnerability in n8n

2026-01-15 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/01/new-vulnerability-in-n8n.html

This isn’t good:

We discovered a critical vulnerability (CVE-2026-21858, CVSS 10.0) in n8n that enables attackers to take over locally deployed instances, impacting an estimated 100,000 servers globally. No official workarounds are available for this vulnerability. Users should upgrade to version 1.121.0 or later to remediate the vulnerability.

Three technical links and two news links.

Will ICE Get Away With This?

2026-01-15 The Atlantic

Post Syndicated from The Atlantic original https://www.youtube.com/watch?v=KCFbXjqq0U4

Opening the AWS European Sovereign Cloud

2026-01-15 Sébastien Stormacq

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/opening-the-aws-european-sovereign-cloud/

Deutsch | English | Español | Français | Italiano

As a European citizen, I understand first-hand the importance of digital sovereignty, especially for our public sector organisations and highly regulated industries. Today, I’m delighted to share that the AWS European Sovereign Cloud is now generally available to all customers. We first announced our plans to build this new independent cloud infrastructure in 2023, and today it’s ready to meet the most stringent sovereignty requirements of European customers with a comprehensive set of AWS services.

Berlin, Brandenburg Gate at sunset

Meeting European sovereignty requirements
Organizations across Europe face increasingly complex regulatory requirements around data residency, operational control, and governance independence. Too often today, European organisations with the highest sovereignty requirements are stuck in legacy on-premises environments or offerings with reduced services and functionality. In response to this critical need, the AWS European Sovereign Cloud is the only fully featured and independently operated sovereign cloud backed by strong technical controls, sovereign assurances, and legal protections. Public sector entities and businesses in highly regulated industries need cloud infrastructure that provides enhanced sovereignty controls that maintain the innovation, security, and reliability they expect from modern cloud services. These organisations require assurance that their data and operations remain under European jurisdiction, with clear governance structures and operational autonomy within the European Union (EU).

A new independent cloud infrastructure for Europe
The AWS European Sovereign Cloud represents a physically and logically separate cloud infrastructure, with all components located entirely within the EU. The first AWS Region in the AWS European Sovereign Cloud is located in the state of Brandenburg, Germany, and is generally available today. This Region operates independently from existing AWS Regions. The infrastructure features multiple Availability Zones with redundant power and networking, designed to operate continuously even if connectivity with the rest of the world is interrupted.

We plan to extend the AWS European Sovereign Cloud footprint from Germany across the EU to support stringent isolation, in-country data residency, and low latency requirements. This will start with new sovereign Local Zones located in Belgium, the Netherlands, and Portugal. In addition, you will be able to extend the AWS European Sovereign Cloud infrastructure with AWS Dedicated Local Zones, AWS AI Factories, or AWS Outposts in locations you select, including your own on-premises data centres.

The AWS European Sovereign Cloud and its Local Zones provide enhanced sovereign controls through its unique operational model. The AWS European Sovereign Cloud will be operated exclusively by EU residents located in the EU. This covers activities such as day-to-day operations, technical support, and customer service. We’re gradually transitioning the AWS European Sovereign Cloud to be operated exclusively by EU citizens located in the EU. During this transition period, we will continue to work with a blended team of EU residents and EU citizens located in the EU.

The infrastructure is managed through dedicated European legal entities established under German law. In October 2025, AWS appointed Stéphane Israël, an EU citizen residing in the EU, as managing director. Stéphane will be responsible for the management and operations of the AWS European Sovereign Cloud, including infrastructure, technology, and services, as well as leading AWS broader digital sovereignty efforts. In January 2026, AWS also appointed Stefan Hoechbauer (Vice President, Germany and Central Europe, AWS) as a managing director of the AWS European Sovereign Cloud. He will work alongside Stéphane Israel to lead the AWS European Sovereign Cloud.

An advisory board comprised exclusively of EU citizens, and including two independent third-party representatives, provides additional oversight and expertise on sovereignty matters.

Enhanced data residency and control
The AWS European Sovereign Cloud provides comprehensive data residency assurances so you can meet the most stringent data residency requirements. As with our existing AWS Regions around the world, all your content remains within the Region you select unless you choose otherwise. Beyond content, customer-created metadata including roles, permissions, resource labels, and configurations also stays within the EU. The infrastructure features its own dedicated AWS Identity and Access Management (IAM) and billing system, all operating independently within European borders.

Technical controls built into the infrastructure prevent access to the AWS European Sovereign Cloud from outside the EU. The infrastructure includes a dedicated European trust service provider for certificate authority operations and uses dedicated Amazon Route 53 name servers. These servers will only use European Top-Level Domains (TLDs) for their own names. The AWS European Sovereign Cloud has no critical dependencies on non-EU personnel or infrastructure.

Security and compliance framework
The AWS European Sovereign Cloud maintains the same core security capabilities you expect from AWS, including encryption, key management, access governance, and the AWS Nitro System for compute isolation. This means your EC2 instances benefit from cryptographically verified platform integrity and hardware-enforced boundaries that prevent unauthorized access to your data without compromising on performance, giving you both the sovereignty controls and the computational power your workloads require. The infrastructure undergoes independent third-party audits, with compliance programs including ISO/IEC 27001:2013, SOC 1/2/3 reports, and Federal Office for Information Security (BSI) C5 attestation.

The AWS European Sovereign Cloud: Sovereign Reference Framework defines the specific sovereignty controls across governance independence, operational control, data residency, and technical isolation. This framework is available in AWS Artifact and provides end-to-end visibility through SOC 2 attestation.

Comprehensive service availability
You can access a broad range of AWS services in the AWS European Sovereign Cloud from launch, including Amazon SageMaker and Amazon Bedrock for artificial intelligence and machine learning (AI/ML) workloads. For compute, you can use Amazon Elastic Compute Cloud (Amazon EC2) and AWS Lambda. Container orchestration is available through Amazon Elastic Kubernetes Service (Amazon EKS) and Amazon Elastic Container Service (Amazon ECS). Database services include Amazon Aurora, Amazon DynamoDB, and Amazon Relational Database Service (Amazon RDS). Storage options include Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Block Store (Amazon EBS), with networking through Amazon Virtual Private Cloud (Amazon VPC) and security services including AWS Key Management Service (AWS KMS) and AWS Private Certificate Authority. For an up-to-date list of services, refer to the AWS Capabilities matrix recently published on the AWS Builder Center.

The AWS European Sovereign Cloud is supported by AWS Partners who are committed to helping you meet your sovereignty requirements. Partners including Adobe, Cisco, Cloudera, Dedalus, Esri, Genesys, GitLab, Mendix, Pega, SAP, SnowFlake, Trend Micro, and Wiz are making their solutions available in the AWS European Sovereign Cloud, providing you with the tools and services you need across security, data analytics, application development, and industry-specific workloads. This broad partner support helps you build sovereign solutions that combine AWS services with trusted partner technologies.

Significant investment in European infrastructure
The AWS European Sovereign Cloud is backed by a €7.8 billion investment in infrastructure, jobs creation, and skills development. This investment is expected to contribute €17.2 billion to the European economy through 2040 and support roughly 2,800 full-time equivalent jobs annually in local businesses.

Some technical details
The AWS European Sovereign Cloud is available to all customers, regardless of where they are located. You can access the infrastructure using the partition name aws-eusc and the Region name eusc-de-east-1. A partition is a group of AWS Regions. Each AWS account is scoped to one partition.

The infrastructure supports all standard AWS access methods including the AWS Management Console, AWS SDKs, and the AWS Command Line Interface (AWS CLI), making it straightforward to integrate into your existing workflows and automation. After having created a new root account for the AWS European Sovereign Cloud partition, you start by creating new IAM identities and roles specific to this infrastructure, giving you complete control over access management within the European sovereign environment.

Getting started
The AWS European Sovereign Cloud provides European organisations with enhanced sovereignty controls whilst maintaining access to AWS innovation and capabilities. You can contract for services through Amazon Web Services EMEA SARL, with pricing in EUR and billing in any of the eight currencies we support today. The infrastructure uses familiar AWS architecture, service portfolio, and APIs, making it straightforward to build and migrate applications.

The AWS European Sovereign Cloud addendum contains the additional contractual commitments for the AWS European Sovereign Cloud.

For me as a European, this launch represents the AWS commitment to meeting the specific needs of our continent and providing the cloud capabilities that drive innovation across industries. I invite you to find out more about the AWS European Sovereign Cloud and how it can help your organisation meet its sovereignty requirements. Read Overview of the AWS European Sovereign Cloud to learn more about the design goals and approach, sign up for a new account, and plan for the deployment of your first workload today.

— seb

German version

Start der AWS European Sovereign Cloud

Als Bürger Europas weiß ich aus eigener Erfahrung, wie wichtig digitale Souveränität ist, insbesondere für unsere öffentlichen Einrichtungen und stark regulierten Branchen. Ich freue mich, Ihnen heute mitteilen zu können, dass die AWS European Sovereign Cloud nun für alle Kunden allgemein verfügbar ist. Wir haben unsere Pläne zum Aufbau dieser neuen unabhängigen Cloud-Infrastruktur erstmals im Jahr 2023 vorgestellt. Heute ist diese Infrastruktur bereit, mit einem umfassenden Angebot an AWS-Services die strengsten Souveränitätsanforderungen europäischer Kunden zu erfüllen.

Berlin, Brandenburger Tor bei Sonnenuntergang

Erfüllung europäischer Souveränitätsanforderungen
Organisationen in ganz Europa sehen sich mit zunehmend komplexen regulatorischen Anforderungen in Bezug auf Datenresidenz, operative Kontrolle und Unabhängigkeit der Governance konfrontiert. Europäische Organisationen mit höchsten Souveränitätsanforderungen sind heutzutage allzu oft in veralteten lokalen Umgebungen oder Angeboten mit eingeschränkten Services und Funktionen gefangen. Die AWS European Sovereign Cloud ist die Antwort auf diesen dringenden Bedarf. Sie ist die einzige unabhängig betriebene souveräne Cloud mit vollem Funktionsumfang, die durch strenge technische Kontrollen, Souveränitätszusicherungen und rechtlichen Schutz abgesichert ist. Einrichtungen des öffentlichen Sektors und Unternehmen in stark regulierten Branchen benötigen eine Cloud-Infrastruktur, die erweiterte Souveränitätskontrollen bietet und gleichzeitig die von modernen Cloud-Services erwartete Innovation, Sicherheit und Zuverlässigkeit gewährleistet. Diese Organisationen benötigen die Zusicherung, dass ihre Daten und Aktivitäten unter europäischer Zuständigkeit bleiben, mit klaren Governance-Strukturen und operativer Autonomie innerhalb der Europäischen Union (EU).

Eine neue unabhängige Cloud-Infrastruktur für Europa
Die AWS European Sovereign Cloud ist eine physisch und logisch getrennte Cloud-Infrastruktur, deren Komponenten sich vollständig innerhalb der EU befinden. Die erste AWS-Region in der AWS European Sovereign Cloud befindet sich im deutschen Bundesland Brandenburg und ist ab heute allgemein verfügbar. Diese Region arbeitet unabhängig von bestehenden AWS-Regionen. Die Infrastruktur umfasst mehrere Availability Zones mit redundanter Stromversorgung und Netzwerkverbindung, die auch bei einer Unterbrechung der Verbindung zum Rest der Welt einen kontinuierlichen Betrieb gewährleisten.

Wir beabsichtigen, die Präsenz der AWS European Sovereign Cloud von Deutschland aus EU-weit auszuweiten, um strenge Anforderungen hinsichtlich Isolierung, Datenresidenz innerhalb einzelner Länder und geringer Latenz zu erfüllen. Dies beginnt mit neuen souveränen Local Zones in Belgien, den Niederlanden und Portugal. Darüber hinaus können Sie die Infrastruktur der AWS European Sovereign Cloud mit dedizierten AWS Local Zones, AWS AI Factories oder AWS Outposts an Standorten Ihrer Wahl, einschließlich Ihrer eigenen lokalen Rechenzentren, erweitern.

Dank ihres einzigartigen Betriebsmodells bieten die AWS European Sovereign Cloud und ihre Local Zones erweiterte Souveränitätskontrollen. Der Betrieb der AWS European Sovereign Cloud wird ausschließlich von EU-Bürgern mit Wohnsitz in der EU sichergestellt. Dies umfasst Aktivitäten wie den täglichen Betrieb, den technischen Support und den Kundenservice. Wir stellen die AWS European Sovereign Cloud schrittweise so um, dass als Betriebspersonal ausschließlich EU-Bürger mit Wohnsitz in der EU zum Einsatz kommen. Während dieser Übergangsphase werden wir weiterhin mit einem gemischten Team aus in der EU ansässigen Personen und in der EU lebenden EU-Bürgern arbeiten.

Die Infrastruktur wird durch spezielle europäische juristische Personen nach deutschem Recht verwaltet. Im Oktober 2025 berief AWS Stéphane Israël, einen in der EU ansässigen EU-Bürger, zum Geschäftsführer. Stéphane wird für das Management und den Betrieb der AWS European Sovereign Cloud verantwortlich zeichnen. Dies umfasst die Bereiche Infrastruktur, Technologie und Services sowie die Federführung bei den breit angelegten Initiativen von AWS auf dem Gebiet der digitalen Souveränität. Im Januar 2026 ernannte AWS zudem Stefan Hoechbauer (Vice President, Germany and Central Europe, AWS) zum Geschäftsführer der AWS European Sovereign Cloud. Er wird gemeinsam mit Stéphane Israel die Leitung der AWS European Sovereign Cloud innehaben.

Ein Beirat, dem ausschließlich EU-Bürger, einschließlich zwei unabhängigen externen Vertretern, angehören, fungiert als zusätzliche Kontrollinstanz und bringt Fachwissen in Fragen der Souveränität ein.

Verbesserte Datenresidenz und -kontrolle
Die AWS European Sovereign Cloud bietet umfassende Garantien hinsichtlich der Datenresidenz, sodass Sie selbst die strengsten Anforderungen in diesem Bereich erfüllen können. Wie auch bei unseren bestehenden AWS-Regionen weltweit verbleiben alle Ihre Inhalte in der von Ihnen ausgewählten Region, sofern Sie keine anderen Einstellungen vornehmen. Neben den Inhalten verbleiben auch die vom Kunden erstellten Metadaten, einschließlich Rollen, Berechtigungen, Ressourcenbezeichnungen und Konfigurationen, innerhalb der EU. Die Infrastruktur verfügt über ein eigenes AWS Identity and Access Management (IAM) und ein eigenes Abrechnungssystem – beides wird innerhalb der europäischen Grenzen unabhängig betrieben.

In die Infrastruktur integrierte technische Kontrollen verhindern den Zugriff auf die AWS European Sovereign Cloud von außerhalb der EU. Die Infrastruktur umfasst einen dedizierten europäischen Trust Service Provider für Zertifizierungsstellen und nutzt dedizierte Amazon-Route-53-Namenserver. Diese Server verwenden ausschließlich europäische Top-Level-Domains (TLDs) für ihre eigenen Namen. Die AWS European Sovereign Cloud unterliegt keinen kritischen Abhängigkeiten hinsichtlich Personal oder Infrastruktur außerhalb der EU.

Sicherheits- und Compliance-Framework
Die AWS European Sovereign Cloud bietet dieselben zentralen Sicherheitsfunktionen, die Sie von AWS erwarten. Dazu gehören Verschlüsselung, Schlüsselverwaltung, Zugriffskontrolle und das AWS Nitro System für die Isolierung von Rechenressourcen. Dies bedeutet, dass Ihre EC2-Instanzen von einer kryptografisch verifizierten Plattformintegrität und hardwaregestützten Grenzen profitieren, die unbefugten Zugriff auf Ihre Daten verhindern, ohne die Leistung zu beeinträchtigen. So erhalten Sie sowohl die Souveränitätskontrollen als auch die Rechenleistung, die Ihre Workloads erfordern. Die Infrastruktur wird unabhängigen Audits durch Dritte unterzogen. Die Compliance-Programme umfassen ISO/IEC 27001:2013, SOC-1/2/3-Berichte und das C5-Zertifikat des Bundesamtes für Sicherheit in der Informationstechnik (BSI).

Das AWS European Sovereign Cloud: Sovereign Reference Framework definiert spezifische Souveränitätskontrollen in den Bereichen Governance-Unabhängigkeit, operative Kontrolle, Datenresidenz und technische Isolierung. Dieses Framework ist in AWS Artifact verfügbar und bietet durch SOC-2-Zertifizierung durchgängige Transparenz.

Umfassende Serviceverfügbarkeit
Von Beginn an steht Ihnen in der AWS European Sovereign Cloud eine breite Palette von AWS-Services zur Verfügung – darunter Amazon SageMaker und Amazon Bedrock für Workloads im Bereich Künstliche Intelligenz und Machine Learning (KI/ML). Für die Rechenleistung stehen Ihnen Amazon Elastic Compute Cloud (Amazon EC2) und AWS Lambda zur Verfügung. Die Container-Orchestrierung ist über den Amazon Elastic Kubernetes Service (Amazon EKS) und den Amazon Elastic Container Service (Amazon ECS) verfügbar. Zu den Datenbank-Services gehören Amazon Aurora, Amazon DynamoDB und Amazon Relational Database Service (Amazon RDS). Die Speicheroptionen umfassen Amazon Simple Storage Service (Amazon S3) und Amazon Elastic Block Store (Amazon EBS) mit Vernetzung über Amazon Virtual Private Cloud (Amazon VPC) und Sicherheits-Services wie AWS Key Management Service (AWS KMS) und AWS Private Certificate Authority. Eine aktuelle Liste der Services finden Sie in der AWS-Funktionsmatrix, die kürzlich im AWS Builder Center veröffentlicht wurde.

Die AWS European Sovereign Cloud wird von AWS-Partnern unterstützt, die es sich zur Aufgabe gemacht haben, Ihnen bei der Erfüllung Ihrer Souveränitätsanforderungen zur Seite zu stehen. Partner wie Adobe, Cisco, Cloudera, Dedalus, Esri, Genesys, GitLab, Mendix, Pega, SAP, SnowFlake, Trend Micro und Wiz stellen ihre Lösungen in der AWS European Sovereign Cloud zur Verfügung und bieten Ihnen die Tools und Services, die Sie in den Bereichen Sicherheit, Datenanalyse, Entwicklung von Anwendungen und branchenspezifische Workloads benötigen. Dank dieser umfassenden Unterstützung durch Partner können Sie eigenständige Lösungen, die AWS-Services mit bewährten Partnertechnologien kombinieren, entwickeln.

Erhebliche Investitionen in die europäische Infrastruktur
Hinter der AWS European Sovereign Cloud steht eine Investition in Höhe von 7,8 Milliarden Euro in Infrastruktur, die Schaffung von Arbeitsplätzen und die Entwicklung von Kompetenzen. Diese Investition wird bis 2040 voraussichtlich 17,2 Milliarden Euro zur europäischen Wirtschaftsleistung beitragen und jährlich rund 2 800 Vollzeitstellen in lokalen Unternehmen sichern.

Einige technische Details
Die AWS European Sovereign Cloud steht allen Kunden zur Verfügung, unabhängig davon, wo sie sich befinden. Sie können mit dem Partitionsnamen aws-eusc und dem Regionsnamen eusc-de-east-1 auf die Infrastruktur zugreifen. Eine Partition ist eine Gruppe von AWS-Regionen. Jedes AWS-Konto ist auf eine Partition beschränkt.

Die Infrastruktur unterstützt alle gängigen AWS-Zugriffsmethoden, einschließlich der AWS Management Console, AWS SDKs und der AWS Command Line Interface (AWS CLI), sodass sie sich problemlos in Ihre bestehenden Workflows und Automatisierungsprozesse integrieren lässt. Nachdem Sie ein neues Root-Konto für die Partition der AWS European Sovereign Cloud erstellt haben, beginnen Sie mit der Erstellung neuer IAM-Identitäten und -Rollen, die speziell für diese Infrastruktur vorgesehen sind. Dadurch erhalten Sie die vollständige Kontrolle über die Zugriffsverwaltung innerhalb der europäischen souveränen Umgebung.

Erste Schritte
Die AWS European Sovereign Cloud bietet europäischen Organisationen erweiterte Souveränitätskontrollen und gewährleistet gleichzeitig den Zugriff auf die Innovationen und Funktionen von AWS. Sie können Services über Amazon Web Services EMEA SARL in Auftrag geben. Die Preise werden in Euro angegeben und die Abrechnung erfolgt in einer der acht Währungen, die wir derzeit unterstützen. Die Infrastruktur basiert auf der bekannten AWS-Architektur, dem AWS-Serviceportfolio und den AWS-APIs, wodurch die Entwicklung und Migration von Anwendungen vereinfacht wird.

Der AWS European Sovereign Cloud Addendum enthält die zusätzlichen vertraglichen Verpflichtungen für die AWS European Sovereign Cloud.

Für mich als Europäer symbolisiert dieser Launch das Engagement von AWS, den spezifischen Anforderungen unseres Kontinents gerecht zu werden und Cloud-Funktionen bereitzustellen, die Innovationen in allen Branchen vorantreiben. Ich lade Sie ein, mehr über die AWS European Sovereign Cloud zu erfahren und zu entdecken, wie sie Ihrer Organisation dabei helfen kann, ihre Souveränitätsanforderungen zu erfüllen. Lesen Sie die Übersicht über die AWS European Sovereign Cloud und erfahren Sie mehr über die Designziele und den Ansatz. Registrieren Sie sich für ein neues Konto und planen Sie noch heute die Bereitstellung Ihrer ersten Workload.

— seb

French version

Ouverture de l’AWS European Souvereign Cloud

En tant que citoyen européen, je mesure personnellement l’importance de la souveraineté numérique, en particulier pour nos organisations du secteur public et les industries fortement réglementées. Aujourd’hui, j’ai le plaisir d’annoncer que l’AWS European Sovereign Cloud est désormais disponible pour l’ensemble de nos clients. Nous avions annoncé pour la première fois notre projet de construction de cette nouvelle infrastructure cloud indépendante en 2023, et elle est aujourd’hui prête à répondre aux exigences de souveraineté les plus strictes des clients européens, avec un large ensemble de services AWS.

Berlin, porte de Brandebourg au coucher du soleil

Répondre aux exigences européennes en matière de souveraineté
Partout en Europe, les organisations sont confrontées à des exigences réglementaires de plus en plus complexes en matière de résidence des données, de contrôle opérationnel et d’indépendance de la gouvernance. Trop souvent aujourd’hui, les organisations européennes ayant les besoins de souveraineté les plus élevés se retrouvent contraintes de rester sur des environnements sur site, ou de recourir à des offres cloud aux services et fonctionnalités limités. Pour répondre à cet enjeu critique, l’AWS European Sovereign Cloud est le seul cloud souverain entièrement fonctionnel, exploité de manière indépendante, et reposant sur des contrôles techniques robustes, des garanties de souveraineté et des protections juridiques solides. Les acteurs du secteur public et les entreprises des secteurs fortement réglementés ont besoin d’une infrastructure cloud offrant des contrôles de souveraineté renforcés, sans renoncer à l’innovation, à la sécurité et à la fiabilité attendues des services cloud modernes. Ces organisations doivent avoir l’assurance que leurs données et leurs opérations restent sous juridiction européenne, avec des structures de gouvernance claires et une autonomie opérationnelle au sein de l’Union européenne (UE).

Une nouvelle infrastructure cloud indépendante pour l’Europe
L’AWS European Sovereign Cloud repose sur une infrastructure cloud physiquement et logiquement distincte, dont l’ensemble des composants est situé exclusivement au sein de l’UE. La première région AWS de l’AWS European Sovereign Cloud est implantée dans le Land de Brandebourg, en Allemagne, et est disponible dès aujourd’hui. Cette région fonctionne de façon indépendante par rapport aux autres régions AWS existantes. L’infrastructure comprend plusieurs zones de disponibilité, avec des systèmes redondants d’alimentation et de réseau, conçus pour fonctionner en continu même en cas d’interruption de la connectivité avec le reste du monde.

Nous prévoyons d’étendre l’empreinte de l’AWS European Sovereign Cloud depuis l’Allemagne à l’ensemble de l’UE, afin de répondre aux exigences strictes d’isolation, de résidence des données dans certains pays et de faible latence. Cette extension débutera avec de nouvelles Local Zones souveraines situées en Belgique, aux Pays-Bas et au Portugal. En complément, vous pourrez étendre l’infrastructure de l’AWS European Sovereign Cloud à l’aide des AWS Dedicated Local Zones, des AWS AI Factories ou d’AWS Outposts, dans les sites de votre choix, y compris au sein de vos propres centres de données sur site.

L’AWS European Sovereign Cloud et ses Local Zones offrent des contrôles de souveraineté renforcés grâce à un modèle opérationnel unique. L’AWS European Sovereign Cloud est exploité exclusivement par des résidents de l’UE basés dans l’UE. Cela couvre notamment les opérations quotidiennes, le support technique et le service client. Nous sommes en train d’opérer une transition progressive afin que l’AWS European Sovereign Cloud soit exploité exclusivement par des citoyens de l’UE résidant dans l’UE. Durant cette période de transition, nous continuons de travailler avec une équipe mixte composée de résidents de l’UE et de citoyens de l’UE basés dans l’UE.

L’infrastructure est gérée par des entités juridiques européennes dédiées, établies conformément au droit allemand. En octobre 2025, AWS a nommé Stéphane Israël, citoyen de l’UE résidant dans l’UE, au poste de directeur général. Stéphane est responsable de la gestion et de l’exploitation de l’AWS European Sovereign Cloud, couvrant l’infrastructure, la technologie et les services, ainsi que de la direction des initiatives plus larges d’AWS en matière de souveraineté numérique. En janvier 2026, AWS a également nommé Stefan Hoechbauer (Vice-président, Allemagne et Europe centrale, AWS) comme directeur général de l’AWS European Sovereign Cloud. Il travaillera aux côtés de Stéphane Israël pour piloter l’AWS European Sovereign Cloud.

Un conseil consultatif, composé exclusivement de citoyens de l’UE et incluant deux représentants indépendants, apporte un niveau supplémentaire de supervision et d’expertise sur les questions de souveraineté.

Résidence des données et contrôle renforcés
L’AWS European Sovereign Cloud fournit des garanties complètes en matière de résidence des données afin de répondre aux exigences les plus strictes. Comme dans les régions AWS existantes à travers le monde, l’ensemble de vos contenus reste dans la région que vous sélectionnez, sauf indication contraire de votre part. Au-delà des contenus, les métadonnées créées par les clients — telles que les rôles, les autorisations, les étiquettes de ressources et les configurations — restent également au sein de l’UE. L’infrastructure dispose de son propre système dédié de AWS Identity and Access Management (IAM) et de facturation, opérant de manière totalement indépendante à l’intérieur des frontières européennes.

Des contrôles techniques intégrés à l’infrastructure empêchent tout accès à l’AWS European Sovereign Cloud depuis l’extérieur de l’UE. L’infrastructure comprend un prestataire européen de services de confiance dédié pour les opérations d’autorité de certification et utilise des serveurs de noms Amazon Route 53 dédiés. Ces serveurs n’utilisent que des domaines de premier niveau (TLD) européens pour leurs propres noms. L’AWS European Sovereign Cloud ne présente aucune dépendance critique vis-à-vis de personnels ou d’infrastructures situés en dehors de l’UE.

Cadre de sécurité et de conformité
L’AWS European Sovereign Cloud conserve les capacités de sécurité fondamentales attendues d’AWS, notamment le chiffrement, la gestion des clés, la gouvernance des accès et le système AWS Nitro pour l’isolation des charges de calcul. Concrètement, vos instances EC2 bénéficient d’une intégrité de plateforme vérifiée cryptographiquement et de frontières matérielles, empêchant tout accès non autorisé à vos données sans compromis sur les performances. Vous bénéficiez ainsi à la fois des contrôles de souveraineté et de la puissance de calcul nécessaires à vos charges de travail. L’infrastructure fait l’objet d’audits indépendants réalisés par des tiers, et s’inscrit dans des programmes de conformité incluant ISO/IEC 27001:2013, les rapports SOC 1/2/3, ainsi que l’attestation C5 de l’Office fédéral allemand pour la sécurité de l’information (BSI).

Le AWS European Sovereign Cloud: Sovereign Reference Framework définit précisément les contrôles de souveraineté couvrant l’indépendance de la gouvernance, le contrôle opérationnel, la résidence des données et l’isolation technique. Ce cadre est disponible dans AWS Artifact et offre une visibilité de bout en bout via une attestation SOC 2.

Disponibilité étendue des services
Dès son lancement, l’AWS European Sovereign Cloud donne accès à un large éventail de services AWS, notamment Amazon SageMaker et Amazon Bedrock pour les charges de travail d’intelligence artificielle et de machine learning (IA/ML). Pour le calcul, vous pouvez utiliser Amazon Elastic Compute Cloud (Amazon EC2) et AWS Lambda. L’orchestration de conteneurs est disponible via Amazon Elastic Kubernetes Service (Amazon EKS) et Amazon Elastic Container Service (Amazon ECS). Les services de bases de données incluent Amazon Aurora, Amazon DynamoDB et Amazon Relational Database Service (Amazon RDS). Les options de stockage comprennent Amazon Simple Storage Service (Amazon S3) et Amazon Elastic Block Store (Amazon EBS), avec des capacités réseau via Amazon Virtual Private Cloud (Amazon VPC) et des services de sécurité tels que AWS Key Management Service (AWS KMS) et AWS Private Certificate Authority. Pour obtenir la liste la plus à jour des services disponibles, consultez la matrice des capacités AWS récemment publiée sur l’AWS Builder Center.

L’AWS European Sovereign Cloud est supporté par de nombreux partenaires AWS engagés à vous aider à répondre à vos exigences de souveraineté. Des partenaires tels qu’Adobe, Cisco, Cloudera, Dedalus, Esri, Genesys, GitLab, Mendix, Pega, SAP, SnowFlake, Trend Micro et Wiz rendent leurs solutions disponibles sur l’AWS European Sovereign Cloud, vous offrant ainsi les outils et services nécessaires dans les domaines de la sécurité, de l’analyse de données, du développement applicatif et des charges de travail spécifiques à certains secteurs industriels. Cet ensemble de partenaires vous permet de construire des solutions souveraines combinant les services AWS et des technologies de partenaires de confiance.

Un investissement majeur dans l’infrastructure européenne
L’AWS European Sovereign Cloud s’appuie sur un investissement de 7,8 milliards d’euros dans l’infrastructure, la création d’emplois et le développement des compétences. Cet investissement devrait contribuer à hauteur de 17,2 milliards d’euros à l’économie européenne d’ici 2040 et soutenir environ 2.800 emplois équivalent temps plein par an au sein des entreprises locales.

Quelques détails techniques
L’AWS European Sovereign Cloud est accessible à tous les clients, quel que soit leur lieu d’implantation. Vous pouvez accéder à l’infrastructure en utilisant le nom de partition aws-eusc et le nom de région eusc-de-east-1. Une partition correspond à un ensemble de régions AWS. Chaque compte AWS est rattaché à une seule partition.

L’infrastructure prend en charge toutes les méthodes d’accès AWS standards, y compris la console de gestion AWS, les AWS SDKs et la AWS Command Line Interface (AWS CLI), ce qui facilite son intégration dans vos flux de travail et vos automatisations existants. Après avoir créé un nouveau compte racine pour la partition AWS European Sovereign Cloud, vous commencez par définir de nouvelles identités et rôles IAM spécifiques à cette infrastructure, vous donnant un contrôle total sur la gestion des accès au sein de l’environnement souverain européen.

Pour commencer
L’AWS European Sovereign Cloud offre aux organisations européennes des contrôles de souveraineté renforcés tout en leur permettant de continuer à bénéficier de l’innovation et des capacités d’AWS. Vous pouvez contractualiser les services via Amazon Web Services EMEA SARL, avec une tarification en euros et une facturation possible dans l’une des huit devises que nous prenons en charge aujourd’hui. L’infrastructure repose sur une architecture, un portefeuille de services et des API AWS familiers, ce qui simplifie le développement et la migration des applications.

L’avenant AWS European Sovereign Cloud précise les engagements contractuels supplémentaires propres à l’AWS European Sovereign Cloud.

En tant qu’Européen, ce lancement illustre l’engagement d’AWS à répondre aux besoins spécifiques de notre continent et à fournir les capacités cloud qui stimulent l’innovation dans tous les secteurs. Je vous invite à découvrir l’AWS European Sovereign Cloud et à comprendre comment il peut aider votre organisation à satisfaire ses exigences de souveraineté. Consultez Overview of the AWS European Sovereign Cloud (en anglais) pour en savoir plus sur les objectifs de conception et l’approche retenue, créez un nouveau compte et planifiez dès aujourd’hui le déploiement de votre première charge de travail.

— seb

Italian version

Lancio di AWS European Sovereign Cloud

Come cittadino europeo, conosco benissimo l’importanza della sovranità digitale, in particolare per le nostre organizzazioni del settore pubblico e dei settori altamente regolamentati. Oggi sono lieto di annunciare che AWS European Sovereign Cloud è ora generalmente disponibile per tutti i clienti. Abbiamo annunciato per la prima volta i nostri piani per la creazione di questa nuova infrastruttura cloud indipendente nel 2023. Finalmente oggi è pronta a soddisfare i più rigorosi requisiti di sovranità dei clienti europei con un set completo di servizi AWS.

Berlino, Porta di Brandeburgo al tramonto

Soddisfare i requisiti di sovranità europea
Le organizzazioni di tutta Europa devono far fronte a requisiti normativi sempre più complessi in materia di residenza dei dati, controllo operativo e indipendenza della governance. Troppo spesso, le organizzazioni europee con i più elevati requisiti di sovranità sono bloccate a causa di offerte o ambienti on-premises legacy con funzionalità e servizi ridotti. In risposta a questa esigenza fondamentale, l’AWS European Sovereign Cloud rappresenta l’unico cloud sovrano con funzionalità complete e a gestione autonoma. supportato da solidi controlli tecnici, garanzie di sovranità e protezioni legali. Gli enti pubblici e le aziende di settori altamente regolamentati necessitano di un’infrastruttura cloud che fornisca controlli di sovranità avanzati che garantiscano l’innovazione, la sicurezza e l’affidabilità che si aspettano dai moderni servizi cloud. Queste organizzazioni devono essere certe che i propri dati e le proprie operazioni restino sotto la giurisdizione europea, con chiare strutture di governance e autonomia operativa nell’ambito dell’Unione europea (UE).

Una nuova infrastruttura cloud indipendente per l’Europa
L’AWS European Sovereign Cloud rappresenta un’infrastruttura cloud separata fisicamente e logicamente, con tutti i componenti situati interamente all’interno dell’UE. La prima Regione AWS nell’AWS European Sovereign Cloud, situata nello stato di Brandeburgo (Germania) e ora disponibile a livello generale, opera indipendentemente dalle Regioni AWS esistenti. L’infrastruttura presenta diverse zone di disponibilità con risorse di alimentazione e rete ridondanti, progettate per funzionare continuamente anche in caso di interruzione della connettività con il resto del mondo.

Abbiamo intenzione di estendere la presenza dell’AWS European Sovereign Cloud dalla Germania all’intera UE per supportare i rigorosi requisiti di isolamento, residenza dei dati all’interno di un determinato Paese e bassa latenza. Inizieremo con nuove zone locali sovrane situate in Belgio, nei Paesi Bassi e in Portogallo. Inoltre, sarà possibile estendere l’infrastruttura AWS European Sovereign Cloud con Zone locali AWS dedicate, AWS AI Factories o AWS Outposts in posizioni selezionate, inclusi i data center on-premises.

L’AWS European Sovereign Cloud e le relative zone locali forniscono controlli sovrani avanzati tramite un modello operativo esclusivo. L’AWS European Sovereign Cloud sarà gestito esclusivamente da residenti UE che si trovano nell’UE. Ciò copre attività come operazioni quotidiane, supporto tecnico e servizio clienti. Stiamo gradualmente trasformando l’AWS European Sovereign Cloud in modo che sia gestito esclusivamente da cittadini UE che si trovano nell’UE. Durante questo periodo di transizione, continueremo a lavorare con un team misto di residenti e cittadini comunitari che si trovano nell’UE.

L’infrastruttura è gestita da entità giuridiche europee dedicate, costituite secondo il diritto tedesco. Nell’ottobre 2025, AWS ha assegnato l’incarico di managing director a Stéphane Israël, cittadino comunitario residente nell’UE. Sarà responsabile della gestione e delle operazioni dell’AWS European Sovereign Cloud, inclusi infrastruttura, tecnologia e servizi, oltre a guidare le più ampie iniziative di sovranità digitale di AWS. Nel gennaio 2026, AWS ha inoltre nominato managing director dell’AWS European Sovereign Cloud Stefan Höchbauer (vicepresidente, Germania ed Europa centrale, AWS). Collaborerà con Stéphane Israël per guidare l’AWS European Sovereign Cloud.

Un comitato consultivo composto esclusivamente da cittadini comunitari, inclusi due rappresentanti terzi indipendenti, fornirà ulteriore supervisione e competenza in materia di sovranità.

Ottimizzazione del controllo e della residenza dei dati
L’AWS European Sovereign Cloud offre garanzie complete sulla residenza dei dati in modo da poter soddisfare i requisiti più rigorosi in materia. Come per le nostre Regioni AWS esistenti a livello mondiale, tutti i contenuti restano all’interno della Regione selezionata, a meno che non si scelga diversamente. Oltre ai contenuti, anche i metadati creati dai clienti, tra cui ruoli, autorizzazioni, etichette delle risorse e configurazioni, restano nell’ambito dell’UE. L’infrastruttura è dotata di un proprio sistema AWS Identity and Access Management (AWS IAM) e di fatturazione dedicato, completamente gestito in modo indipendente all’interno dei confini europei.

I controlli tecnici integrati nell’infrastruttura impediscono l’accesso all’AWS European Sovereign Cloud dall’esterno dell’UE. L’infrastruttura include un provider di servizi fiduciari europeo dedicato per le operazioni delle autorità di certificazione e utilizza nameserver Amazon Route 53 dedicati. Questi server utilizzeranno solo domini di primo livello (TLD) europei per i propri nomi. L’AWS European Sovereign Cloud non ha dipendenze fondamentali da personale o infrastrutture non UE.

Framework di sicurezza e conformità
L’AWS European Sovereign Cloud mantiene le stesse funzionalità di sicurezza di base di AWS, tra cui crittografia, gestione delle chiavi, governance degli accessi e AWS Nitro System per l’isolamento computazionale. Ciò significa che le istanze EC2 beneficiano dell’integrità della piattaforma verificata a livello di crittografia e dei limiti imposti dall’hardware che impediscono l’accesso non autorizzato ai dati senza compromettere le prestazioni, offrendo i controlli di sovranità e la potenza di calcolo richiesti dai carichi di lavoro. L’infrastruttura è sottoposta ad audit di terze parti indipendenti, con programmi di conformità che includono ISO/IEC 27001:2013, report SOC 1/2/3 e attestazione C5 dell’Ufficio Federale per la Sicurezza Informatica (BSI).

L’AWS European Sovereign Cloud: Sovereign Reference Framework definisce i controlli di sovranità specifici in termini di indipendenza della governance, controllo operativo, residenza dei dati e isolamento tecnico. Questo framework è disponibile in AWS Artifact e fornisce visibilità end-to-end tramite l’attestazione SOC 2.

Disponibilità completa del servizio
Dal momento del lancio, sarà possibile accedere a un’ampia gamma di servizi AWS nell’AWS European Sovereign Cloud, tra cui Amazon SageMaker e Amazon Bedrock per carichi di lavoro di intelligenza artificiale e machine learning (IA/ML). Per i calcoli, è possibile utilizzare Amazon Elastic Compute Cloud (Amazon EC2) e AWS Lambda. L’orchestrazione di container è disponibile tramite Amazon Elastic Kubernetes Service (Amazon EKS) e Amazon Elastic Container Service (Amazon ECS). I servizi di database includono Amazon Aurora, Amazon DynamoDB e Amazon Relational Database Service (Amazon RDS). Le opzioni di storage includono Amazon Simple Storage Service (Amazon S3) e Amazon Elastic Block Store (Amazon EBS), con connessione in rete tramite Amazon Virtual Private Cloud (Amazon VPC) e servizi di sicurezza tra cui Servizio AWS di gestione delle chiavi (AWS KMS) e Autorità di certificazione privata AWS (AWS Private CA). Per un elenco aggiornato dei servizi, è possibile consultare l’elenco delle funzionalità AWS pubblicato di recente su AWS Builder Center.

L’AWS European Sovereign Cloud è supportato dai partner AWS, che aiutano a soddisfare i propri requisiti di sovranità. Partner come Adobe, Cisco, Cloudera, Dedalus, Esri, Genesys, GitLab, Mendix, Pega, SAP, SnowFlake, Trend Micro e Wiz stanno rendendo disponibili le loro soluzioni nell’AWS European Sovereign Cloud, fornendo gli strumenti e i servizi necessari per la sicurezza, l’analisi dei dati, lo sviluppo di applicazioni e i carichi di lavoro specifici del settore. Questo ampio supporto dei partner aiuta a creare soluzioni sovrane che combinano i servizi AWS con tecnologie di partner affidabili.

Investimenti significativi nelle infrastrutture europee
L’AWS European Sovereign Cloud è sostenuto da un investimento di 7,8 miliardi di euro destinati a infrastrutture, creazione di posti di lavoro e sviluppo delle competenze. Si prevede che questo investimento contribuirà con 17,2 miliardi di euro all’economia europea fino al 2040 e sosterrà circa 2.800 posti di lavoro equivalenti a tempo pieno all’anno nelle aziende locali.

Alcuni dettagli tecnici
L’AWS European Sovereign Cloud è disponibile per tutti i clienti, indipendentemente da dove si trovino. È possibile accedere all’infrastruttura utilizzando il nome della partizione aws-eusc e il nome della Regione eusc-de-east-1. Una partizione è un gruppo di Regioni AWS. Ogni account AWS è associato a una partizione.

L’infrastruttura supporta tutti i metodi di accesso AWS standard, tra cui la Console di gestione AWS, gli SDK AWS e l’interfaccia della linea di comando AWS (AWS CLI), semplificando l’integrazione nell’automazione e nei flussi di lavoro esistenti. Dopo aver creato un nuovo account root per la partizione Di AWS European Sovereign Cloud, si inizia creando nuove identità e ruoli IAM specifici per questa infrastruttura, che consentiranno di avere il controllo completo sulla gestione degli accessi all’interno dell’ambiente sovrano europeo.

Nozioni di base
L’AWS European Sovereign Cloud fornisce alle organizzazioni europee controlli di sovranità avanzati pur mantenendo l’accesso all’innovazione e alle funzionalità di AWS. È possibile contrattare servizi tramite Amazon Web Services EMEA SARL, con prezzi in EUR e fatturazione in una delle otto valute supportate oggi. L’infrastruttura utilizza l’architettura AWS, il portafoglio di servizi e le API tradizionali, semplificando la creazione e la migrazione delle applicazioni.

L’addendum di AWS European Sovereign Cloud include gli impegni contrattuali aggiuntivi per l’AWS European Sovereign Cloud.

Per me come europeo, questo lancio rappresenta l’impegno di AWS per soddisfare le esigenze specifiche del nostro continente e fornire le funzionalità cloud che guidano l’innovazione in tutti i settori. Invito tutti a scoprire di più sull’AWS European Sovereign Cloud e su come può aiutare le organizzazioni a soddisfare i requisiti di sovranità. Nella Panoramica di AWS European Sovereign Cloud sono disponibili maggiori informazioni sugli obiettivi e sull’approccio di progettazione, creare un nuovo account e pianificare l’implementazione del primo carico di lavoro oggi stesso.

— seb

Spanish version

Apertura de AWS European Sovereign Cloud

Como ciudadano europeo, comprendo de primera mano la importancia de la soberanía digital, especialmente para las organizaciones de nuestro sector público y las industrias altamente reguladas. Hoy me complace anunciar que la AWS European Sovereign Cloud ya está disponible de forma generalizada para todos los clientes. Anunciamos por primera vez nuestros planes de crear esta nueva infraestructura de nube independiente en 2023, y hoy está lista para cumplir los requisitos de soberanía más estrictos de los clientes europeos con un exhaustivo conjunto de servicios de AWS.

Berlín, Puerta de Brandeburgo al atardecer

Cumplimiento de los requisitos de soberanía europeos
Las organizaciones de toda Europa se enfrentan a unos requisitos normativos cada vez más complejos en relación con la residencia de los datos, el control operativo y la independencia de la gobernanza. Hoy en día, con demasiada frecuencia, las organizaciones europeas con los requisitos de soberanía más estrictos se ven atrapadas en entornos locales heredados u ofertas con servicios y funcionalidades reducidos. En respuesta a esta necesidad crítica, la AWS European Sovereign Cloud es la única nube soberana independiente con todas las características que está respaldada por sólidos controles técnicos, garantías de soberanía y protecciones legales. Las entidades del sector público y las empresas de industrias altamente reguladas necesitan una infraestructura en la nube que proporcione controles de soberanía mejorados que mantengan la innovación, la seguridad y la fiabilidad que se esperan de los servicios modernos en la nube. Estas organizaciones necesitan la garantía de que sus datos y operaciones permanecen bajo la jurisdicción europea, con estructuras de gobernanza claras y autonomía operativa dentro de la Unión Europea (UE).

Una nueva infraestructura de nube independiente para Europa
La AWS European Sovereign Cloud es una infraestructura de nube separada de manera física y lógica, en la que todos los componentes están ubicados íntegramente dentro de la UE. La primera región de AWS de la AWS European Sovereign Cloud se encuentra en el estado de Brandeburgo (Alemania) y ya está disponible para el público en general. Esta región opera de forma independiente de las regiones de AWS existentes. La infraestructura cuenta con varias zonas de disponibilidad con fuentes de alimenacion y redes redundantes, diseñadas para funcionar de forma continua incluso si se interrumpe la conectividad con el resto del mundo.

Tenemos previsto ampliar la presencia de la AWS European Sovereign Cloud de Alemania a toda la UE para cumplir los requisitos de aismlamiento estrictos, residencia de los datos dentro del país y baja latencia. Esto comenzará con nuevas zonas locales soberanas ubicadas en Bélgica, los Países Bajos y Portugal. Además, podrá ampliar la infraestructura de la AWS European Sovereign Cloud con zonas locales dedicadas de AWS, AWS AI Factories o AWS Outposts en las ubicaciones que elija, incluidos sus propios centros de datos locales.

La AWS European Sovereign Cloud y sus zonas locales proporcionan controles soberanos mejorados a través de su modelo operativo único. La AWS European Sovereign Cloud será operada exclusivamente por residentes de la UE ubicados en la UE. Abarca actividades como las operaciones diarias, la asistencia técnica y el servicio de atención al cliente. Estamos realizando una transición gradual de la AWS European Sovereign Cloud para que se opere exclusivamente por ciudadanos de la UE ubicados en la UE. Durante este período de transición, seguiremos trabajando con un equipo mixto de residentes de la UE y ciudadanos de la UE ubicados en la UE.

La infraestructura se administra a través de entidades jurídicas europeas especializadas constituidas en el marco de la legislación alemana. En octubre de 2025, AWS nombró director general a Stéphane Israël, ciudadano de la UE que reside en la UE. Stéphane será responsable de la administración y las operaciones de la AWS European Sovereign Cloud, lo que incluye la infraestructura, la tecnología y los servicios, además de liderar los esfuerzos más amplios de AWS en materia de soberanía digital. En enero de 2026, AWS también nombró a Stefan Hoechbauer (vicepresidente de AWS para Alemania y Europa Central) director general de la AWS European Sovereign Cloud. Stefan dirigirá la AWS European Sovereign Cloud junto con Stéphane Israël.

Un consejo consultivo compuesto exclusivamente por ciudadanos de la UE, que incluye a dos representantes independientes externos, proporciona supervisión y experiencia adicional en materia de soberanía.

Mejor control y residencia de datos
La AWS European Sovereign Cloud ofrece amplias garantías de residencia de datos para que pueda cumplir los requisitos más estrictos en materia de residencia de datos. Al igual que ocurre con nuestras regiones de AWS existentes en todo el mundo, todo el contenido permanece dentro la región que elija, a menos que decida lo contrario. Además del contenido, los metadatos creados por los clientes, incluidos los roles, los permisos, las etiquetas de recursos y las configuraciones, también permanecen dentro de la UE. La infraestructura cuenta con su propio sistema dedicado de AWS Identity and Access Management (AWS IAM) y facturación, que funciona de forma independiente dentro de las fronteras europeas.

Los controles técnicos integrados en la infraestructura impiden el acceso a la AWS European Sovereign Cloud desde fuera de la UE. La infraestructura incluye un proveedor de servicios de confianza europeo dedicado para las operaciones de las autoridades de certificación y utiliza servidores de nombres de Amazon Route 53 dedicados. Estos servidores solo usarán dominios de nivel superior europeos para sus propios nombres. La AWS European Sovereign Cloud no tiene dependencias críticas de personal o infraestructura fuera de la UE.

Marco de seguridad y cumplimiento
La AWS European Sovereign Cloud mantiene las mismas capacidades de seguridad básicas que cabe esperar de AWS, como el cifrado, la administración de claves, la gobernanza del acceso y AWS Nitro System para el aislamiento de la computación. Esto significa que sus instancias de EC2 se benefician de la integridad de la plataforma verificada criptográficamente y de los límites impuestos por el hardware que impiden el acceso no autorizado a sus datos sin comprometer el rendimiento, lo que le proporciona tanto los controles de soberanía como la potencia computacional que requieren sus cargas de trabajo. La infraestructura se somete a auditorías independientes externas, con programas de cumplimiento que incluyen la norma ISO/IEC 27001:2013, informes SOC 1/2/3 y la certificación C5 de la Oficina Federal de Seguridad de la Información.

El marco de referencia de soberanía de la AWS European Sovereign Cloud define los controles de soberanía específicos en relación con la independencia de la gobernanza, el control operativo, la residencia de datos y el aislamiento técnico. Este marco está disponible en AWS Artifact y proporciona visibilidad total a través de la certificación SOC 2.

Disponibilidad total del servicio
Puede acceder a una amplia variedad de servicios de AWS en la AWS European Sovereign Cloud desde su lanzamiento, incluidos Amazon SageMaker y Amazon Bedrock para cargas de trabajo de inteligencia artificial y machine learning. Para el procesamiento, puede usar Amazon Elastic Compute Cloud (Amazon EC2) y AWS Lambda. La orquestación de contenedores está disponible a través de Amazon Elastic Kubernetes Service (Amazon EKS) y Amazon Elastic Container Service (Amazon ECS). Los servicios de bases de datos incluyen Amazon Aurora, Amazon DynamoDB y Amazon Relational Database Service (Amazon RDS). Dispone de opciones de almacenamiento como Amazon Simple Storage Service (Amazon S3) y Amazon Elastic Block Store (Amazon EBS), con redes a través de Amazon Virtual Private Cloud (Amazon VPC) y servicios de seguridad como AWS Key Management Service (AWS KMS) y AWS Private Certificate Authority. Para obtener una lista actualizada de los servicios, consulte la matriz de capacidades de AWS que se ha publicado recientemente en AWS Builder Center.

La AWS European Sovereign Cloud cuenta con el respaldo de los socios de AWS, que se comprometen a ayudarle a cumplir sus requisitos de soberanía. Socios como Adobe, Cisco, Cloudera, Dedalus, Esri, Genesys, GitLab, Mendix, Pega, SAP, SnowFlake, Trend Micro y Wiz ofrecen sus soluciones en la AWS European Sovereign Cloud, lo que le proporciona las herramientas y los servicios que necesita en materia de seguridad, análisis de datos, desarrollo de aplicaciones y cargas de trabajo específicas del sector. Este amplio apoyo de los socios le ayuda a crear soluciones soberanas que combinan los servicios de AWS con tecnologías de socios de confianza.

Inversión importante. en la infraestructura europea
La AWS European Sovereign Cloud está respaldada por una inversión de 7.800 millones de EUR en infraestructura, creación de empleo y desarrollo de habilidades. Se espera que esta inversión aporte 17.200 millones de EUR a la economía europea para 2040 y ayude a crear el equivalente a aproximadamente 2.800 puestos de trabajo a tiempo completo por año en empresas locales.

Algunos detalles técnicos
La AWS European Sovereign Cloud está disponible para todos los clientes, independientemente de dónde se encuentren. Puede acceder a la infraestructura utilizando el nombre de partición aws-eusc y el nombre de región eusc-de-east-1. Una partición es un grupo de regiones de AWS. Cada cuenta de AWS tiene limitado su alcance a una sola partición.

La infraestructura admite todos los métodos de acceso estándar de AWS, como la Consola de administración de AWS, los AWS SDK y la Interfaz de la línea de comandos de AWS (AWS CLI), lo que facilita la integración en sus flujos de trabajo y automatización existentes. Tras crear una nueva cuenta raíz para la partición de la AWS European Sovereign Cloud, primero deberá crear nuevas identidades y roles de IAM específicos para esta infraestructura, lo que le permitirá tener un control total sobre la administración del acceso en el entorno soberano europeo.

Cómo empezar
La AWS European Sovereign Cloud proporciona a las organizaciones europeas controles de soberanía mejorados, al tiempo que mantiene el acceso a la innovación y las capacidades de AWS. Puede contratar los servicios a través de Amazon Web Services EMEA SARL, con precios en EUR y facturación en cualquiera de las ocho divisas que admitimos actualmente. La infraestructura utiliza la arquitectura, la cartera de servicios y las API habituales de AWS, lo que facilita la creación y la migración de aplicaciones.

La adenda de la AWS European Sovereign Cloud contiene los compromisos contractuales adicionales para la AWS European Sovereign Cloud.

Para mí, como europeo, este lanzamiento representa el compromiso de AWS de satisfacer las necesidades específicas de nuestro continente y proporcionar las capacidades en la nube que impulsan la innovación en todos los sectores. Le invito a descubrir más sobre la AWS European Sovereign Cloud y cómo puede ayudar a su organización a cumplir sus requisitos de soberanía. Lea la descripción general de la AWS European Sovereign Cloud para obtener más información sobre los objetivos y el enfoque del diseño, regístrese para obtener una nueva cuenta y planifique hoy mismo el despliegue de su primera carga de trabajo.

— seb

[$] LWN.net Weekly Edition for January 15, 2026

2026-01-15 jzb

Post Syndicated from jzb original https://lwn.net/Articles/1053201/

Inside this week’s LWN.net Weekly Edition:

Front: SFC v. VIZIO; GPLv2 requirements; Debian and GTK 2; OpenZL; kernel scheduler QoS; Rust concurrent data access; Asciinema.
Briefs: OpenSSL and Python; LSFMM+BPF 2026; Fedora elections; Gentoo retrospective; EU lawmaking; Git data model; Firefox 147; Radicle 1.6.0; Quotes; …
Announcements: Newsletters, conferences, security updates, patches, and more.

6-day and IP Address Certificates are Generally Available

2026-01-15 Let's Encrypt

Post Syndicated from Let's Encrypt original https://letsencrypt.org/2026/01/15/6day-and-ip-general-availability.html

Short-lived and IP address certificates are now generally available from Let’s Encrypt. These certificates are valid for 160 hours, just over six days. In order to get a short-lived certificate subscribers simply need to select the ‘shortlived’ certificate profile in their ACME client.

Short-lived certificates improve security by requiring more frequent validation and reducing reliance on unreliable revocation mechanisms. If a certificate’s private key is exposed or compromised, revocation has historically been the way to mitigate damage prior to the certificate’s expiration. Unfortunately, revocation is an unreliable system so many relying parties continue to be vulnerable until the certificate expires, a period as long as 90 days. With short-lived certificates that vulnerability window is greatly reduced.

Short-lived certificates are opt-in and we have no plan to make them the default at this time. Subscribers that have fully automated their renewal process should be able to switch to short-lived certificates easily if they wish, but we understand that not everyone is in that position and generally comfortable with this significantly shorter lifetime. We hope that over time everyone moves to automated solutions and we can demonstrate that short-lived certificates work well.

Our default certificate lifetimes will be going from 90 days down to 45 days over the next few years, as previously announced.

IP address certificates allow server operators to authenticate TLS connections to IP addresses rather than domain names. Let’s Encrypt supports both IPv4 and IPv6. IP address certificates must be short-lived certificates, a decision we made because IP addresses are more transient than domain names, so validating more frequently is important. You can learn more about our IP address certificates and the use cases for them from our post announcing our first IP Certificate.

We’d like to thank the Open Technology Fund and Sovereign Tech Agency, along with our Sponsors and Donors, for supporting the development of this work.

A Conversation with Arizona Secretary of State Adrian Fontes

2026-01-15 The Atlantic

Post Syndicated from The Atlantic original https://www.youtube.com/watch?v=MtN34CRs2SM

What is CDC?

Why Apache Hudi?

Hudi MultiTable Delta Streamer

Solution overview

Mechanism

Prerequisites

Set up Amazon MSK and create Kafka topics

Schema setup

Create the Kafka topics

Configure Apache Hudi

Create common configuration file

Create table-specific configurations

Launch Amazon EMR cluster

Invoke the Hudi MultiTable DeltaStreamer

Verify and query data

Real-world impact of Hudi CDC pipelines

Cleanup

Conclusion

About the authors

What users reported

What we found

Tracing through the stack

The lifecycle of incident mitigations

What we did

What we’re building

Evaluations

Why are evals important?

What makes evals challenging?

Visualize trajectories

Intentional changes

Establish success criteria first

Read production samples

Closing thoughts

Queue-based vs. workgroup-level monitoring

Solution overview

Architecture Components

Prerequisites

Identify workload types

Define queue configuration

Best practices for queue management

Clean up

Conclusion

About the authors

Overview of solution

Prerequisites

Solution walkthrough

Deploying the solution

Testing the solution

Clean up

Conclusion

About the authors

Колко толерантно е населението на България към ЛГБТИ хората?

Двама души от един и същи пол, които… се държат за ръце.

52,7% пък не биха пуснали детето си да си играе в дома на приятелче, чиито родители са от един и същи пол.

Едва една трета от хората в България казват, че биха променили отношението си към близък човек, ако научат, че е ЛГБТИ.

Добрите новини

Така че държавата има все по-малко основание да не изпълнява решенията на европейски съдилища,

Най-добрата новина всъщност е, че толерантността продължава да е ценност.

Human Native x Cloudflare

The Internet needs new economic models

New tools for AI developers

Building the foundation for these new business models

What’s next

German version

French version

Italian version

Spanish version

The collective thoughts of the interwebz