Your serverless attendee’s guide to AWS re:Invent 2026

Post Syndicated from Nicolas Tarzia original https://aws.amazon.com/blogs/compute/your-serverless-attendees-guide-to-aws-reinvent-2026/

AWS re:Invent 2026 offers an extensive selection of serverless and application integration content. This year’s lineup includes new AWS Lambda execution models: durable functions, Lambda Managed Instances, and Lambda MicroVMs. It also covers AI agents as a new kind of serverless consumer and maturing patterns for production event-driven applications.

Don’t have a ticket yet? Register for re:Invent 2026 and join us in Las Vegas from November 30 to December 4.

For detailed descriptions and schedule, visit the AWS re:Invent Session Catalog. Sessions are held across several venues, so allow extra time to travel between properties.

We’ve curated this guide to help you navigate the serverless sessions and find the content most relevant to you.

Session types

  • Breakout Sessions are lecture-style presentations covering architecture, best practices, and deep dives into AWS services.
  • Workshops are 2-hour hands-on sessions where you work through tasks in AWS accounts using AWS services. Laptops are required.
  • Chalk Talks are highly interactive 60-minute sessions with smaller audiences, focused on technical deep dives with whiteboards for architectural discussions.
  • Builders’ Sessions are 60-minute small-group sessions led by an AWS expert who guides you through a technical problem using AWS services.
  • Code Talks are 60-minute live coding sessions where AWS experts show how to build solutions using AWS services.
  • Lightning Talks are 20-minute sessions focused on a single idea.

What’s new in AWS Serverless

Start here to get oriented on what has changed in serverless heading into re:Invent 2026.

SVS207 | Modernizing your compute stack with Lambda’s new execution models – Breakout Session

Choose a Lambda execution model for each workload class, run mixed deployments, and compare monitoring across models. Bring your workload portfolio.

SVS305 | What’s new in AWS Serverless – Workshop

Build with the newest serverless releases and work through the trade-offs that matter in real systems.

SVS339 | What’s new with Amazon API Gateway – Breakout Session

See what’s new in Amazon API Gateway, including direct service integrations that replace Lambda glue code. Learn how to design APIs for both developers and AI agents, including Model Context Protocol (MCP) servers.

Getting started with serverless

Are you new to serverless or taking your first steps? These sessions cover fundamentals and hands-on introductions.

SVS202 | Building your first event-driven application – Builders’ Session

Build a retail order system on Amazon EventBridge, Lambda, and Amazon DynamoDB where each step runs independently and one failure doesn’t cascade. No event-driven experience required.

SVS301 | AWS Serverless developer experience – Workshop

Build a realistic serverless solution while covering event-driven principles, observability, open-source tools, and deployment automation, in Python, TypeScript, Java, or .NET.

SVS302 | Building serverless applications with Terraform – Workshop

Define, deploy, and maintain serverless architectures with Terraform, including reusable modules, state management, and deployments across environments.

New Lambda execution models

Three Lambda execution models cover different workload shapes: durable functions, Lambda Managed Instances, and Lambda MicroVMs.

Lambda durable functions

API314 | A closer look at AWS Lambda durable functions – Chalk Talk

Learn how checkpointing and replay-based recovery work, see patterns such as sagas, fan-out/fan-in, and human-in-the-loop waits, and get a framework for choosing between durable functions and AWS Step Functions.

API316 | Coding an AI agent from scratch with AWS Lambda durable functions – Code Talk

Build an AI agent from scratch with durable functions, including a tool-use loop, retries with exponential backoff, and human approval that doesn’t bill for idle time.

SVS325 | Distributed transactions with Step Functions and durable functions – Builders’ Session

Build a payment processing flow with the saga pattern, using Step Functions and durable functions for compensation when part of a transaction fails.

Lambda Managed Instances

SVS332 | Lambda Managed Instances: Always-on serverless compute – Chalk Talk

Lambda Managed Instances offer Amazon Elastic Compute Cloud (Amazon EC2) economics with multi-concurrency while Lambda manages instance lifecycle, patching, and scaling. Model your break-even point, and bring your Lambda bill.

Lambda MicroVMs

SVS307 | Running untrusted code in sandboxes with AWS Lambda MicroVMs – Builders’ Session

Build sandboxes that run untrusted code, such as AI-generated code and third-party plugins, with resource limits, permission boundaries, and tested failure modes.

SVS337 | AWS Lambda MicroVMs: Strong isolation for agents and beyond – Breakout Session

Learn how Lambda MicroVMs work: VM-level isolation per agent, tenant, or task, with state that persists across invocations and sandboxes that suspend when idle and resume on demand.

Serverless and AI agents

A major theme this year is building, orchestrating, and operating AI agents on serverless infrastructure.

SVS338 | Serverless developers in the agentic era: What changes, what doesn’t – Breakout Session

See what changes when AI agents call your functions as tools, and what stays the same. Topics include orchestration for non-deterministic workflows and cost governance when one reasoning chain calls 200 functions.

SVS306 | Building agentic AI architectures with AWS Serverless – Builders’ Session

Build a multi-agent system that uses Lambda for tool execution, Amazon Bedrock for reasoning, and Amazon EventBridge for communication between agents.

SVS324 | Building agentic apps with serverless: Lambda as your agent’s toolbox – Chalk Talk

Design Lambda functions as agent tools, with single-responsibility tools, clear input and output schemas, MCP-compatible interfaces, and multi-tenant isolation.

SVS342 | Coding the tool layer for AI agents on serverless – Code Talk

Retrofit a serverless API into an agent tool, live. The build covers an MCP server with versioned tools, discovery through the Gateway capability of Amazon Bedrock AgentCore, a Step Functions workflow exposed as one tool, and scoped agent identity.

API312 | Building event-driven, multi-agent architectures on serverless – Chalk Talk

Map agent coordination patterns on Amazon EventBridge, such as peer-to-peer messaging, supervisor-worker hierarchies, and broadcast, and the failure modes they introduce, including circular delegation.

COM325 | The agent orchestration spectrum: agents, Step Functions, or Lambda? – Code Talk

Live-code the same AI workload with AgentCore, Step Functions, and durable functions, and compare latency, cost, and debuggability on a production content platform.

SVS328 | Guardrails for AI-generated serverless infrastructure – Chalk Talk

Coding agents can generate over-permissioned AWS Identity and Access Management (IAM) policies and skip encryption. Learn which guardrails to automate and which need human review.

API402 | Building event-driven architectures in the age of AI agents – Breakout Session

See how familiar patterns such as queues, events, and retries evolve when your consumers are autonomous agents instead of microservices.

Building event-driven architectures

Event-driven architectures remain the backbone of serverless applications. These sessions cover patterns from foundational to advanced.

API302 | Serverlesspresso: Build an event-driven app from the ground up – Workshop

Build a serverless coffee-ordering app with Lambda, Amazon EventBridge, and durable functions, which checkpoint progress and recover from failures without custom state management.

API305 | Building enterprise-scale event buses with Amazon EventBridge – Builders’ Session

Configure an event bus with routing policies, schema discovery, and access controls so platform teams govern events while application teams ship independently.

API311 | How event-driven architectures go wrong and how to fix them – Chalk Talk

Dissect costly anti-patterns such as unbounded fan-out, missing dead-letter queues, chatty event schemas, and orchestration gaps that silently drop work, and fix them.

API317 | Coding event-driven retry patterns that prevent outages – Code Talk

Live-code exponential backoff with jitter, circuit breakers, poison-pill isolation, and replay from archive for Amazon EventBridge and Step Functions.

Orchestration: Step Functions and durable functions

Both Step Functions and Lambda durable functions handle retries, long-running execution, and failure recovery. These sessions help you choose.

API318 | Composing the right orchestration for serverless workflows – Lightning Talk

A decision framework for human approvals, long-running AI agent workflows, and multi-service sagas, plus a hybrid pattern that uses durable functions inside Step Functions.

ARC329 | The coordination decision: when to orchestrate and when to let go – Breakout Session

Decide per flow whether to orchestrate or let services react to events, using a logistics platform that runs both patterns on Step Functions and Amazon EventBridge.

Building APIs and serverless web applications

Serverless is increasingly used to build full-stack web applications, not only API backends.

SVS348 | Serverless compute built for the web – Breakout Session

Build full-stack web applications entirely on serverless services, with patterns for HTTP routing, server-side rendering, static assets, and real-time features.

SVS327 | Build interactive web apps with AWS Lambda – Chalk Talk

Run web frameworks such as Express, Hono, and Next.js on Lambda with runtime adapters, including migration paths for containerized web apps.

API304 | API Gateway unleashed – Workshop

Create, deploy, and manage APIs with API Gateway, including message transformation, request validation, security, Infrastructure as Code, and WebSocket backends.

API306 | Build secure APIs faster with Amazon API Gateway and Kiro CLI – Builders’ Session

Implement and compare API Gateway authorization with Amazon Cognito user pools, Amazon Verified Permissions, Lambda custom authorizers, and IAM authorizers. Then go from spec to deployed API with Kiro CLI, OpenAPI, and MCP servers.

API309 | Amazon API Gateway: Migration strategies and agent-ready APIs – Chalk Talk

Learn migration strategies from legacy API platforms, then design APIs that AI agents can discover, authenticate against, and call, including MCP-compatible tool definitions.

Serverless developer experience

AI-assisted development is changing how serverless applications are built, tested, and deployed.

SVS303 | AI-driven serverless development with Kiro – Workshop

Use Kiro to build a serverless application with authentication, tests, observability, and an optional continuous integration and continuous delivery (CI/CD) pipeline.

SVS343 | A day in the life of a serverless builder – Code Talk

Follow one serverless application from first line of code to production, live: an AI coding agent generates the scaffolding, and you debug a failing request with OpenTelemetry.

SVS329 | Testing serverless and event-driven applications – Chalk Talk

Whiteboard testing strategies for serverless and event-driven apps, from unit and contract tests to chaos techniques such as duplicate and out-of-order delivery.

SVS322 | Build self-healing serverless applications with AWS DevOps Agent – Builders’ Session

Inject failures such as Lambda throttling, timeout cascades, and dead-letter queue buildup, then configure AWS DevOps Agent to find root causes and apply fixes with approval gates.

Performance, cost, and operations at scale

Tune your Lambda functions, optimize costs, and learn what changes when you operate at enterprise scale.

SVS341 | Lambda performance tuning and cost optimization – Chalk Talk

Go beyond memory tuning with SnapStart and provisioned concurrency, event source mapping tuning, and guidance on when Lambda Managed Instances beat per-invocation pricing.

SVS335 | Operating serverless at scale: What changes at 1,000+ functions – Chalk Talk

Learn what changes between 10 and 1,000 functions: safe deployments, account structure and concurrency management, observability without cardinality overload, and platform patterns.

ARC325 | Serverless at 1M RPS: Lambda, DynamoDB, and SQS scaling lessons – Chalk Talk

Push Lambda, DynamoDB, and Amazon Simple Queue Service (Amazon SQS) to one million requests per second and see what breaks. The session shares real numbers on cold start budgets, retry amplification, and backpressure.

SVS334 | Serverless observability: OpenTelemetry and vended logs – Chalk Talk

Set up OpenTelemetry and API Gateway vended logs for tracing across Step Functions, Amazon EventBridge, and SQS, plus observability for agentic workloads such as token use.

SVS320 | Same code, any compute: Portable architecture on AWS – Builders’ Session

Apply Hexagonal Architecture and 12-factor patterns so compute is a deployment decision, not an architectural one, using the AWS Lambda Web Adapter and runtime-neutral handlers.

Serverless data processing

Process data at scale without provisioning clusters or managing batch infrastructure.

SVS309 | Building serverless distributed data processing workloads – Builders’ Session

Build a pipeline with Step Functions Distributed Map, Lambda, and Amazon Simple Storage Service (Amazon S3) that processes thousands of objects in parallel and handles partial failures.

SVS321 | Serverless streaming data processing with AWS Lambda and Apache Kafka – Builders’ Session

Build a real-time location tracking app that ingests events from Amazon Managed Streaming for Apache Kafka (Amazon MSK), processes them with Lambda, and updates a dashboard through AWS AppSync.

Community sessions

Sessions led by AWS community members and customers sharing production experience with the preceding execution models and patterns.

COM303 | Lambda for every scale and workload shape – Breakout Session

Honeycomb shares how it runs its query backend across all three Lambda execution models. Honeycomb reports that Managed Instances absorb steady-state traffic at roughly 7.5x lower cost than Functions, while MicroVMs take spikes and Functions catch bursts.

COM326 | Observable agents with Lambda durable functions and OpenTelemetry – Code Talk

Build a multi-agent purchasing coordinator on durable functions and instrument it with OpenTelemetry for one coherent trace across all invocations, plus Amazon CloudWatch insights for fleet-wide queries.

COM304 | Hidden trade-offs in distributed architectures – Breakout Session

Investigate unexpected system behavior in a real-world case study built on Lambda, DynamoDB, DynamoDB Streams, and Step Functions, and learn the trade-offs behind it.

Additional resources

Many sessions run more than once, so if a session is full, check the catalog for another time.

If you cannot join in person, we upload breakout sessions to the AWS Events YouTube channel after the event.

For more serverless learning resources, visit Serverless Land.

We look forward to seeing you at re:Invent 2026!

Upcoming Speaking Engagements

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/10/upcoming-speaking-engagements-61.html

This is a current list of where and when I am scheduled to speak:

  • I’m speaking at ATTENTION: Democracy, Rebuilt in Montreal, Canada. The event runs October 21–23, 2026, and my talk is on Wednesday, October 21.
  • I’m speaking at the University of Toronto in Toronto, Ontario, Canada, at 5 PM ET on October 22, 2026.
  • I’m speaking at [un]prompted in San Francisco, California. The event runs October 27–29, 2026, and my talk is at 11:35 AM PT on Thursday, October 29.
  • I’m speaking at the Catalyst Summit in Brampton, Ontario, Canada, on Wednesday, November 18, 2026.
  • I’m speaking at QT Qonference 2026 in Toronto, Ontario, Canada, on Thursday, November 19, 2026.

The list is maintained on this page.

Multi-region disaster recovery for Amazon API Gateway WebSocket APIs

Post Syndicated from Shubham Dhoka original https://aws.amazon.com/blogs/architecture/multi-region-disaster-recovery-for-amazon-api-gateway-websocket-apis/

WebSocket connections power some of the most interactive experiences on the web: financial trading applications streaming live market data, collaborative editing tools where multiple users work on the same document, gaming leaderboards updating in real time, and IoT dashboards monitoring thousands of devices. These applications depend on persistent, bidirectional communication between clients and servers.

Real-time applications using WebSocket connections face a unique challenge during Regional outages. WebSocket connections are persistent and bound to a specific AWS Region. A Regional failure severs every active connection. For applications that also maintain shared state across those connections, such as a live auction or collaborative editor, users lose both their connection and their session context. This post presents a design pattern for multi-Region disaster recovery with Amazon API Gateway WebSocket APIs that reconnects clients and restores state automatically during failover.

In this post, you learn how to build this pattern using Amazon Route 53 for DNS-based failover, Amazon DynamoDB Global Tables for cross-Region state replication, a canary health probe for traffic-independent failure detection, and client-side reconnection logic to recover from Regional failures while preserving application state.

Solution overview

This pattern creates an active-passive multi-Region architecture where the primary Region serves WebSocket connections and automatically fails over to a secondary Region during a disaster. The core idea is separating application state from the connection itself, so the client can restore its session after reconnection.

The architecture deploys identical WebSocket API stacks in two Regions, a primary and a secondary. Both Regions share application state through DynamoDB Global Tables, so the secondary Region is consistently ready to serve traffic. A single custom domain name fronts both Regions. Amazon Route 53 failover routing policy directs traffic to the primary Region under normal conditions. When the primary becomes unhealthy, Route 53 automatically shifts DNS resolution to the secondary Region. Clients reconnect to the same domain and resume operations without changing their configuration.

Figure 1 shows the architecture and the normal operation and failover flow.

Architecture diagram showing a client connecting through a Route 53 failover policy to the primary Region us-east-1, which runs API Gateway WebSocket, Lambda, and DynamoDB, with an Amazon EventBridge rule triggering a canary health check and a CloudWatch alarm that drives the Route 53 health check; DynamoDB Global Tables replicate to the secondary Region us-west-2 for failover

Figure 1: Multi-region WebSocket API failover architecture

  1. A user opens the application, which establishes a WebSocket connection to the application’s domain name.
  2. Amazon Route 53 resolves the domain using a failover routing policy. Under normal conditions, it returns the primary Region’s endpoint.
  3. The WebSocket connection reaches API Gateway in the primary Region, which invokes an AWS Lambda function for each connection event and message.
  4. Lambda reads and writes application state to DynamoDB. Connected clients receive updates through their open WebSocket connections.
  5. DynamoDB Global Tables replicates the application state to the secondary Region asynchronously, keeping it ready to serve traffic.
  6. Independently, an Amazon EventBridge rule triggers a canary Lambda function every minute.
  7. The canary opens a WebSocket connection to the API Gateway endpoint, sends a heartbeat, and publishes the result as a custom Amazon CloudWatch metric.
  8. A CloudWatch alarm evaluates the metric. If the canary reports consecutive failures or stops publishing, the alarm fires and the associated Route 53 health check marks the primary as unhealthy.
  9. Route 53 detects the unhealthy primary and begins resolving the domain to the secondary Region instead.
  10. Clients reconnect through the same domain, now reaching the secondary Region, and recover their application state from the local DynamoDB replica.

Prerequisites

To follow along with this post, you need:

Deploy the solution

The complete source code and deployment instructions are available on the GitHub repo. The repository includes AWS CloudFormation templates for the WebSocket APIs, canary health probe, and a React client application with the reconnection logic described in this post.

The demo application

The sample application is a real-time auction where multiple users bid on an item simultaneously. Each bid updates a shared counter through WebSocket messages, and connected clients see the current bid in real time. The application displays which AWS Region is serving responses, making failover visible during testing. Although minimal, this pattern represents real-time workloads where multiple clients share mutable state over WebSocket connections: chat applications, collaborative editors, live dashboards, or multiplayer games.

Managing connection state across Regions

WebSocket connections are persistent, but the application state built on top of them is what matters during failover. As users interact with the application, the server builds up session data tied to their identity. When a failover occurs and the client reconnects to a different Region, that session data needs to be available for the experience to continue uninterrupted. This solution stores state in DynamoDB Global Tables, which replicates it to the secondary Region asynchronously, typically within a second. Because this replication is asynchronous, writes that occur immediately before a failure may not reach the secondary Region. For most applications, this window is small enough to be acceptable.

After failover, the client reconnects and restores its state. In this demo, the client sends a getState message after reconnection to retrieve the latest application state from the secondary Region’s DynamoDB replica.

Production applications add two considerations on reconnection. First, clients authenticate again: WebSocket APIs authenticate on the $connect route, so after failover the secondary Region re-authenticates the client through a Lambda authorizer. Deploy that authorizer in both Regions so reconnecting clients can authenticate against either one. Second, to restore state, identify the user from the validated token, such as the sub claim. Key the stored state by the user ID rather than the connection ID, which changes on every reconnect. Applications model session state in different ways, so adapt this to your design.

If your application uses a relational database, you need to set up similar cross-region state replication. For instance, Amazon Aurora Global Database provides sub-second replication latency across Regions. Note that Aurora requires promoting a read replica to primary during failover, which typically adds 1–2 minutes to the recovery process. This promotion can also be automated based on a CloudWatch alarm action.

Heartbeat

Both the client and the canary use application-level heartbeat to verify their WebSocket connections are alive. At a regular interval, they send a lightweight message and expect an acknowledgment. A missing or invalid response signals a broken connection. The client uses this to trigger reconnection. The canary uses it to drive the Route 53 health check. This mechanism is described in RFC 6455 and is the standard approach for WebSocket connection health monitoring.

The client code snippets in this post use react-use-websocket, a React library that provides built-in support for reconnection and heartbeat. You can implement the same patterns in other WebSocket client libraries or languages.

heartbeat: {
  message: JSON.stringify({ action: 'heartbeat' }),
  returnMessage: 'pong',
  timeout: 35000,   // Close connection if no message received within 35s
  interval: 30000,  // Send heartbeat every 30s
}

Detecting regional failures automatically

When the primary Region becomes unavailable, Route 53 needs to detect the failure and redirect traffic to the secondary. To do this, the solution creates a health check that actively monitors the WebSocket endpoint. A Lambda function called the canary, deployed in the primary Region, powers this health check. It probes the endpoint every minute by opening a WebSocket connection, sending a heartbeat message, and confirming it receives a valid response. The canary publishes the result as a custom CloudWatch metric (App/WebSocketHealth, dimension ApiId): 1 for healthy, 0 for unhealthy.

A CloudWatch alarm monitors this metric and treats both failures and missing data as unhealthy (TreatMissingData: breaching). After two consecutive unhealthy datapoints, the alarm fires and the associated Route 53 health check (configured with InsufficientDataHealthStatus: Unhealthy) marks the primary as down. Route 53 shifts DNS to the secondary Region.

Failback works through the same mechanism. When the primary recovers, the canary starts succeeding again, the alarm clears, and Route 53 shifts traffic back. For production workloads, consider asymmetric thresholds: use a low threshold (two datapoints) for fast failover and a higher threshold (over five) for slower failback, which helps prevent flapping if the primary is unstable during recovery. A DNS time-to-live (TTL) of 60–120 seconds provides a good balance between fast propagation and lookup efficiency.

Client reconnection strategy

When a connection drops, the client needs to reconnect without overwhelming the recovering service. The heartbeat described earlier handles detection in most cases. If no messages arrive within the timeout window, the library closes the connection. However, the backend can be degraded rather than fully unreachable. For example, API Gateway returns error responses when the Lambda function is throttled, so the connection remains technically open while the application is non-functional. To handle this, the client tracks consecutive failed heartbeats: responses that aren’t the expected pong message. After 2 consecutive failures, the client force-closes the connection and triggers reconnection. With a 30-second heartbeat interval, this detects a degraded backend within 60 seconds.

After the client identifies a broken connection, it begins reconnecting using exponential backoff with full jitter, a pattern described in the AWS Architecture Blog post Exponential Backoff And Jitter. The reconnection interval doubles with each attempt, and full jitter randomizes the actual delay within that window. This spreads reconnection attempts over time, helping prevent a thundering herd where many clients retry simultaneously and overwhelm the secondary Region the moment DNS shifts.

reconnectInterval: (attempt) => {
  // Exponential backoff with full jitter
  const expDelay = Math.min(2000 * Math.pow(2, attempt), 20000);
  return Math.random() * expDelay;
}

After each successful reconnection, the client requests the current application state from the server. Because the client reconnects to the same custom domain, Route 53 resolves it to whichever Region is currently healthy. The client doesn’t need to know which Region it’s connecting to. The server includes the Region in its response, and the application can update its UI or logging accordingly.

Failover in action

To see the full failover sequence, start the client application and connect to your WebSocket endpoint:

cd client
cp .env.example .env.local
# Edit .env.local with your custom domain URL
npm install
npm run dev

Open the application in your browser and note the serving Region displayed in the UI.

Auction app showing Connected status and serving Region US-EAST-1 with a current bid of $9

Figure 2: Client connected to the primary Region (us-east-1)

Next, simulate a failure by setting the primary Lambda function’s reserved concurrency to zero, which prevents it from processing requests:

aws lambda put-function-concurrency \
  --function-name counter-handler-us-east-1 \
  --reserved-concurrent-executions 0 \
  --region us-east-1

Within about two minutes, the canary detects the failure after two consecutive failed probes at one-minute intervals. Route 53 then marks the primary as unhealthy, and DNS begins resolving to the secondary Region.

The client detects the dropped connection and begins retrying with exponential backoff and jitter.

Auction app showing Disconnected status and reconnecting after a connection failure, with connection errors in the activity log

Figure 3: Client detecting failure and reconnecting with exponential backoff

When DNS propagation completes, the next reconnection attempt reaches the secondary Region. The client requests the current state, recovers it from DynamoDB Global Tables, and the UI updates to show the new serving Region.

Auction app reconnected to serving Region US-WEST-2 after DNS failover, with the current bid of $9 preserved

Figure 4: Client after failover to the secondary Region (us-west-2) with state preserved

When the primary Region recovers, the canary confirms that it’s healthy again. After the alarm clears, Route 53 shifts DNS back and clients reconnect to the primary through the same automatic process. No manual intervention is needed for failback.

Clean up

To avoid ongoing charges, delete the resources created during this walkthrough:

  1. Delete the CloudFormation stacks: 
    aws cloudformation delete-stack --stack-name websocket-health-check --region us-east-1
aws cloudformation delete-stack --stack-name websocket-counter-secondary --region us-west-2
aws cloudformation delete-stack --stack-name websocket-counter-primary --region us-east-1

  2. Delete the Route 53 failover records and API Gateway custom domain names manually from the console.

Conclusion

In this post, you learned how to build a multi-region disaster recovery pattern for API Gateway WebSocket APIs. By separating application state from the connection itself, you can restore sessions automatically after a regional failover. This pattern applies to real-time applications where WebSocket continuity matters. Adapt the health check thresholds, reconnection timing, and state recovery approach to fit your workload.

To learn more about the services used in this solution, refer to the following resources:

 

About the authors

How UTS established a modern curriculum data services foundation on AWS

Post Syndicated from Oleg Polianski original https://aws.amazon.com/blogs/architecture/how-uts-established-a-modern-curriculum-data-services-foundation-on-aws/

Students and staff experience the University of Technology Sydney (UTS) as a single institution, not as the collection of curriculum and student systems behind it. They expect the information underpinning teaching, learning, course design and institutional decision-making to be accurate, timely and coherent, irrespective of its source.

Behind that experience, UTS had reached a natural inflection point in its integration landscape. Critical teaching and learning capabilities relied upon established vendor platforms with disparate data models, interfaces, stability profiles and scaling characteristics. Although those platforms continued to perform essential operational functions, integrating and operating the surrounding estate had become progressively more intricate and costly.

UTS’s purpose remained constant: excellence in teaching, learning and the student experience. Technology exists to advance that purpose, not to supplant it. UTS Information Technology Unit (ITU) therefore sought to remove impediments between authoritative data and the people, processes and digital services that depend upon it.

UTS made that transition practical with AWS. With managed, elastic and serverless capabilities, UTS reduced undifferentiated operational effort and improved fault tolerance. This established a sustainable model for governed data services. Curriculum became the first bounded business domain in which UTS applied that model.

The challenge

UTS ITU required an architecture that could:

  • Make authoritative curriculum information available through reusable services without displacing the Curriculum Management System, a third-party software as a service (SaaS) application, as the system of record.
  • Replace tightly coupled, consumer-specific interfaces with governed contracts that insulate consumers from source schemas and implementation detail.
  • Accommodate routine change and transient high-volume activity, including complete domain reloads, without permanently provisioned capacity.
  • Support graph traversal across deeply nested and inverse curriculum relationships, exact and full-text retrieval, and semantic discovery from the same authoritative source.
  • Operate reliably and economically within finite engineering capacity whilst establishing a repeatable pattern for subsequent data domains.

Domain boundaries were non-negotiable. Curriculum describes what the university teaches and remains the responsibility of the Curriculum Management System. Preserving that bounded context prevents ownership, validation and governance from migrating into non-authoritative services.

A domain-oriented approach

UTS ITU organised the capability according to domain-driven design. Rather than aligning data services with the applications that produce or consume data, they aligned them with:

Each data product has explicit ownership, consumers, quality expectations, interface contracts and lifecycle management. Institutional data is therefore treated as a governed asset designed for reuse, not an incidental application by-product.

Declarative architecture as code provides the executable realisation of the design and a reproducible source of truth. Configuration is separated from implementation where practicable, delivery is automated and managed AWS services are preferred to bespoke infrastructure software. The discipline is “just enough code”: engineering effort focuses on building differentiated domain logic (the business rules and data services unique to higher education) rather than undifferentiated infrastructure.

Stateless execution, asynchronous communication, purpose-built persistence and independently recoverable components avoid a tightly coupled failure domain. Consumption-based services align expenditure with workload demand instead of maintaining an estate permanently sized for intermittent peaks.

Solution overview

The architecture comprises five layers: edge, ingestion, persistence, application and business logic, and interface. Each layer has discrete responsibilities, as shown in Figure 1.

Identity, encryption, network controls, observability and operational governance apply across the stack. Curriculum data services capture complete changes from the Curriculum Management System as events, preserve them in a durable event log, materialise purpose-built representations and expose governed synchronous and asynchronous services.

Five-layer UTS curriculum data services architecture connecting the Curriculum Management System through the edge, ingestion, persistence, applications, and interface layers to consumers, with observability and security spanning all layers

Figure 1: UTS curriculum data services layered architecture

The next five sections examine each layer of the realised architecture in turn.

Edge layer

The edge layer is the controlled ingress boundary between the SaaS Curriculum Management System and the curriculum domain. Its remit is deliberately narrow: receive a complete curriculum change, establish the internal event contract and submit it to the ingestion layer with minimal processing.

The source system sends change events to an Amazon API Gateway HTTP API, which invokes an AWS Lambda function acting as a technical adapter because no suitable managed connector satisfies the source system’s proprietary API contract and payload format (for example, vendor-specific REST endpoints, authentication schemes, and data structures). The adapter encapsulates the complete vendor event in a CloudEvent envelope, appends UTS operational metadata and publishes it to the appropriate Amazon Managed Streaming for Apache Kafka (Amazon MSK) Serverless topic.

The edge neither reinterprets curriculum nor performs validation or consumer-specific transformation. Validation remains within the system of record. This preserves source fidelity, prevents downstream concerns from contaminating the ingress boundary and allows the stateless edge to scale automatically during complete domain reloads.

Ingestion layer

Amazon MSK Serverless forms the durable, append-only event backbone. Curriculum changes are segregated into domain-aligned topics by academic item type, retaining the complete source event whilst allowing independent consumers to progress, retry and recover without obstructing one another.

MSK Serverless scales on demand. Routine changes consume little capacity. A domain reload fans out across the event backbone and parallel processors, then contracts after completion. Model or schema changes can be accommodated through idempotent reprocessing or a complete reload, without introducing a second curriculum authority.

Persistence layer

The persistence layer uses purpose-built data services selected according to the access patterns and business questions each representation must support. Amazon Neptune Serverless stores curriculum structures and inverse relationships as a property graph, whilst Amazon OpenSearch Service stores complete source documents and indices for exact, full-text and vector retrieval.

In Neptune, programmes of learning, areas of learning and units of learning are connected academic items. Consumers can traverse from a programme to its constituent structures or begin with a unit and identify every programme and area through which it is taught. These inverse enquiries are native graph traversals rather than application-specific joins repeatedly reconstructed by consumers.

Property graph in Amazon Neptune showing programme, major, and subject nodes connected by TEACHES and IS_TAUGHT_BY relationships

Figure 2: Curriculum structures relationships in Amazon Neptune

OpenSearch provides exact lookup, filtering and document retrieval alongside k-nearest-neighbour vector search. Selected data processors implemented as AWS Lambda functions invoke an embedding model through Amazon Bedrock and append vector representations of relevant curriculum text before persistence. Curriculum can consequently be discovered by semantic proximity rather than lexical coincidence. UTS also validated multilingual retrieval against English-language curriculum content.

Neptune and OpenSearch are independently and eventually consistent. Failure of one materialisation doesn’t block another, and neither store is exposed directly to consumers. Both are disposable projections rather than competing systems of record and can be reconstructed from the Curriculum Management System through a complete reload. Data is encrypted with a dedicated domain AWS Key Management Service (AWS KMS) key whose use is restricted to authorised domain services.

Applications layer

The applications layer converts those materialisations into reusable domain capabilities. Dozens of narrowly scoped Lambda-backed services query Neptune or OpenSearch, whilst persistence remains private and is exposed solely through governed contracts.

Amazon EventBridge Pipes acts as a set of transport adapters between MSK Serverless and the Lambda functions acting as processors and services consuming the event stream. Pipes provide routing and delivery, not curriculum validation, enrichment or business transformation, keeping processing responsibilities explicit, independently deployable and observable.

Stateless Lambda function execution allows services to scale with demand and recover independently through retries and re-invocation. Signature Version 4 (SigV4) authentication and fine-grained AWS Identity and Access Management (IAM) roles provide least-privilege access to Amazon Neptune and Amazon OpenSearch Service without distributing or managing database credentials. The services provide authoritative retrieval, graph traversal and semantic discovery without transferring responsibility for curriculum validity into the data domain.

Interface layer

The interface layer provides pull-based access through REST and GraphQL and push-based delivery of domain events, whilst keeping persistence and implementation private. Amazon API Gateway supplies the REST boundary. AWS AppSync provides GraphQL access to frequently consumed capabilities with attribute-based access control at schema level. Both interfaces invoke the same governed service contracts.

Approved consumers might also receive curriculum events in the internal CloudEvent envelope. Amazon EventBridge Pipes manages last-mile delivery without bespoke routing middleware, and consumers determine which academic item events are relevant rather than UTS maintaining a filtered stream for each consumer.

Amazon Cognito authenticates approved external consumers through OAuth 2.0 and OpenID Connect, whilst AWS native workloads use IAM. AWS Key Management Service, AWS Secrets Manager, AWS WAF, and AWS Certificate Manager provide encryption, secrets management, perimeter protection and certificate lifecycle management. Amazon Route 53 supplies naming for a single shared vanity domain, and Amazon CloudWatch and Amazon Simple Notification Service (Amazon SNS) provide observability, telemetry and targeted alerting across the architecture.

Full solution architecture

The following diagram shows the complete solution architecture, including the identity, encryption, network, observability and governance controls that apply across all five layers.

End-to-end architecture of the UTS curriculum data services solution across the edge, ingestion, persistence, application, and interface layers, with the security, operational, and ancillary services that span them

Figure 3: End-to-end architecture of the curriculum data services solution

Results and measurable outcomes

The outcome isn’t a replacement for the Curriculum Management System as the system of record. It’s a governed data foundation that leaves the system of record authoritative whilst making curriculum information reusable through APIs, graph traversal, semantic search and event delivery. Since the production launch in November 2024, the domain has recorded zero production incidents.

  • Latency – Changes become available to downstream consumers in an average of 1.6 seconds from crossing the edge layer, through synchronous and asynchronous interfaces.
  • Curriculum graph – The domain models 43,794 relationships across approximately 800 programs of learning, 3,500 units of learning and 2,500 areas of learning.
  • Consumer adoption – 11 institutional consumers have adopted the services, replacing bespoke point-to-point integrations with reusable, governed contracts.
  • Operational footprint – One engineer operates the domain. Managed, serverless and self-healing components make routine intervention negligible and remove the need for dedicated Level 3 support.
  • Cost efficiency – UTS achieved a material reduction in the total cost of ownership of curriculum data integration and delivery, consolidating capabilities previously spread across commercial products, bespoke integrations, databases and multiple support teams.
  • Availability and resilience – The platform operates continuously and has maintained downstream access to trusted curriculum information when the upstream system of record was unavailable.

Collectively, these outcomes show how a small engineering contingent can operate an institutional-scale data domain using managed AWS services, whilst increasing capability, reducing operational complexity and aligning expenditure with demand.

What’s next

The curriculum domain validated the production architecture and operating model. The student data domain followed, providing a single governed point of access for student profiles, admissions, offers, enrolments and related information. UTS is now preparing the learning activities domain, a substantially larger capability built around a lifelong learner graph using Amazon Neptune and Amazon Neptune Analytics, extending the same domain-driven pattern across the teaching and learning landscape.

Each bounded context can select the AWS services appropriate to its business questions whilst retaining common controls for identity, security, observability, delivery and operational sustainability.

Conclusion

UTS’s integration landscape had reached the point at which cumulative complexity and operational overhead constrained institutional change. The response was not technology for its own sake. UTS ITU used AWS to remove the impediments between authoritative institutional data and the teaching, learning and administrative capabilities that depend upon it.

The significance of AWS extended beyond technology selection, but the outcome was principally an architectural and operating-model achievement by UTS. Curriculum data services became UTS’s first production data domain. UTS combined domain-driven design, asynchronous processing, purpose-built persistence and governed interfaces with managed AWS capabilities that made those patterns practicable at institutional scale.

The resulting service foundation supports event streaming, graph traversal, document and semantic retrieval, compute, identity, security and governed data exposure. UTS operates it without a conventional infrastructure estate. This allowed UTS ITU to convert an ambitious architectural design into a sustainable production capability whilst keeping engineering effort concentrated on domain services rather than undifferentiated infrastructure.

Curriculum data services embody the resulting transition: from a burdensome integration model to a domain-aligned data foundation that’s reusable, resilient and operationally sustainable. UTS now possesses a repeatable model through which institutional data can be treated as a strategic asset rather than a by-product of individual systems. The university will continue to operate many systems. Students and staff should never experience the boundaries between them.

For more information or support, contact the AWS Public Sector team.

About the authors

The Future of AI Cryptanalysis

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/10/the-future-of-ai-cryptanalysis.html

Matthew Green has been writing a rather pessimistic series of posts about the potential for AI systems to vastly outperform humans in cryptanalysis of both symmetric and public-key ciphers.

I am more optimistic than he is. That is, I have stronger faith in the over-engineering cryptographers tend to do. I am trying to write this up in more detail.

Cyberattack on Connected Fridges in France

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/10/cyberattack-on-connected-fridges-in-france.html

Another IoT cyberattack, this one in France:

Hackers gained control of the software that regulates the temperatures of the refrigeration units, forcing some to warm to 50C (120F) overnight, according to one supermarket chain affected.

[…]

Normally, supermarket freezers go through a short period of defrosting overnight, in which temperatures rise slightly. But the hackers ordered the units to heat up and stay hot, although Danfoss did not confirm any reports about just how warm the fridges and freezers could become in that process.

Danfoss said the affected cold units were older models whose security software could not be updated to ward off a risk they had identified.

Any IoT device that can’t be updated is vulnerable. And, probably, any IoT device that can be updated is also vulnerable.

[$] Collaborating with out-of-tree tracers

Post Syndicated from jake original https://lwn.net/Articles/1099300/

LTTng creator Mathieu Desnoyers seems a
bit trapped in something of a catch-22.
At the 2026 Linux Plumbers Conference
(LPC), he led a discussion on the problem he is having in trying to
collaborate with other tracing projects (e.g. ftrace, perf) in the kernel. The
feedback he has gotten, both over the years and more recently, does not
show a path that would allow him to contribute the 21-year-old
GPLv2-licensed LTTng tracing framework, or, incrementally, features from
it, to the mainline kernel.

Linus Torvalds’s first LLM-assisted patch series

Post Syndicated from corbet original https://lwn.net/Articles/1100449/

At the recently concluded Maintainers
Summit, Linus Torvalds said that he had been using a large language
model to work on a patch series addressing a longstanding problem; he has
now posted
that series for review.

The fairly short series of ten patches basically enhances folio
lookup by noticing when folios are hot, and setting a “delay
releasing this folio until after a RCU period”, and then using that
RCU delay it does not just the folio lookup itself under RCU (which
we’ve done forever), but also the actual folio data accesses
themselves under RCU. No refcount taking needed for the fastpath
where we can just read the data from the folio entirely in an
RCU-protected region when the user copy doesn’t cause page faults.

The end result is an order-of-magnitude speedup for a read()-heavy
benchmark. Review has already identified a race
condition in the code, though, so further work will be needed.

KDE Plasma 6.8 released

Post Syndicated from jzb original https://lwn.net/Articles/1100429/

Version 6.8 of
the KDE Plasma desktop has been
released. The The KDE project has timed the release to coincide with the 30th
anniversary of Matthias Ettrich’s email announcing the
project:

Unix popularity grows thanks to the free variants, mostly Linux. But still a
consistent, nice looking free desktop-environment is missing. There are several
nice either free or low-priced applications available, so that Linux/X11 would
almost fit everybody needs if we could offer a real GUI.

Notable changes include accessibility improvements, inclusion
of the Kup backup
tool as part of Plasma, as well as the ability to include system audio and
microphone input in Spectacle
screen recordings. Plasma 6.8 also drops support for Plasma X11 sessions, as was announced
in November 2025. X11 applications are still supported via XWayland, and KDE
applications will continue to support X11 “for the foreseeable
future“.

See the KDE wiki for a
list of major user-facing changes, and the changelog
for all changes since Plasma 6.7.5.

Security updates for Wednesday

Post Syndicated from jzb original https://lwn.net/Articles/1100426/

Security updates have been issued by AlmaLinux (bind9.18, dovecot, flatpak, gd, jackson-core, jackson-databind, jackson-jaxrs-providers, and jackson-modules-base, kernel, opentelemetry-collector, pcre2, and tftp), Debian (curl and keystone), Fedora (aegisub, arm-none-eabi-binutils-cs, avr-binutils, chromium, curl, firefox, kernel, python-jupytext, and xdg-dbus-proxy), Mageia (apache, freeciv, gnumeric, mingw-gdk-pixbuf, and patool), Red Hat (cockpit-image-builder, fence-agents, firefox, ipa, kernel, kernel-rt, pcre2, python-tornado, qt6-qtdeclarative, thunderbird, and webkit2gtk3), Slackware (expat), SUSE (bind, busybox, containerd, cups-filters, fixme, gimp, glibc, google-osconfig-agent, kernel, libtcnative-2-0, openssh, openssl-3, php8, podman, python-oauthlib, python-PyJWT, python-urllib3, python-virtualenv, python311, python313-jupyter-server, python313-kubernetes, python313-logilab-common, python313-markdown-it-py, python313-markdown2, python313-marshmallow, python313-mpmath, python313-mysql-connector-python, python313-oci-sdk, python313-oic, unbound, wpa_supplicant, xen, and xorg-x11-server), and Ubuntu (kernel, libyaml-perl, linux, linux-aws, linux-aws-fips, linux-aws-hwe, linux-azure, linux-azure-4.15, linux-azure-fips, linux-fips, linux-gcp, linux-gcp-4.15, linux-gcp-fips, linux-hwe, linux-kvm, linux, linux-aws, linux-aws-fips, linux-fips, linux-gcp, linux-gcp-fips, linux-gke, linux-gkeop, linux-ibm, linux-lowlatency, linux-oracle, linux-realtime, linux, linux-aws, linux-azure, linux-azure-5.4, linux-fips, linux-gcp, linux-gcp-5.4, linux-gcp-fips, linux-hwe-5.4, linux-iot, linux-kvm, linux-oracle, linux-oracle-5.4, linux-raspi, linux-raspi-5.4, linux-xilinx-zynqmp, linux, linux-aws, linux-fips, linux-kvm, linux-lts-xenial, linux-aws-5.4, linux-aws-fips, linux-azure-fips, linux-ibm, linux-ibm-5.4, linux-azure, linux-azure-fde, linux-azure-fips, linux-oracle, and pyjwt).

Inside Hyperdrive: How we solved a rare Hyperdrive MySQL caching bug

Post Syndicated from Pranoy Dutta original https://blog.cloudflare.com/hyperdrive-mysql/

Hyperdrive support for MySQL is now generally available (GA). Hyperdrive, Cloudflare’s database connection pooler for Workers, initially only supported Postgres. Since we announced the beta for MySQL protocol support last year, we have been working on making our MySQL handling as reliable as our Postgres support.

We’ve learned some things from building and running a MySQL connection pooler on Cloudflare’s network. It turns out that the difference between beta and GA is just 20 bug fixes in a trenchcoat.

During the beta, our goal was to validate that real-world customer usage hit our production availability target of 99.99%. We knew there were gaps in our reliability since we track how often client connections from a Worker end in an error related to Hyperdrive. In addition, we had feedback from customers in the beta that led to the discovery of a rare and subtle bug in our production environment. To convey the complexity of this particular bug, we first need to explain how Hyperdrive works.

What is Hyperdrive?

Hyperdrive is a distributed database connection pooler that connects Cloudflare Workers to SQL databases. Hyperdrive holds warm SQL connections open on our global network and shares these connections across many Worker requests. In addition to connection pooling, Hyperdrive can also cache common queries in our global CDN cache, reducing latency and load on the origin database.

Establishing a connection to a SQL database is expensive, taking a minimum of five network round trips. On a serverless compute platform like Workers, establishing a connection for each request would be impractical, both because of the amount of latency added to each request, but also because a large number of open connections can incur high per-connection resource costs. While MySQL fares better here than Postgres since MySQL doesn’t use an entire OS process per connection, a typical deployment can still run into resource issues with a large number of connections.

For more details, see our blog post introducing Hyperdrive.

A journey to stability

Some of the issues we ran into were small and easy to fix once reported. One of the first problems reported in the beta was that emojis written to the database were not being correctly returned. This was a bug related to character set encoding. Hyperdrive negotiates the encoding for the connection in the startup handshake and we negotiate utf8 as the encoding. As it turns out, utf8 is a deprecated alias for utf8mb3 which is not all of UTF-8: it only supports 1-3 bytes per Unicode code point, while full UTF-8 has a max of 4 bytes per code point. To get all of UTF-8, we needed to change our handshake to use utf8mb4 , which fixed the emoji issue.

Another encoding-related bug occurred when we added connection cleanup via COM_RESET_CONNECTION. This is a lightweight MySQL command to clear any session state in a given connection, which is exactly what Hyperdrive needs to ensure that no session-level state leaks between checkouts. But adding this connection cleanup step caused encoding issues once again! It turns out that COM_RESET_CONNECTION does not reset the connection state to the character set encoding we negotiated at connection startup; it instead resets the connection to the server default. This means that after negotiating utf8mb4 on a MySQL server instance that defaults to latin1, once we run COM_RESET_CONNECTION, our connection now has an encoding of latin1. The fix is to always run a command setting the encoding to utf8mb4 after a connection reset.

A hard-to-find bug

During the beta, our highest-volume MySQL customer reported a small but consistent rate of empty query responses for which a non-empty response was always expected. We started the investigation by combing through all of our telemetry and understanding the user’s logic. Then we also increased the trace rate for this Hyperdrive instance, hoping to see the execution flow of this bug. We searched for network issues that could explain the issue — but nothing seemed to account for this behavior. This error occurred about 0.001% of the time, which made it very hard to find relevant end-to-end execution traces.

But we did have one clue: the bug only occurred on Hyperdrive instances with caching enabled. This made us suspect an issue with Hyperdrive’s MySQL caching.

Hyperdrive caching

To understand Hyperdrive's caching, we first need to talk about Hyperdrive's topology and how it interacts with the CDN cache. Every Hyperdrive instance has a split architecture with two nodes: a Client and an Endpoint. The Client node runs on the same machine as the user’s Worker, handles local caching (if enabled), and forwards queries to the Endpoint if needed. The Endpoint node holds the connection pool and is placed in the data center closest to the origin database. Each node uses the Cloudflare CDN cache local to the data center it is running in. Importantly, this means that if the two nodes are running in the same data center, they will share a cache.

Sharing the cache between Client and Endpoint is generally good for performance. We use a hash of the contents of the query’s wire messages with some normalization and store the result bytes in the cache. If a Hyperdrive Client running in the same data center as its Endpoint receives a request, and that Endpoint has recently cached the results of the same query, the Client can serve the response directly from the shared data center cache. To work correctly, this scheme requires that the same bytes are cached for the same response, whether stored by the Client or the Endpoint.

MySQL client/server protocol

Since we couldn’t get a direct trace of the issue, we decided to spend time looking at the MySQL client / server protocol. Perhaps we had an assumption about Postgres’ protocol built into our MySQL handling that was causing these issues.

The protocol describes how a MySQL client communicates with a MySQL server. This can be somewhat confusing in the case of database connection pooling since we have two components acting like clients and two acting like servers. The first client is the Worker using a MySQL client library, and it connects to the first component acting like a server, the Hyperdrive Client. The Hyperdrive Client connects to the Hyperdrive Endpoint which acts as a MySQL client connecting to the actual MySQL origin server.

This protocol starts with the Connection Phase where the MySQL client and the MySQL server agree upon a common authentication method and capabilities. Once a connection is established, the client and server begin the Command Phase, where the client sends commands (e.g. SQL statements) to be executed with the server responding and waiting for the next command.

Capabilities are exchanged during the Connection Phase to support protocol evolution while remaining backwards compatible. Each capability is a feature, like supporting compression during the Command Phase. The clients and servers must only use features which the other supports, otherwise a protocol desync can occur, leading to unexpected behavior.

This capability negotiation can be tricky for Hyperdrive, since Hyperdrive acts as a client to the origin database, and acts as a server to the Worker. To ensure that Hyperdrive can parse the messages from the origin MySQL server, Hyperdrive has a fixed set of capability flags which are used to communicate to the origin database. Since it is possible for Hyperdrive’s set of capabilities to be different from the Worker client’s set of capabilities, Hyperdrive needs to translate all origin responses encoded by its capabilities to the capabilities that the Worker expects.

While trying to debug this issue, we found that a popular MySQL client library for Node.js (mysql2) does not currently support the CLIENT_DEPRECATE_EOF capability, which changes how result sets are terminated. Without this capability, the server sends an EOF packet between the column definitions and rows, then another EOF packet at the end of the result set. When CLIENT_DEPRECATE_EOF is negotiated, the metadata EOF packet is omitted and the final EOF packet is replaced with an OK packet. MySQL server 5.7.5 and later supports this capability, but the newer OK-packet behavior is only used when the client negotiates it.

While Hyperdrive does support the CLIENT_DEPRECATE_EOF capability, it’s possible that the Worker’s MySQL driver does not. To handle this, Hyperdrive adjusts its behavior to support the capabilities that the Worker client driver expects. This means rewriting result sets to include EOFs and to translate all OK packets to EOFs when required. 

After days of looking through telemetry, documentation, and Hyperdrive code, everything clicked. We finally came across the last missing piece: a faulty assumption in our Hyperdrive Client caching code. We baked in an assumption when implementing the Client caching for Postgres: that we can cache exactly the bytes we wrote back to the Worker. While this was a reasonable assumption to make with Postgres’ wire protocol, it no longer holds due to the mismatch in MySQL capabilities. If a Client node is caching exactly what it writes back to the Worker, these bytes are in a different format than what an Endpoint node would be expecting. That’s because the Endpoint node is always working with Hyperdrive’s capabilities, while the Client node writes back bytes that are translated to match the Worker’s capabilities.

Now if we put all these clues together, we can understand scenarios where Hyperdrive can return a MySQL response encoded with the wrong capabilities, leading to the unexpected empty results and hangs. The bug occurs when both the Client and Endpoint are in the same data center, sharing a cache, and when one reads a value cached by the other.

Here’s a closer look at two such scenarios.

Endpoint caches, Client reads

  1. The Endpoint is in the middle of servicing a request for query Q. It gets back a response R from the origin server and caches response result bytes R_hyperdrive (Response encoded with Hyperdrive capabilities) into the shared cache.
  2. Client receives another request for query Q, checks the shared cache and finds R_hyperdrive.
  3. Client responds to the Worker with R_hyperdrive without translating to the Worker’s capabilities.

This diagram shows that flow:

Client caches, Endpoint reads

  1. Client is finishing serving a request for query Q, and it writes the translated response bytes R_worker to cache.
  2. Endpoint is in the middle of servicing a request for Q, and checks the cache to find R_worker.
  3. Endpoint sends R_worker to a different Client, and this Client tries to translate R_worker to the Worker’s capabilities, and ends up sending an early EOF or hangs.

You can see that here:

Once we found this type of failure mode, the fix was simple. We now make sure that we always write back a canonical version of the query result bytes in the cache encoded using Hyperdrive’s capabilities, no matter which node stored it. After releasing this change, the empty row problem went away — and our success rate for Client connections reached 99.99%.

Reliability is a feature

You can use Hyperdrive support for MySQL today! Create a Hyperdrive to an existing MySQL-compatible database via the dashboard or wrangler, or connect to a new PlanetScale database. If you run into any issues or have questions, reach out on our Discord. 

npx wrangler hyperdrive create HYPERDRIVE_NAME --connection-string="mysql://user:[email protected]:3306/dbname"

From Backup to Assurance: Monitoring Veeam with Zabbix

Post Syndicated from Michael Kammer original https://blog.zabbix.com/from-backup-to-assurance-monitoring-veeam-with-zabbix/33694/

Your data isn’t truly protected if you don’t know when that protection fails. Today’s organizations depend on backup infrastructure to safeguard critical workloads, yet a single backup failure, dwindling repository capacity, expired license, security incident, or infrastructure issue can quietly undermine that protection. The key isn’t just having backups – it’s knowing, as quickly and reliably as possible, when something has gone wrong.

Simply having a backup solution in place is not enough. IT teams need continuous visibility into health and operational status. That’s where Zabbix and Veeam come together.

With the latest integrations for Zabbix 8.0 and Veeam Backup & Replication v13, organizations can bring Veeam backup monitoring into their broader infrastructure monitoring strategy, gaining centralized visibility, proactive alerting, and a clearer picture of whether their data protection environment is ready when it is needed. Veeam protects your data. Zabbix makes sure you always know when that protection is at risk.

Two ways to monitor your Veeam environment

Zabbix provides integrations for monitoring Veeam environments through two approaches:

  • Veeam Backup & Replication by HTTP
  • Veeam Backup Enterprise Manager by HTTP

Both approaches extend Zabbix monitoring into the backup environment, helping teams detect operational and security issues before they become business-impacting incidents.

The Veeam Backup & Replication by HTTP integration provides direct monitoring capabilities for Veeam environments and offers administrators detailed visibility into backup operations and the infrastructure that supports them.

The Veeam Backup Enterprise Manager by HTTP integration is an alternative for organizations that use Veeam Backup Enterprise Manager to centrally manage multiple Veeam Backup & Replication deployments. It uses the Enterprise Manager REST API to bring information from multiple deployments into Zabbix.

It’s important to note that Enterprise Manager is an additional Veeam component and requires a separate installation. For organizations that already have Enterprise Manager deployed, its centralized architecture can make it a valuable option for bringing multiple Veeam environments into a single Zabbix monitoring view.

Veeam Backup & Replication by HTTP: Direct visibility into your backup environment

The Veeam Backup & Replication integration focuses on the operational health of the backup infrastructure and the results of backup activities. With Zabbix, administrators can monitor important components and events, including:

  • Backup job status and execution results
  • Backup sessions and failures
  • Backup repositories and available capacity
  • Backup proxies and managed servers
  • License status and expiration
  • Security Analyzer findings
  • Malware detection events
  • Authentication and security-related events

NOTE: License, Authorization Events, and Security Analyzer Data require the “Backup Administrator” role; without it, these items are not supported.

This means backup monitoring becomes part of the same monitoring workflow used for servers, applications, networks, storage, and other critical infrastructure. Instead of discovering a failed backup when a recovery is already required, teams can receive an alert as soon as the failure occurs.

Veeam Backup Enterprise Manager by HTTP: Centralized observability across deployments

For organizations managing multiple Veeam Backup & Replication deployments, Veeam Backup Enterprise Manager offers another way to centralize backup information. This Zabbix integration communicates with Enterprise Manager through its REST API and provides centralized monitoring and alerting across multiple Veeam Backup & Replication environments. This gives administrators visibility into areas like:

  • Backup infrastructure health
  • Backup job status
  • Repository utilization
  • Operational events
  • Job run results
  • VM protection status

The result is a consolidated view of backup operations from a single Zabbix dashboard. The integration also works without external scripts and uses a Zabbix script item, helping simplify deployment and ongoing maintenance. This means that Enterprise Manager can be especially useful when an organization already operates it as a centralized management layer for multiple Veeam deployments.

Why monitor Veeam with Zabbix?

Veeam is responsible for protecting data. Zabbix adds a monitoring and alerting layer that helps ensure the protection itself remains healthy. This distinction is important!

A backup job can exist without actually providing reliable protection. Repositories can run out of capacity. Backup jobs can fail repeatedly. Infrastructure components can become unavailable. Licenses can approach expiration. Security-related events can require immediate attention.

Without proactive monitoring, these conditions can remain unnoticed until they have serious consequences. By integrating Veeam into Zabbix, organizations can continuously monitor these conditions and incorporate them into their existing operational processes.

A single pane of glass. IT teams often manage complex environments containing physical and virtual servers, cloud services, networks, storage, applications, and backup infrastructure. Bringing Veeam into Zabbix means that backup operations can be monitored alongside the rest of the environment rather than through an isolated monitoring workflow. This provides a single pane of glass for infrastructure and backup operations and gives administrators a more complete picture of overall service health.

Immediate detection of backup failures. A successful backup is one of the most important indicators that data protection is working as expected. Zabbix can alert teams when backup jobs or sessions fail, allowing them to investigate while there is still time to resolve the problem and complete a successful backup. This changes backup monitoring from a reactive process into a proactive one.

Proactive capacity monitoring. Backup repositories require sufficient capacity to continue protecting data. Monitoring repository utilization and available capacity allows administrators to identify trends and potential capacity problems before they cause backup failures. Instead of reacting to a full repository, teams can plan capacity increases or optimize retention policies ahead of time.

Faster troubleshooting. When something goes wrong, knowing that a backup failed is only the first step. Monitoring the surrounding infrastructure and Veeam components in the same platform can provide valuable context. Zabbix can help teams correlate backup-related events with infrastructure conditions, making it easier to identify potential causes and accelerate troubleshooting.

Reduced recovery risk. The real value of a backup becomes apparent when an organization needs to recover data.
A continuously monitored backup environment provides greater confidence that backup jobs complete successfully, repositories have sufficient capacity, and critical components remain operational. Proactive monitoring, therefore, directly reduces recovery risks and improves both business continuity and disaster recovery readiness.

Security-aware monitoring. Modern backup environments are also an important part of an organization’s security strategy.
Monitoring Veeam security-related information, including malware detection events, Security Analyzer findings, and authentication or security-related events, helps teams identify conditions that may require immediate investigation. Combining these signals with the rest of the organization’s monitoring data can provide additional context when responding to potential threats.

Zabbix 8.0 and Veeam Backup & Replication v13: Keeping the integration current

The latest integration updates for Zabbix 8.0 and Veeam Backup & Replication v13 make it easier for organizations running current versions of these platforms to incorporate Veeam into their monitoring strategy. Keeping integrations aligned with current platform versions is important for organizations that want to modernize their infrastructure while maintaining consistent monitoring and alerting.

Whether you’re observing a single Veeam deployment directly through Veeam Backup & Replication or using Enterprise Manager to centralize multiple deployments, the goal is the same – to
make the state of your backup environment visible and actionable.

From backup monitoring to backup assurance

The biggest benefit of monitoring Veeam with Zabbix is not simply having more metrics or dashboards. It’s the peace of mind that comes from knowing that your data is fully protected at all times. After all, a backup system shouldn’t be something that IT teams think about only when they need to restore data. Its health should be continuously visible, and potential problems should generate alerts before they become emergencies.

Together, Veeam and Zabbix provide a proactive approach to backup monitoring. Veeam protects your data, and Zabbix helps you verify that the protection remains healthy. By combining backup monitoring with broader infrastructure observability, organizations can detect failures earlier, respond faster, plan capacity more effectively, and increase confidence in their business continuity and disaster recovery readiness.

For organizations running Veeam, monitoring the backup environment should be an essential part of the protection strategy, because knowing that your backups are working is just as important as having backups in the first place.

Visit Zabbix to find out more about open source observability, and check out Veeam’s website to discover data protection and resilience solutions.

The post From Backup to Assurance: Monitoring Veeam with Zabbix appeared first on Zabbix Blog.

На второ четене: „Горски цар“

Post Syndicated from Стефан Иванов original https://www.toest.bg/na-vtoro-chetene-gorski-tsar/

„Горски цар“ от Мишел Турние

На второ четене: „Горски цар“

превод от френски Изабела Георгиева, София: изд. „Парадокс“, 2024

Има книги, които човек препрочита, и книги, които препрочитат него. „Горски цар“ на Мишел Турние принадлежи към втория вид. За първи път го прочетох в гимназията, на онази възраст, когато всеки роман е или откровение, или скука, и той се оказа много важен за мен с някаква смущаваща, почти неприлична настойчивост. После, по инерцията на заразата, изчетох всичко останало на Турние на български и на френски – от „Петкан“ до „Метеорите“ и „Златната капка“. Днес възприемам това като един от решаващите етапи в читателското си развитие, врата с надпис, който се разчита едва след като си я затворил зад себе си. Новото издание на „Парадокс“ в превода на Изабела Георгиева е добър повод за второ четене, което по дефиниция е и разследване на първото.

Сюжетът изглежда като биография, сглобена от случайности, и точно в това е капанът. Абел Тифож, собственик на автосервиз на площад „Порт дьо Терн“, пише своите „Опаки записки“ с лявата си ръка след злополука с манивелата. Заглавието е решение на преводачката, което запазва двусмислието на френското sinistre – едновременно ляво и зловещо. Детството му в колежа „Свети Христофор“ е белязано от Нестор, чудовищно развито дете, което го посвещава в тайнописа на знаците. Обвинението в посегателство над момиченцето Мартина го изпраща към съда. Войната от 1939 г. идва като deus ex machina, която го спасява и отнася. Следват гълъбите на Рейн, пленът в Източна Прусия и ловните гори на Роминтен, където Гьоринг царува като карикатурен людоед сред трофеите си. Накрая идва Калтенборн – националполитическа школа за елитни момчета. Там Тифож, облечен в пълната си митологическа униформа, обикаля околните села на кон и събира деца за Райха. Главите се казват „Роминтенският човекоядец“ и „Калтенборнският човекоядец“ и не хитруват.

Сърцевината на романа е понятието фория, носенето. Тифож открива екстаза на тялото, което носи друго тяло.

Около него изгражда цяла теология, опряна на свети Христофор, пренесъл Христос през реката, и на баладата на Гьоте, в която бащата носи сина си през нощта, а Горският цар го отнема. Турние, възпитаник на Башлар и слушател на Леви-Строс, строи символна машина с часовникарска прецизност. Всеки знак има своя обратен образ и благословията се среща със своята злокачествена инверсия. Носенето спасява и поглъща. Роговете на елена са едновременно триумф и огледален фалос. Дори името на героя е шифър. Тифож е замъкът на Жил дьо Ре, спътника на Жана д’Арк и детеубиеца, когото Юисманс описва в „Там долу“ и Жорж Батай го анализира в свое изследване.

Прочетен в контекста на цялото творчество на Турние, романът разкрива и своята генеалогия. В „Петкан“ той обръща Дефо и превръща Робинзон от колонизатор в ученик на дивака. В „Метеорите“ разиграва мита за близнаците като утопия на затвореното тяло. „Горски цар“ е най-тъмният панел в този триптих от пренаписани митове. Тук методът е подложен на най-тежкото изпитание, защото материалът му е историята, която не търпи алегория. Турние го знае и затова подковава фикцията с бележки под линия към Нюрнбергския процес, сякаш документът е призван да удържи мита да не отлети в чиста поезия.

Тук критическото четене трябва да бъде по-внимателно от възхищението. Силата на книгата и нейният проблем са едно и също нещо. Тифож чете света като йероглифи, изписани лично за него, и романът му дава право, защото съдбата наистина му намига.

Читателят е поканен да опознае една херменевтика, която логически съвпада с параноята и естетически съвпада с фашизма.

Сюзан Зонтаг пише своето есе „Очарователният фашизъм“ пет години след романа. Тя описва точно този култ към здравото тяло, ритуала и подчинението като еротика. Турние го показва отвътре, с лупа и без морален коментар и затова е неудобен. Педофилският подтекст се мярка като сянка, достатъчно ясна, за да я видиш, и достатъчно неназована, за да те направи съучастник в мълчанието. Финалът е изкупление с почти католическа хореография. Тифож носи еврейското момче Ефраим през блатото и потъва под тежестта му, докато над него се върти златна шестолъчна звезда. Красиво е, може би прекалено красиво. Символната система се затваря като ковчеже и изкушението да я приемеш като прошка е силно. При първото четене тази звезда ме разплака. При второто ме накара да се запитам кой всъщност е спасен – дали това е детето, героят или авторът.

На второ четене: „Горски цар“

Паралелите със световната литература се предлагат сами. „Тенекиеният барабан“ на Грас разказва същата епоха през погледа на детето, което отказва да порасне. „Горски цар“ я разказва през погледа на великана, който отказва на детето да порасне. „Доктор Фаустус“ на Томас Ман проследява как германският мит се превръща в катастрофа. Турние стига до същото, но с френска наслада от германския мит.

„Лолита“ на Набоков е най-близкият роднина по техника и отново е с чудовищен разказвач, чиято реторика е по-обаятелна от моралния ни рефлекс.

„Доброжелателните“ на Джонатан Лител, трийсет и шест години по-късно, повтарят жеста с есесовец вместо с френски пленник и показват колко далеч е отишъл Турние пръв. Над всички тях стои Перо, защото приказката за людоеда е първият текст, в който децата се броят като запаси. Шльондорф екранизира романа през 1996 г. с Джон Малкович и доказа неволно, че тази книга не понася добре буквалността на образа. Митът оцелява в изречението и умира в кадъра.

Днешният свят предлага на романа неочаквано добри условия за четене. Фразата на Тифож „Всичко на този свят е знак“ звучи като мото на алгоритмичната епоха. Всеки има персонален поток от знамения, скроени лично за него, и лесно се убеждава, че небето не е пусто. Апофенията („немотивирано виждане за взаимовръзка, съпровождаща се с характерното чувство за неадекватна важност“) вече е бизнес модел. Култът към младото тяло, униформата и строя се връща в новите войни, в които детските военно-патриотични организации отново маршируват в камуфлаж. Дигиталната икономика на вниманието е може би най-ефективният людоед, който човечеството е изобретило, защото отвлича децата, без да ги изнася от стаята им.

За българския читател Калтенборн звучи смущаващо познато. Пионерските лагери, чавдарчетата с връзките, бригадирските движения, спартакиадите с масовите гимнастически фигури бяха българската версия на форията. Колективното тяло беше носено от идеологията, която претендираше, че го обича. Днешните патриотични лагери с бойна подготовка и възторжената реторика за „здрав дух в здраво тяло“ показват, че тази хореография е оцеляла и е сменила единствено знамената.

Още по-точен е паралелът с Роминтен. Гьоринг, който се изживява като пазител на гората и убива в нея, има своя български двойник в социалистическите ловни стопанства. Има го и в днешните резервати, където властта продължава да трофеизира природата и да се снима до убития елен. Отделно, че страстта към знаците е почти национален спорт от пророчествата на Ванга до конспиративните канали в социалните мрежи. Почти всеки у нас е малко Тифож, убеден, че съдбата му намига персонално.

Второто четене прави очарованието на „Горски цар“ по-подозрително и затова по-ценно. Книгата, която в гимназията ми изглеждаше като тайно познание, днес ми изглежда като изпит по бдителност. Турние ни учи да четем знаците и едновременно с това ни показва какво става с онзи, който ги чете твърде добре. Постижение е да се напише красив, умен и опасен роман, срещу който читателят трябва да се защитава със същата интелигентност, с която му се наслаждава.

Никой от нас не чете единствено най-новите книги. Тогава защо само за тях се пише? „На второ четене“ е рубрика, в която отваряме списъците с книги, публикувани преди поне година, четем ги и препоръчваме любимите си от тях. За нея медията „Тоест“ е отличена с Националната награда „Христо Г. Данов“ (2025) за принос в представянето на българската книга.

Рубриката е част от партньорската програма Читателски клуб „Тоест“, благодарение на която активните дарители на „Тоест“ получават 20% отстъпка от коричната цена на всички книги на включените издателства. Изборът на заглавия обаче е единствено на авторите Стефан Иванов, Севда Семер и Антония Апостолова, които биха ви препоръчали тези книги и ако имаше как да се разходите с тях в книжарницата. 

Patch Tuesday – October 2026

Post Syndicated from Adam Barnett original https://www.rapid7.com/blog/post/em-patch-tuesday-october-2026

Microsoft is publishing 1,413 own-product vulnerabilities on October 2026 Patch Tuesday, including 1,102 vulnerabilities in Windows. Anyone who enjoyed last month’s record-setting Patch Tuesday will love the October 2026 edition, because it’s much bigger still. As Rapid7 has noted over the course of 2026, there is every reason to suppose that vulnerability volumes will continue to rise. Microsoft is aware of exploitation in the wild for just one of the vulnerabilities published today, as well as public disclosure for two others.

Windows Shell: zero-day security feature bypass

The Windows Shell is Microsoft’s term for the GUI interaction logic for the entire OS provided by explorer.exe and associated libraries and APIs, and for the second time this year, the Windows Shell is the site of an exploited-in-the-wild zero-day vulnerability. Today’s specimen is CVE-2026-96951. Once again, the immediate prize for an attacker is a Mark of the Web (MOTW) bypass. What this means is that Windows won’t provide the user with a dialog explaining that Windows protected their PC, because in this case, it didn’t. Exploitation requires that a user opens a specially crafted malicious file or visits an attacker-controlled site, so the evergreen advice about being careful what you click remains valid. Fans of vintage operating systems should note that Windows Server 2012 doesn’t receive a patch, although Server 2012 R2 does. The advisory acknowledges reporters from multiple organizations, although none from within Microsoft. The vulnerability also affected Microsoft’s Outlook.com cloud services, although Outlook.com was patched in a build dated several weeks ago.

.NET Framework: zero-day security feature bypass

Microsoft is aware of some level of public disclosure of CVE-2026-108398, a .NET Framework security feature bypass. Exploitation requires an attacker with an existing foothold to register a malicious .NET assembly for loading by a protected process, bypassing Windows Defender Application Control (WDAC) enforcement. Microsoft rebranded WDAC as App Control for Business several years ago, so you may know it under its new name. The Microsoft Learn page for Application Control for Windows points out that “your organization’s data is one of its most valuable assets… and threat actors want it”, and this is, of course, correct. The patch presumably removes an attacker-controlled location from the list of places .NET Framework searches for registered assemblies, which are typically contained within DLLs and executables. If any further indications were needed that vulnerability volumes are increasing industry-wide, this is only the second month that Microsoft has published vulnerabilities with six-digit CVE numbers, but these extra-large CVEs will seem unremarkable from now on.

Outlook: multiple no-interaction RCEs

The sheer volume of vulnerabilities in recent Patch Tuesday releases can easily include several examples of potentially concerning vulnerabilities, each of which might have been more remarkable in its own right back when volumes were lower. Today, Microsoft has published five separate remote code execution vulnerabilities in Outlook where the preview pane is a vector, meaning that simply scrolling past an email in the Outlook reading pane would be sufficient to trigger exploitation. Worse still, Outlook might helpfully display the preview without any user interaction at all. All five of CVE-2026-70242, CVE-2026-97749, CVE-2026-97016, CVE-2026-88156, and CVE-2026-70291 receive fixes in the same patches. No-interaction Outlook preview pane vulnerabilities can sometimes feature prominently in state-backed actor campaigns, with CVE-2023-23397 as a prominent example in this category.

Edge: security updates unclear

In September 2026, Microsoft did not appear to publish any desktop browser security advisories between the start of the month and Patch Tuesday. In October, a slightly different approach has emerged: several recent entries in the Edge security updates release notes mention new versions of Edge which incorporate the latest updates from the Chromium project, but without detailing which vulnerabilities have been fixed, stating instead, “CVE’s will be added as soon as available,” and directing readers to the Security Update Guide (SUG). At the time of writing, the SUG lists a total of zero vulnerabilities addressed in Edge in the first 13 days of October, and the “CVE’s added soon” message has been up for over a week. As a result, defenders have no obvious way of knowing which recent vulnerabilities Microsoft has addressed in Edge.

Microsoft lifecycle update

In Microsoft product lifecycle news, today sees Windows 11 24H2 Home & Pro reach end of servicing, while Windows Server 2022 moves to extended support, with free critical security updates continuing but no further feature development. Also this month, the final curtain falls for Windows Server 2012 and 2012 R2 with the expiry of the third and final year of cash-for-updates Extended Security Update (ESU) program for these aging workhorses. Microsoft lists the end of ESU Year 3 on October 14, 2026, in both cases. Microsoft has a long history of providing at least some security patches for products beyond the end of support, with Server 2008 being a case in point, but the time to move past Server 2012 is now. Office 2021 also moves into retirement on October 14; this includes the Long-Term Servicing Channel, with no ESU available in that case.

Also on October 15, 2026, Exchange Server 2016 and Exchange Server 2019 finally join the “no ESU” club, after two previous six-month reprieves. Microsoft seems to really mean it this time, since any future Exchange Server CVEs released by MSRC will stop mentioning Exchange 2016 and 2019. However, Microsoft points out that a lack of warnings does not mean those unsupported versions are not vulnerable to future security issues.

Summary charts

2026-10-vuln_count_component.png

2026-10-vuln_count_impact-component-heatmap.png

2026-10-vuln_count_impact.png

Scope workloads and share resilience policies with AWS Resilience Hub

Post Syndicated from Sinisa Mikasinovic original https://aws.amazon.com/blogs/architecture/scope-workloads-and-share-resilience-policies-with-aws-resilience-hub/

A resilience plan is useful only while it reflects the application you run. After a migration or deployment, a service can acquire a new dependency while its documented recovery objectives stay the same. In a shared Kubernetes cluster, even deciding which resources belong to that service can take investigation.

On September 18, 2026, the Next generation of AWS Resilience Hub introduced three capabilities to help you keep workload scope and resilience requirements aligned with your running environment:

  • Amazon Elastic Kubernetes Service (Amazon EKS) label support for service input sources selects Kubernetes resources using the labels that already describe your workloads.
  • Generative AI-powered Dependency Insights analyzes a service’s discovered dependencies and highlights the patterns worth reviewing.
  • Resilience policy sharing through AWS Organizations lets a central team define resilience requirements once and make them available to member accounts.

In this post, you follow a team reviewing the resilience of a sample workload named checkout on Amazon EKS, representing an online store’s checkout service. The team starts by defining which Kubernetes resources belong to checkout, using labels to distinguish it from inventory in the same namespace. This establishes the resource boundary for the resilience review.

With that boundary defined, the team examines discovered dependencies and uses Dependency Insights to prioritize findings for investigation. A cross-Region dependency becomes a concrete question about whether checkout’s recovery design accounts for the services it requires to operate. The service owner then works with a central resilience team to validate the recovery requirements and apply a shared policy. Maintaining those requirements centrally gives teams a common definition to adopt across accounts, while policy events help the central team track adoption.

You perform these steps using the console or the AWS Command Line Interface (AWS CLI). The workload generates synthetic traffic to demonstrate dependency discovery.

About the application model: In the Next generation of Resilience Hub, the primary unit of assessment is a service. Services can be grouped into systems, and user journeys describe paths through them. See the key changes and the overview of the new hierarchy in the Next generation of Resilience Hub.

Prerequisites

To follow the examples, you need:

  • Access to the Next generation of AWS Resilience Hub in a supported AWS Region.
  • An Amazon EKS cluster with sample, labeled workloads and permission to configure access to the cluster.
  • If you follow the CLI examples, install or update the AWS CLI using the installation and update instructions. Your installation must support the resiliencehubv2 commands. These examples use Bash.
  • An AWS Identity and Access Management (IAM) role that Resilience Hub can assume, with the AWSResilienceHubV2AssessmentExecutionPolicy managed policy attached. The console can create this role when you create a service.
  • Kubernetes RBAC permissions and an EKS access entry that grant this role access to Kubernetes resources. The permissions guide provides the manifest and commands.
  • For policy sharing, an organization with all features enabled and access to its management account.

The examples use two accounts: a workload account containing the EKS cluster and Resilience Hub service, and a management account that owns the shared policy.

For the CLI examples, configure profiles named workload and management for the respective accounts. Replace the example cluster ARN and other placeholders with your own values, and set the Region:

export AWS_REGION=us-east-1

Each command uses --profile to select the account. Follow either the console steps or their CLI equivalents for each operation.

Amazon EKS label support for service input sources

A service input source tells Resilience Hub where to discover a service’s resources. Input sources can be AWS CloudFormation stacks, Terraform state files, resource tags, or Amazon EKS clusters. For an EKS input source, you select the namespaces to include. With this launch, you can also use Kubernetes label selectors to filter resources within those namespaces.

A namespace can contain workloads owned by different teams. With labels, you can select checkout’s resources without including inventory or reorganizing the namespace for an assessment. You use the same labels your teams already maintain.

How it works

Specify the cluster and one or more namespaces, then add label selectors to filter resources within each namespace. A selector contains a key, an operator, and values. Selectors within a namespace use AND logic: an object must match all of them. Without a selector, Resilience Hub discovers every supported object in the selected namespaces.

Checkout and inventory share the production namespace. Their resources carry app=checkout and app=inventory, respectively. Selecting production with app In checkout includes checkout’s Deployment and ReplicaSet and excludes inventory.

A label selector chooses checkout resources from a namespace shared with inventory

Figure 1: Selecting checkout resources with a Kubernetes label

The resource list also includes supporting AWS infrastructure, such as the cluster, node group, virtual private cloud (VPC), and subnets. Labels select Kubernetes objects. They don’t make that shared infrastructure exclusive to checkout.

How to use it

In the Resilience Hub console:

  1. Choose Create service and enter a name. Under Permission, choose Create default role to create an IAM role with the required AWS permissions, or choose Use another role to select an existing role.
  2. Under Service resource discovery, choose Amazon EKS cluster and enter the cluster ARN.
  3. Choose Configure namespaces, enter production, and add a label selector with key app, operator In, and value checkout.
  4. Choose Save, select Enable dependency discovery for this service, then choose Create service. Resource discovery starts automatically.
Configure namespaces dialog selecting production with app In checkout

Figure 2: Scoping the checkout service to one namespace and one label

If you prefer to use the AWS CLI instead of the console, run the following commands to create the service and add its EKS input source.

Skip this block if you already created the service using the preceding console steps.

The console displays selector operators as In, NotIn, Exists, and NotExists. In the API, use IN, NOT_IN, EXISTS, and DOES_NOT_EXIST, respectively.

In the following command, replace ResilienceHubRole with your service’s IAM role name.

# Create the checkout service with dependency discovery enabled and save its ARN
SERVICE_ARN=$(aws resiliencehubv2 create-service \
  --profile workload \
  --name checkout \
  --regions "$AWS_REGION" \
  --permission-model '{"invokerRoleName": "ResilienceHubRole"}' \
  --dependency-discovery ENABLED \
  --query 'service.serviceArn' --output text)

# Add the EKS input source, selecting resources labeled app=checkout in production
aws resiliencehubv2 create-input-source \
  --profile workload \
  --service-arn "$SERVICE_ARN" \
  --resource-configuration '{
    "eks": {
      "clusterArn": "arn:aws:eks:us-east-1:111122223333:cluster/resilience-demo",
      "namespaces": ["production"],
      "labelSelector": {
        "matchExpressions": [
          {"key": "app", "operator": "IN", "values": ["checkout"]}
        ]
      }
    }
  }'

After resource discovery completes, you can use the following command to list the discovered resource types. If you created the service on the console, first set SERVICE_ARN to that service’s ARN.

# List discovered resource types after resource discovery completes
aws resiliencehubv2 list-resources \
  --profile workload \
  --service-arn "$SERVICE_ARN" \
  --query 'serviceResources[].resource.resourceType'

Generative AI-powered Dependency Insights

Dependency Insights uses generative AI to summarize discovered dependencies and highlight patterns that warrant resilience review. It provides an overview and categorized findings.

The Resources list describes the service’s components. The Dependencies list shows endpoints discovered through DNS activity, including AWS service endpoints in other Regions and external APIs. When that list grows to hundreds of entries, Dependency Insights helps you prioritize what to investigate.

How it works

Dependency Insights builds on dependency discovery, an optional, separately priced feature. Dependency discovery analyzes DNS activity through Amazon Route 53 Resolver and does not require installing an agent in your workload. It examines up to 35 days of available history and continues monitoring for changes. The dependency list shows each endpoint’s name, location, and when it was first and last seen.

Dependency Insights analyzes this list, filters out common service activity that typically does not indicate resilience risk, and highlights patterns such as these:

Pattern What to investigate
Cross-Region dependencies Whether a call to another Region is intentional, and how it affects recovery
New dependencies Whether an endpoint first observed within the last seven days changes the documented architecture
Third-party dependencies How the service behaves when an external provider is unavailable
Uneven usage Whether activity reflects a scheduled job or a dependency used only at certain times
Discovered dependency data feeds a generated overview and findings that a service owner reviews

Figure 3: Turning dependency data into a focused review

Insights prioritize review. They don’t prescribe fixes or list every dependency. Use the full dependency list when you need detail. A new workload’s insights reflect only the history it has generated.

How to use it

In the Resilience Hub console, wait for initial dependency discovery to complete before generating insights:

  1. Open the service. On the Assessment tab, scroll to the Dependencies section to review discovered dependencies.
  2. Choose Generate insights.
  3. Read the overview and compare each finding with the intended architecture.

You can regenerate insights once every 24 hours. This limit applies whether you generate them through the console or the AWS CLI.

Generated Insights overview and categorized findings for the checkout service

Figure 4: Dependency Insights for the checkout service

If you prefer the AWS CLI, use the following commands instead of the console steps. Set SERVICE_ARN to your service’s ARN.

Enable dependency discovery only if it is not already enabled. After initial discovery completes, review the dependency list and generate insights. If you already generated insights on the console, skip start-dependency-insights and use get-dependency-insights to retrieve the existing result.

Generation can complete asynchronously. Run get-dependency-insights again if the returned status is IN_PROGRESS.

# Enable dependency discovery if it is not already enabled
aws resiliencehubv2 update-service \
  --profile workload \
  --service-arn "$SERVICE_ARN" \
  --dependency-discovery ENABLED

# Review the dependency list
aws resiliencehubv2 list-dependencies \
  --profile workload \
  --service-arn "$SERVICE_ARN" \
  --query 'dependencySummaries[].[dependencyName, location, dependencyId]' \
  --output table

# Start generating dependency insights
aws resiliencehubv2 start-dependency-insights \
  --profile workload \
  --service-arn "$SERVICE_ARN"

# Retrieve insights; if status is IN_PROGRESS, run this command again later
aws resiliencehubv2 get-dependency-insights \
  --profile workload \
  --service-arn "$SERVICE_ARN"

The checkout service produced five insights across 29 dependencies. The following response shows the generated overview and all five findings:

{
    "overview": "5 insights generated for service 'checkout' across 29 dependencies, surfacing cross-region, third-party, and uneven usage patterns.",
    "insights": [
        {
            "category": "CROSS_REGION",
            "description": "DynamoDB in us-west-2 is reached cross-region with hard criticality, coupling this service's availability to a remote region. This dependency carries medium DNS query volume with consistent usage."
        },
        {
            "category": "CROSS_REGION",
            "description": "PagerDuty (api.pagerduty.com) is reached cross-region to us-west-2, creating an availability dependency on that remote region."
        },
        {
            "category": "THIRD_PARTY",
            "description": "Three third-party dependencies — example.com, example.net, and example.org — are present at medium DNS query volume with consistent usage patterns."
        },
        {
            "category": "THIRD_PARTY",
            "description": "GitHub is a third-party dependency with low DNS query volume."
        },
        {
            "category": "UNEVEN_USAGE",
            "description": "Two in-region dependencies show highly uneven daily usage: an S3 bucket (amzn-s3-demo-bucket) and an ECR registry (602401143452.dkr.ecr.us-east-1.amazonaws.com)."
        }
    ],
    "status": "COMPLETED",
    "createdAt": "2026-10-08T14:24:27.372000-05:00"
}

The first finding identifies a cross-Region Amazon DynamoDB dependency. If checkout requires that dependency to accept orders, a disruption in the remote Region could prevent it from operating even when its Amazon EKS resources remain healthy. The service owner should verify which operations require the table and review how those operations behave when it is unavailable. That review determines whether the dependency fits the recovery design or whether the architecture needs to change. DNS activity alone cannot answer that question.

The uneven-usage finding points to container image infrastructure. Image downloads during pod startup can produce bursts of DNS activity. This is a pattern to investigate, rather than evidence that your application has an outage.

Classify a dependency as hard if the service fails without it, or soft if the service degrades but keeps running. In the sample environment, we had classified the DynamoDB dependency as hard using the following command. The comment describes an illustrative failure assumption, which requires validation through testing:

# Classify the selected dependency as hard and record the reason
aws resiliencehubv2 update-dependency \
  --profile workload \
  --service-arn "$SERVICE_ARN" \
  --dependency-id "<dependency-id>" \
  --criticality HARD \
  --comment "Checkout writes orders to this table and cannot accept orders without it."

Resilience policy sharing through AWS Organizations

After reviewing checkout’s dependencies, the service owner can validate its recovery requirements with the business and the central site reliability engineering (SRE) team. Dependency findings inform that discussion. They don’t automatically determine the policy targets.

The central SRE team can codify the agreed requirements into a resilience policy, defining availability and recovery targets for critical services. The management account or a delegated administrator can share that policy across the organization so service owners can apply the centrally maintained requirements without creating separate copies.

The policy owner can review attachment and detachment events to track adoption. If a service stops using the policy, the SRE team can engage its owner to understand why and determine whether different requirements are appropriate. Application owners remain responsible for meeting the targets.

How it works

A policy can combine four target types: availability, recovery from an Availability Zone disruption, recovery from a Regional disruption, and data recovery. The recovery time objective (RTO) specifies the targeted recovery time. The recovery point objective (RPO) specifies the tolerable period of data loss.

When sharing is on, the policy appears in every member account alongside the account’s own policies, and its ARN shows which account owns it. Each service owner chooses whether to apply it. If a requirement must apply regardless of that choice, associate the policy with a system containing the relevant services instead, as described in Applying resilience policies across accounts.

The owning account can review a timeline of the services that start or stop using the policy. Turning sharing off stops evaluating member services against the policy and removes it from the services that used it, so coordinate changes with the teams involved.

A centrally owned resilience policy is shared with service owners in two member accounts

Figure 5: One policy definition, shared with member accounts

How to use it

The organization integration is a one-time setup in the management account:

  1. In the Resilience Hub console, open AWS Organizations settings.
  2. Select Enable trusted access and create a service-linked role, then choose Create integration.

To confirm that the service-linked role exists in a member account:

# Confirm the service-linked role exists in the workload account
aws iam get-role \
  --profile workload \
  --role-name AWSServiceRoleForResilienceHub \
  --query 'Role.Arn'

Next, create the policy in the management account with sharing turned on. Policy names can contain letters, numbers, hyphens, and underscores, but no spaces. The following targets are examples. Agree on real values with the business and service owners. This example uses an availability target of 99.9 percent.

# Create an organization-shared resilience policy and save its ARN
POLICY_ARN=$(aws resiliencehubv2 create-policy \
  --profile management \
  --name tier-1-critical \
  --availability-slo '{"target": 99.9}' \
  --multi-az '{"rtoInMinutes": 5, "rpoInMinutes": 1, "disasterRecoveryApproach": "ACTIVE_ACTIVE"}' \
  --multi-region '{"rtoInMinutes": 15, "rpoInMinutes": 5, "disasterRecoveryApproach": "WARM_STANDBY"}' \
  --data-recovery '{"timeBetweenBackupsInMinutes": 60}' \
  --sharing-enabled \
  --query 'policy.policyArn' --output text)

In the workload account, find the shared policy and apply it to the service. On the console, choose Change policy on the service and select the shared policy. With the AWS CLI:

# List policies available in the workload account, including shared policies
aws resiliencehubv2 list-policies \
  --profile workload \
  --query 'policySummaries[].[name, policyArn, sharingEnabled]' \
  --output table

# Apply the shared resilience policy to the checkout service
aws resiliencehubv2 update-service \
  --profile workload \
  --service-arn "$SERVICE_ARN" \
  --policy-arn "<shared-policy-arn>"
Lifecycle panel showing Policy associated and four targets pending assessment

Figure 6: Policy targets are defined. Assessment of those targets is still pending

Back in the management account, the policy owner can confirm adoption:

# Review service attachment events for the shared policy
aws resiliencehubv2 list-policy-events \
  --profile management \
  --policy-arn "$POLICY_ARN" \
  --event-types POLICY_ATTACHED_TO_SERVICE

An attachment event confirms adoption, not compliance. After we applied the shared policy, the console changed each target from Not defined in policy to Pending failure mode assessment. A failure mode assessment and appropriate resilience testing are needed to determine whether the service meets those targets.

Things to know

  • Shared clusters. Labels decide which Kubernetes objects belong to a service. On a cluster that several workloads share, review the discovered dependencies to confirm they belong to the service, as described in the guidance for shared EKS clusters.
  • What DNS-based discovery can’t see. Connections made directly by IP address aren’t discovered, AWS Lambda functions need VPC connectivity to be covered, and very infrequent calls might not appear in initial results. See Coverage and known limitations.
  • Timing. Results reflect the history available to the service. Give a new workload time to build history before drawing conclusions from its insights. You can regenerate insights once every 24 hours.

Cleanup

If you created resources to try these capabilities, remove them when you finish. Turning sharing off also removes the policy from every member service that uses it, so only do this for a demonstration policy:

# Stop sharing the demo policy; this also removes it from member services
aws resiliencehubv2 update-policy --profile management --policy-arn "$POLICY_ARN" --no-sharing-enabled
# Delete the demo policy
aws resiliencehubv2 delete-policy --profile management --policy-arn "$POLICY_ARN"
# Delete the demo Resilience Hub service
aws resiliencehubv2 delete-service --profile workload --service-arn "$SERVICE_ARN"

Delete the EKS cluster as well if you created it only for this walkthrough.

Conclusion

Start with a service whose architecture and ownership you know. Use EKS labels to check its Kubernetes resource boundary, then review Dependency Insights against its intended behavior. If several teams maintain the same requirements separately, consider a shared policy.

You can use the capabilities together or independently. Label support applies to EKS workloads. Dependency Insights and policy sharing also support services beyond EKS. Insights focus the investigation. Your architecture review determines what to change.

Further reading

About the author

The collective thoughts of the interwebz