Post Syndicated from Vaibhav Jain original https://aws.amazon.com/blogs/compute/introducing-queue-consumption-mode-for-apache-kafka-in-aws-lambda/

Queue consumption mode for AWS Lambda event source mappings (ESMs) that subscribe to Apache Kafka is now available. Queue consumption mode is available on the self-managed Apache Kafka ESM. It covers both Kafka clusters you operate yourself and managed Kafka offerings such as Confluent Cloud and other providers that offer Kafka 4.2 or later with Kafka Queue support.

With this launch, you can build queue-style event-driven workloads on Kafka that process records with high parallelism that scales concurrency to match your backlog instead of your topic layout. Queue mode achieves this using Kafka Share Groups (KIP-932), introduced in Kafka 4.0. With share groups, multiple consumers read concurrently from the same partitions, so Lambda can process records from a single partition across many parallel invocations. The result is parallelism that is decoupled from partition count, without head-of-line blocking and without introducing a second messaging system into your architecture.

Many Kafka workloads treat each event as an independent unit of work rather than a step in an ordered sequence: generating image and video renditions after upload, offloading slow work from a user-facing API, scoring transactions for fraud, delivering notifications, enriching records for inference, processing bulk files, buffering ahead of a rate-limited dependency, and ingesting telemetry. Correctness for these workloads depends on processing records quickly, not in order. The Kafka consumer group protocol, however, assigns each partition to exactly one consumer, so a topic with 10 partitions supports up to 10 concurrent consumers. Over-partitioning raises that ceiling but increases broker resource usage, slows rebalances, and still leaves head-of-line blocking, where one slow record stalls every record behind it. Previously, teams moved this traffic to a separate queuing system, adding a platform to operate and new producer code.

In this post, you learn how queue consumption mode works compared to the existing stream mode, what broker and cluster requirements your Kafka 4.2 cluster must meet, and how to scale from on-demand defaults to provisioned mode for high-throughput workloads.

What’s new

This launch adds a ConsumptionMode parameter to the Kafka event source mappings in Lambda. It accepts two values: Stream (the default, matching today’s behavior) and Queue, which uses Kafka Share Groups for parallel consumption. The core primitive in both modes is the event poller, a compute resource that fetches records from Kafka and invokes your Lambda function. Queue mode changes the protocol an event poller uses to obtain those records. Because pollers are no longer bound to individual partitions, queue mode supports higher concurrency. A single poller reaches up to 10 concurrent invocations compared to 5 in stream mode, and queue mode scales independently of partition count. We cover the scaling model in detail later in this post.

Stream mode: One consumer per partition, ordered processing

ConsumptionMode defaults to Stream, so existing ESMs are unaffected by this launch. In Stream mode, event pollers join a Kafka consumer group, and the broker assigns each partition exclusively to one consumer. This guarantees ordered processing within each partition.

Queue mode: Multiple consumers per partition, unordered parallel processing

Setting ConsumptionMode to Queue directs Lambda to use the Kafka Share Consumer. Event pollers join a share group, where partitions can be assigned to multiple event pollers for parallel processing without guarantees about order. The broker manages a record lifecycle of Available, Acquired, Acknowledged, and Archived. Fetched records are marked Acquired under a visibility timeout called the record lock duration. A successful invocation acknowledges them. A failed invocation releases them back to Available for any poller to acquire. While an invocation is still running, the poller renews the lock without incrementing the delivery attempt count.

Choosing between the modes

Use Stream mode when per-partition ordering matters: change data capture, event sourcing, or per-entity state machines. Use Queue mode when records are independent, and you want parallelism that is not bounded by partition count. You can run a Queue mode ESM and a Stream mode ESM against the same topic. Each operates independently through its own protocol.

Minimum requirements

Before you configure an ESM with ConsumptionMode: Queue, confirm that your Kafka cluster and Lambda function meet the following requirements.

Cluster requirements

• Apache Kafka 4.2 or later, with share groups enabled.

Four broker settings interact with your ESM configuration:

• group.share.partition.max.record.locks – The maximum in-flight records per partition. In Queue mode, multiple pollers might acquire records from the same partition concurrently, so the total in-flight count can exceed a single poller’s batch size. Set this value high enough to accommodate your expected concurrency. When the limit is reached, pollers can’t acquire new records from that partition until existing records are acknowledged, or their locks expire.
• group.share.record.lock.duration.ms – The visibility timeout for acquired records. In most cases, the default of 30 seconds is sufficient. The poller renews the lock during processing. A long-running function does not typically require a duration longer than 30 seconds.
• group.share.delivery.count.limit – The maximum number of times the broker attempts to deliver a record before archiving it. This works alongside MaximumRetryAttempts to govern retry behavior.
• group.share.max.size – The maximum consumers allowed in the share group. When provisioned mode is active, keep MinimumPollers ≤ MaximumPollers ≤ group.share.max.size so that Lambda scaling isn’t capped by the broker. In on-demand mode, Lambda internally manages poller counts within broker-safe defaults.

Getting started

In the Lambda console, choose your function, choose Add trigger, and select Self-managed Apache Kafka. Enter your bootstrap servers and topic name, set Consumption mode to Queue, and configure your virtual private cloud (VPC) subnets and security groups. Optionally set Minimum event pollers and Maximum event pollers to use provisioned mode. For on-demand scaling leave those empty.

The following AWS CLI command creates the equivalent Queue mode ESM in on-demand mode:

aws lambda create-event-source-mapping
--function-name my-function
--self-managed-event-source '{"Endpoints":{"KAFKA_BOOTSTRAP_SERVERS":["broker1.example.com:9092","broker2.example.com:9092"]}}'
--topics my-queue-topic
--batch-size 100
--self-managed-kafka-event-source-config '{"ConsumptionMode": "Queue"}'

ConsumptionMode is the only parameter required to enable Queue mode. BatchSize sets the maximum records per batch sent to your function, accepts values from 1–10,000, and defaults to 100.

With AWS CloudFormation, the equivalent resource looks as follows:

MyKafkaQueueESM:
  Type: AWS::Lambda::EventSourceMapping
  Properties:
    FunctionName: !Ref MyFunction
    SelfManagedEventSource:
      Endpoints:
        KafkaBootstrapServers:
          - broker1.example.com:9092
          - broker2.example.com:9092
    Topics:
      - my-queue-topic
    BatchSize: 100
    SelfManagedKafkaEventSourceConfig:
      ConsumptionMode: Queue

Queue mode is also configurable through the AWS SDKs, AWS Serverless Application Model (AWS SAM), and AWS Cloud Development Kit (AWS CDK). For the complete setup walkthrough, see the AWS Lambda Developer Guide.

Scaling with on-demand mode

Queue mode ESMs use on-demand mode for scaling by default. Lambda evaluates share group consumer lag at one-minute intervals and adds or removes event pollers as needed, completing scaling actions within three minutes. On-demand mode scales to five event pollers per ESM, each supporting up to 1 MB/s. It reaches its ceiling at either 5 MB/s aggregate throughput or 50 concurrent invocations, whichever comes first. This suits lower-concurrency workloads and development environments.

Scaling with provisioned mode

For workloads with strict latency requirements or spiky traffic, provisioned mode for scaling gives you direct control over the event pollers dedicated to your ESM. Set MinimumPollers from 1 –200 and MaximumPollers from 1 to 2,000. However, the broker’s group.share.max.size setting caps how many members can join a share group, and Kafka 4.2 limits this value to 1,000. Set MaximumPollers to 1,000 or below unless your broker supports a higher limit. Each provisioned event poller supports up to 5 MB/s.

aws lambda create-event-source-mapping
--function-name my-function
--self-managed-event-source '{"Endpoints":{"KAFKA_BOOTSTRAP_SERVERS":["broker1.example.com:9092","broker2.example.com:9092"]}}'
--topics my-queue-topic
--batch-size 100
--self-managed-kafka-event-source-config '{"ConsumptionMode": "Queue"}'
--provisioned-poller-config '{"MinimumPollers": 5, "MaximumPollers": 100}'
--source-access-configurations '[{"Type":"VPC_SUBNET","URI":"subnet-0123456789abcdef0"},{"Type":"VPC_SUBNET","URI":"subnet-0123456789abcdef1"},{"Type":"VPC_SECURITY_GROUP","URI":"sg-0123456789abcdef0"}]'

To size MinimumPollers, divide your baseline ingestion rate by 5 MB/s. If your workload is concurrency-bound, divide your target concurrent invocations by 10 instead. A workload needing 300 concurrent invocations requires at least 30 pollers. Setting MinimumPollers equal to MaximumPollers pins capacity and disables autoscaling. For detailed sizing guidance, see Introducing provisioned mode for Kafka event source mappings with AWS Lambda on the AWS Compute Blog.

Low-latency processing, schema registry filtering for Avro and Protobuf, and ESM grouping are available only in provisioned mode. Provisioned mode also removes the requirement to configure AWS PrivateLink VPC endpoints, so you are not charged for those endpoints. To switch an ESM to on-demand scaling, pass an empty configuration:

aws lambda update-event-source-mapping \
--uuid a1b2c3d4-5678-90ab-cdef-EXAMPLE11111 \
--provisioned-poller-config '{}'

Cost optimization with provisioned mode

In provisioned mode, ESMs in the same VPC can share Event Poller Unit (EPU) capacity through a common PollerGroupName, letting low-throughput ESMs pack onto shared EPUs instead of each holding dedicated capacity. For details, see Cost optimization for provisioned mode. A poller group supports up to 100 ESMs and 2,000 aggregate maximum pollers.

Queue mode ESMs share EPUs only with other Queue mode ESMs. The two modes autoscale on different signals, and isolation keeps a new Queue mode ESM from destabilizing existing ordered consumers. Use a unique PollerGroupName for your Queue mode ESMs. If you reuse a PollerGroupName across modes, CreateEventSourceMapping returns a validation error.

Handling failures

MaximumRetryAttempts accepts values from 0 to 10,000 and defaults to -1, which retries indefinitely. BisectBatchOnFunctionError splits a batch in half when the function returns an error, isolating a poison-pill record from an otherwise healthy batch. To capture records that exhaust their retries, configure an on-failure destination: a Kafka topic, Amazon Simple Queue Service (Amazon SQS), or Amazon Simple Notification Service (Amazon SNS). ESM logging to Amazon CloudWatch Logs is available for diagnostics.

Key considerations for Queue mode

Keep the following in mind when configuring your event source mapping for Queue mode.

• Make your function idempoten – Lambda event source mappings process each record at least once, and duplicate processing can occur. See How do I make my Lambda function idempotent in the AWS Knowledge Center.
• No ordering guarantees – Records can arrive out of order, including records from the same partition.
• Batching window is not supported – MaximumBatchingWindowInSeconds returns a validation error. The Kafka Share Consumer requires every record in a batch to be acknowledged before fetching new records, which is incompatible with a time-based window.
• Starting position is not supported – StartingPosition and StartingPositionTimestamp can’t be used. Queue mode reads from the share group’s committed position. To control where a new share group begins reading, configure share.auto.offset.reset on the broker.
• Event filtering supports up to five filter patterns per event source mapping.

Pricing and availability

Queue consumption mode is available in all AWS Regions where Lambda event source mappings for self-managed Apache Kafka is supported. On-demand mode carries no charge beyond standard Lambda invocation charges. Provisioned mode is billed on EPU usage. For details, see AWS Lambda pricing.

Conclusion

Queue consumption mode brings queue-style processing to Apache Kafka topics without additional infrastructure and without changes to your producers, topic configuration, or function code. By replacing exclusive partition assignment with share groups, it removes the partition count as a ceiling on parallelism and eliminates head-of-line blocking for workloads built on independent records.

To get started, create an event source mapping with ConsumptionMode set to Queue on a self-managed Apache Kafka 4.2 or later cluster. For configuration details, see Polling Kafka queue topics in the AWS Lambda Developer Guide.

For more serverless learning resources, visit Serverless Land.

About the authors

Post Syndicated from Pooya Paridel original https://aws.amazon.com/blogs/compute/observing-aws-lambda-durable-functions-with-the-durable-execution-telemetry-exporter/

A durable function is an AWS Lambda function that might run for minutes, days, or even up to a year. You build these functions with the Durable Execution SDK, which provides operations such as steps, waits, and parallel. A single durable execution might be spread across many Lambda invocations, with waits and retries in between. So observing a durable execution, made of many invocations, needs different tools. The question that you want answered is rarely “what did this invocation do?” It’s “how far did my durable execution get, which step failed, how many times did it retry, and how long did the whole thing take?”

After you run thousands of executions, the questions widen from one execution to all of them. Some questions are operational. Which step fails most often, and with which error? Which step causes the most retries? Did a step get slower after last week’s deployment? Other questions are about the business the workflow runs. Which customers did last night’s payment failures affect? How long does each manual approval step take? Of the orders that started, how many reached the payment step, and where did the rest stop? Answering these questions needs per execution telemetry data that includes relevant data such as status, timing, steps, and the input and output fields that you choose to emit.

Traditional Lambda observability works well for a single invocation. Lambda writes a log stream per invocation, so a durable execution that suspends four times writes to four separate log streams, interleaved with every other concurrent execution. Amazon CloudWatch metrics tell you the function’s error rate, but not which of your workflow’s steps is responsible. Distributed tracing, such as AWS X-Ray or OpenTelemetry, shows the timing of one execution. It isn’t built to answer questions across many executions, such as which step failed most often this week. For that, you need one queryable record per execution.

Today we’re introducing the Durable Execution Telemetry Exporter, an observability plugin for AWS Lambda durable functions. We’re also introducing the Durable Execution Telemetry MCP server, a Model Context Protocol (MCP) server that lets an AI coding agent query the telemetry data. The exporter is open source for TypeScript, Python, Java, and Go, in each language’s Durable Execution SDK repository. AWS provides the MCP server as open source in the aws-durable-execution-sdk-js repository.

With a one-line configuration change, you get a record of each execution, with every step, wait, callback, and invoke, including timing, retry attempts, status, and errors. The exporter delivers it to a destination you choose. There’s no custom instrumentation to write and no separate agent to run.

How the durable execution telemetry exporter works

The Durable Execution Telemetry Exporter is a plugin for the durable execution SDK. It runs inside your Lambda function, in the same process as your handler, and subscribes to the SDK’s instrumentation lifecycle. It doesn’t run in your request path.

On every invocation of a durable execution, the plugin builds a cumulative snapshot of that execution, rather than a delta or per-invocation view. Because the durable execution SDK replays your handler from the beginning on each resume and the SDK passes the checkpointed state to the plugin, the plugin can reconstruct the whole execution so far from any invocation. That snapshot is a single JSON document called a DurableExecutionTelemetryRecord:

{
  "recordType": "DurableExecutionTelemetry",
  "schemaVersion": "1.0",
  "executionArn": "arn:aws:lambda:us-east-1:111122223333:function:order-processor:$LATEST/durable-execution/abc123/inv456",
  "executionName": "abc123",
  "functionName": "order-processor",
  "status": "SUCCEEDED",
  "startTime": "2026-06-16T17:00:22.100Z",
  "endTime": "2026-06-16T17:00:27.514Z",
  "durationMs": 5414,
  "input": { "orderId": "order-12345", "customerId": "cust-789" },
  "output": { "orderId": "order-12345", "status": "completed", "charged": 99.99 },
  "operations": [
    { "name": "validate-order",  "type": "STEP", "status": "SUCCEEDED", "durationMs": 250 },
    { "name": "check-inventory", "type": "STEP", "status": "SUCCEEDED", "durationMs": 350 },
    { "name": "cool-down",       "type": "WAIT", "status": "SUCCEEDED", "durationMs": 5000 },
    { "name": "charge-payment",  "type": "STEP", "status": "SUCCEEDED", "durationMs": 80 }
  ]
}

Each record is complete on its own. One row in a table is one durable execution, which means the questions you ask (slowest step, most-retried step, executions that failed after the payment step) become ordinary queries instead of requiring you to correlate logs across streams.

Getting started

Install the plugin alongside the durable execution SDK:

npm install @aws/durable-execution-sdk-js-telemetry-exporter

Then pass it to withDurableExecution:

import { DurableContext, withDurableExecution } from "@aws/durable-execution-sdk-js";
import { durableExecutionTelemetry } from "@aws/durable-execution-sdk-js-telemetry-exporter";

const handler = async (event: OrderEvent, context: DurableContext) => {
  const order = await context.step("validate-order", async () =>
    validateOrder(event.orderId),
  );

  const inventory = await context.step("check-inventory", async () =>
    reserveInventory(order),
  );

  await context.wait({ seconds: 30 });

  return await context.step("charge-payment", async () =>
    chargeCustomer(order, inventory),
  );
};

export const lambdaHandler = withDurableExecution(handler, {
  plugins: [durableExecutionTelemetry({})],
});

That is the entire integration. With no configuration, the default LambdaLogExporter writes records as JSON to your function’s own CloudWatch log group. You don’t need to create any infrastructure or grant additional AWS Identity and Access Management (IAM) permissions, because your function already writes its own logs. You can query the result immediately with Amazon CloudWatch Logs Insights:

fields executionName, status, durationMs
| filter recordType = "DurableExecutionTelemetry" and status = "FAILED"
| sort durationMs desc

Choosing where records go

The plugin includes 13 destination exporters so you can send records where they are useful:

The plugin sends records to all configured exporters in parallel. This allows you to route data to different stores based on their characteristics, such as using a fast store for real-time lookups and a less expensive, high-capacity store for long-term storage:

import {
  durableExecutionTelemetry,
  S3Exporter,
  DynamoDBExporter,
  EventBridgeExporter,
} from "@aws/durable-execution-sdk-js-telemetry-exporter";

const telemetry = durableExecutionTelemetry({
  emitMode: "on-complete",
  samplingRate: 1.0,
  exporters: [
    new S3Exporter({ bucket: string }), // Use valid bucket name you own
    new DynamoDBExporter({ tableName: "durable-execution-telemetry-live" }),
    new EventBridgeExporter({}),
  ],
});

With the S3 exporter, the plugin writes records under Hive-style partitions (year=YYYY/month=MM/day=DD/) that an AWS Glue crawler discovers automatically, so Athena queries work without additional modeling:

SELECT op.name, COUNT(*) AS failures, AVG(op.durationms) AS avg_ms
FROM durable_execution_telemetry
CROSS JOIN UNNEST(operations) AS t(op)
WHERE status = 'FAILED' AND year = '2026' AND month = '06'
GROUP BY op.name
ORDER BY failures DESC;

Controlling frequency and content

Three settings govern what a production deployment costs and exposes.

emitMode determines when the plugin sends a record. The default, on-complete, emits once when an execution reaches SUCCEEDED or FAILED. Use on-failure for the lowest overhead when you only care about errors, or on-change when you want to watch executions progress in near real time.

samplingRate reduces frequency. The plugin derives the sampling decision from a hash of the execution ARN. The execution ARN is stable across replays, so the decision is deterministic: a sampled-in execution emits all of its records, and a sampled-out execution emits none. You never receive a half-recorded workflow.

content controls what leaves your function. By default, the plugin includes execution input and output as-is, which is convenient in development and often unacceptable in production. Pass a transform to redact:

const telemetry = durableExecutionTelemetry({
  samplingRate: 0.1,
  content: {
    input: (event) => ({ orderId: event.orderId }),
    output: (result) => ({ status: result.status }),
  },
});

Each destination supports different payload sizes: CloudWatch Logs events at 256 KB, DynamoDB items at 400 KB. Each exporter has a maxRecordSizeBytes, and when a record exceeds it the plugin truncates for one destination without affecting other destinations: operation results first, then whole operations oldest-first, then execution input and output last. The plugin never drops identity and timeline fields, and it marks a trimmed record truncated: true, so a truncated record is always distinguishable from a complete one. We recommend using content transforms to bound payloads deliberately rather than relying on truncation.

Querying execution history from an AI agent

Answering “why did last night’s batch fail” requires knowing which data store you configured, which query dialect it uses, and what the record schema looks like. DynamoDB PartiQL, Athena’s Trino SQL, Aurora PostgreSQL, Amazon Redshift, OpenSearch, and CloudWatch Logs Insights all differ.

The Durable Execution Telemetry MCP server removes that requirement. It’s a local stdio server that exposes your configured destination to any MCP-capable agent, including Kiro, Claude Code, and Cursor. Add it with one command:

kiro-cli mcp add \
  --name durable-telemetry \
  --command npx \
  --args "-y,@aws/durable-execution-sdk-js-telemetry-mcp" \
  --env DURABLE_TELEMETRY_DESTINATION_TYPE=dynamodb \
  --env DURABLE_TELEMETRY_DYNAMODB_TABLE_NAME=durable-execution-telemetry-live \
  --env DURABLE_TELEMETRY_REGION=us-east-1 \
  --scope workspace

Configuration is environment-only and credentials come from the standard AWS SDK provider chain, so there’s no state file and no secret. The server exposes five tools:

• test_destination runs read-only connectivity checks. If a required variable is unset, it names the exact variable and returns without making any AWS call.
• describe_schema returns the record schema, query dialect, and row cap for whichever destination is configured. Agents should call this before writing a query.
• list_executions filters by status, function name, and time range with no query language involved. This handles most questions.
• get_execution fetches one record by execution ARN.
• query handles questions the filters cannot express.

You ask a question and the agent picks the tools:

Which steps in order-processor failed most often in the last 24 hours, and what was the average retry count?

On the SQL destinations, write each query as a SELECT statement, or as a WITH statement that ends in a SELECT. These are the only forms the server runs. It checks a query before it makes any AWS call, so a statement that would change data never reaches your destination. On the CloudWatch Logs destinations, the Logs Insights query language has no statements that write data. So every query through the server is read-only.

Best practices and considerations

Choose operationDetail deliberately. The default, top-level, records only top-level operations and produces the same set of operations no matter when a record is emitted. full-tree includes children of contexts such as map items and parallel branches, but by default the durable execution backend prunes finished contexts’ children from the state passed to later invocations. To keep the full tree across a suspend and resume, also set pluginsConfig.childOperationsDepth on withDurableExecution. That preservation has a cost: it forces a replay pass over each preserved context on resume, so enable only the depth you need.

Choose operation names that your destination can query. The plugin uses names verbatim as keys and identifiers and never sanitizes them. A . in a name is a field-path separator in CloudWatch Logs Insights and OpenSearch. Letters, digits, -, and _ are portable everywhere.

Understand what the plugin cannot see. The Durable Execution Telemetry Exporter runs inside your function, so it can only emit records when your function is invoked. Terminal states that originate in the Lambda service without an invocation never reach the plugin: STOPPED from a manual stop, and TIMED_OUT when the execution timeout is exceeded. For complete lifecycle coverage, subscribe to Lambda durable execution lifecycle events on the default Amazon EventBridge bus and update your destination from a small handler:

{
  "source": ["aws.lambda"],
  "detail-type": ["Lambda Durable Execution State Change"],
  "detail": { "status": ["STOPPED", "TIMED_OUT"] }
}

Account for the cost of the destination. The plugin adds at most one export each time it emits a record, because it sends only the latest of any updates that pile up. Each record is a complete snapshot, and the latest supersedes all earlier ones. If updates arrive faster than an exporter can handle, the plugin drops the intermediate snapshots. The charges that matter are the ones your destination incurs: CloudWatch Logs ingestion, DynamoDB writes, Aurora capacity, and so on. Combine samplingRate and emitMode to control them.

Conclusion

The Durable Execution Telemetry Exporter gives AWS Lambda durable functions an observability model that matches how they run: one complete record per execution, sent to a destination that fits your team. The Durable Execution Telemetry MCP server lets an AI agent answer questions against that data directly.

To get started with durable functions, see the AWS Lambda Developer Guide. For the plugin’s full exporter reference, configuration options, and an AWS Cloud Development Kit (AWS CDK) stack that deploys every destination for testing, see the package README.

About the author

Post Syndicated from Julian Wood original https://aws.amazon.com/blogs/compute/exploring-aws-lambda-web-functions-for-building-web-applications/

You can now run a standard Node.js web server on AWS Lambda with Lambda Web Functions, a new type of Lambda function. Deploy it with one command or AWS CloudFormation and get a public HTTPS endpoint with no adapter layer and no servers or load balancers to manage.

Lambda Web Functions suit server-side rendered web apps, REST and GraphQL APIs, streaming endpoints, and microservices. Building on the launch announcement, this post uses a sample application to show how one execution environment handles many requests at once and how the same app serves a global audience. See how active-CPU billing keeps the cost of waiting low while your code waits, with memory and requests billed separately.

Overview

Lambda launched in 2014 with a minimal model: upload your code, and the service handles everything else. For over a decade that code ran as an event handler, invoked once per event such as a batch of queue messages or an object landing in Amazon Simple Storage Service (Amazon S3). We now call this an Event Function and it remains the right choice for event-driven work.

Web Functions are a second type of Lambda function, running on the same Firecracker-based isolation as Event Functions. Instead of a handler that receives an event, your code runs as a standard HTTP server that listens on a port, which is what lets a web framework, or in this case a plain Node.js server, run unmodified.

A Web Function is made of three resources:

• Web Function is the top-level resource with a name and configuration.
• Revision is an immutable snapshot of your code and configuration which Lambda creates each time you deploy code or change revision configuration, so you can roll back by routing traffic to an earlier revision.
• Endpoint is a network entry point with a unique HTTPS domain, an authentication type, and the rules that route traffic to your revisions.

At launch, Web Functions run Node.js 24 on Arm64, with 2 vCPUs and 2 GB of memory per execution environment. Web Functions support Amazon Virtual Private Cloud (VPC) connectivity, allowing your function to access resources that exist within a VPC.

Walkthrough

Node.js frameworks that start an HTTP server, such as Express, Fastify, and Hono, run without adapters. Next.js in standalone mode needs a small index.js that sets its host and port. This post uses a plain Node.js server to stay dependency-free.

The full source is available in the Serverless Patterns collection along with other Web Functions patterns.

Prerequisites

You need an AWS account with the AWS Command Line Interface (AWS CLI) installed and configured and Node.js installed locally.

The pattern instructions show how to clone the repo and change to the pattern directory or download the pattern .zip file to run the example.

The app

The dashboard is a single Node.js HTTP server that streams live server metrics to the browser over server-sent events (SSE).

The runtime provides the address to listen on in the AWS_LAMBDA_HTTP_ENDPOINT environment variable, in host:port form. The server reads it and starts listening, exactly as it would anywhere else:

const [host, port] = (process.env.AWS_LAMBDA_HTTP_ENDPOINT || "0.0.0.0:3000").split(":");
server.listen(Number(port), host);

The runtime is nodejs24.x, and it loads index.js at the package root by default. To use a different entry file, set AWS_LAMBDA_NODEJS_ENTRYPOINT to its path.

Run locally

You can run the server locally with npm start.

Browse to http://localhost:3000/ to view the dashboard streaming from the local server.

Deploy

1. Deploy using the single command:

   aws lambda-web deploy --name lambda-web-sse-dashboard --code .

   For this example, you can also use npm run deploy defined in the package.json file.

The AWS CLI bundles your code, creates the function and an endpoint named dev with a live HTTPS URL. It also creates an execution role and a versioned S3 bucket, if one does not exist, that all CLI-deployed Web Functions in the Region share.

2. Press Enter to create the resources.

The AWS CLI output shows the URL in the format: https://<id>.<shard>.lambda-web.<region>.on.aws

Figure 1: AWS CLI showing the aws lambda-web deploy command.

Browse to the URL and the dashboard loads, already streaming. This endpoint is public and the sample has no authentication, so use it for evaluation and delete it when you finish.

Figure 2: The app streaming dashboard showing function information.

The dashboard shows how long the execution environment has been alive and how many streaming connections it is holding open. It shows how much CPU it has used with a button to generate real CPU load on demand. The function runs in the US East (N. Virginia) Region (us-east-1). From my laptop in the United Kingdom, latency is 90 ms.

The dashboard updates every two seconds over a single HTTP streaming response. A Web Function environment is a long-lived process, so it keeps the connection open and keeps pushing data to the browser. The long-lived process can also run background tasks outside the main request path, such as emitting telemetry events.

An Event Function environment serves one request at a time. It can stream a response during an invocation, but each open stream occupies a whole environment. Lambda freezes the environment between invocations, so background work stops.

Active-CPU billing

If the environment stays alive for minutes at a time, what are you paying for? You pay for requests, consumed memory, and the CPU time the environment consumes. For this example, compare connection uptime with CPU time. The environment keeps running but uses only a fraction of a second of CPU.

Choose Generate CPU load, and the app runs a cryptographic hashing loop for about 1.5 seconds. CPU time climbs on the chart, then falls back to near zero when the work stops. When a request waits on a database, downstream API, or AI inference, your code uses little CPU, so the wait costs less than it would under wall-clock billing.

Figure 3: CPU chart mid-spike after choosing Generate CPU load.

For pricing details, read the AWS Lambda pricing page.

Concurrency and scale-out

With Web Functions, each execution environment handles many requests at once, not one at a time. As environments approach their concurrency limit, Lambda adds more environments and removes them again when traffic falls.

You can control how a Web Function scales. maxConcurrencyPerEnvironment sets the maximum number of requests each execution environment handles at once, up to 128, with a default of 64. On the endpoint, maxEnvironments caps the number of execution environments and rateLimit caps requests per second. You can use these two settings to protect downstream dependencies such as databases and external APIs. Both apply in each Region the endpoint serves and maxEnvironments applies to each revision. Requests above a limit receive HTTP 429 and are not queued.

Generate load on the app. Send 300 streaming requests to the /api/events endpoint, with up to 200 open at once, rather than a burst of quick responses.

curl -sN --parallel --parallel-immediate --parallel-max 200 --max-time 90 "https://<ENDPOINT>/api/events?[1-300]"

Figure 4: Concurrency showing increased connections in a single execution environment.

The dashboard updates to reflect the active SSE connections and peak for the currently connected execution environment.

The runtime also writes a per-environment summary record to Amazon CloudWatch Logs every minute with the instance ID, request count, peak concurrency, and CPU time. A CloudWatch Logs Insights query aggregates them across the fleet:

QID=$(aws logs start-query \
    --log-group-name /aws/lambda/web/lambda-web-sse-dashboard \
    --start-time $(($(date +%s)-300)) --end-time $(date +%s) \
    --query-string 'filter type="platform.invokeSummary" | stats count_distinct(record.instanceId) as environments, sum(record.metrics.requests) as totalRequests, max(record.metrics.concurrentRequests) as peakConcurrency, sum(record.metrics.cpuTimeMs)/1000 as totalCpuSeconds' \
    --query queryId --output text) && sleep 6 && aws logs get-query-results --query-id "$QID"

From a sample run of 300 streaming requests with up to 200 open at once, the logs showed the following:

• Execution environments used: 5.
• Total requests handled: 484.
• Peak concurrency on a single environment: 59.
• Total CPU consumed across the whole fleet: 4.75 seconds.

One environment peaked at 59 connections, near the default limit of 64. Lambda adds environments before existing ones reach their limit, so this load used five. Event Functions serve one request per environment, so 200 open streams would need up to 200 environments. Across 484 requests, the whole fleet used about 4.8 seconds of CPU.

Each environment reports only its own connections, because Lambda routes each request independently with no session affinity.

Going multi-Region

The same app can serve each caller from the Region with the lowest latency. Because an endpoint’s type and Regions are fixed when you create it, going global means adding a new endpoint:

aws lambda-web deploy --name lambda-web-sse-dashboard --code . --endpoint-name global --endpoint-type MultiRegion --regions us-east-1,eu-west-2

Figure 5: AWS CLI creating a global endpoint.

Lambda replicates your code and revision configuration, including environment variables, to each Region, and centralizes logs in the home Region. In this case, Lambda replicates to Europe (London) Region (eu-west-2). It provides a single global domain URL and uses Amazon Route 53 latency-based routing to send each request to the lowest-latency Region.

Browse to the new global URL which is now in the format: https://<domain>.lambda-web.global.on.aws

Figure 6: The app dashboard showing routing to the local Region.

From my laptop in the United Kingdom, the request routes to eu-west-2 (London) in 16 ms, where the single-Region endpoint in us-east-1 replied in 90 ms. These are single measurements from one location, not a benchmark. Each request goes to the Region with the lowest latency for the caller through the same URL. Lambda manages the DNS routing, so there are no load balancers or Route 53 records to configure.

Authentication and running behind Amazon CloudFront

Each endpoint has an authentication type that you can change without redeploying code.

With ApplicationManaged, the endpoint is public, and your code handles authentication (OAuth, JWT, a session cookie, or Amazon Cognito). Administrators can prevent public endpoints across an organization with a service control policy that uses the lambda:Request/WebFunctionAuthType condition key.

With IamAuth, Lambda validates each request with AWS Signature Version 4 (SigV4) for single-Region endpoints and SigV4a for multi-Region endpoints. Only callers with the lambda:InvokeWebFunctionEndpoint permission reach your app, which suits service-to-service APIs. Endpoints are always reachable over the public internet, and IamAuth controls who can invoke them.

Endpoints use a Lambda-provided domain. For a custom domain, edge caching, and AWS WAF, place Amazon CloudFront in front of the endpoint. Use IamAuth with origin access control so the Lambda domain cannot bypass CloudFront which still passes your application’s own Authorization header through to your code.

Build it with an agent

Because deployment is a single aws lambda-web deploy command and the code is an ordinary HTTP server, you can scaffold, build, and deploy an app from a prompt using AWS skills for coding agents. The Agent Toolkit for AWS includes Web Functions.

Clean up

To remove the default resources that the aws lambda-web deploy command created, see the cleanup instructions in the repo.

Conclusion

AWS Lambda Web Functions bring the serverless operating model to web applications without adapter layers. You write a standard Node.js HTTP server, deploy it with one command or CloudFormation, and get a scalable, pay-per-use HTTPS endpoint with global routing built in. Deploy the sample application from the Serverless Patterns collection along with other Web Functions patterns. Read the Lambda Web Functions documentation to go deeper and the Lambda roadmap.

For more serverless learning resources, visit Serverless Land.

Post Syndicated from Rahul Shandilya original https://aws.amazon.com/blogs/compute/improving-lambda-function-latency-with-scalable-network-bandwidth/

AWS Lambda now supports scalable network bandwidth for functions configured with 2,048 MB of memory or more, running outside of a virtual private cloud (VPC). Previously, sustained network throughput was capped at 625 Mbps regardless of your function’s memory configuration. Now, sustained throughput scales proportionally from 625 Mbps at configurations below 2,048 MB up to 3,000 Mbps at 10,240 MB, increasing the rate at which data moves to and from your execution environment.

In this post, you learn how to apply this new capability to latency-sensitive data processing workloads, helping reduce function execution times and per-invocation costs while improving the end-user experience through reduced latency. You also walk through a deployable implementation that demonstrates the performance improvements this capability unlocks.

Latency-sensitive data processing

Latency-sensitive data processing applications are data processing workloads that must be completed in a defined period of time. They often experience bursty, ad hoc traffic patterns while being required to download gigabytes or even terabytes of data from a data store, process it in a compute environment, and return a result to a waiting end user.

Latency-sensitive data processing is often highly parallelizable. Data can be divided into smaller pieces with each piece being individually processed before combining them together to obtain a result.

These workloads can be found in multiple industries and verticals. Examples include:

• Log querying engines – An end user initiates an on-demand search across terabytes of log data and expects results within seconds.
• Insurance underwriting – A prospective customer submits an application, triggering real-time evaluation of historical claims and risk data. The underwriting process determines what coverage and premiums to offer to the prospective customer.
• Financial ETL pipelines – An economic announcement triggers an unexpected burst of market data that must be ingested, transformed, and made available to downstream trading systems before the next market tick.
• Genomics platforms – A clinician orders a diagnostic test, requiring gigabytes of DNA or RNA sequencing data to pass through a bioinformatics pipeline and be compared against a reference genome while the patient awaits results.

These workloads are challenging to build on traditional compute clusters. Their spiky and unpredictable nature forces you to choose between under-provisioning compute to optimize costs (and risk missing your SLA) or over-provisioning and paying for idle capacity.

Why Lambda fits latency-sensitive data processing

Lambda eliminates this tradeoff. Instead of pre-provisioning a compute cluster, Lambda scales compute capacity in response to incoming requests, matching processing power to unpredictable traffic patterns. Because latency-sensitive data processing is highly parallelizable, the ability of Lambda to rapidly scale out execution environments makes it a natural fit. You can fan out across thousands of concurrent functions to process data in parallel, paying only for the compute you use.

However, as data volume and performance requirements grow, network bandwidth to and from the compute environment can become the limiting factor in minimizing workload latency.

Scalable network bandwidth directly addresses this limitation by raising the per-environment network throughput ceiling, improving the rate at which data can be transferred to and from the execution environment. Each execution environment can now drive up to 3,000 Mbps of sustained throughput when configured with 10,240 MB of memory, a 4.8x increase from the previous ceiling of 625 Mbps. Combined with the ability of Lambda to scale out at a rate of 1,000 execution environments every 10 seconds, you can download more than 3 TB of data in under 10 seconds.

New network throughput behavior for Lambda functions

Scalable network bandwidth applies to both data ingress to and egress from an execution environment for functions outside of a VPC. For functions configured with 2 GB of memory or more, network bandwidth scales by approximately 280 Mbps increments for every 1 GB of additional memory allocated.

The following table shows the maximum sustained bandwidth available to each execution environment at each memory configuration.

Table 1. Lambda sustained network bandwidth by memory configuration. Bandwidth scales at ~280 Mbps per additional GB of memory above 2 GB.

In the following section, you learn how scalable network bandwidth improves end-user latency by building an ETL pipeline that demonstrates it. You can find the source code in the GitHub repository.

Solution overview

Consider a SaaS analytics platform where users submit ad hoc queries against a data store. The application must extract the relevant data, apply a filter or transformation, and return an aggregate result while the user waits. In this example, the result needs to be returned in 8 seconds or less.

The following diagram illustrates the architecture of the solution.

Figure 1. ETL fan-out pattern: an orchestrator Lambda function distributes work to multiple worker Lambda functions that read from Amazon S3 in parallel, with bandwidth scaling callouts per memory tier.

A client initiates an ad hoc query by calling the orchestrator Lambda function through the Lambda API. The orchestrator function determines how to split the work. To process the data in parallel, the orchestrator function uses a ThreadPoolExecutor to issue synchronous invoke requests to the Lambda worker function, fanning out the worker across multiple execution environments at the same time.

Each Lambda worker function is configured with 10,240 MB of memory, so it has access to up to 3,000 Mbps of sustained network throughput. After the data is processed, the aggregated result is returned to the client.

Prerequisites

Before you start the deployment process, make sure that you have completed the following steps:

1. Install the AWS SAM CLI on your computer and confirm that you are running Python 3.12 or later.
2. Have your AWS account credentials ready.
3. Submit a request to AWS Service Quotas to turn on scalable network bandwidth for your Lambda functions. This quota is listed under Network bandwidth per execution environment.

Clone the source code from the GitHub repo and deploy the application within your AWS account. Creating the 10 GB test dataset and running the benchmark can incur charges to your AWS account.

git clone https://github.com/aws-samples/sample-lambda-enhanced-bandwidth
cd sample-lambda-enhanced-bandwidth
sam build
sam deploy --guided

After the AWS CloudFormation stack is deployed, record the DataBucketName and orchestrator function name from the stack outputs to use in subsequent commands.

To simulate data for the end user to query, the GitHub repo has a script that creates 10 GB of synthetic data and uploads it to your S3 bucket.

Mode 1: Processing pre-partitioned data

In Mode 1, the 10 GB of synthetic data is pre-partitioned. Pre-partitioned data is typically produced incrementally by many sources over a period of time, which can be the case with IoT data or access logs. The following command creates 10 GB of data divided into 20 partitions that are 512 MB each.

python scripts/generate_data.py \
    --bucket <DATA_BUCKET_NAME> \
    --total-gb 10 \
    --chunk-mb 512

Turning on scalable network bandwidth does not, on its own, make your downloads faster. A single download request only opens one connection to Amazon S3, and one connection does not move data fast enough to fill all the bandwidth now available to your Lambda function. To actually use your full allotment of network bandwidth, the execution environment has to pull the data over several connections at once. It does this by preferring the AWS Common Runtime (CRT) transfer client, a high-performance download engine built into Boto3. When the worker calls download_fileobj, the CRT client automatically breaks the 512 MB object into smaller parts and downloads them in parallel across multiple Amazon S3 requests. Those parallel downloads are what let a single Lambda worker take advantage of its full network bandwidth.

The following command runs the benchmark on the pre-partitioned data.

python scripts/run_fanout_benchmark.py \
    --orchestrator-name <STACK_NAME>-orchestrator \
    --bucket <DATA_BUCKET_NAME> \
    --iterations 5

Mode 2: Processing single large objects

Mode 2 generates 10 GB of data in one large object. This arrangement is more common when data is produced or delivered as one complete unit, such as database backups or genomic datasets. The following command creates 10 GB of data in a single large object.

python scripts/generate_data.py \
    --bucket <DATA_BUCKET_NAME> \
    --single-object-gb 10 \
    --key large-object/large-file.bin

In Mode 1, the CRT preference applies to Boto3 managed transfer methods such as download_file and download_fileobj. Mode 2 takes a different approach. Each worker reads a specific byte range of a single large object using get_object. The CRT preference setting has no effect on these calls. Instead, you can control concurrency by explicitly tuning the number of Lambda workers and using a bounded ThreadPoolExecutor to issue multiple byte-range requests at the same time.

When you run the following command, the orchestrator takes the single large object and divides it into consecutive byte ranges of 512 MB each. Each of the individual ranges is then processed by a Lambda worker execution environment in parallel.

python scripts/run_fanout_benchmark.py \
    --orchestrator-name <STACK_NAME>-orchestrator \
    --bucket <DATA_BUCKET_NAME> \
    --key large-object/large-file.bin \
    --slice-size-mb 512 \
    --iterations 5

The benchmark reports wall-clock duration, client-observed duration, aggregate throughput across workers, worker completion counts, and target compliance. When comparing memory configurations, keep the code, dataset, AWS Region, partition count, warm-up policy, and measurement count identical. You should run the benchmark multiple times in your account because placement, cold starts, concurrency, S3 behavior, and execution-environment reuse could affect results.

Results

To compare results, we ran the benchmark using a baseline configuration where the worker Lambda function is configured with only 1,024 MB of memory, well below the 2,048 MB threshold required for scalable network bandwidth to take effect. The 1,024 MB configuration limits network throughput to the previous sustained ceiling of 625 Mbps.

In our baseline test run, a worker downloaded and processed a single 512 MB partition with a 6.61-second download time at a 649.8 Mbps throughput (at p50). The 649.8 Mbps throughput exceeds the 625 Mbps ceiling because Lambda is capable of bursts in network throughput over a short period of time. Across twenty measured fan-out queries, the complete 10 GB query was completed with a 7.113-second wall-clock at p50. This fits within the 8-second SLA but leaves very little headroom.

To run our scalable network bandwidth benchmark, we re-deployed our worker Lambda function with a 10,240 MB memory configuration and re-ran the application. At a 10,240 MB memory configuration, each execution environment can now access up to 3,000 Mbps in sustained throughput. Direct 512 MB downloads achieved a 1.70-second download time and 2,521.3 Mbps throughput (both at p50). The complete 10 GB query was completed with a 2.640-second wall-clock at p50. That is 2.69 times faster, or 62.9% lower median latency, than the 1,024 MB configuration.

Table 2 summarizes the direct worker and end-to-end fan-out measurements for the same 10 GB dataset and 20 × 512 MB orchestration pattern. Aggregate throughput is the total data transfer rate across all twenty execution environments spun up to run the benchmark.

Table 2. Measured 1,024 MB baseline tier and 10,240 MB scalable bandwidth performance for a 10 GB fan-out ETL query.

Using scalable network bandwidth, the customer’s SLA headroom has improved by nearly 5 seconds. The Lambda function can now handle larger partitions within the same SLA window, reducing costs while still remaining comfortably within the customer’s SLA.

Clean up

To clean up the resources you created for the benchmark test, run the following commands:

aws s3 rm s3://<DATA_BUCKET_NAME> --recursive
sam delete --stack-name <STACK_NAME>

Best practices

After scalable network bandwidth is turned on for your AWS account, the following practices help you get the most out of it.

Profiling and planning

• Test before you tune. Not every function is network-bound. Before increasing memory, profile your function to confirm that network I/O is the primary contributor to invocation duration and not CPU or application logic. Use Amazon CloudWatch Lambda Insights to inspect rx_bytes, tx_bytes, and duration. Functions where network I/O dominates invocation time are prime candidates for tuning.
• Design for parallelism. Break your data into parallelizable chunks that can be processed independently in a fan-out pattern across multiple execution environments. You can use Amazon S3 byte-range reads to split large files into independently downloadable partitions. For implementation details, see Downloading an object with part numbers in the Amazon S3 User Guide.
• Run AWS Lambda Power Tuning. Lambda Power Tuning is a state machine that helps you optimize your Lambda functions for cost and performance. Use Power Tuning to sweep memory configurations from 1,024 MB to 10,240 MB and identify the optimal cost-vs-latency point for your workload.

Implementation

• Check upstream and downstream limits. Check the throughput limits of your data sources. For example, a Lambda function running at 3,000 Mbps can exceed the throughput capacity of a single S3 prefix, which supports up to 5,500 GET requests per second. When this happens, you will see HTTP 503 (Slow Down) errors in your application logs. Distribute your S3 objects across multiple prefixes to parallelize reads and avoid per-prefix throttling.
• Balance bandwidth and CPU. Lambda allocates CPU proportionally to memory. For example, at a 1.7 GB memory configuration you are allocated 1 vCPU while a 10 GB memory configuration is allocated up to 6 vCPU. If your function processes data in parallel threads, the higher memory tiers give you both more network bandwidth and more CPU to process it. Use the concurrent.futures module in Python or worker_threads in Node.js to process data across parallel threads and maximize both CPU and network utilization.
• Turn on Amazon S3 CRT for Boto3. If your function uses the Python runtime, initialize your Amazon S3 client with preferred_transfer_client: 'crt' to maximize single-connection throughput. The AWS Common Runtime automatically parallelizes requests across multiple TCP connections, which matters because individual TCP connections have a throughput ceiling.
• Use SnapStart for JVM workloads. If you use Lambda SnapStart for Java functions, scalable network bandwidth reduces afterRestore hook latency. Network activity that occurs during function restore, such as pre-warming connections or pre-fetching configuration data, can complete faster.

Conclusion

Scalable network bandwidth raises the per-environment sustained throughput ceiling of AWS Lambda from 625 Mbps to 3,000 Mbps, directly reducing end-to-end latency for data-intensive workloads. Combined with the Lambda scaling rate, you can now move terabytes of data in seconds, without provisioning or managing infrastructure.

To get started, request the Network bandwidth per execution environment quota increase through AWS Service Quotas and deploy the sample application from the GitHub repository to see the improvement firsthand.