Post Syndicated from James Beswick original https://aws.amazon.com/blogs/compute/migrating-aws-lambda-functions-from-the-go1-x-runtime-to-the-custom-runtime-on-amazon-linux-2/

This post is written by Micah Walter, Senior Solutions Architect, Yanko Bolanos, Senior Solutions Architect, and Ramesh Mathikumar, Senior DevOps Consultant.

This blog post describes our plans to improve performance and streamline the user experience for customers writing AWS Lambda functions using Go.

Today, customers using Go with Lambda can either use the go1.x runtime, or use the provided.al2 runtime. Going forward, we plan to deprecate the go1.x runtime in line with the end-of-life of Amazon Linux 1, currently scheduled for December 31, 2023.

Customers using the go1.x runtime should migrate their functions to the provided.al2 runtime to continue to benefit from the latest runtime updates and security patches. Customers who deploy Go functions using container images who are currently using the go1.x base container image should similarly migrate to the provided.al2 base image.

Using the provided.al2 runtime offers several benefits over the go1.x runtime. First, it supports running Lambda functions on AWS Graviton2 processors, offering up to 34% better price-performance compared to functions running on x86_64 processors. Second, it offers a streamlined implementation with a smaller deployment package and faster function invoke path. Finally, this change aligns Go with other languages that also compile to native code such as Rust or C++, which also run on the provided.al2 runtime.

This migration does not require any code changes. The only changes relate to how you build your deployment package and configure your function. This blog post outlines the steps required to update your build scripts and tooling to use the provided.al2 runtime for your Go functions.

There is a difference in Lambda billing between the go1.x runtime and the provided.al2 runtime. With the go1.x runtime, Lambda does not bill for time spent during function initialization (cold start), whereas with the provided.al2 runtime Lambda includes function initialization time in the billed function duration. Since Go functions typically initialize very quickly, and since Lambda reduces the number of initializations by re-using function execution environments for multiple function invokes, in practice the difference in your Lambda bill should be very small.

Compiling for the provided.al2 runtime

In order to run a compiled Go application on Lambda, you must compile your code for Linux. While the go1.x runtime allows you to use any executable name, the provided.al2 runtime requires you to use bootstrap as the executable name. On macOS and Linux, here’s the simplest form of the build command:

GOARCH=amd64 GOOS=linux go build -o bootstrap main.go

This build command creates a Go binary file called bootstrap compatible with the x86_64 instruction set for Lambda. To compile for AWS Graviton processors, set GOARCH=arm64 in the preceding command.

The final step is to compress this binary into a ZIP file deployment package, ready to deploy to Lambda:

zip myFunction.zip bootstrap

For users compiling on Windows, Go supports compiling for Linux without using a Linux virtual machine or build container. However, Lambda uses POSIX file permissions, which must be set correctly. Lambda provides a helper tool which builds a deployment package that is valid for Lambda—see the Lambda documentation for details. Existing users of this tool should update to the latest version to make sure their build scripts are up-to-date.

Removing the RPC dependency

The go1.x runtime uses two processes within the Lambda execution environment to route requests to your handler function. The first process, which is included in the runtime, retrieves function invocation requests from the Lambda runtime API, and uses RPC to pass the invoke to the second process. This second process runs the executable which you deploy, and comprises the aws-lambda-go package and your function code. The aws-lambda-go package receives the RPC request and executes your function.

The following runtime architecture diagram for the go1.x runtime shows the runtime client process calling the runtime API to retrieve a function invocation and using RPC to call a separate process containing the function code.

Go functions deployed to the provided.al2 runtime use a simpler, single-process architecture. When building the Go executable, you include the same aws-lambda-go package as before. However, in this case the aws-lambda-go package acts as the runtime client, retrieving invocation requests from the runtime API directly.

The following runtime architecture diagram shows a Go function running on the provided.al2 runtime. A single process retrieves the function invocation from the runtime API and executes the function code.

Removing the additional process and RPC hop streamlines the function execution path, resulting in faster invokes. You can also remove the RPC component from the aws-lambda-go package, giving a smaller binary size and faster code loading during cold starts. To remove the RPC dependency, add the lambda.norpc tag to your build command:

GOARCH=amd64 GOOS=linux go build -tags lambda.norpc -o bootstrap main.go

Creating a new Lambda function

Once your deployment package is ready, you can create a new Lambda function using the provided.al2 runtime using the Lambda console:

Migrating existing functions

If you have existing Lambda functions that use the go1.x runtime, you can migrate these functions by following these steps:

1. Recompile your binary using the preceding commands, making sure to name your binary bootstrap.
2. If you are using the same instruction set architecture, open the runtime settings and switch the runtime to “Provide your own bootstrap on Amazon Linux 2”.
3. Upload the new version of your binary as a zip file.
   

Note: The handler value is not used by the provided.al2 runtime, nor the aws-lambda-go library, and may be set to any value. We recommend the setting the value to bootstrap to help with migrating between go1.x and provided.al2.

To switch instruction set architecture to Graviton (arm64), save your changes, and then re-open the runtimes settings to make the architecture change.

Migrating Go1.x Lambda container images

Lambda allows you to run your Go code as a container image. Customers that are using the go1.x base image for Lambda containers must migrate to the provided.al2 base image. The Lambda documentation includes instructions on how to build and deploy Go functions using the provided.al2 base image.

The following Dockerfile uses a two-stage build to avoid unnecessary layers and files in your final image. The first stage of the process builds the application. This stage installs Go, downloads the dependencies for the code, and compiles the binary. The second stage copies the executable to a new container without the dependencies of the build process.

1. Create a Dockerfile in your project directory:

   FROM public.ecr.aws/lambda/provided:al2 as build
   # install compiler
   RUN yum install -y golang
   RUN go env -w GOPROXY=direct
   # cache dependencies
   ADD go.mod go.sum ./
   RUN go mod download
   # build
   ADD . .
   RUN go build -tags lambda.norpc -o /main
   # copy artifacts to a clean image
   FROM public.ecr.aws/lambda/provided:al2
   COPY --from=build /main /main
   ENTRYPOINT [ "/main" ]           
   

2. Build your Docker image with the Docker build command:

   docker build -t hello-world .

3. Authenticate the Docker CLI to your Amazon ECR registry:

   aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com 

4. Tag and push your image to the Amazon ECR registry:

   docker tag hello-world:latest 123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest
   
   docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest

You can now create or update your Go Lambda function to use the new container image.

Changes to tooling

To migrate your functions from the go1.x runtime to the provided.al2 runtime, you must make configuration changes to your build scripts or CI/CD configurations. Here are some common examples.

Makefiles and build scripts

If you use Makefiles or custom build scripts to build Go functions, you must modify to ensure the executable file is named bootstrap when deploying to the provided.al2 runtime.

Here is an example Makefile which compiles the main.go file into an executable called bootstrap in the bin folder. It also creates a zip file, which you can deploy to Lambda using the console or via the AWS CLI.

GOARCH=arm64 GOOS=linux go build -tags lambda.norpc -o ./bin/bootstrap
(cd bin && zip -FS bootstrap.zip bootstrap)

CloudFormation

If you deploy your Lambda functions using AWS CloudFormation templates, change the Handler and Runtime settings under Properties:

Resources:
  Function:
    Type: AWS::Serverless::Function
    Properties:
      Handler: bootstrap
      Runtime: provided.al2
      ... # Other required properties

AWS Serverless Application Model

If you use the AWS Serverless Application Model (AWS SAM) to build and deploy your Go functions, make the same changes to the Handler and Runtime settings as for CloudFormation. You must also add the BuildMethod:

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: go1.x 
    Properties:
      CodeUri: hello-world/ # folder where your main program resides
      Handler: bootstrap
      Runtime: provided.al2
      Architectures:
        - x86_64

Cloud Development Kit (CDK)

If you use the AWS Cloud Development Kit (AWS CDK), you can compile your Go executable and place it under a folder in your project. Next, specify the location by using awslambda.Code_FromAsset, and AWS CDK packages the binary into a zip file and uploads it.

// Go CDK
awslambda.NewFunction(stack, jsii.String("HelloHandler"), &awslambda.FunctionProps{
    Code:         awslambda.Code_FromAsset(jsii.String("lambda"), nil), //folder where bootstrap executable is located
    Runtime:      awslambda.Runtime_PROVIDED_AL2(),
    Handler:      jsii.String("bootstrap"), // Handler named bootstrap
    Architecture: awslambda.Architecture_ARM_64(),
})

Taking this further, AWS CDK can perform build commands as part of your AWS CDK build process by using the native AWS CDK bundling functionality. With the bundling parameter, AWS CDK can perform steps before staging the files in the cloud assembly. Instead of placing the binary file, place the Go code in a folder and use the Bundling option to compile the code in a Docker container.

This example uses the golang:1.20.1 Docker image. After compilation, AWS CDK creates a zip file with the binary and creates the Lambda function:

// Go CDK
awslambda.NewFunction(stack, jsii.String("HelloHandler"), &awslambda.FunctionProps{
        Code: awslambda.Code_FromAsset(jsii.String("go-lambda"), &awss3assets.AssetOptions{
                Bundling: &awscdk.BundlingOptions{
                        Image: awscdk.DockerImage_FromRegistry(jsii.String("golang:1.20.1")),
                        Command: &[]*string{
                                jsii.String("bash"),
                                jsii.String("-c"),
                                jsii.String("GOCACHE=/tmp go mod tidy && GOCACHE=/tmp GOARCH=arm64 GOOS=linux go build -tags lambda.norpc -o /asset-output/bootstrap"),
                        },
                },
        }),
        Runtime:      awslambda.Runtime_PROVIDED_AL2(),
        Handler:      jsii.String("bootstrap"),
        Architecture: awslambda.Architecture_ARM_64(),
})

Conclusion

Lambda is deprecating the go1.x runtime in line with Amazon Linux 1 end-of-life, scheduled for December 31, 2023. Customers using Go with Lambda should migrate their functions to the provided.al2 runtime. Benefits include support for AWS Graviton2 processors with better price-performance, and a streamlined invoke path with faster performance.

For more serverless learning resources, visit Serverless Land.

Post Syndicated from James Beswick original https://aws.amazon.com/blogs/compute/understanding-aws-lambdas-invoke-throttle-limits/

This post is written by Archana Srikanta, Principal Engineer, AWS Lambda.

When you call AWS Lambda’s Invoke API, a series of throttle limits are evaluated to decide if your call is let through or throttled with a 429 “Too Many Requests” exception. This blog post explains the most common invoke throttle limits and the relationship between them, so you can better understand scaling workloads on Lambda.

Overview

The throttle limits exist to protect the following components of Lambda’s internal service architecture, and your workload, from noisy neighbors:

• Execution environment: An execution environment is a Firecracker microVM where your function code runs. A given execution environment only hosts one invocation at a time, but it can be reused for subsequent invocations of the same function version.
• Invoke data plane: These are a series of internal web services that, on an invoke, select (or create) a sandbox and route your request to it. This is also responsible for enforcing the throttle limits.

When you make an Invoke API call, it transits through some or all of the Invoke Data Plane services, before reaching an execution environment where your function code is downloaded and executed.

There are three distinct but related throttle limits which together decide if your invoke request is accepted by the data plane or throttled.

Concurrency

Concurrent means “existing, happening, or done at the same time”. Accordingly, the Lambda concurrency limit is a limit on the simultaneous in-flight invocations allowed at any given time. It is not a rate or transactions per second (TPS) limit in and of itself, but instead a limit on how many invocations can be inflight at the same time. This documentation visually explains the concept of concurrency.

Under the hood, the concurrency limit roughly translates to a limit on the maximum number of execution environments (and thus Firecracker microVMs) that your account can claim at any given point in time. Lambda runs a fleet of multi-tenant bare metal instances, on which Firecracker microVMs are carved out to serve as execution environments for your functions. AWS constantly monitors and scales this fleet based on incoming demand and shares the available capacity fairly among customers.

The concurrency limit helps protect Lambda from a single customer exhausting all the available capacity and causing a denial of service to other customers.

Transactions per second (TPS)

Customers often ask how their concurrency limit translates to TPS. The answer depends on how long your function invocations last.

The diagram above considers three cases, each with a different function invocation duration, but a fixed concurrency limit of 1000. In the first case, invocations have a constant duration of 1 second. This means you can initiate 1000 invokes and claim all 1000 execution environments permitted by your concurrency limit. These execution environments remain busy for the entire second, and you cannot start any more invokes in that second because your concurrency limit prevents you from claiming any more execution environments. So, the TPS you can achieve with a concurrency limit of 1000 and a function duration of 1 second is 1000 TPS.

In case 2, the invocation duration is halved to 500ms, with the same concurrency limit of 1000. You can initiate 1000 concurrent invokes at the start of the second as before. These invokes keep the execution environments busy for the first half of the second. Once finished, you can start an additional 1000 invokes against the same execution environments while still being within your concurrency limit. So, by halving the function duration, you doubled your TPS to 2000.

Similarly, in case 3, if your function duration is 100ms, you can initiate 10 rounds of 1000 invokes each in a second, achieving a TPS of 10K.

Codifying this as an equation, the TPS you can achieve given a concurrency limit is:

TPS = concurrency / function duration in seconds

Taken to an extreme, for a function duration of only 1ms and at a concurrency limit of 1000 (the default limit), an account can drive an invoke TPS of one million. For every additional unit of concurrency granted via a limit increase, it implicitly grants an additional 1000 TPS per unit of concurrency increased. The high TPS doesn’t require any additional execution environments (Firecracker microVMs), so it’s not problematic from a fleet capacity perspective. However, driving over a million TPS from a single account puts stress on the Invoke Data Plane services. They must be protected from noisy neighbor impact as well so all customers have a fair share of the services’ bandwidth. A concurrency limit alone isn’t sufficient to protect against this – the TPS limit provides this protection.

As of this writing, the invoke TPS is capped at 10 times your concurrency. Added to the previous equation:

TPS = min( 10 x concurrency, concurrency / function duration in seconds)

The concurrency factor is common across both terms in the min function, so the key comparison is:

min(10, 1 / function duration in seconds)

If the function duration is exactly 100ms (or 1/10th of a second), both terms in the min function are equal. If the function duration is over 100ms, the second term is lower and TPS is limited as per concurrency/function duration. If the function duration is under 100ms, the first term is lower and TPS is limited as per 10 x concurrency.

To summarize, the TPS limit exists to protect the Invoke Data Plane from the high churn of short-lived invocations, for which the concurrency limit alone affords too high of a TPS. If you drive short invocations of under 100ms, your throughput is capped as though the function duration is 100ms (at 10 x concurrency) as shown in the diagram above. This implies that short lived invocations may be TPS limited, rather than concurrency limited. However, if your function duration is over 100ms you can effectively ignore the 10 x concurrency TPS limit and calculate your available TPS as concurrency/function duration.

Burst

The third throttle limit is the burst limit. Lambda does not keep execution environments provisioned for your entire concurrency limit at all times. That would be wasteful, especially if usage peaks are transient, as is the case with many workloads. Instead, the service spins up execution environments just-in-time as the invoke arrives, if one doesn’t already exist. Once an execution environment is spun up, it remains “warm” for some period of time and is available to host subsequent invocations of the same function version.

However, if an invoke doesn’t find a warm execution environment, it experiences a “cold start” while we provision a new execution environment. Cold starts involve certain additional operations over and above the warm invoke path, such as downloading your code or container and initializing your application within the execution environment. These initialization operations are typically computationally heavy and so have a lower throughput compared to the warm invoke path. If there are sudden and steep spikes in the number of cold starts, it can put pressure on the invoke services that handle these cold start operations, and also cause undesirable side effects for your application such as increased latencies, reduced cache efficiency and increased fan out on downstream dependencies. The burst limit exists to protect against such surges of cold starts, especially for accounts that have a high concurrency limit. It ensures that the climb up to a high concurrency limit is gradual so as to smooth out the number of cold starts in a burst.

The algorithm used to enforce the burst limit is the Token Bucket rate-limiting algorithm. Consider a bucket that holds tokens. The bucket has a maximum capacity of B tokens (burst). The bucket starts full. Each time you send an invoke request that requires an additional unit of concurrency, it costs a token from the bucket. If the token exists, you are granted the additional concurrency and the token is removed from the bucket. The bucket is refilled at a constant rate of r tokens per minute (rate) until it reaches its maximum capacity.

What this means is that the rate of climb of concurrency is limited to r tokens per minute. Even though the algorithm allows you to collect up to B tokens and burst, you must wait for the bucket to refill before you can burst again, effectively limiting your average rate to r per minute.

The chart above shows the burst limit in action with a maximum concurrency limit of 3000, a maximum burst(B) of 1000 and a refill rate(r) of 500/minute. The token bucket starts full with 1000 tokens, as is the available burst headroom.

There is a burst activity between minute one and two, which consumes all tokens in the bucket and claims all 1000 concurrent execution environments allowed by the burst limit. At this point the bucket is empty and any attempt to claim additional concurrent execution environments is burst throttled, in spite of max concurrency not being reached yet.

The token bucket and the burst headroom are replenished at minutes two and three with 500 tokens each minute to bring it back up to its maximum capacity of 1000. At minute four, there is no additional refill because the bucket is at maximum capacity. Between minutes four and five, there is a second burst activity which empties the bucket again and claims an additional 1000 execution environments, bringing the total number of active execution environments to 2000.

The bucket continues to replenish at a rate of 500/minute at minutes five and six. At this point, sufficient tokens have been accumulated to cover the entire concurrency limit of 3000, and so the bucket isn’t refilled anymore even when you have the third burst activity at minute seven. At minute ten, when all the usage ramps down, the available burst headroom slowly stair steps back down to the maximum initial burst of 1K.

The actual numbers for maximum burst and refill rate vary by Region and are subject to change, please visit the Lambda burst limits page for specific values.

It is important to distinguish that the burst limit isn’t a rate limit on the invoke itself, but a rate limit on how quickly concurrency can rise. However, since invoke TPS is a function of concurrency, it also clamps how quickly TPS can rise (a rate limit for a rate limit). The following chart shows how the TPS burst headroom follows a similar stair step pattern as the concurrency burst headroom, only with a multiplier.

Conclusion

This blog explains three key throttle limits applied on Lambda invokes: the concurrency limit, TPS limit and burst limit. It outlines the relationship between these limits and how each one protects the system and your workload from noisy neighbors. Equipped with this knowledge you can better interpret any 429 throttling exceptions you may receive while scaling your applications on Lambda. For more information on getting started with Lambda visit the Developer Guide.

For more serverless learning resources, visit Serverless Land.

Post Syndicated from James Beswick original https://aws.amazon.com/blogs/compute/building-private-serverless-apis-with-aws-lambda-and-amazon-vpc-lattice/

This post was written by Josh Kahn, Tech Leader, Serverless.

Amazon VPC Lattice is a new, generally available application networking service that simplifies connectivity between services. Builders can connect, secure, and monitor services on instances, containers, or serverless compute in a simplified and consistent manner.

VPC Lattice supports AWS Lambda functions as both a target and a consumer of services. This blog post explores how to incorporate VPC Lattice into your serverless workloads to simplify private access to HTTP-based APIs built with Lambda.

Overview

VPC Lattice is an application networking service that enables discovery and connectivity of services across VPCs and AWS accounts. VPC Lattice includes features that allow builders to define policies for network access, traffic management, and monitoring. It also supports custom domain names for private endpoints.

VPC Lattice is composed of several key components:

• Service network – a logical grouping mechanism for a collection of services on which you can apply common policies. Associate one or more VPCs to allow access from services in the VPC to the service network.
• Service – a unit of software that fulfills a specific task or function. Services using VPC Lattice can run on instances, containers, or serverless compute. This post focuses on services built with Lambda functions.
• Target group – in a serverless application, a Lambda function that performs business logic in response to a request. Routing rules within the service route requests to the appropriate target group.
• Auth policy – an AWS Identity and Access Management (IAM) resource policy that can be associated with a service network and a service that defines access to those services.

VPC Lattice enables connectivity across VPC and account boundaries, while alleviating the complexity of the underlying networking. It supports HTTP/HTTPS and gRPC protocols, though gRPC is not currently applicable for Lambda target groups.

VPC Lattice and Lambda

Lambda is one of the options to build VPC Lattice services. The AWS Lambda console supports VPC Lattice as a trigger, similar to previously existing triggers such as Amazon API Gateway and Amazon EventBridge. You can also connect VPC Lattice as an event source using infrastructure as code, such as AWS CloudFormation and Terraform.

To configure VPC Lattice as a trigger for a Lambda function in the Console, navigate to the desired function and select the Configuration tab. Select the Triggers menu on the left and then choose Add trigger.

The trigger configuration wizard allows you to define a new VPC Lattice service provided by the Lambda function or to add to an existing service. When adding to an existing service, the wizard allows configuration of path-based routing that sends requests to the target group that includes the function. Path-based and other routing mechanisms available from VPC Lattice are useful in migration scenarios.

This example shows creating a new service. Provide a unique name for the service and select the desired VPC Lattice service network. If you have not created a service network, follow the link to create a new service network in the VPC Console (to create a new service network, read the VPC Lattice documentation).

The listener configuration allows you to configure the protocol and port on which the service is accessible. HTTPS (port 443) is the default configuration, though you can also configure the listener for HTTP (port 80). Note that configuring the listener for HTTP does not change the behavior of Lambda: it is still invoked by VPC Lattice over an HTTPS endpoint, but the service endpoint is available as HTTP. Choose Add to complete setup.

In addition to configuring the VPC Lattice service and target group, the Lambda wizard also adds a resource policy to the function that allows the VPC Lattice target group to invoke the function.

VPC Lattice integration

When a client sends a request to a VPC Lattice service backed by a Lambda target group, VPC Lattice synchronously invokes the target Lambda function. During a synchronous invocation, the client waits for the result of the function and all retry handling is performed by the client. VPC Lattice has an idle timeout of one minute and connection timeout of ten minutes to both the client and target.

The event payload received by the Lambda function when invoked by VPC Lattice is similar to the following example. Note that base64 encoding is dependent on the content type.

{
    "body": "{ "\userId\": 1234, \"orderId\": \"5C71D3EB-3B8A-457B-961D\" }",
    "headers": {
        "accept": "application/json, text/plain, */*",
        "content-length": "156",
        "user-agent": "axios/1.3.4",
        "host": "myvpclattice-service-xxxx.xxxx.vpc-lattice-svcs.us-east-2.on.aws",
        "x-forwarded-for": "10.0.129.151"
    },
    "is_base64_encoded": false,
    "method": "PUT",
    "query_string_parameters": {
        "action": "add"
    },
    "raw_path": "/points?action=add"
}

The response payload returned by the Lambda function includes a status code, headers, base64 encoding, and an optional body as shown in the following example. A response payload that does not meet the required specification results in an error. To return binary content, you must set isBase64Encoded to true.

{
    "isBase64Encoded": false,
    "statusCode": 200,
    "statusDescription": "200 OK",
    "headers": {
        "Set-Cookie": "cookies",
        "Content-Type": "application/json"
    },
    "body": "Hello from Lambda (optional)"
}

For more details on the integration between VPC Lattice and Lambda, visit the Lambda documentation.

Calling VPC Lattice services from Lambda

VPC Lattice services support connectivity over HTTP/HTTPS and gRPC protocols as well as open access or authorization using IAM. To call a VPC Lattice service, the Lambda function must be attached to a VPC that is associated to a VPC Lattice service network:

While a function that calls a VPC Lattice service must be associated with an appropriate VPC, a Lambda function that is part of a Lattice service target group does not need to be attached to a VPC. Remember that Lambda functions are always invoked via an AWS endpoint with access controlled by AWS IAM.

Calls to a VPC Lattice service are similar to sending a request to other HTTP/HTTPS services. VPC Lattice allows builders to define an optional auth policy to enforce authentication and perform context-specific authorization and implement network-level controls with security groups. Callers of the service must meet networking and authorization requirements to access the service. VPC Lattice blocks traffic if it does not explicitly meet all conditions before your function is invoked.

A Lambda function that calls a VPC Lattice service must have explicit permission to invoke that service, unless the auth type for the service is NONE. You provide that permission through a policy attached to the Lambda function’s execution role, for example:

{
    "Action": "vpc-lattice-svcs:Invoke",
    "Resource": "arn:aws:vpc-lattice:us-east-2:123456789012:service/svc-123abc/*",
    "Effect": "Allow"
}

If the auth policy associated with your service network or service requires authenticated requests, any requests made to that service must contain a valid request signature computed using Signature Version 4 (SigV4). An example of computing a SigV4 signature can be found in the VPC Lattice documentation. VPC Lattice does not support payload signing at this time. In TypeScript, you can sign a request using the AWS SDK and Axios library as follows:

import { SignatureV4 } from "@aws-sdk/signature-v4";
import { Sha256 } from "@aws-crypto/sha256-js";
import axios from "axios";

const endpointUrl = new URL(VPC_LATTICE_SERVICE_ENDPOINT);
const sigv4 = new SignatureV4({
    service: "vpc-lattice-svcs",
    region: process.env.AWS_REGION!,
    credentials: {
        accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
        secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
        sessionToken: process.env.AWS_SESSION_TOKEN
    },
    sha256: Sha256
});

const signedRequest = await sigv4.sign({
    method: "PUT",
    hostname: endpointUrl.host,
    path: endpointUrl.pathname,
    protocol: endpointUrl.protocol,
    headers: {
        'Content-Type': 'application/json',
        host: endpointUrl.hostname,
        // Include following header as VPC Lattice does not support signed payloads
        "x-amz-content-sha256": "UNSIGNED-PAYLOAD"
    }
  });    
  const { data } = await axios({
    ...signedRequest,
    data: {
        // some data
    },
    url: VPC_LATTICE_SERVICE_ENDPOINT
  });

VPC Lattice provides several layers of security controls, including network-level and auth policies, that allow (or deny) access from a client to your service. These controls can be implemented at the service network, applying those controls across all services in the network.

Connecting to any VPC Lattice service

VPC Lattice supports services built using Amazon EKS and Amazon EC2 in addition to Lambda. Calling services built using these other compute options looks exactly the same to the caller as the preceding sample. VPC Lattice provides an endpoint that abstracts how the service itself is actually implemented.

A Lambda function configured to access resources in a VPC can potentially access VPC Lattice services that are part of the service network associated with that VPC. IAM permissions, the auth policy associated with the service, and security groups may also impact whether the function can invoke the service (see VPC Lattice documentation for details on securing your services).

Services deployed to an Amazon EKS cluster can also invoke Lambda functions exposed as VPC Lattice services using native Kubernetes semantics. They can use either the VPC Lattice-generated domain name or a configured custom domain name to invoke the Lambda function instead of API Gateway or an Application Load Balancer (ALB). Refer to this blog post on the AWS Container Blog for details on how an Amazon EKS service invokes a VPC Lattice service with access control enabled.

Building private serverless APIs

With the launch of VPC Lattice, AWS now offers several options to build serverless APIs accessible only within your customer VPC. These options include API Gateway, ALB, and VPC Lattice. Each of these services offers a unique set of features and trade-offs that may make one a better fit for your workload than others.

Private APIs with API Gateway provide a rich set of features, including throttling, caching, and API keys. API Gateway also offers a rich set of authorization and routing options. Detailed networking and DNS knowledge may be required in complex environments. Both network-level and resource policy controls are available to control access and the OpenAPI specification allows schema sharing.

Application Load Balancer provides flexibility and a rich set of routing options, including to a variety of targets. ALB also can offer a static IP address via AWS Global Accelerator. Detailed networking knowledge is required to configure cross-VPC/account connectivity. ALB relies on network-level controls.

Service networks in VPC Lattice simplify access to services on EC2, EKS, and Lambda across VPCs and accounts without requiring detailed knowledge of networking and DNS. VPC Lattice provides a centralized means of managing access control and guardrails for service-to-service communication. VPC Lattice also readily supports custom domain names and routing features (path, method, header) that enable customers to build complex private APIs without the complexity of managing networking. VPC Lattice can be used to provide east-west interservice communication in combination with API Gateway and AWS AppSync to provide public endpoints for your services.

Conclusion

We’re excited about the simplified connectivity now available with VPC Lattice. Builders can focus on creating customer value and differentiated features instead of complex networking in much the same way that Lambda allows you to focus on writing code. If you are interested in learning more about VPC Lattice, we recommend the VPC Lattice User Guide.

To learn more about serverless, visit Serverless Land for a wide array of reusable patterns, tutorials, and learning materials.

Post Syndicated from James Beswick original https://aws.amazon.com/blogs/compute/understanding-techniques-to-reduce-aws-lambda-costs-in-serverless-applications/

This post is written by Josh Kahn, Tech Leader, and Chloe Jeon, Senior PMTes Lambda.

Serverless applications can lower the total cost of ownership (TCO) when compared to a server-based cloud execution model because it effectively shifts operational responsibilities such as managing servers to a cloud provider. Deloitte research on serverless TCO with Fortune 100 clients across industries show that serverless applications can offer up to 57% cost savings compared with server-based solutions.

Serverless applications can offer lower costs in:

• Initial development: Serverless enables builders to be more agile, deliver features more rapidly, and focus on differentiated business logic.
• Ongoing maintenance and infrastructure: Serverless shifts operational burden to AWS. Ongoing maintenance tasks, including patching and operating system updates.

This post focuses on options available to reduce direct AWS costs when building serverless applications. AWS Lambda is often the compute layer in these workloads and may comprise a meaningful portion of the overall cost.

To help optimize your Lambda-related costs, this post discusses some of the most commonly used cost optimization techniques with an emphasis on configuration changes over code updates. This post is intended for architects and developers new to building with serverless.

Building with serverless makes both experimentation and iterative improvement easier. All of the techniques described here can be applied before application development, or after you have deployed your application to production. The techniques are roughly by applicability: The first can apply to any workload; the last applies to a smaller number of workloads.

Right-sizing your Lambda functions

Lambda uses a pay-per-use cost model that is driven by three metrics:

• Memory configuration: allocated memory from 128MB to 10,240MB. CPU and other resources available to the function are allocated proportionally to memory.
• Function duration: time function runs, measures in milliseconds.
• Number of invocations: the number of times your function runs.

Over-provisioning memory is one of the primary drivers of increased Lambda cost. This is particularly acute among builders new to Lambda who are used to provisioning hosts running multiple processes. Lambda scales such that each execution environment of a function only handles one request at a time. Each execution environment has access to fixed resources (memory, CPU, storage) to complete work on the request.

By right-sizing the memory configuration, the function has the resources to complete its work and you are paying for only the needed resources. While you also have direct control of function duration, this is a less effective cost optimization to implement. The engineering costs to create a few milliseconds of savings may outweigh the cost savings. Depending on the workload, the number of times your function is invoked may be outside your control. The next section discusses a technique to reduce the number of invocations for some types of workloads.

Memory configuration is accessible via the AWS Management Console or your favorite infrastructure as code (IaC) option. The memory configuration setting defines allocated memory, not memory used by your function. Right-sizing memory is an adjustment that can reduce the cost (or increase performance) of your function. However, lowering the function-memory may not always result in cost savings. Lowering function memory means lowering available CPU for the Lambda function, which could increase the function duration, resulting in either no cost savings or higher cost. It is important to identify the optimal memory configuration for cost savings while preserving performance.

AWS offers two approaches to right-sizing memory allocation: AWS Lambda Power Tuning and AWS Compute Optimizer.

AWS Lambda Power Tuning is an open-source tool that can be used to empirically find the optimal memory configuration for your function by trading off cost against execution time. The tool runs multiple concurrent versions of your function against mock input data at different memory allocations. The result is a chart that can help you find the “sweet spot” between cost and duration/performance. Depending on the workload, you may prioritize one over the other. AWS Lambda Power Tuning is a good choice for new functions and can also help select between the two instruction set architectures offered by Lambda.

AWS Compute Optimizer uses machine learning to recommend an optimal memory configuration based on historical data. Compute Optimizer requires that your function be invoked at least 50 times over the trailing 14 days to provide a recommendation based on past utilization, so is most effective once your function is in production.

Both Lambda Power Tuning and Compute Optimizer help derive the right-sized memory allocation for your function. Use this value to update the configuration of your function using the AWS Management Console or IaC.

This post includes AWS Serverless Application Model (AWS SAM) sample code throughout to demonstrate how to implement optimizations. You can also use AWS Cloud Development Kit (AWS CDK), Terraform, Serverless Framework, and other IaC tools to implement the same changes.

MyFunction:
  Type: AWS::Serverless::Function
  Properties:
    Runtime: nodejs18.x
    Handler: app.handler
    MemorySize: 1024   # Set memory configuration to optimize for cost or performance

Setting a realistic function timeout

Lambda functions are configured with a maximum time that each invocation can run, up to 15 minutes. Setting an appropriate timeout can be beneficial in containing costs of your Lambda-based application. Unhandled exceptions, blocking actions (for example, opening a network connection), slow dependencies, and other conditions can lead to longer-running functions or functions that run until the configured timeout. Proper timeouts are the best protection against both slow and erroneous code. At some point, the work the function is performing and the per-millisecond cost of that work is wasted.

Our recommendation is to set a timeout of less than 29-seconds for all synchronous invocations, or those in which the caller is waiting for a response. Longer timeouts are appropriate for asynchronous invocations, but consider timeouts longer than 1-minute to be an exception that requires review and testing.

Using Graviton

Lambda offers two instruction set architectures in most AWS Regions: x86 and arm64.

Choosing Graviton can save money in two ways. First, your functions may run more efficiently due to the Graviton2 architecture. Second, you may pay less for the time that they run. Lambda functions powered by Graviton2 are designed to deliver up to 19 percent better performance at 20 percent lower cost. Consider starting with Graviton when developing new Lambda functions, particularly those that do not require natively compiled binaries.

If your function relies on native compiled binaries or is packaged as a container image, you must rebuild to move between arm64 and x86. Lambda layers may also include dependencies targeted for one architecture or the other. We encourage you to review dependencies and test your function before changing the architecture. The AWS Lambda Power Tuning tool also allows you to compare the price and performance of arm64 and x86 at different memory settings.

You can modify the architecture configuration of your function in the console or your IaC of choice. For example, in AWS SAM:

MyFunction:
  Type: AWS::Serverless::Function
  Properties:
    Architectures:
      - arm64. # Set architecture to use Graviton2
    Runtime: nodejs18.x
    Handler: app.handler

Filtering incoming events

Lambda is integrated with over 200 event sources, including Amazon SQS, Amazon Kinesis Data Streams, Amazon DynamoDB Streams, Amazon Managed Streaming for Apache Kafka, and Amazon MQ. The Lambda service integrates with these event sources to retrieve messages and invokes your function as needed to process those messages.

When working with one of these event sources, builders can configure filters to limit the events sent to your function. This technique can greatly reduce the number of times your function is invoked depending on the number of events and specificity of your filters. When not using event filtering, the function must be invoked to first determine if an event should be processed before performing the actual work. Event filtering alleviates the need to perform this upfront check while reducing the number of invocations.

For example, you may only want a function to run when orders of over $200 are found in a message on a Kinesis data stream. You can configure an event filtering pattern using the console or IaC in a manner similar to memory configuration.

To implement the Kinesis stream filter using AWS SAM:

MyFunction:
  Type: AWS::Serverless::Function
  Properties:
    Runtime: nodejs18.x
    Handler: app.handler
    Events:
      Type: Kinesis
      Properties:
        StartingPosition: LATEST
        Stream: "arn:aws:kinesis:us-east-1:0123456789012:stream/orders"
        FilterCriteria:
          Filters:
            - Pattern: '{ "data" : { "order" : { "value" : [{ "numeric": [">", 200] }] } } }'

If an event satisfies one of the event filters associated with the event source, Lambda sends the event to your function for processing. Otherwise, the event is discarded as processed successfully without invoking the function.

If you are building or running a Lambda function that is invoked by one of the previously mentioned event sources, it’s recommended that you review the filtering options available. This technique requires no code changes to your Lambda function – even if the function performs some preprocessing check, that check still completes successfully with filtering implemented.

To learn more, read Filtering event sources for AWS Lambda functions.

Avoiding recursion

You may be familiar with the programming concept of a recursive function or a function/routine that calls itself. Though rare, customers sometimes unintentionally build recursion in their architecture, so a Lambda function continuously calls itself.

The most common recursive pattern is between Lambda and Amazon S3. Actions in an S3 bucket can trigger a Lambda function, and recursion can occur when that Lambda function writes back to the same bucket.

Consider a use case in which a Lambda function is used to generate a thumbnail of user-submitted images. You configure the bucket to trigger the thumbnail generation function when a new object is put in the bucket. What happens if the Lambda function writes the thumbnail to the same bucket? The process starts anew and the Lambda function then runs on the thumbnail image itself. This is recursion and can lead to an infinite loop condition.

While there are multiple ways to prevent against this condition, it’s best practice to use a second S3 bucket to store thumbnails. This approach minimizes changes to the architecture as you do not need to change the notification settings nor the primary S3 bucket. To learn about other approaches, read Avoiding recursive invocation with Amazon S3 and AWS Lambda.

If you do encounter recursion in your architecture, set Lambda reserved concurrency to zero to stop the function from running. Allow minutes to hours before lifting the reserved concurrency cap. Since S3 events are asynchronous invocations that have automatic retries, you may continue to see recursion until you resolve the issue or all events have expired.

Conclusion

Lambda offers a number of techniques that you can use to minimize infrastructure costs whether you are just getting started with Lambda or have numerous functions already deployed in production. When combined with the lower costs of initial development and ongoing maintenance, serverless can offer a low total cost of ownership. Get hands-on with these techniques and more with the Serverless Optimization Workshop.

To learn more about serverless architectures, find reusable patterns, and keep up-to-date, visit Serverless Land.

Post Syndicated from James Beswick original https://aws.amazon.com/blogs/compute/python-3-10-runtime-now-available-in-aws-lambda/

This post is written by Suresh Poopandi, Senior Solutions Architect, Global Life Sciences.

AWS Lambda now supports Python 3.10 as both a managed runtime and container base image. With this release, Python developers can now take advantage of new features and improvements introduced in Python 3.10 when creating serverless applications on Lambda.

Enhancements in Python 3.10 include structural pattern matching, improved error messages, and performance enhancements. This post outlines some of the benefits of Python 3.10 and how to use this version in your Lambda functions.

AWS has also published a preview Lambda container base image for Python 3.11. Customers can use this image to get an early look at Python 3.11 support in Lambda. This image is subject to change and should not be used for production workloads. To provide feedback on this image, and for future updates on Python 3.11 support, see https://github.com/aws/aws-lambda-base-images/issues/62.

What’s new in Python 3.10

Thanks to its simplicity, readability, and extensive community support, Python is a popular language for building serverless applications. The Python 3.10 release includes several new features, such as:

• Structural pattern matching (PEP 634): Structural pattern matching is one of the most significant additions to Python 3.10. With structural pattern matching, developers can use patterns to match against data structures such as lists, tuples, and dictionaries and run code based on the match. This feature enables developers to write code that processes complex data structures more easily and can improve code readability and maintainability.
• Parenthesized context managers (BPO-12782): Python 3.10 introduces a new syntax for parenthesized context managers, making it easier to read and write code that uses the “with” statement. This feature simplifies managing resources such as file handles or database connections, ensuring they are released correctly.
• Writing union types as X | Y (PEP 604): Python 3.10 allows writing union types as X | Y instead of the previous versions’ syntax of typing Union[X, Y]. Union types represent a value that can be one of several types. This change does not affect the functionality of the code and is backward-compatible, so code written with the previous syntax will still work. The new syntax aims to reduce boilerplate code, and improve readability and maintainability of Python code by providing a more concise and intuitive syntax for union types.
• User-defined type guards (PEP 647): User-defined type guards allow developers to define their own type guards to handle custom data types or to refine the types of built-in types. Developers can define their own functions that perform more complex type checks as user-defined typed guards. This feature improves Python code readability, maintainability, and correctness, especially in projects with complex data structures or custom data types.
• Improved error messages: Python 3.10 has improved error messages, providing developers with more information about the source of the error and suggesting possible solutions. This helps developers identify and fix issues more quickly. The improved error messages in Python 3.10 include more context about the error, such as the line number and location where the error occurred, as well as the exact nature of the error. Additionally, Python 3.10 error messages now provide more helpful information about how to fix the error, such as suggestions for correct syntax or usage.

Performance improvements

The faster PEP 590 vectorcall calling convention allows for quicker and more efficient Python function calls, particularly those that take multiple arguments. The specific built-in functions that benefit from this optimization include map(), filter(), reversed(), bool(), and float(). By using the vectorcall calling convention, according to Python 3.10 release notes, these inbuilt functions’ performance improved by a factor of 1.26x.

When a function is defined with annotations, these are stored in a dictionary that maps the parameter names to their respective annotations. In previous versions of Python, this dictionary was created immediately when the function was defined. However, in Python 3.10, this dictionary is created only when the annotations are accessed, which can happen when the function is called. By delaying the creation of the annotation dictionary until it is needed, Python can avoid the overhead of creating and initializing the dictionary during function definition. This can result in a significant reduction in CPU time, as the dictionary creation can be a time-consuming operation, particularly for functions with many parameters or complex annotations.

In Python 3.10, the LOAD_ATTR instruction, which is responsible for loading attributes from objects in the code, has been improved with a new mechanism called the “per opcode cache”. This mechanism works by storing frequently accessed attributes in a cache specific to each LOAD_ATTR instruction, which reduces the need for repeated attribute lookups. As a result of this improvement, according to Python 3.10 release notes, the LOAD_ATTR instruction is now approximately 36% faster when accessing regular attributes and 44% faster when accessing attributes defined using the slots mechanism.

In Python, the str(), bytes(), and bytearray() constructors are used to create new instances of these types from existing data or values. Based on the result of the performance tests conducted as part of  BPO-41334, constructors str(), bytes(), and bytearray() are around 30–40% faster for small objects.

Lambda functions developed with Python that read and process Gzip compressed files can gain a performance improvement. Adding _BlocksOutputBuffer for the bz2/lzma/zlib module eliminated the overhead of resizing bz2/lzma buffers, preventing excessive memory footprint of the zlib buffer. According to Python 3.10 release notes, bz2 decompression is now 1.09x faster, lzma decompression 1.20x faster, and GzipFile read is 1.11x faster

Using Python 3.10 in Lambda

AWS Management Console

To use the Python 3.10 runtime to develop your Lambda functions, specify a runtime parameter value Python 3.10 when creating or updating a function. Python 3.10 version is now available in the Runtime dropdown in the Create function page.

To update an existing Lambda function to Python 3.10, navigate to the function in the Lambda console, then choose Edit in the Runtime settings panel. The new version of Python is available in the Runtime dropdown:

AWS Serverless Application Model (AWS SAM)

In AWS SAM, set the Runtime attribute to python3.10 to use this version.

AWSTemplateFormatVersion: ‘2010-09-09’
Transform: AWS::Serverless-2016-10-31
Description: Simple Lambda Function

Resources:
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      Description: My Python Lambda Function
      CodeUri: my_function/
      Handler: lambda_function.lambda_handler
      Runtime: python3.10

AWS SAM supports the generation of this template with Python 3.10 out of the box for new serverless applications using the sam init command. Refer to the AWS SAM documentation here.

AWS Cloud Development Kit (AWS CDK)

In the AWS CDK, set the runtime attribute to Runtime.PYTHON_3_10 to use this version. In Python:

from constructs import Construct
from aws_cdk import (
    App, Stack,
    aws_lambda as _lambda
)


class SampleLambdaStack(Stack):

    def __init__(self, scope: Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        base_lambda = _lambda.Function(self, 'SampleLambda',
                                       handler='lambda_handler.handler',
                                       runtime=_lambda.Runtime.PYTHON_3_10,
                                       code=_lambda.Code.from_asset('lambda'))

In TypeScript:

import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda'
import * as path from 'path';
import { Construct } from 'constructs';

export class CdkStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // The code that defines your stack goes here

    // The python3.10 enabled Lambda Function
    const lambdaFunction = new lambda.Function(this, 'python310LambdaFunction', {
      runtime: lambda.Runtime.PYTHON_3_10,
      memorySize: 512,
      code: lambda.Code.fromAsset(path.join(__dirname, '/../lambda')),
      handler: 'lambda_handler.handler'
    })
  }
}

AWS Lambda – Container Image

Change the Python base image version by modifying FROM statement in the Dockerfile:

FROM public.ecr.aws/lambda/python:3.10

# Copy function code
COPY lambda_handler.py ${LAMBDA_TASK_ROOT}

To learn more, refer to the usage tab on building functions as container images.

Conclusion

You can build and deploy functions using Python 3.10 using the AWS Management Console, AWS CLI, AWS SDK, AWS SAM, AWS CDK, or your choice of Infrastructure as Code (IaC). You can also use the Python 3.10 container base image if you prefer to build and deploy your functions using container images.

We are excited to bring Python 3.10 runtime support to Lambda and empower developers to build more efficient, powerful, and scalable serverless applications. Try Python 3.10 runtime in Lambda today and experience the benefits of this updated language version and take advantage of improved performance.

For more serverless learning resources, visit Serverless Land.

Post Syndicated from James Beswick original https://aws.amazon.com/blogs/compute/optimizing-aws-lambda-extensions-in-c-and-rust/

This post is written by Siarhei Kazhura, Senior Specialist Solutions Architect, Serverless.

Customers use AWS Lambda extensions to integrate monitoring, observability, security, and governance tools with their Lambda functions. AWS, along with AWS Lambda Ready Partners, like Datadog, Dynatrace, New Relic, provides ready-to-run extensions. You can also develop your own extensions to address your specific needs.

External Lambda extensions are designed as a companion process running in the same execution environment as the function code. That means that the Lambda function shares resources like memory, CPU, and disk I/O, with the extension. Improperly designed extensions can result in a performance degradation and extra costs.

This post shows how to measure the impact an extension has on the function performance using key performance metrics on an Amazon CloudWatch dashboard.

This post focuses on Lambda extensions written in C# and Rust. It shows the benefits of choosing to write Lambda extensions in Rust. Also, it explains how you can optimize a Lambda extension written in C# to deliver three times better performance. The solution can be converted to the programming languages of your choice.

Overview

A C# Lambda function (running on .NET 6) called HeaderCounter is used as a baseline. The function counts the number of headers in a request and returns the number in the response. A static delay of 500 ms is inserted in the function code to simulate extra computation. The function has the minimum memory setting (128 MB), which magnifies the impact that extension has on performance.

A load test is performed via a curl command that is issuing 5000 requests (with 250 requests running simultaneously) against a public Amazon API Gateway endpoint backed by the Lambda function. A CloudWatch dashboard, named lambda-performance-dashboard, displays performance metrics for the function.

Metrics captured by the dashboard:

1. The Max Duration, and Average Duration metrics allow you to assess the impact the extension has on the function execution duration.
2. The PostRuntimeExtensionsDuration metric measures the extra time that the extension takes after the function invocation.
3. The Average Memory Used, and Memory Allocated metrics allow you to assess the impact the extension has on the function memory consumption.
4. The Cold Start Duration, and Cold Starts metrics allow you to assess the impact the extension has on the function cold start.

Running the extensions

There are a few differences between how the extensions written in C# and Rust are run.

The extension written in Rust is published as an executable. The advantage of an executable is that it is compiled to native code, and is ready to run. The extension is environment agnostic, so it can run alongside with a Lambda function written in another runtime.

The disadvantage of an executable is the size. Extensions are served as Lambda layers, and the size of the extension counts towards the deployment package size. The maximum unzipped deployment package size for Lambda is 250 MB.

The extension written in C# is published as a dynamic-link library (DLL). The DLL contains the Common Intermediate Language (CIL), that must be converted to native code via a just-in-time (JIT) compiler. The .NET runtime must be present for the extension to run. The dotnet command runs the DLL in the example provided with the solution.

Blank extension

Three instances of the HeaderCounter function are deployed:

1. The first instance, available via a no-extension endpoint, has no extensions.
2. The second instance, available via a dotnet-extension endpoint, is instrumented with a blank extension written in C#. The extension does not provide any extra functionality, except logging the event received to CloudWatch.
3. The third instance, available via a rust-extension endpoint, is instrumented with a blank extension written in Rust. The extension does not provide any extra functionality, except logging the event received to CloudWatch.

The dashboard shows that the extensions add minimal overhead to the Lambda function. The extension written in C# adds more overhead in the higher percentile metrics, such as the Maximum Cold Start Duration and Maximum Duration.

EventCollector extension

Three instances of the HeaderCounter function are deployed:

1. The first instance, available via a no-extension endpoint, has no extensions.
2. The second instance, available via a dotnet-extension endpoint, is instrumented with an EventCollector extension written in C#. The extension is pushing all the extension invocation events to Amazon S3.
3. The third instance, available via a rust-extension endpoint, is instrumented with an EventCollector extension written in Rust. The extension is pushing all the extension invocation events to S3.

The Rust extension adds little overhead in terms of the Duration, number of Cold Starts, and Average PostRuntimeExtensionDuration metrics. Yet there is a clear performance degradation for the function that is instrumented with an extension written in C#. Average Duration jumped almost three times, and the Maximum Duration is now around six times higher.

The function is now consuming almost all the memory allocated. CPU, networking, and storage for Lambda functions are allocated based on the amount of memory selected. Currently, the memory is set to 128 MB, the lowest setting possible. Constrained resources influence the performance of the function.

Increasing the memory to 512 MB and re-running the load test improves the performance. Maximum Duration is now 721 ms (including the static 500 ms delay).

For the C# function, the Average Duration is now only 59 ms longer than the baseline. The Average PostRuntimeExtensionDuration is at 36.9 ms (compared with 584 ms previously). This performance gain is due to the memory increase without any code changes.

You can also use the Lambda Power Tuning to determine the optimal memory setting for a Lambda function.

Garbage collection

Unlike C#, Rust is not a garbage collected language. Garbage collection (GC) is a process of managing the allocation and release of memory for an application. This process can be resource intensive, and can affect higher percentile metrics. The impact of GC is visible with the blank extension’s and EventCollector extension’s metrics.

Rust uses ownership and borrowing features, allowing for safe memory release without relying on GC. This makes Rust a good runtime choice for tools like Lambda extensions.

EventCollector native AOT extension

Native ahead-of-time (Native AOT) compilation (available in .NET 7 and .NET 8), allows for the extensions written in C# to be delivered as executables, similar to the extensions written in Rust.

Native AOT does not use a JIT compiler. The application is compiled into a self-contained (all the resources that it needs are encapsulated) executable. The executable runs in the target environment (for example, Linux x64) that is specified at compilation time.

These are the results of compiling the .NET extension using Native AOT and re-running the performance test (with function memory set to 128 MB):

For the C# extension, Average Duration is now close the baseline (compared to three times the baseline as a DLL). Average PostRuntimeExtensionDuration is now 0.77 ms (compared with 584 ms as a DLL). The C# extension also outperforms the Rust extension for the Maximum PostRuntimeExtensionDuration metric – 297 ms versus 497 ms.

Overall, the Rust extension still has better Average/Maximum Duration, Average/Maximum Cold Start Duration, and Memory Consumption. The Lambda function with the C# extension still uses almost all the allocated memory.

Another metric to consider is the binary size. The Rust extension compiles into a 12.3 MB binary, while the C# extension compiles into a 36.4 MB binary.

Example walkthroughs

To follow the example walkthrough, visit the GitHub repository. The walkthrough explains:

1. The prerequisites required.
2. A detailed solution deployment walkthrough.
3. The cleanup process.
4. Cost considerations.

Conclusion

This post demonstrates techniques that can be used for running and profiling different types of Lambda extensions. This post focuses on Lambda extensions written in C# and Rust. This post outlines the benefits of writing Lambda extensions in Rust and shows the techniques that can be used to improve Lambda extension written in C# to deliver better performance.

Start writing Lambda extensions with Rust by using the Runtime extensions for AWS Lambda crate. This is a part of a Rust runtime for AWS Lambda.

For more serverless learning resources, visit Serverless Land.

Post Syndicated from James Beswick original https://aws.amazon.com/blogs/compute/introducing-new-asynchronous-invocation-metrics-for-aws-lambda/

This post is written by Arthi Jaganathan, Principal SA, Serverless and Dhiraj Mahapatro, Principal SA, Serverless.

Today, AWS is announcing three new Amazon CloudWatch metrics for asynchronous AWS Lambda function invocations: AsyncEventsReceived, AsyncEventAge, and AsyncEventsDropped. These metrics provide visibility for asynchronous Lambda function invocations.

Previously, customers found it challenging to monitor the processing of asynchronous invocations. With these new metrics for asynchronous function invocations, you can identify the root cause of processing issues. These issues include throttling, concurrency limit, function errors, processing latency because of retries, missing events, and taking corrective action.

This blog and the sample application provide examples that highlight the usage of the new metrics.

Overview

AWS services such as Amazon S3, Amazon SNS, and Amazon EventBridge invoke Lambda functions asynchronously. Lambda uses an internal queue to store events. A separate process reads events from the queue and sends them to the function.

By default, Lambda discards events from its event queue if the retry policy has exceeded the number of configured retries or the event reached its maximum age. However, the event once discarded from the event queue goes to the destination or DLQ, if configured.

This table summarizes the retry behavior. Visit the asynchronous Lambda invocations documentation to learn more:

What’s new

The AsyncEventsReceived metric is a measure of the number of events enqueued in Lambda’s internal queue. You can track events from the client using custom CloudWatch metrics or extract it from logs using Embedded Metric Format (EMF). In case this metric is lower than the number of events that you expect, it shows that the source did not emit events or events did not arrive at the Lambda service. This is possible because of transient networking issues. Lambda does not emit this metric for retried events.

The AsyncEventAge metric is a measure of the difference between the time that an event is first enqueued in the internal queue and the time the Lambda service invokes the function. With retries, Lambda emits this metric every time it attempts to invoke the function with the event. An increasing value shows retries because of error or throttles. Customers can set alarms on this metric to alert on SLA breaches.

The AsyncEventsDropped metric is a measure of the number of events dropped because of processing failure.

How to use the new async event metrics

This flowchart shows the way that you can combine the new metrics with existing metrics to troubleshoot problems with asynchronous processing:

Example application

You can deploy a sample Lambda function to show how to use the new metrics for troubleshooting.

To test the following scenarios, you must install:

• AWS CLI
• AWS SAM CLI
• jq

To set up the application:

1. Set your AWS Region:

   export REGION=<your AWS region>

2. Clone the GitHub repository:

   git clone https://github.com/aws-samples/lambda-async-metrics-sample.git
   cd lambda-async-metrics-sample

3. Build and deploy the application. Provide lambda-async-metric as the stack name when prompted. Keep everything else as default values:

   sam build
   sam deploy --guided --region $REGION

4. Save the name of the function in an environment variable:

   FUNCTION_NAME=$(aws cloudformation describe-stacks \
     --region $REGION \
     --stack-name lambda-async-metric \
     --query 'Stacks[0].Outputs[?OutputKey==`HelloWorldFunctionResourceName`].OutputValue' --output text)
   

5. Invoke the function using AWS CLI:

   aws lambda invoke \
     --region $REGION \
     --function-name $FUNCTION_NAME \
     --invocation-type Event out_file.txt
   

6. Choose All Metrics under Metrics in the left-hand panel in the CloudWatch console and search for “async”:
   
7. Choose Lambda > By Function Name and choose AsyncEventsReceived for the function you created. Under “Graphed metrics”, change the statistic to sum and “Period” to 1 minute. You see one record. After waiting a few seconds, refresh if you don’t see the metric immediately.

Scenarios

These scenarios show how you can use the three new metrics.

Scenario 1: Troubleshooting delays due to function error

Lambda retries processing the asynchronous invocation event for a maximum of two times, in case of function error or exception. Lambda drops the event from its internal queue if the retries are exhausted.

To simulate a function error, throw an exception from the Lambda handler:

1. Edit the function code in hello_world/app.py to raise an exception:

   def handler(event, context):
     print(“Hello from AWS Lambda”)
     raise Exception(“Lambda function throwing exception”)
   

2. Build and deploy:

   sam build && sam deploy –region $REGION

3. Invoke the function:

   aws lambda invoke \
     --region $REGION \
     --function-name $FUNCTION_NAME \
     --invocation-type Event out_file.txt
   

It is best practice to alert on function errors using the error metric and use the metrics to get better insights into retry behavior, such as interval between retries. For example, if a function errors because of a downstream system being overwhelmed, you can use AsyncEventAge and Concurrency metrics.

If you received an alert for function error, you see data points for AsyncEventsDropped. It is 1 for this scenario. Overlaying the Errors and Throttles metrics reconfirms function error causes this.

There are two retries before the Lambda service drops the event. No throttling confirms the function error. Next, you can confirm that the AsyncEventAge is increasing. Lambda publishes this metric every time it polls from the event queue and sends it to the function. This creates multiple data points for the metric.

You can duplicate the metric to see both statistics on a single graph. Here, the two lines overlap because there is only one data point published in each 1-minute interval.

The event spent 37ms in the internal queue before the first invoke attempt. Lambda’s first retry happens after 63.5 seconds. The second and final retry happens after 189.6 seconds.

Scenario 2: Troubleshooting delays because of concurrency limits

In case of throttling or system errors, Lambda retries invoking the event up to the configured MaximumEventAgeInSeconds (the maximum is 6 hours). To simulate the throttling error without hitting the account concurrency limit, you can:

• Set the function reserved concurrency to 1
• Introduce a 90 seconds sleep in the function code to simulate a lengthy function execution.

The Lambda service throttles new invocations while the first request is in progress. You invoke the function in quick succession from the command line to simulate throttling and observe the retry behavior:

1. Set the function reserved concurrency to 1 by updating the AWS SAM template.

   HelloWorldFunction:
     Type: AWS::Serverless::Function
     Properties:
       CodeUri: hello_world/
       Handler: app.lambda_handler
       Runtime: python3.9
       Timeout: 100
       MemorySize: 128
       Architectures:
       - x86_64
       ReservedConcurrentExecutions: 1
   

2. Edit the function code in hello_world/app.py to introduce a 90-second sleep. Replace existing code with the following in app.py:

   import time
   
   def handler(event, context):
     time.sleep(90)
     print("Hello from AWS Lambda")
   

3. Build and deploy:

   sam build && sam deploy --region $REGION

4. Invoke the function twice in succession from the command line:

   for i in {1..2}; do aws lambda invoke \
     --region $REGION \
     --function-name $FUNCTION_NAME \
     --invocation-type Event out_file.txt; done

In a real-world use case, missing the processing SLA should trigger the troubleshooting workflow. Start with AsyncEventsReceived to confirm events enqueued by the Lambda service. This is 2 in this scenario. Look for dropped events using AsyncEventsDropped metric.

AsyncEventAge verifies delays in processing. As mentioned in the previous section, there can be multiple data points for this metric in a one-minute interval. You can duplicate the metric to compare minimum and maximum values.

There are 2 data points in the first minute. The event age increases to 31 seconds during this period.

There is only one data point for the metric in the remaining one-minute intervals, so the lines overlap. The event age increases to 59,153ms (~59 seconds) in one interval and then to 130,315ms (~130 seconds) in the next one-minute interval. Since the function sleeps for 90 seconds, it explains why the final retry is at around 2 minutes since the function received the event.

Checking function throttling, this screenshot confirms throttling six times in the first minute (07:12 UTC timestamp) and once in the subsequent minute (07:13 UTC timestamp).

This is because of the back-off behavior of Lambda’s internal queue. The data for AsyncEventAge shows that there is only one throttle in the second interval. Lambda delivers the event during the next one-minute interval after spending around 2 minutes in the internal queue.

Overlaying the ConcurrentExecutions and AsyncEventsReceived metrics provides more information. You see receipt of two events, but concurrency stayed at 1. This results in one event being throttled:

There are multiple ways to resolve throttling errors. You optimize the function to run faster or increase the function or account concurrency limits to address throttling errors.

The sample application covers other scenarios such as troubleshooting dropped events on event expiration, and troubleshooting dropped events when a function’s reserved concurrency is set to zero.

Cleaning up

Use the following command and follow the prompts to clean up the resources:

sam delete lambda-async-metric --region $REGION

Conclusion

Using these new CloudWatch metrics, you can gain visibility into the processing of Lambda asynchronous invocations. This blog explained the new metrics AsyncEventsReceived, AsyncEventAge, and AsyncEventsDropped and how to use them to troubleshoot issues. With these new metrics, you can track the asynchronous invocation requests sent to Lambda functions. You monitor any delays in processing, and take corrective actions if required.

The Lambda service sends these new metrics to CloudWatch at no cost to you. However, charges apply for CloudWatch Metric Streams and CloudWatch Alarms. See CloudWatch pricing for more information.

For more serverless learning resources, visit Serverless Land.