Tag Archives: serverless

ICYMI: Serverless Q4 2019

Post Syndicated from Rob Sutter original https://aws.amazon.com/blogs/compute/icymi-serverless-q4-2019/

Welcome to the eighth edition of the AWS Serverless ICYMI (in case you missed it) quarterly recap. Every quarter, we share the most recent product launches, feature enhancements, blog posts, webinars, Twitch live streams, and other interesting things that you might have missed!

In case you missed our last ICYMI, checkout what happened last quarter here.

The three months comprising the fourth quarter of 2019

AWS re:Invent

AWS re:Invent 2019

re:Invent 2019 dominated the fourth quarter at AWS. The serverless team presented a number of talks, workshops, and builder sessions to help customers increase their skills and deliver value more rapidly to their own customers.

Serverless talks from re:Invent 2019

Chris Munns presenting 'Building microservices with AWS Lambda' at re:Invent 2019

We presented dozens of sessions showing how customers can improve their architecture and agility with serverless. Here are some of the most popular.

Videos

Decks

You can also find decks for many of the serverless presentations and other re:Invent presentations on our AWS Events Content.

AWS Lambda

For developers needing greater control over performance of their serverless applications at any scale, AWS Lambda announced Provisioned Concurrency at re:Invent. This feature enables Lambda functions to execute with consistent start-up latency making them ideal for building latency sensitive applications.

As shown in the below graph, provisioned concurrency reduces tail latency, directly impacting response times and providing a more responsive end user experience.

Graph showing performance enhancements with AWS Lambda Provisioned Concurrency

Lambda rolled out enhanced VPC networking to 14 additional Regions around the world. This change brings dramatic improvements to startup performance for Lambda functions running in VPCs due to more efficient usage of elastic network interfaces.

Illustration of AWS Lambda VPC to VPC NAT

New VPC to VPC NAT for Lambda functions

Lambda now supports three additional runtimes: Node.js 12, Java 11, and Python 3.8. Each of these new runtimes has new version-specific features and benefits, which are covered in the linked release posts. Like the Node.js 10 runtime, these new runtimes are all based on an Amazon Linux 2 execution environment.

Lambda released a number of controls for both stream and async-based invocations:

  • You can now configure error handling for Lambda functions consuming events from Amazon Kinesis Data Streams or Amazon DynamoDB Streams. It’s now possible to limit the retry count, limit the age of records being retried, configure a failure destination, or split a batch to isolate a problem record. These capabilities help you deal with potential “poison pill” records that would previously cause streams to pause in processing.
  • For asynchronous Lambda invocations, you can now set the maximum event age and retry attempts on the event. If either configured condition is met, the event can be routed to a dead letter queue (DLQ), Lambda destination, or it can be discarded.

AWS Lambda Destinations is a new feature that allows developers to designate an asynchronous target for Lambda function invocation results. You can set separate destinations for success and failure. This unlocks new patterns for distributed event-based applications and can replace custom code previously used to manage routing results.

Illustration depicting AWS Lambda Destinations with success and failure configurations

Lambda Destinations

Lambda also now supports setting a Parallelization Factor, which allows you to set multiple Lambda invocations per shard for Kinesis Data Streams and DynamoDB Streams. This enables faster processing without the need to increase your shard count, while still guaranteeing the order of records processed.

Illustration of multiple AWS Lambda invocations per Kinesis Data Streams shard

Lambda Parallelization Factor diagram

Lambda introduced Amazon SQS FIFO queues as an event source. “First in, first out” (FIFO) queues guarantee the order of record processing, unlike standard queues. FIFO queues support messaging batching via a MessageGroupID attribute that supports parallel Lambda consumers of a single FIFO queue, enabling high throughput of record processing by Lambda.

Lambda now supports Environment Variables in the AWS China (Beijing) Region and the AWS China (Ningxia) Region.

You can now view percentile statistics for the duration metric of your Lambda functions. Percentile statistics show the relative standing of a value in a dataset, and are useful when applied to metrics that exhibit large variances. They can help you understand the distribution of a metric, discover outliers, and find hard-to-spot situations that affect customer experience for a subset of your users.

Amazon API Gateway

Screen capture of creating an Amazon API Gateway HTTP API in the AWS Management Console

Amazon API Gateway announced the preview of HTTP APIs. In addition to significant performance improvements, most customers see an average cost savings of 70% when compared with API Gateway REST APIs. With HTTP APIs, you can create an API in four simple steps. Once the API is created, additional configuration for CORS and JWT authorizers can be added.

AWS SAM CLI

Screen capture of the new 'sam deploy' process in a terminal window

The AWS SAM CLI team simplified the bucket management and deployment process in the SAM CLI. You no longer need to manage a bucket for deployment artifacts – SAM CLI handles this for you. The deployment process has also been streamlined from multiple flagged commands to a single command, sam deploy.

AWS Step Functions

One powerful feature of AWS Step Functions is its ability to integrate directly with AWS services without you needing to write complicated application code. In Q4, Step Functions expanded its integration with Amazon SageMaker to simplify machine learning workflows. Step Functions also added a new integration with Amazon EMR, making EMR big data processing workflows faster to build and easier to monitor.

Screen capture of an AWS Step Functions step with Amazon EMR

Step Functions step with EMR

Step Functions now provides the ability to track state transition usage by integrating with AWS Budgets, allowing you to monitor trends and react to usage on your AWS account.

You can now view CloudWatch Metrics for Step Functions at a one-minute frequency. This makes it easier to set up detailed monitoring for your workflows. You can use one-minute metrics to set up CloudWatch Alarms based on your Step Functions API usage, Lambda functions, service integrations, and execution details.

Step Functions now supports higher throughput workflows, making it easier to coordinate applications with high event rates. This increases the limits to 1,500 state transitions per second and a default start rate of 300 state machine executions per second in US East (N. Virginia), US West (Oregon), and Europe (Ireland). Click the above link to learn more about the limit increases in other Regions.

Screen capture of choosing Express Workflows in the AWS Management Console

Step Functions released AWS Step Functions Express Workflows. With the ability to support event rates greater than 100,000 per second, this feature is designed for high-performance workloads at a reduced cost.

Amazon EventBridge

Illustration of the Amazon EventBridge schema registry and discovery service

Amazon EventBridge announced the preview of the Amazon EventBridge schema registry and discovery service. This service allows developers to automate discovery and cataloging event schemas for use in their applications. Additionally, once a schema is stored in the registry, you can generate and download a code binding that represents the schema as an object in your code.

Amazon SNS

Amazon SNS now supports the use of dead letter queues (DLQ) to help capture unhandled events. By enabling a DLQ, you can catch events that are not processed and re-submit them or analyze to locate processing issues.

Amazon CloudWatch

Amazon CloudWatch announced Amazon CloudWatch ServiceLens to provide a “single pane of glass” to observe health, performance, and availability of your application.

Screenshot of Amazon CloudWatch ServiceLens in the AWS Management Console

CloudWatch ServiceLens

CloudWatch also announced a preview of a capability called Synthetics. CloudWatch Synthetics allows you to test your application endpoints and URLs using configurable scripts that mimic what a real customer would do. This enables the outside-in view of your customers’ experiences, and your service’s availability from their point of view.

CloudWatch introduced Embedded Metric Format, which helps you ingest complex high-cardinality application data as logs and easily generate actionable metrics. You can publish these metrics from your Lambda function by using the PutLogEvents API or using an open source library for Node.js or Python applications.

Finally, CloudWatch announced a preview of Contributor Insights, a capability to identify who or what is impacting your system or application performance by identifying outliers or patterns in log data.

AWS X-Ray

AWS X-Ray announced trace maps, which enable you to map the end-to-end path of a single request. Identifiers show issues and how they affect other services in the request’s path. These can help you to identify and isolate service points that are causing degradation or failures.

X-Ray also announced support for Amazon CloudWatch Synthetics, currently in preview. CloudWatch Synthetics on X-Ray support tracing canary scripts throughout the application, providing metrics on performance or application issues.

Screen capture of AWS X-Ray Service map in the AWS Management Console

X-Ray Service map with CloudWatch Synthetics

Amazon DynamoDB

Amazon DynamoDB announced support for customer-managed customer master keys (CMKs) to encrypt data in DynamoDB. This allows customers to bring your own key (BYOK) giving you full control over how you encrypt and manage the security of your DynamoDB data.

It is now possible to add global replicas to existing DynamoDB tables to provide enhanced availability across the globe.

Another new DynamoDB capability to identify frequently accessed keys and database traffic trends is currently in preview. With this, you can now more easily identify “hot keys” and understand usage of your DynamoDB tables.

Screen capture of Amazon CloudWatch Contributor Insights for DynamoDB in the AWS Management Console

CloudWatch Contributor Insights for DynamoDB

DynamoDB also released adaptive capacity. Adaptive capacity helps you handle imbalanced workloads by automatically isolating frequently accessed items and shifting data across partitions to rebalance them. This helps reduce cost by enabling you to provision throughput for a more balanced workload instead of over provisioning for uneven data access patterns.

Amazon RDS

Amazon Relational Database Services (RDS) announced a preview of Amazon RDS Proxy to help developers manage RDS connection strings for serverless applications.

Illustration of Amazon RDS Proxy

The RDS Proxy maintains a pool of established connections to your RDS database instances. This pool enables you to support a large number of application connections so your application can scale without compromising performance. It also increases security by enabling IAM authentication for database access and enabling you to centrally manage database credentials using AWS Secrets Manager.

AWS Serverless Application Repository

The AWS Serverless Application Repository (SAR) now offers Verified Author badges. These badges enable consumers to quickly and reliably know who you are. The badge appears next to your name in the SAR and links to your GitHub profile.

Screen capture of SAR Verifiedl developer badge in the AWS Management Console

SAR Verified developer badges

AWS Developer Tools

AWS CodeCommit launched the ability for you to enforce rule workflows for pull requests, making it easier to ensure that code has pass through specific rule requirements. You can now create an approval rule specifically for a pull request, or create approval rule templates to be applied to all future pull requests in a repository.

AWS CodeBuild added beta support for test reporting. With test reporting, you can now view the detailed results, trends, and history for tests executed on CodeBuild for any framework that supports the JUnit XML or Cucumber JSON test format.

Screen capture of AWS CodeBuild

CodeBuild test trends in the AWS Management Console

Amazon CodeGuru

AWS announced a preview of Amazon CodeGuru at re:Invent 2019. CodeGuru is a machine learning based service that makes code reviews more effective and aids developers in writing code that is more secure, performant, and consistent.

AWS Amplify and AWS AppSync

AWS Amplify added iOS and Android as supported platforms. Now developers can build iOS and Android applications using the Amplify Framework with the same category-based programming model that they use for JavaScript apps.

Screen capture of 'amplify init' for an iOS application in a terminal window

The Amplify team has also improved offline data access and synchronization by announcing Amplify DataStore. Developers can now create applications that allow users to continue to access and modify data, without an internet connection. Upon connection, the data synchronizes transparently with the cloud.

For a summary of Amplify and AppSync announcements before re:Invent, read: “A round up of the recent pre-re:Invent 2019 AWS Amplify Launches”.

Illustration of AWS AppSync integrations with other AWS services

Q4 serverless content

Blog posts

October

November

December

Tech talks

We hold several AWS Online Tech Talks covering serverless tech talks throughout the year. These are listed in the Serverless section of the AWS Online Tech Talks page.

Here are the ones from Q4:

Twitch

October

There are also a number of other helpful video series covering Serverless available on the AWS Twitch Channel.

AWS Serverless Heroes

We are excited to welcome some new AWS Serverless Heroes to help grow the serverless community. We look forward to some amazing content to help you with your serverless journey.

AWS Serverless Application Repository (SAR) Apps

In this edition of ICYMI, we are introducing a section devoted to SAR apps written by the AWS Serverless Developer Advocacy team. You can run these applications and review their source code to learn more about serverless and to see examples of suggested practices.

Still looking for more?

The Serverless landing page has much more information. The Lambda resources page contains case studies, webinars, whitepapers, customer stories, reference architectures, and even more Getting Started tutorials. We’re also kicking off a fresh series of Tech Talks in 2020 with new content providing greater detail on everything new coming out of AWS for serverless application developers.

Throughout 2020, the AWS Serverless Developer Advocates are crossing the globe to tell you more about serverless, and to hear more about what you need. Follow this blog to keep up on new launches and announcements, best practices, and examples of serverless applications in action.

You can also follow all of us on Twitter to see latest news, follow conversations, and interact with the team.

Chris Munns: @chrismunns
Eric Johnson: @edjgeek
James Beswick: @jbesw
Moheeb Zara: @virgilvox
Ben Smith: @benjamin_l_s
Rob Sutter: @rts_rob
Julian Wood: @julian_wood

Happy coding!

Prototyping optimizations with Cloudflare Workers and WebPageTest

Post Syndicated from Andrew Galloni original https://blog.cloudflare.com/workers-and-webpagetest/

Prototyping optimizations with Cloudflare Workers and WebPageTest

This article was originally published as part of  Perf Planet’s 2019 Web Performance Calendar.

Have you ever wanted to quickly test a new performance idea, or see if the latest performance wisdom is beneficial to your site? As web performance appears to be a stochastic process, it is really important to be able to iterate quickly and review the effects of different experiments. The challenge is to be able to arbitrarily change requests and responses without the overhead of setting up another internet facing server. This can be straightforward to implement by combining two of my favourite technologies : WebPageTest and Cloudflare Workers. Pat Meenan sums this up with the following slide from a recent getting the most of WebPageTest presentation:

Prototyping optimizations with Cloudflare Workers and WebPageTest

So what is Cloudflare Workers and why is it ideally suited to easy prototyping of optimizations?

Cloudflare Workers

From the documentation :

Cloudflare Workers provides a lightweight JavaScript execution environment that allows developers to augment existing applications or create entirely new ones without configuring or maintaining infrastructure.A Cloudflare Worker is a programmable proxy which brings the simplicity and flexibility of the Service Workers event-based fetch API from the browser to the edge. This allows a worker to intercept and modify requests and responses.

Prototyping optimizations with Cloudflare Workers and WebPageTest

With the Service Worker API you can add an EventListener to any fetch event that is routed through the worker script and modify the request to come from a different origin.

Cloudflare Workers also provides a streaming HTMLRewriter to enable on the fly modification of HTML as it passes through the worker. The streaming nature of this parser ensures latency is minimised as the entire HTML document does not have to be buffered before rewriting can happen.

Setting up a worker

It is really quick and easy to sign up for a free subdomain at workers.dev which provides you with 100,000 free requests per day. There is a quick-start guide available here.To be able to run the examples in this post you will need to install Wrangler, the CLI tool for deploying workers. Once Wrangler is installed run the following command to download the example worker project:    

wrangler generate wpt-proxy https://github.com/xtuc/WebPageTest-proxy

You will then need to update the wrangler.toml with your account_id, which can be found in the dashboard in the right sidebar. Then configure an API key with the command:

wrangler config

Finally, you can publish the worker with:  

wrangler publish

At this the point, the worker will be active at

https://wpt-proxy.<your-subdomain>.workers.dev.

WebPageTest OverrideHost  

Now that your worker is configured, the next step is to configure WebPageTest to redirect requests through the worker. WebPageTest has a feature where it can re-point arbitrary origins to a different domain. To access the feature in WebPageTest, you need to use the WebPageTest scripting language “overrideHost” command, as shown:

Prototyping optimizations with Cloudflare Workers and WebPageTest

This example will redirect all network requests to www.bbc.co.uk to wpt-proxy.prf.workers.dev instead. WebPageTest also adds an x-host header to each redirected request so that the destination can determine for which host the request was originally intended:    

x-host: www.bbc.co.uk

The script can process multiple overrideHost commands to override multiple different origins. If HTTPS is used, WebPageTest can use HTTP/2 and benefit from connection coalescing:  

overrideHost www.bbc.co.uk wpt-proxy.prf.workers.dev    
overrideHost nav.files.bbci.co.uk wpt-proxy.prf.workers.dev
navigate https://www.bbc.co.uk

 It also supports wildcards:  

overrideHost *bbc.co.uk wpt-proxy.prf.workers.dev    
navigate https://www.bbc.co.uk

There are a few special strings that can be used in a script when bulk testing, so a single script can be re-used across multiple URLs:

  • %URL% – Replaces with the URL of the current test
  • %HOST% – Replaces with the hostname of the URL of the current test
  • %HOSTR% – Replaces with the hostname of the final URL in case the test URL does a redirect.

A more generic script would look like this:    

overrideHost %HOSTR% wpt-proxy.prf.workers.dev    
navigate %URL% 

Basic worker

In the base example below, the worker listens for the fetch event, looks for the x-host header that WebPageTest has set and responds by fetching the content from the orginal url:

/* 
* Handle all requests. 
* Proxy requests with an x-host header and return 403
* for everything else
*/

addEventListener("fetch", event => {    
   const host = event.request.headers.get('x-host');        
   if (host) {          
      const url = new URL(event.request.url);          
      const originUrl = url.protocol + '//' + host + url.pathname + url.search;             
      let init = {             
         method: event.request.method,             
         redirect: "manual",             
         headers: [...event.request.headers]          
      };          
      event.respondWith(fetch(originUrl, init));        
   } 
   else {           
     const response = new Response('x-Host headers missing', {status: 403});                
     event.respondWith(response);        
   }    
});

The source code can be found here and instructions to download and deploy this worker are described in the earlier section.

So what happens if we point all the domains on the BBC website through this worker, using the following config:  

overrideHost    *bbci.co.uk wpt.prf.workers.dev    
overrideHost    *bbc.co.uk  wpt.prf.workers.dev    
navigate    https://www.bbc.co.uk

configured to a 3G Fast setting from a UK test location.

Prototyping optimizations with Cloudflare Workers and WebPageTest
Comparison of BBC website if when using a single connection. 

BeforeAfter
Prototyping optimizations with Cloudflare Workers and WebPageTestPrototyping optimizations with Cloudflare Workers and WebPageTest

The potential performance improvement of loading a page over a single connection, eliminating the additional DNS lookup, TCP connection and TLS handshakes, can be seen  by comparing the filmstrips and waterfalls. There are several reasons why you may not want or be able to move everything to a single domain, but at least it is now easy to see what the performance difference would be.  

HTMLRewriter

With the HTMLRewriter, it is possible to change the HTML response as it passes through the worker. A jQuery-like syntax provides CSS-selector matching and a standard set of DOM mutation methods. For instance you could rewrite your page to measure the effects of different preload/prefetch strategies, review the performance savings of removing or using different third-party scripts, or you could stock-take the HEAD of your document. One piece of performance advice is to self-host some third-party scripts. This example script invokes the HTMLRewriter to listen for a script tag with a src attribute. If the script is from a proxiable domain the src is rewritten to be first-party, with a specific path prefix.

async function rewritePage(request) {  
  const response = await fetch(request);    
    return new HTMLRewriter()      
      .on("script[src]", {        
        element: el => {          
          let src = el.getAttribute("src");          
          if (PROXIED_URL_PREFIXES_RE.test(src)) {
            el.setAttribute("src", createProxiedScriptUrl(src));
          }           
        }    
    })    
    .transform(response);
}

Subsequently, when the browser makes a request with the specific prefix, the worker fetches the asset from the original URL. This example can be downloaded with this command:    

wrangler generate test https://github.com/xtuc/rewrite-3d-party.git

Request Mangling

As well as rewriting content, it is also possible to change or delay a request. Below is an example of how to randomly add a delay of a second to a request:

addEventListener("fetch", event => {    
    const host = event.request.headers.get('x-host');    
    if (host) { 
//....     
    // Add the delay if necessary     
    if (Math.random() * 100 < DELAY_PERCENT) {       
      await new Promise(resolve => setTimeout(resolve, DELAY_MS));     
    }    
    event.respondWith(fetch(originUrl, init));
//...
}

HTTP/2 prioritization

What if you want to see what the effect of changing the HTTP/2 prioritization of assets would make to your website? Cloudflare Workers provide custom http2 prioritization schemes that can be applied by setting a custom header on the response. The cf-priority header is defined as <priority>/<concurrency> so adding:    

response.headers.set('cf-priority', “30/0”);    

would set the priority of that response to 30 with a concurrency of 0 for the given response. Similarly, “30/1” would set concurrency to 1 and “30/n” would set concurrency to n. With this flexibility, you can prioritize the bytes that are important for your website or run a bulk test to prove that your new  prioritization scheme is better than any of the existing browser implementations.

Summary

A major barrier to understanding and innovation, is the amount of time is takes to get feedback. Having a quick and easy framework, to try out a new idea and comprehend the impact, is key. I hope this post has convinced you that combining WebPageTest and Cloudflare Workers is an easy solution to this problem and is indeed magic

Orchestrating a security incident response with AWS Step Functions

Post Syndicated from Benjamin Smith original https://aws.amazon.com/blogs/compute/orchestrating-a-security-incident-response-with-aws-step-functions/

In this post I will show how to implement the callback pattern of an AWS Step Functions Standard Workflow. This is used to add a manual approval step into an automated security incident response framework. The framework could be extended to remediate automatically, according to the individual policy actions defined. For example, applying alternative actions, or restricting actions to specific ARNs.

The application uses Amazon EventBridge to trigger a Step Functions Standard Workflow on an IAM policy creation event. The workflow compares the policy action against a customizable list of restricted actions. It uses AWS Lambda and Step Functions to roll back the policy temporarily, then notify an administrator and wait for them to approve or deny.

Figure 1: High-level architecture diagram.

Important: the application uses various AWS services, and there are costs associated with these services after the Free Tier usage. Please see the AWS pricing page for details.

You can deploy this application from the AWS Serverless Application Repository. You then create a new IAM Policy to trigger the rule and run the application.

Deploy the application from the Serverless Application Repository

  1. Find the “Automated-IAM-policy-alerts-and-approvals” app in the Serverless Application Repository.
  2. Complete the required application settings
    • Application name: an identifiable name for the application.
    • EmailAddress: an administrator’s email address for receiving approval requests.
    • restrictedActions: the IAM Policy actions you want to restrict.

      Figure 2 Deployment Fields

  3. Choose Deploy.

Once the deployment process is completed, 21 new resources are created. This includes:

  • Five Lambda functions that contain the business logic.
  • An Amazon EventBridge rule.
  • An Amazon SNS topic and subscription.
  • An Amazon API Gateway REST API with two resources.
  • An AWS Step Functions state machine

To receive Amazon SNS notifications as the application administrator, you must confirm the subscription to the SNS topic. To do this, choose the Confirm subscription link in the verification email that was sent to you when deploying the application.

EventBridge receives new events in the default event bus. Here, the event is compared with associated rules. Each rule has an event pattern defined, which acts as a filter to match inbound events to their corresponding rules. In this application, a matching event rule triggers an AWS Step Functions execution, passing in the event payload from the policy creation event.

Running the application

Trigger the application by creating a policy either via the AWS Management Console or with the AWS Command Line Interface.

Using the AWS CLI

First install and configure the AWS CLI, then run the following command:

aws iam create-policy --policy-name my-bad-policy1234 --policy-document '{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketObjectLockConfiguration",
                "s3:DeleteObjectVersion",
                "s3:DeleteBucket"
            ],
            "Resource": "*"
        }
    ]
}'

Using the AWS Management Console

  1. Go to Services > Identity Access Management (IAM) dashboard.
  2. Choose Create policy.
  3. Choose the JSON tab.
  4. Paste the following JSON:
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "VisualEditor0",
                "Effect": "Allow",
                "Action": [
                    "s3:GetBucketObjectLockConfiguration",
                    "s3:DeleteObjectVersion",
                    "s3:DeleteBucket"
                ],
                "Resource": "*"
            }
        ]
    }
  5. Choose Review policy.
  6. In the Name field, enter my-bad-policy.
  7. Choose Create policy.

Either of these methods creates a policy with the permissions required to delete Amazon S3 buckets. Deleting an S3 bucket is one of the restricted actions set when the application is deployed:

Figure 3 default restricted actions

This sends the event to EventBridge, which then triggers the Step Functions state machine. The Step Functions state machine holds each state object in the workflow. Some of the state objects use the Lambda functions created during deployment to process data.

Others use Amazon States Language (ASL) enabling the application to conditionally branch, wait, and transition to the next state. Using a state machine decouples the business logic from the compute functionality.

After triggering the application, go to the Step Functions dashboard and choose the newly created state machine. Choose the current running state machine from the executions table.

Figure 4 State machine executions.

You see a visual representation of the current execution with the workflow is paused at the AskUser state.

Figure 5 Workflow Paused

These are the states in the workflow:

ModifyData
State Type: Pass
Re-structures the input data into an object that is passed throughout the workflow.

ValidatePolicy
State type: Task. Services: AWS Lambda
Invokes the ValidatePolicy Lambda function that checks the new policy document against the restricted actions.

ChooseAction
State type: Choice
Branches depending on input from ValidatePolicy step.

TempRemove
State type: Task. Service: AWS Lambda
Creates a new default version of the policy with only permissions for Amazon CloudWatch Logs and deletes the previously created policy version.

AskUser
State type: Choice
Sends an approval email to user via SNS, with the task token that initiates the callback pattern.

UsersChoice
State type: Choice
Branch based on the user action to approve or deny.

Denied
State type: Pass
Ends the execution with no further action.

Approved
State type: Task. Service: AWS Lambda
Restores the initial policy document by creating as a new version.

AllowWithNotification
State type: Task. Services: AWS Lambda
With no restricted actions detected, the user is still notified of change (via an email from SNS) before execution ends.

The callback pattern

An important feature of this application is the ability for an administrator to approve or deny a new policy. The Step Functions callback pattern makes this possible.

The callback pattern allows a workflow to pause during a task and wait for an external process to return a task token. The task token is generated when the task starts. When the AskUser function is invoked, it is passed a task token. The task token is published to the SNS topic along with the API resources for approval and denial. These API resources are created when the application is first deployed.

When the administrator clicks on the approve or deny links, it passes the token with the API request to the receiveUser Lambda function. This Lambda function uses the incoming task token to resume the AskUser state.

The lifecycle of the task token as it transitions through each service is shown below:

Figure 6 Task token lifecycle

  1. To invoke this callback pattern, the askUser state definition is declared using the .waitForTaskToken identifier, with the task token passed into the Lambda function as a payload parameter:
    "AskUser":{
     "Type": "Task",
     "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
     "Parameters":{  
     "FunctionName": "${AskUser}",
     "Payload":{  
     "token.$":"$$.Task.Token"
      }
     },
      "ResultPath":"$.taskresult",
      "Next": "usersChoice"
      },
  2. The askUser Lambda function can then access this token within the event object:
    exports.handler = async (event,context) => {
        let approveLink = `process.env.APIAllowEndpoint?token=${JSON.stringify(event.token)}`
        let denyLink = `process.env.APIDenyEndpoint?token=${JSON.stringify(event.token)}
    //code continues
  3. The task token is published to an SNS topic along with the message text parameter:
        let params = {
     TopicArn: process.env.Topic,
     Message: `A restricted Policy change has been detected Approve:${approveLink} Or Deny:${denyLink}` 
    }
     let res = await sns.publish(params).promise()
    //code continues
  4. The administrator receives an email with two links, one to approve and one to deny. The task token is appended to these links as a request query string parameter named token:

    Figure 7 Approve / deny email.

  5. Using the Amazon API Gateway proxy integration, the task token is passed directly to the recieveUser Lambda function from the API resource, and accessible from within in the function code as part of the event’s queryStringParameter object:
    exports.handler = async(event, context) => {
    //some code
        let taskToken = event.queryStringParameters.token
    //more code
    
  6.  The token is then sent back to the askUser state via an API call from within the recieveUser Lambda function.  This API call also defines the next course of action for the workflow to take.
    //some code 
    let params = {
            output: JSON.stringify({"action":NextAction}),
            taskToken: taskTokenClean
        }
    let res = await stepfunctions.sendTaskSuccess(params).promise()
    //code continues
    

Each Step Functions execution can last for up to a year, allowing for long wait periods for the administrator to take action. There is no extra cost for a longer wait time as you pay for the number of state transitions, and not for the idle wait time.

Conclusion

Using EventBridge to route IAM policy creation events directly to AWS Step Functions reduces the need for unnecessary communication layers. It helps promote good use of compute resources, ensuring Lambda is used to transform data, and not transport or orchestrate.

Using Step Functions to invoke services sequentially has two important benefits for this application. First, you can identify the use of restricted policies quickly and automatically. Also, these policies can be removed and held in a ‘pending’ state until approved.

Step Functions Standard Workflow’s callback pattern can create a robust orchestration layer that allows administrators to review each change before approving or denying.

For the full code base see the GitHub repository https://github.com/bls20AWS/AutomatedPolicyOrchestrator.

For more information on other Step Functions patterns, see our documentation on integration patterns.

Building an AWS IoT Core device using AWS Serverless and an ESP32

Post Syndicated from Moheeb Zara original https://aws.amazon.com/blogs/compute/building-an-aws-iot-core-device-using-aws-serverless-and-an-esp32/

Using a simple Arduino sketch, an AWS Serverless Application Repository application, and a microcontroller, you can build a basic serverless workflow for communicating with an AWS IoT Core device.

A microcontroller is a programmable chip and acts as the brain of an electronic device. It has input and output pins for reading and writing on digital or analog components. Those components could be sensors, relays, actuators, or various other devices. It can be used to build remote sensors, home automation products, robots, and much more. The ESP32 is a powerful low-cost microcontroller with Wi-Fi and Bluetooth built in and is used this walkthrough.

The Arduino IDE, a lightweight development environment for hardware, now includes support for the ESP32. There is a large collection of community and officially supported libraries, from addressable LED strips to spectral light analysis.

The following walkthrough demonstrates connecting an ESP32 to AWS IoT Core to allow it to publish and subscribe to topics. This means that the device can send any arbitrary information, such as sensor values, into AWS IoT Core while also being able to receive commands.

Solution overview

This post walks through deploying an application from the AWS Serverless Application Repository. This allows an AWS IoT device to be messaged using a REST endpoint powered by Amazon API Gateway and AWS Lambda. The AWS SAR application also configures an AWS IoT rule that forwards any messages published by the device to a Lambda function that updates an Amazon DynamoDB table, demonstrating basic bidirectional communication.

The last section explores how to build an IoT project with real-world application. By connecting a thermal printer module and modifying a few lines of code in the example firmware, the ESP32 device becomes an AWS IoT–connected printer.

All of this can be accomplished within the AWS Free Tier, which is necessary for the following instructions.

An example of an AWS IoT project using an ESP32, AWS IoT Core, and an Arduino thermal printer

An example of an AWS IoT project using an ESP32, AWS IoT Core, and an Arduino thermal printer.

Required steps

To complete the walkthrough, follow these steps:

  • Create an AWS IoT device.
  • Install and configure the Arduino IDE.
  • Configure and flash an ESP32 IoT device.
  • Deploying the lambda-iot-rule AWS SAR application.
  • Monitor and test.
  • Create an IoT thermal printer.

Creating an AWS IoT device

To communicate with the ESP32 device, it must connect to AWS IoT Core with device credentials. You must also specify the topics it has permissions to publish and subscribe on.

  1. In the AWS IoT console, choose Register a new thing, Create a single thing.
  2. Name the new thing. Use this exact name later when configuring the ESP32 IoT device. Leave the remaining fields set to their defaults. Choose Next.
  3.  Choose Create certificate. Only the thing cert, private key, and Amazon Root CA 1 downloads are necessary for the ESP32 to connect. Download and save them somewhere secure, as they are used when programming the ESP32 device.
  4. Choose Activate, Attach a policy.
  5. Skip adding a policy, and choose Register Thing.
  6. In the AWS IoT console side menu, choose Secure, Policies, Create a policy.
  7. Name the policy Esp32Policy. Choose the Advanced tab.
  8. Paste in the following policy template.
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": "iot:Connect",
          "Resource": "arn:aws:iot:REGION:ACCOUNT_ID:client/THINGNAME"
        },
        {
          "Effect": "Allow",
          "Action": "iot:Subscribe",
          "Resource": "arn:aws:iot:REGION:ACCOUNT_ID:topicfilter/esp32/sub"
        },
    	{
          "Effect": "Allow",
          "Action": "iot:Receive",
          "Resource": "arn:aws:iot:REGION:ACCOUNT_ID:topic/esp32/sub"
        },
        {
          "Effect": "Allow",
          "Action": "iot:Publish",
          "Resource": "arn:aws:iot:REGION:ACCOUNT_ID:topic/esp32/pub"
        }
      ]
    }
  9. Replace REGION with the matching AWS Region you’re currently operating in. This can be found on the top right corner of the AWS console window.
  10.  Replace ACCOUNT_ID with your own, which can be found in Account Settings.
  11. Replace THINGNAME with the name of your device.
  12. Choose Create.
  13. In the AWS IoT console, choose Secure, Certification. Select the one created for your device and choose Actions, Attach policy.
  14. Choose Esp32Policy, Attach.

Your AWS IoT device is now configured to have permission to connect to AWS IoT Core. It can also publish to the topic esp32/pub and subscribe to the topic esp32/sub. For more information on securing devices, see AWS IoT Policies.

Installing and configuring the Arduino IDE

The Arduino IDE is an open-source development environment for programming microcontrollers. It supports a continuously growing number of platforms including most ESP32-based modules. It must be installed along with the ESP32 board definitions, MQTT library, and ArduinoJson library.

  1. Download the Arduino installer for the desired operating system.
  2. Start Arduino and open the Preferences window.
  3. For Additional Board Manager URLs, add
    https://dl.espressif.com/dl/package_esp32_index.json.
  4. Choose Tools, Board, Boards Manager.
  5. Search esp32 and install the latest version.
  6. Choose Sketch, Include Library, Manage Libraries.
  7. Search MQTT, and install the latest version by Joel Gaehwiler.
  8. Repeat the library installation process for ArduinoJson.

The Arduino IDE is now installed and configured with all the board definitions and libraries needed for this walkthrough.

Configuring and flashing an ESP32 IoT device

A collection of various ESP32 development boards.

A collection of various ESP32 development boards.

For this section, you need an ESP32 device. To check if your board is compatible with the Arduino IDE, see the boards.txt file. The following code connects to AWS IoT Core securely using MQTT, a publish and subscribe messaging protocol.

This project has been tested on the following devices:

  1. Install the required serial drivers for your device. Some boards use different USB/FTDI chips for interfacing. Here are the most commonly used with links to drivers.
  2. Open the Arduino IDE and choose File, New to create a new sketch.
  3. Add a new tab and name it secrets.h.
  4. Paste the following into the secrets file.
    #include <pgmspace.h>
    
    #define SECRET
    #define THINGNAME ""
    
    const char WIFI_SSID[] = "";
    const char WIFI_PASSWORD[] = "";
    const char AWS_IOT_ENDPOINT[] = "xxxxx.amazonaws.com";
    
    // Amazon Root CA 1
    static const char AWS_CERT_CA[] PROGMEM = R"EOF(
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
    )EOF";
    
    // Device Certificate
    static const char AWS_CERT_CRT[] PROGMEM = R"KEY(
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
    )KEY";
    
    // Device Private Key
    static const char AWS_CERT_PRIVATE[] PROGMEM = R"KEY(
    -----BEGIN RSA PRIVATE KEY-----
    -----END RSA PRIVATE KEY-----
    )KEY";
  5. Enter the name of your AWS IoT thing, as it is in the console, in the field THINGNAME.
  6. To connect to Wi-Fi, add the SSID and PASSWORD of the desired network. Note: The network name should not include spaces or special characters.
  7. The AWS_IOT_ENDPOINT can be found from the Settings page in the AWS IoT console.
  8. Copy the Amazon Root CA 1, Device Certificate, and Device Private Key to their respective locations in the secrets.h file.
  9. Choose the tab for the main sketch file, and paste the following.
    #include "secrets.h"
    #include <WiFiClientSecure.h>
    #include <MQTTClient.h>
    #include <ArduinoJson.h>
    #include "WiFi.h"
    
    // The MQTT topics that this device should publish/subscribe
    #define AWS_IOT_PUBLISH_TOPIC   "esp32/pub"
    #define AWS_IOT_SUBSCRIBE_TOPIC "esp32/sub"
    
    WiFiClientSecure net = WiFiClientSecure();
    MQTTClient client = MQTTClient(256);
    
    void connectAWS()
    {
      WiFi.mode(WIFI_STA);
      WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
    
      Serial.println("Connecting to Wi-Fi");
    
      while (WiFi.status() != WL_CONNECTED){
        delay(500);
        Serial.print(".");
      }
    
      // Configure WiFiClientSecure to use the AWS IoT device credentials
      net.setCACert(AWS_CERT_CA);
      net.setCertificate(AWS_CERT_CRT);
      net.setPrivateKey(AWS_CERT_PRIVATE);
    
      // Connect to the MQTT broker on the AWS endpoint we defined earlier
      client.begin(AWS_IOT_ENDPOINT, 8883, net);
    
      // Create a message handler
      client.onMessage(messageHandler);
    
      Serial.print("Connecting to AWS IOT");
    
      while (!client.connect(THINGNAME)) {
        Serial.print(".");
        delay(100);
      }
    
      if(!client.connected()){
        Serial.println("AWS IoT Timeout!");
        return;
      }
    
      // Subscribe to a topic
      client.subscribe(AWS_IOT_SUBSCRIBE_TOPIC);
    
      Serial.println("AWS IoT Connected!");
    }
    
    void publishMessage()
    {
      StaticJsonDocument<200> doc;
      doc["time"] = millis();
      doc["sensor_a0"] = analogRead(0);
      char jsonBuffer[512];
      serializeJson(doc, jsonBuffer); // print to client
    
      client.publish(AWS_IOT_PUBLISH_TOPIC, jsonBuffer);
    }
    
    void messageHandler(String &topic, String &payload) {
      Serial.println("incoming: " + topic + " - " + payload);
    
    //  StaticJsonDocument<200> doc;
    //  deserializeJson(doc, payload);
    //  const char* message = doc["message"];
    }
    
    void setup() {
      Serial.begin(9600);
      connectAWS();
    }
    
    void loop() {
      publishMessage();
      client.loop();
      delay(1000);
    }
  10. Choose File, Save, and give your project a name.

Flashing the ESP32

  1. Plug the ESP32 board into a USB port on the computer running the Arduino IDE.
  2. Choose Tools, Board, and then select the matching type of ESP32 module. In this case, a Sparkfun ESP32 Thing was used.
  3. Choose Tools, Port, and then select the matching port for your device.
  4. Choose Upload. Arduino reads Done uploading when the upload is successful.
  5. Choose the magnifying lens icon to open the Serial Monitor. Set the baud rate to 9600.

Keep the Serial Monitor open. When connected to Wi-Fi and then AWS IoT Core, any messages received on the topic esp32/sub are logged to this console. The device is also now publishing to the topic esp32/pub.

The topics are set at the top of the sketch. When changing or adding topics, remember to add permissions in the device policy.

// The MQTT topics that this device should publish/subscribe
#define AWS_IOT_PUBLISH_TOPIC   "esp32/pub"
#define AWS_IOT_SUBSCRIBE_TOPIC "esp32/sub"

Within this sketch, the relevant functions are publishMessage() and messageHandler().

The publishMessage() function creates a JSON object with the current time in milliseconds and the analog value of pin A0 on the device. It then publishes this JSON object to the topic esp32/pub.

void publishMessage()
{
  StaticJsonDocument<200> doc;
  doc["time"] = millis();
  doc["sensor_a0"] = analogRead(0);
  char jsonBuffer[512];
  serializeJson(doc, jsonBuffer); // print to client

  client.publish(AWS_IOT_PUBLISH_TOPIC, jsonBuffer);
}

The messageHandler() function prints out the topic and payload of any message from a subscribed topic. To see all the ways to parse JSON messages in Arduino, see the deserializeJson() example.

void messageHandler(String &topic, String &payload) {
  Serial.println("incoming: " + topic + " - " + payload);

//  StaticJsonDocument<200> doc;
//  deserializeJson(doc, payload);
//  const char* message = doc["message"];
}

Additional topic subscriptions can be added within the connectAWS() function by adding another line similar to the following.

// Subscribe to a topic
  client.subscribe(AWS_IOT_SUBSCRIBE_TOPIC);

  Serial.println("AWS IoT Connected!");

Deploying the lambda-iot-rule AWS SAR application

Now that an ESP32 device has been connected to AWS IoT, the following steps walk through deploying an AWS Serverless Application Repository application. This is a base for building serverless integration with a physical device.

  1. On the lambda-iot-rule AWS Serverless Application Repository application page, make sure that the Region is the same as the AWS IoT device.
  2. Choose Deploy.
  3. Under Application settings, for PublishTopic, enter esp32/sub. This is the topic to which the ESP32 device is subscribed. It receives messages published to this topic. Likewise, set SubscribeTopic to esp32/pub, the topic on which the device publishes.
  4. Choose Deploy.
  5. When creation of the application is complete, choose Test app to navigate to the application page. Keep this page open for the next section.

Monitoring and testing

At this stage, two Lambda functions, a DynamoDB table, and an AWS IoT rule have been deployed. The IoT rule forwards messages on topic esp32/pub to TopicSubscriber, a Lambda function, which inserts the messages on to the DynamoDB table.

  1. On the application page, under Resources, choose MyTable. This is the DynamoDB table that the TopicSubscriber Lambda function updates.
  2. Choose Items. If the ESP32 device is still active and connected, messages that it has published appear here.

The TopicPublisher Lambda function is invoked by the API Gateway endpoint and publishes to the AWS IoT topic esp32/sub.

1.     On the application page, find the Application endpoint.

2.     To test that the TopicPublisher function is working, enter the following into a terminal or command-line utility, replacing ENDPOINT with the URL from above.

curl -d '{"text":"Hello world!"}' -H "Content-Type: application/json" -X POST https://ENDPOINT/publish

Upon success, the request returns a copy of the message.

Back in the Serial Monitor, the message published to the topic esp32/sub prints out.

Creating an IoT thermal printer

With the completion of the previous steps, the ESP32 device currently logs incoming messages to the serial console.

The following steps demonstrate how the code can be modified to use incoming messages to interact with a peripheral component. This is done by wiring a thermal printer to the ESP32 in order to physically print messages. The REST endpoint from the previous section can be used as a webhook in third-party applications to interact with this device.

A wiring diagram depicting an ESP32 connected to a thermal printer.

A wiring diagram depicting an ESP32 connected to a thermal printer.

  1. Follow the product instructions for powering, wiring, and installing the correct Arduino library.
  2. Ensure that the thermal printer is working by holding the power button on the printer while connecting the power. A sample receipt prints. On that receipt, the default baud rate is specified as either 9600 or 19200.
  3. In the Arduino code from earlier, include the following lines at the top of the main sketch file. The second line defines what interface the thermal printer is connected to. &Serial2 is used to set the third hardware serial interface on the ESP32. For this example, the pins on the Sparkfun ESP32 Thing, GPIO16/GPIO17, are used for RX/TX respectively.
    #include "Adafruit_Thermal.h"
    
    Adafruit_Thermal printer(&Serial2);
  4. Replace the setup() function with the following to initialize the printer on device bootup. Change the baud rate of Serial2.begin() to match what is specified in the test print. The default is 19200.
    void setup() {
      Serial.begin(9600);
    
      // Start the thermal printer
      Serial2.begin(19200);
      printer.begin();
      printer.setSize('S');
    
      connectAWS();
    }
    
  5. Replace the messageHandler() function with the following. On any incoming message, it parses the JSON and prints the message on the thermal printer.
    void messageHandler(String &topic, String &payload) {
      Serial.println("incoming: " + topic + " - " + payload);
    
      // deserialize json
      StaticJsonDocument<200> doc;
      deserializeJson(doc, payload);
      String message = doc["message"];
    
      // Print the message on the thermal printer
      printer.println(message);
      printer.feed(2);
    }
  6. Choose Upload.
  7. After the firmware has successfully uploaded, open the Serial Monitor to confirm that the board has connected to AWS IoT.
  8. Enter the following into a command-line utility, replacing ENDPOINT, as in the previous section.
    curl -d '{"message": "Hello World!"}' -H "Content-Type: application/json" -X POST https://ENDPOINT/publish

If successful, the device prints out the message “Hello World” from the attached thermal printer. This is a fully serverless IoT printer that can be triggered remotely from a webhook. As an example, this can be used with GitHub Webhooks to print a physical readout of events.

Conclusion

Using a simple Arduino sketch, an AWS Serverless Application Repository application, and a microcontroller, this post demonstrated how to build a basic serverless workflow for communicating with a physical device. It also showed how to expand that into an IoT thermal printer with real-world applications.

With the use of AWS serverless, advanced compute and extensibility can be added to an IoT device, from machine learning to translation services and beyond. By using the Arduino programming environment, the vast collection of open-source libraries, projects, and code examples open up a world of possibilities. The next step is to explore what can be done with an Arduino and the capabilities of AWS serverless. The sample Arduino code for this project and more can be found at this GitHub repository.

Top 10 Architecture Blog Posts of 2019

Post Syndicated from Annik Stahl original https://aws.amazon.com/blogs/architecture/top-10-architecture-blog-posts-of-2019/

As we wind our way toward 2020, I want to take a moment to first thank you, our readers, for spending time on our blog. We grew our audience quite a bit this year and the credit goes to our hard-working Solutions Architects and other blog post writers. Below are the top 10 Architecture blog posts written in 2019.

#10: How to Architect APIs for Scale and Security

by George Mao

George Mao, a Specialist Solutions Architect at AWS, focuses on serverless computing and has FIVE posts in the top ten this year. Way to go, George!

This post was the first in a series that focused on best practices and concepts you should be familiar with when you architect APIs for your applications.

Read George’s post.

#9: From One to Many: Evolving VPC Guidance

by Androski Spicer

Since its inception, the Amazon Virtual Private Cloud (VPC) has acted as the embodiment of security and privacy for customers who are looking to run their applications in a controlled, private, secure, and isolated environment.

This logically isolated space has evolved, and in its evolution has increased the avenues that customers can take to create and manage multi-tenant environments with multiple integration points for access to resources on-premises.

Read Androski’s post.

#8: Things to Consider When You Build REST APIs with Amazon API Gateway

by George Mao

REST API 2

This post dives deeper into the things an API architect or developer should consider when building REST APIs with Amazon API Gateway.

Read George’s post.

#7: How to Design Your Serverless Apps for Massive Scale

by George Mao

Serverless at scale-1

Serverless is one of the hottest design patterns in the cloud today, allowing you to focus on building and innovating, rather than worrying about the heavy lifting of server and OS operations. In this series of posts, we’ll discuss topics that you should consider when designing your serverless architectures. First, we’ll look at architectural patterns designed to achieve massive scale with serverless.

Read George’s post.

#6: Best Practices for Developing on AWS Lambda

by George Mao

RDS instance: When to VPC enable a Lambda function

One of the benefits of using Lambda, is that you don’t have to worry about server and infrastructure management. This means AWS will handle the heavy lifting needed to execute your AWS Lambda functions. Take advantage of this architecture with the tips in this post.

Read George’s post.

#5: Stream Amazon CloudWatch Logs to a Centralized Account for Audit and Analysis

by David Bailey

Figure 1 - Initial Landing Zone logging account resources

A key component of enterprise multi-account environments is logging. Centralized logging provides a single point of access to all salient logs generated across accounts and regions, and is critical for auditing, security and compliance. While some customers use the built-in ability to push Amazon CloudWatch Logs directly into Amazon Elasticsearch Service for analysis, others would prefer to move all logs into a centralized Amazon Simple Storage Service (Amazon S3) bucket location for access by several custom and third-party tools. In this blog post, David Bailey will show you how to forward existing and any new CloudWatch Logs log groups created in the future to a cross-account centralized logging Amazon S3 bucket.

Read David’s post.

#4: Updates to Serverless Architectural Patterns and Best Practices

by Drew Dennis

Drew wrote this post at about the halfway point between re:Invent 2018 and re:Invent 2019, where he revisited some of the recent serverless announcements we’ve made. These are all complimentary to the patterns discussed in the re:Invent architecture track’s Serverless Architectural Patterns and Best Practices session.

Read Drew’s post.

#3: Understanding the Different Ways to Invoke Lambda Functions

by George Mao

Invoking Lambda

In George’s first post of this series (#7 on this list), he talked about general design patterns to enable massive scale with serverless applications. In this post, he’ll review the different ways you can invoke Lambda functions and what you should be aware of with each invocation model.

Read George’s post.

#2: Using API Gateway as a Single Entry Point for Web Applications and API Microservices

by Anandprasanna Gaitonde and Mohit Malik

In this post, Anand and Mohit talk about a reference architecture that allows API Gateway to act as single entry point for external-facing, API-based microservices and web applications across multiple external customers by leveraging a different subdomain for each one.

Read Anand’s and Mohit’s post.

#1: 10 Things Serverless Architects Should Know

by Justin Pirtle

Building on the first three parts of the AWS Lambda scaling and best practices series where you learned how to design serverless apps for massive scale, AWS Lambda’s different invocation models, and best practices for developing with AWS Lambda, Justin invited you to take your serverless knowledge to the next level by reviewing 10 topics to deepen your serverless skills.

Read Justin’s post.

Thank You

Thanks again to all our readers and blog post writers. We look forward to learning and building amazing things together in the coming year.

Best of 2019

ICYMI: Serverless re:Invent re:Cap 2019

Post Syndicated from Eric Johnson original https://aws.amazon.com/blogs/compute/icymi-serverless-reinvent-recap-2019/

Thank you for attending re:Invent 2019

In the week before AWS re:Invent 2019 we wrote about a number of service and feature launches leading up to the biggest event of the year for us at AWS. These included new features for AWS Lambda, integrations for AWS Step Functions, and other exciting service and feature launches for related product areas. But this was just the warm-up – AWS re:Invent 2019 itself saw several new serverless or serverless related announcements.

Here’s what’s new.

AWS Lambda

For developers needing greater control over performance of their serverless applications at any scale, AWS Lambda announced Provisioned Concurrency. This feature enables Lambda functions to execute with consistent start-up latency making them ideal for building latency sensitive applications.

AWS Step Functions

Express work flows

AWS Step Functions released AWS Step Functions Express Workflows. With the ability to support event rates greater than 100,000 per second, this feature is designed for high performance workloads at a reduced cost.

Amazon EventBridge

EventBridge schema registry and discovery

Amazon EventBridge announced the preview of the Amazon EventBridge schema registry and discovery service. This service allows developers to automate discovery and cataloging event schemas for use in their applications. Additionally, once a schema is stored in the registry, you can generate and download a code binding that represents the schema as an object in your code.

Amazon API Gateway

HTTP API

Amazon API Gateway announced the preview of HTTP APIs. With HTTP APIs most customers will see an average cost saving up to 70%, when compared to API Gateway REST APIs. In addition, you will see significant performance improvements in the API Gateway service overhead. With HTTP APIs, you can create an API in four simple steps. Once the API is created, additional configuration for CORS and JWT authorizers can be added.

Databases

Amazon Relational Database Services (RDS) announced a previews of Amazon RDS Proxy to help developers manage RDS connection strings for serverless applications.

RDS Proxy

The RDS proxy maintains a pool of established connections to your RDS database instances. This pool enables you to support a large number of application connections so your application can scale without compromising performance. It also increases security by enabling IAM authentication for database access and enabling you to centrally manage database credentials using AWS Secrets Manager.

AWS Amplify

Amplify platform choices

AWS Amplify has expanded their delivery platforms to include iOS and Android. Developers can now build iOS and Android applications using the Amplify Framework with the same category-based programming model that they use for JavaScript apps.

The Amplify team has also improved offline data access and synchronization by announcing Amplify DataStore. Developers can now create applications that allow users to continue to access and modify data, without an internet connection. Upon connection, the data synchronizes transparently with the cloud.

Amazon CodeGuru

Whether you are a team of one or an enterprise with thousands of developers, code review can be difficult. At re:Invent 2019, AWS announced a preview of Amazon CodeGuru, a machine learning based service to help make code reviews more effective and aid developers in writing code that is secure, performant, and consistent.

Serverless talks from re:Invent 2019

re:Invent presentation recordings

We presented dozens of sessions showing how customers can improve their architecture and agility with serverless. Here are some of the most popular.

Videos

Decks

You can also find decks for many of the serverless presentations and other re:Invent presentations on our AWS Events Content.

Conclusion

Prior to AWS re:Invent, AWS serverless had many service and feature launches and the pace continued throughout re:Invent itself. As we head towards 2020, follow this blog to keep up on new launches and announcements, best practices, and examples of serverless applications in action

Additionally, the AWS Serverless Developer Advocates will be crossing the globe to tell you more about serverless, and to hear more about what you need. You can also follow all of us on Twitter to see latest news, follow conversations, and interact with the team.

Chris Munns: @chrismunns
Eric Johnson: @edjgeek
James Beswick: @jbesw
Moheeb Zara: @virgilvox
Ben Smith: @benjamin_l_s
Rob Sutter: @rts_rob
Julian Wood: @julian_wood

Happy coding!

Integrating B2B using event notifications with Amazon SNS

Post Syndicated from Rachel Richardson original https://aws.amazon.com/blogs/compute/integrating-b2b-using-event-notifications-with-amazon-sns/

This post is courtesy of Murat Balkan, AWS Solutions Architect

Event notification patterns are popular among B2B integrations. Their scalable and decoupled structure helps implement complex integration scenarios in a variety of enterprises.

This post introduces a generic serverless architecture that applies to external integrations that use event notifications with Amazon SNS and Event Fork Pipelines. Some business scenarios involving B2B integrations include:

  • Inventory information sourcing to customers
  • Catalog sourcing to suppliers or partners
  • Real-time event sourcing to customers (for example, in an online auction)

External integration use cases vary, but a fundamental fact unifies them: External integration is difficult because target systems are not under your control. For example, your IT capabilities may differ. You may not have an internal development team and might rely on tools that can read data from a specific source type.

Alternatively, you may have a large development team and therefore have more data-processing needs and capabilities. These systems may replicate the information from the source, run complex machine learning algorithms against historical data, and must act upon real-time data.

Overview of event notification

Event notification is the sharing of state changes that occur in an application or domain with other applications or domains. You can relate events to any domain object such as orders, products, shipments, or financial transactions. The owner of these entities publishes changes to subscribers. The subscribers subscribe to different events and receive notifications accordingly when a new event is available.

After receiving an event, the integrating party application must determine what to do with the event. The application can store the event, enrich it with additional information, relay it to another party, or ignore it.

To ensure scalability, the publishing application must write or publish to a durable and scalable destination, for you to read from there. These destinations can be message queues (such as Amazon MQ and Amazon SQS) or data streams (such as Amazon Kinesis and Apache Kafka, and the managed AWS version Amazon MSK). To choose between streams and queues, evaluate the traffic characteristics and business use cases. However, the main principles are the same for both.

While creating your B2B external integration architecture, consider different needs, and introduce a mechanism to subscribe to only events of interest. In the example of an online auction site, those that perform active and automated bidding might be interested in real-time bidding events. Others, such as shippers, are only interested in tomorrow’s auction inventory. For the latter, an InventoryItemCreated event can be enough.

Events reflect the nature of a business environment, which can be unpredictable. If a worldwide event affects the markets, event counts might soar dramatically. A marketing event can also cause order events to rise. You need a scalable infrastructure to support your architecture. Serverless is a perfect fit for these kinds of scenarios, and this post’s architecture leverages several AWS serverless components.

In this architecture, you interact with a self-subscription application that exposes a REST API. To start interacting with the system, you also select one or more integration channels for receiving the events. You may prefer SFTP, while others prefer webhooks or multiple channels at the same time. Your IT and development capabilities play an essential role in this selection.

After you determine the integration channels, optionally select types of events channels. The self-subscription application knows all possible event types. As part of its development process, the application provisions them each time a new event appears.

The architecture’s notification channels are as follows:

  • Submission of read-time updates using webhook integration
  • Direct S3 access or SFTP integration
  • Access to Kinesis directly from other AWS accounts

Main data flow

The data flow begins when the publisher applications publish all of their events to a single Amazon SNS topic. Amazon SNS follows the publish and subscribe pattern to fan out a published message to all subscribers of that message topic.

It is worth mentioning Amazon’s new serverless service offering for event-based integrations, Amazon EventBridge. Amazon EventBridge is an event bus that makes it easy to connect applications together using data from your own applications, Software-as-a-Service (SaaS) applications, and AWS services. Amazon EventBridge comes with a powerful rules engine which allows you to put the business logic onto the bus. It can manipulate the payload of the messages and deliver specific payloads to specific consumer applications. Native event integration capabilities with AWS services make it a good candidate for event-based integrations.

For this architecture, I used SNS because AWS offers a quick deployment option through Event Fork Pipelines, a collection of open-source event handling pipelines, based on the AWS Serverless Application Model (AWS SAM).

You can deploy Event Fork Pipelines directly from the AWS Serverless Application Repository into your AWS account. The proposed SNS based architecture also allows the use of custom message payloads in any JSON format, including raw.

SNS has a powerful feature called subscription filter policies. These policies serve as intercepting filters and pass only the desired types of messages to subscribers. Because SNS performs the filtering, you don’t have to implement filtering logic, which decreases their complexity. The policies look for specific attributes and their values in the message. You can use the message attribute Event_Type for filtering.

After filtering the events, route them to the previously selected notification channels. The events land in a queue at each channel before the channel’s logic processes. SNS has built-in integration with SQS, a powerful serverless queueing service. SQS holds your events and acts as a buffer. Every delivery channel’s characteristics and handlers are different. You need a different SQS queue per delivery channel type.

Lambda functions subscribed to the webhook queue handle the webhook notification mechanism. You can convert the polled events to external HTTPS calls against your web servers. Internet delivery over the HTTP protocol is always slower than internal message propagation.

To keep up with the constant flow of events and increase message throughput, webhooks are sent in parallel. SQS provides different features for handling errors that might occur on the subscribing side. For example, the visibility timeout mechanism causes messages to be available after a specified time period, which serves as an auto-retry mechanism for consumed but not properly processed messages. You can also reject functional errors, which cause SQS to send you to dead-letter queues (DLQ) for further troubleshooting.

Amazon S3 handles the file-based notification mechanism. In this integration, a Lambda function polls a dedicated queue that integrates with S3. This function forwards the events to Amazon Kinesis Data Firehose. Kinesis Data Firehose acts as a buffer to consolidate individual messages into bigger files. SQS provides up to 10 messages in a single batch.

After reaching the Kinesis Data Firehose batch size or batch interval, Kinesis Data Firehose delivers the files to an S3 bucket. You can share this bucket with accounts using cross-account access. If you rely on SFTP for file transfers, AWS Transfer for SFTP can expose the objects over SFTP.

Kinesis Data Firehose also lets you define Lambda functions for the transformation of data before your Lambda function writes it to S3. You can use this part of the process to cleanse, filter, or enrich your data. For direct system integration, SQS cross-account access is always an option if you have an account. For more information, see Basic Examples of Amazon SQS Policies.

Example architecture that uses different event types for different delivery channels

Figure 1: Example architecture that uses different event types for different delivery channels

Self-subscription application

A self-subscription application collects channel and event type selections. You can use a single-page application that interacts with a REST API that Amazon API Gateway hosts. API Gateway uses AWS Lambda for backend processing and Amazon DynamoDB for user profile persistence. After collecting integration channel selections and optionally event type filters for these channels, the self-subscription application also orchestrates cloud provisioning tasks.

As subscriptions occur, the self-subscription application’s backend Lambda function triggers AWS CloudFormation to update the subscriptions, subscription filters, and other notification infrastructure components. A different AWS CloudFormation stack manages every integrating party.

Because the whole architecture is serverless, you can use AWS SAM during your provisioning and let AWS SAM interact with AWS CloudFormation. AWS SAM aims to simplify infrastructure as code practices for serverless resources. AWS SAM also allows you to inject application resources from AWS Serverless Application Repository.

AWS provides a set of serverless applications via AWS Serverless Application Repository to cover common integration scenarios for event-driven architectures. These off-the-shelf applications speed up the development and delivery of common event-driven mechanisms such as Command Query Responsibility Segregation (CQRS), Event Replay, and Event Storage or Backup. You can reference Event Fork   applications within AWS SAM templates to use in your applications.

While designing the provisioning pipeline of the proposed architecture, reuse two of the Event Fork Pipeline applications: Event Replay Pipeline (fork-event-replay-pipeline) and Event Storage and Backup Pipeline (fork-event-storage-backup-pipeline). Your webhooks use case is a custom Event Fork Pipeline application.

Because a single AWS SAM template contains these applications, you can deploy and manage the subscription filters and Event Fork Pipelines as a single self-subscription application stack for each integrating party.

CI/CD Pipeline that deploys the serverless application via AWS SAM

Figure 2: CI/CD Pipeline that deploys the serverless application via AWS SAM

In this architecture, Lambda converts the user input into an AWS SAM template and puts it into an S3 bucket. This PUT action triggers an AWS CodePipeline pipeline. The pipeline’s build phase downloads, packages, and deploys the provided AWS SAM template. You can also enhance the pipeline with features such as notifications, manual approvals, or external integrations.

Conclusion

Architectures such as this help you share your business or data events with suppliers, partners, and customers while minimizing integration time and streamlining your business processes. You can try out existing Event Fork Pipelines that was published by AWS, create custom pipelines for your internal use or share them with other AWS users in the AWS Serverless Application Repository.

Coming soon: Updated Lambda states lifecycle for VPC networking

Post Syndicated from Chris Munns original https://aws.amazon.com/blogs/compute/coming-soon-updated-lambda-states-lifecycle-for-vpc-networking/

On November 27, we announced that AWS Lambda now includes additional attributes in the function information returned by several Lambda API actions to better communicate the current “state” of your function, when they are being created or updated. In our post “Tracking the state of AWS Lambda functions”, we covered the various states your Lambda function can be in, the conditions that lead to them, and how the Lambda service transitions the function through those states.

Our first feature using the function states lifecycle is a change to the recently announced improved VPC networking for AWS Lambda functions. As stated in the announcement post, Lambda creates the ENIs required for your function to connect to your VPCs, which can take 60–90 seconds to complete. We are updating this operation to explicitly place the function into a Pending state while pre-creating the required elastic network interface resources, and transitioning to an Active state after that process is completed. By doing this, we can use the lifecycle to complete the creation of these resources, and then reduce inconsistent invokes after the create/update has completed.

Most customers experience no impact from this change except for fewer long cold-starts due to network resource creation. As a reminder, any invocations or other API actions that operate on the function will fail during the time before the function is Active. To better assist you in adopting this behavior, we are rolling out this behavior for VPC configured functions in a phased manner. This post provides further details about timelines and approaches to both test the change before it is 100% live or delay it for your functions using a delay mechanism.

Changes to function create and update

On function create

During creation of new functions configured for VPC, your function remains in the Pending state until all VPC resources are created. You are not able to invoke the function or take any other Lambda API actions against it. After successful completion of the creation of these resources, your function transitions automatically to the Active state and is available for invokes and Lambda API actions. If the network resources fail to create then your function is placed in a Failed state.

On function update

During the update of functions configured for VPC, if there are any modifications to the VPC configuration, the function remains in the Active state, but shows in the InProgress status until all VPC resources are updated. During this time, any invokes go to the previous function code and configuration. After successful completion, the function LastUpdateStatus transitions automatically to Successful and all new invokes use the newly updated code and configuration. If the network resources fail to be created/updated then the LastUpdateStatus shows Failed, but the previous code and configuration remains in the Active state.

It’s important to note that creation or update of VPC resources can take between 60-90 seconds complete.

Change timeframe

As a reminder, all functions today show an Active state only. We are rolling out this change to create resources during the Pending state over a multiple phase period starting with the Begin Testing phase today, December 16, 2019. The phases allow you to update tooling for deploying and managing Lambda functions to account for this change. By the end of the update timeline, all accounts transition to using this new VPC resource create/update Lambda lifecycle.

Update timeline

Update timeline

December 16, 2019 – Begin Testing: You can now begin testing and updating any deployment or management tools you have to account for the upcoming lifecycle change. You can also use this time to update your function configuration to delay the change until the Delayed Update phase.

January 20, 2020 – General Update: All customers without the delayed update configuration begin seeing functions transition as described above under “On function create” and “On function update”.

February 17, 2020 – Delayed Update: The delay mechanism expires and customers now see the new VPC resource lifecycle applied during function create or update.

March 2, 2020 – Update End: All functions now have the new VPC resource lifecycle applied during function create or update.

Opt-in and delayed update configurations

Starting today, we are providing a mechanism for an opt-in, to allow you to update and test your tools and developer workflow processes for this change. We are also providing a mechanism to delay this change until the end of the Delayed Update phase. If you configure your functions for VPC and use the delayed update mechanism after the start of the General Update, your functions continue to experience a delayed first invocation due to VPC resource creation.

This mechanism operates on a function-by-function basis, so you can test and experiment individually without impacting your whole account. Once the General Update phase begins, all functions in an account that do not have the delayed update mechanism in place see the new lifecycle for their functions.

Both mechanisms work by adding a special string in the “Description” parameter of your Lambda functions. This string can be added to the prefix or suffix, or be the entire contents of the field.

To opt in:

aws:states:opt-in

To delay the update:

aws:states:opt-out

NOTE: Delay configuration mechanism has no impact after the Delayed Update phase ends.

Here is how this looks in the console:

  1. I add the opt-in configuration to my function’s Description.

    Opt-in in Description

    Opt-in in Description

  2. When I choose Save at the top, I see the update begin. During this time, I am blocked from executing tests, updating my code, and making some configuration changes against the function.

    Function updating

    Function updating

  3. After the update completes, I can once again run tests and other console commands.

    Function update successful

    Function update successful

Once the opt-in is set for a function, then updates on that function go through the update flow shown above. If I don’t change my function’s VPC configuration, then updates to my function transition almost instantly to the Successful update status.

With this in place, you can now test your development workflow ahead of the General Update phase. Download the latest CLI (version 1.16.291 or greater) or SDKs in order to see function state and related attribute information.

Conclusion

With functions states, you can have better clarity on how the resources required by your Lambda function are being created. This change does not impact the way that functions are invoked or how your code is executed. While this is a minor change to when resources are created for your Lambda function, the result is even better consistency of performance. Combined with the original announcement of improved VPC networking for Lambda, you experience better consistency for invokes, greatly reduced cold-starts, and fewer network resources created for your functions.

Announcing HTTP APIs for Amazon API Gateway

Post Syndicated from George Mao original https://aws.amazon.com/blogs/compute/announcing-http-apis-for-amazon-api-gateway/

Amazon API Gateway helps developers easily create, publish, and maintain secure APIs at any scale.  API Gateway handles all of the heavy lifting of managing thousands of API calls.  There are no minimum fees and you only pay for the API calls you receive.

Announcing the API Gateway HTTP API

We talk to customers every day that use API Gateway for critical production applications. This includes everything ranging from simple HTTP proxies to full-blown API management with request transformation, authentication, and validation. API Gateway supports REST APIs and WebSocket APIs, but customers have told us they want more features, lower latency, and lower cost.

Customers have explained their need for the core features of API Gateway at a lower price along with an easier developer experience. Based on this feedback, we are excited to announce the availability of HTTP APIs (Preview).

HTTP APIs is a new flavor of API Gateway. It focuses on delivering enhanced features, improved performance, and an easier developer experience for customers building with API Gateway. Today, we’re launching the first phase, and we will continue to enhance HTTP APIs over the next few months.

We are introducing a new pricing model for HTTP APIs that better represents customer usage patterns. Staying true to our serverless principles, you will only pay for the requests you receive.  With existing REST APIs, you pay $3.50/million requests plus data transferred out.

With HTTP APIs, we have reduced request pricing to $1.00/million requests for the first 300 million requests, and $0.90/million for all requests after that. Most customers will see an average cost saving up to 70%, when compared to API Gateway REST APIs. In addition, you will see significant performance improvements in the API Gateway service overhead.

Getting started

The API Gateway HTTP API is in preview, so there are a couple of things to keep in mind:

  • The preview is available in the following Regions: US East (Ohio), US East (N. Virginia), US West (N. California), US West (Oregon), Asia Pacific (Sydney), Asia Pacific (Tokyo), EU (Frankfurt), EU (Ireland).
  • The preview is optimized for building APIs that proxy requests to Lambda functions or HTTP endpoints. Over the next few months, we will continue to add functionality to HTTP APIs (see the complete feature overview).

Review the preview guide for a detailed description of the service.

Creating an HTTP API

There are currently two API Gateway namespaces for managing API Gateway deployments. The API V1 namespace represents REST APIs and API V2 represents WebSocket APIs and the new HTTP APIs. You can create an HTTP API by using the AWS Management Console, CLI, APIs, CloudFormation, SDKs, or the Serverless Application Model (SAM).

Let’s review creating an HTTP API via the AWS Management Console. If you are familiar with API Gateway, the first thing you’ll notice is a new console experience.

  1. Start by navigating to the API Gateway console, locate the new HTTP API “Beta” dialog box, and choose Build:
    Create a new HTTP API

    Create a new HTTP API

    For this example, we use Lambda as our backend integration. If you use Lambda, be sure to review the required response attributes. You can use this as an example Lambda function:

    exports.handler = async (event) => {
        // TODO implement
        const response = {
            statusCode: 200,
            body: JSON.stringify('Hello from Lambda!'),
        };
        return response;
    };
  2. Select Add Integration and choose Lambda. Select your Lambda function, provide a name for the API, and choose Next.

    Add an Integration

    Add an Integration

  3. Next, you must configure the routes. A route directs API requests to backend resources (Integrations), and consists of a resource path and an HTTP method that invokes your integration. We use the default ‘ANY’ Method but you can select a more specific HTTP Method if you need. Choose Next.

    Configure a route

    Configure a route

  4. Now you are ready to configure stages. A stage represents a unique version of an API (for example, dev, test or prod). HTTP APIs support a $default stage that is served from the API’s base URL. Accept the $default stage, and leave Auto-deploy enabled. Choose Next.

    Define a stage and enable auto deploy

    Define a stage and enable auto deploy

  5. Review your settings and choose Create.
    Review and create your API

    Review and create your API

    Since Auto deploy was left enabled the API is ready to serve content immediately using the $default stage. Any changes you make to the API are also reflected immediately. After you choose Create, the API is auto-deployed at the $default stage and the assigned Invoke URL.  You are now ready to test.

    API details

    API details

Enter the Invoke URL into your browser and append the route you created earlier (/myHTTPProxyFunction). For example, https://abcdedfg.execute-api.us-west-2.amazonaws.com/myHTTPProxyFunction. This routes an HTTP GET request to the Lambda integration and returns “Hello from Lambda!”

Explore the new console experience

The navigation panel on the left lets you manage your APIs. You can add, remove, or modify routes, and their integrations. Or enable authorization and provide CORS response details for your routes. You can also review your deployed stages. Finally, use the Metrics and Logging sections to see options for monitoring your API.

Explore the new API Gateway console

Explore the new API Gateway console

What’s new?

We encourage you to explore the following new and improved features:

  • JWT Authorizers. This is a new authorization type that supports native OpenID Connect (OIDC) authorization. You can configure API Gateway to parse incoming JWT tokens, and allow or deny requests based on the OAuth scopes in the token. With REST APIs, you must use a Lambda custom authorizer to parse the incoming JWT token. Choose Authorization in the left navigation panel to get started.

    Create a new JWT authorizer

    Create a new JWT authorizer

  • Ability to assign a default Stage or configure a default route. Default Stages and Routes make it easier to work with APIs. When you assign a default stage, you are able to serve your API from the base URL. For example: https://{api_id}.execute-api.{region}.amazonaws.com/. This means you no longer need an explicit stage in your API. Choose the Stages option in the left nav bar to review or manage your Stages.
  • Automatic deployments. You can now optionally enable Auto deploy when you change a stage. By default, you must deploy the API to a stage for changes to go live. This option lets you release changes to your API immediately as the changes are made
  • Fully customizable CORS experience.  Cross-Origin Resource Sharing allows a browser to execute Javascript across different domains or origins by sending a preflight options request to API Gateway. You do not need to configure an options route, as API Gateway manages this. Choose the CORS link in the left nav bar to edit the settings.

    Setup Cross Origin Resource Sharing

    Setup Cross Origin Resource Sharing

Developer experience

In addition to the console, you can use the Serverless Application Model (SAM) or CloudFormation to create your HTTP APIs.  The HTTP API is supported under the AWS::ApigatewayV2 resource type. Here’s an example of how to create a basic HTTP API with CloudFormation. See here for more CloudFormation examples.

Resources:
  HttpApi:
    Type: AWS::ApiGatewayV2::Api
    Properties:
      Name: Lambda Proxy
      Description: Lambda Proxy using Quick Create
      ProtocolType: HTTP
      Target: arn:aws:lambda:us-west-1:[your acct number here]:function:Echo

The CLI also supports HTTP APIs via the apigatewayV2 commands. Here is an example CLI command that creates an HTTP API:

aws apigatewayv2 create-api —name MyAPIname —protocol-type HTTP --target arn:aws:lambda:us-west-1:[your acct number here]:function:Echo

This uses the new Quick Create feature. This feature automatically creates an API with a default route and default stage. The service automatically deploy your changes allowing you to create an API with a simple CloudFormation script or CLI command. Quick create enables the API to be invoked at its base path.

Don’t forget that you will need to add a Lambda resource policy permission to the function to allow API Gateway to invoke your function. For example:

aws lambda add-permission
--statement-id dd0dffb7-971e-5952-9699-38493cf34293
--action lambda:InvokeFunction
--function-name "arn:aws:lambda:us-west-1:[your account number]:function:Echo"
--principal apigateway.amazonaws.com
--source-arn "arn:aws:execute-api:us-west-1:[your accunt number]:[your gateway id]/"

Who should use HTTP APIs

Here is a feature summary of what is available today. Over the next few months, we will enhance HTTP APIs with new features.

You can migrate today if your APIs interact with HTTP or Lambda integrations and your authorization requirements are OIDC/JWT based. You can export your existing REST APIs to an OpenAPI JSON file, and import the API definition as an HTTP API. To learn more about exporting a REST API, see here. Once you have exported your API definition, you can import it with the CLI as follows:

aws apigatewayv2 import-api —body file://api-definition.json

Conclusion

Our goal is to make it as easy as possible for developers to build and manage APIs with API Gateway. We encourage you to try the new HTTP APIs and let us know what you think. Take a look at our developer guide and happy coding!

New Express Workflows for AWS Step Functions

Post Syndicated from Benjamin Smith original https://aws.amazon.com/blogs/compute/new-express-workflows-for-aws-step-functions/

Today, AWS is introducing Express Workflows for AWS Step Functions. This is a new workflow type to orchestrate AWS services at a higher-throughput than existing workflows.

Developers have been using AWS Step Functions since December 2016 to assemble long running workflows to orchestrate AWS Lambda Functions and other serverless services. Customers were looking for better ways to optimize for workloads that require higher event rates and shorter durations.

The new AWS Step Functions Express Workflows type uses fast, in-memory processing for high-event-rate workloads of up to 100,000 state transitions per second, for a total workflow duration of up to 5 minutes. Express Workflows are suited to streaming data processing, IoT data ingestion, mobile backends, and other high-throughput use-cases. Existing workflows in AWS Step Functions are now called Standard Workflows.

Getting started

You can build and run Express Workflows using the AWS Management Console, AWS CLI, or AWS CloudFormation. The steps below explain how to build an Express Workflow from within the AWS Management Console:

1.    In the AWS Step Functions console, choose Create state machine.

Figure 1 Creating a state machine from the AWS Step Functions Console

Figure 1 Creating a state machine from the AWS Step Functions console

2.    In the Type section, choose Express.

Figure 2 New Express Workflow option

Figure 2 New Express Workflow option

3.    Enter a name for the Express Workflow in the State machine name field. In the Definition section, you can generate code snippets from a selection of example use cases.

4.    Leave the default definition and choose Next.

Figure 3 Define your workflow with ASL

Figure 3 Define your workflow with ASL

You can create a new role for the Express Workflow execution or use an existing role.

5.    Leave Create New Role selected, and give it a logical name in the Role name field.

A new section named Logging configuration appears in the AWS Management Console. Here you can choose what level of logging for CloudWatch Logs. For Express Workflows, you must enable logging to inspect and debug executions.

6.    Choose ALL for Log level and leave the defaults for Include execution data and CloudWatch log group fields.

Figure 4 Logging configuration

Figure 4 Logging configuration

7.    Choose Create state machine.

Once the Express Workflow is created, in the Details section the type is Express.

Figure 5 New 'type' in details section

Figure 5 New ‘type’ in details section

Executing an Express Workflow from within the AWS Management Console is the same process as with a Standard Workflow.

Combining workflows

Applications may require a combination of both long-running and high-event-rate workflows. For example, the initial step in a workflow may involve ingesting and processing IoT stream data (Express Workflow), followed by executing a long-running machine learning model to derive insights (Standard Workflow).

Applications can benefit from the wait state of a Step Functions Standard Workflow and the shorter duration cost efficiencies of an Express Workflow when used together. An example support ticket automation application shows the SaaS integration capabilities of Amazon EventBridge, using Amazon Comprehend for sentiment analysis on support tickets.

Figure 6 Support ticket automation application architecture

Figure 6 Support ticket automation application architecture

You can improve part of the workflow by moving the short duration Lambda ‘Set tags’, ’Set category’, and ‘Escalate priority’ functions into a nested Express Workflow to orchestrate the re-occurring ‘Process Ticket’ stage. The Step Functions Standard Workflow uses the ‘Wait’ task to pause for a set amount of time until the Express Workflow completes. The Express Workflow is a distinct child workflow with its own Success or Fail task state from within the parent workflow. The child workflow must complete within the parent’s duration limit and uses the parent retry policy.

Figure 7 Improved support ticket automation application

Figure 7 Improved support ticket automation application

Monitoring and logging with Express Workflows

Standard Workflows show execution history and visual debugging in the Step Functions console. Express Workflows send execution history to CloudWatch Logs. Two new tabs, Monitoring and Logging, have been added to the AWS Step Functions console to gain visibility into Express Workflow executions.

The Monitoring tab shows six different graphs with CloudWatch metrics for Execution errors, Execution Succeeded, Execution Duration, Billed Duration, Billed Memory, and Executions Started.

The Logging tab shows the logging configuration with a quick link to CloudWatch Logs.

Figure 8 New logging and monitoring tabs

Figure 8 New logging and monitoring tabs

State execution details are visible in CloudWatch Logs. CloudWatch Logs Insights provides search and log analysis with a purpose-built query language.

API changes

The Step Functions API has been updated to integrate Express Workflows for both creating and updating a state machine.

New parameter: Type, String
State machines have a new field called Type. Which can be Standard or Express. The default type is Standard to maintain backwards compatibility.

New parameter: Log-level, Integer
Defines the log detail level for CloudWatch Logs.

New exception: InvalidStateMachineType
This is thrown when type is not Express or Standard, or when there’s an activity (.sync or .waitForTaskToken connector).

Creating state machine with these new parameters:

–        API Call without using Type returns a Express Workflow state machine.

$ aws stepfunctions create-state-machine \
 --definition "{}" \
 --name "FlightTicketHandler" \
 --execution-role-arn "arn:aws:iam:::role:roleName/user" \
 --log-level 1 \
 --tags "Key=CreatedBy,Value=Diego,Key=stack,Value=Production"

–        API Call using Type Express

$ aws stepfunctions create-state-machine \
 --definition "{}" \
 --name "FlightTicketHandler" \
 --execution-role-arn "arn:aws:iam:::role:roleName/user" \
 --log-level 1 \
 --tags "Key=CreatedBy,Value=Diego,Key=stack,Value=Production" \
 --type "EXPRESS"

–        API Call using Type Standard

$ aws stepfunctions create-state-machine \
 --definition "{}" \
 --name "FlightTicketHandler" \
 --execution-role-arn "arn:aws:iam:::role:roleName/user" \
 --log-level 1 \
 --tags "Key=CreatedBy,Value=Diego,Key=stack,Value=Production" \
 --type "STANDARD"

–        Updating the log level of a state machine

$ aws stepfunctions update-state-machine \
 --state-machine-arn "arn:aws:states:us-east-1:123456789012:stateMachine:${StateMachineName}" \
 --log-level 0 

For more information regarding the API changes, refer to the API documentation.

Comparing Standard and Express Workflows

Both workflow types use the declarative code semantics of Amazon States Language to build the workflow definition. Here are some of the key differences:

Express WorkflowsStandard Workflows
Supported execution start rateOver 100,000 per secondOver 2,000 per second
Max run time5 minutes1 year
Execution guaranteeAt least onceExactly once
Execution LoggingAvailable in CloudWatch LogsIn Step Functions service
Pricing

$1.00 per million invocations

Tiered pricing based on memory and duration

$25.00 per million state transitions

Service Integrations and Patterns

 

Supports all service integrations. Does not support Job-run (.sync) or Callback (.wait For Callback) integration patterns

Supports all service integrations and patterns

 

Conclusion

Many customers are already using AWS Step Functions Standard Workflows to orchestrate long-running, auditable workloads. The addition of a new Express Workflow type adds a lower-priced, higher-throughput workflow capability to AWS Step Functions.

Express Workflows complement AWS Step Functions Standard Express Workflows. Developers now have the power to choose the workflow type that best suits their needs, or choose to mix and blend as appropriate.

You can get started with Express Workflows via the AWS Management Console, AWS CLI, or AWS CloudFormation. It is available in all AWS Regions where AWS Step Functions is available.

For more information on where AWS Step Functions is available, see the AWS Region Table. For pricing for Express Workflows, see pricing.

New for AWS Lambda – Predictable start-up times with Provisioned Concurrency

Post Syndicated from James Beswick original https://aws.amazon.com/blogs/compute/new-for-aws-lambda-predictable-start-up-times-with-provisioned-concurrency/

Since the launch of AWS Lambda five years ago, thousands of customers such as iRobot, Fender, and Expedia have experienced the benefits of the serverless operational model. Being able to spend less time on managing scaling and availability, builders are increasingly using serverless for more sophisticated workloads with more exacting latency requirements.

As customers have progressed to building mission-critical serverless applications at scale, we have continued to invest in performance at every level of the service. From the updates in Lambda’s execution environment to the major improvements in VPC networking rolled out this year, performance remains a top priority. In addition to changes that broadly impact Lambda customers, we have also been working on a new feature specifically for the most latency-sensitive workloads, to provide even more granular control over application performance.

Currently, when the Lambda service runs “on demand,” it decides when to launch new execution environments for your function in response to incoming requests. As a result, the latency profile of your function will vary as it scales up, which may not meet the requirements of some application workloads. Today, we are providing builders with a significant new feature called Provisioned Concurrency, which allows more precise control over start-up latency when Lambda functions are invoked.

When enabled, Provisioned Concurrency is designed to keep your functions initialized and hyper-ready to respond in double-digit milliseconds at the scale you need. This new feature provides a reliable way to keep functions ready to respond to requests, giving you more precise control over the performance characteristics of your serverless applications. Builders can now choose the concurrency level for each Lambda function version or alias, including when and for how long these levels are in effect.

This powerful feature is controlled via the AWS Management Console, AWS CLI, AWS Lambda API, or AWS CloudFormation, and it’s simple to implement. This blog post introduces how to use Provisioned Concurrency, and how you can gain the benefits for your most latency-sensitive serverless workloads.

Using Provisioned Concurrency

This feature is ideal for where you need predictable function start times. For example, for interactive workloads such as mobile and web backends, synchronously invoked APIs, and latency-sensitive processes. Also, for applications that experience heavy loads based upon a predictable schedule, you can increase the amount of concurrency during times of high demand and decrease it when the demand decreases.

The simplest way to benefit from Provisioned Concurrency is by using AWS Auto Scaling. You can use Application Auto Scaling to configure schedules for concurrency, or have Auto Scaling automatically add or remove concurrency in real time as demand changes.

Turning Provisioned Concurrency on or off does not require any changes to your function’s code, Lambda Layers, or runtime configurations. There is no change to the Lambda invocation and execution model. It’s simply a matter of configuring your function parameters via the AWS Management Console or CLI, and the Lambda service manages the rest.

Provisioned Concurrency adds a pricing dimension to the existing dimensions of Duration and Requests. You only pay for the amount of concurrency that you configure and for the period of time that you configure it. When Provisioned Concurrency is enabled for your function and you execute it, you also pay for Requests and execution Duration.

How it works

Typically, the overhead in starting a Lambda invocation – commonly called a cold-start – consists of two components. The first is the time taken to set up the execution environment for your function’s code, which is entirely controlled by AWS.

The second is the code initialization duration, which is managed by the developer. This is impacted by code outside of the main Lambda handler function, and is often responsible for tasks like setting up database connections, initializing objects, downloading reference data, or loading heavy frameworks like Spring Boot. In our analysis of production usage, this causes by far the largest share of overall cold start latency. It also cannot be automatically optimized by AWS in a typical on-demand Lambda execution.

Provisioned Concurrency targets both causes of cold-start latency. First, the execution environment set-up happens during the provisioning process, rather during execution, which eliminates this issue. Second, by keeping functions initialized, which includes initialization actions and code dependencies, there is no unnecessary initialization in subsequent invocations. Once initialized, functions are hyper-ready to respond in double digit milliseconds of being invoked. This is the key to understanding how this feature helps you obtain predictable start-up latency for both causes of cold-starts.

While all Provisioned Concurrency functions start more quickly than the existing on-demand Lambda execution style, this is particularly beneficial for certain function profiles. Runtimes like C# and Java have much slower initialization times than Node.js or Python, but significantly faster execution times once initialized. With Provisioned Concurrency turned on, users of these runtimes benefit from both the consistent low latency of the function’s start-up, and the runtime’s performance during execution.

When you configure concurrency on your functions, the Lambda service initializes execution environments for running your code. If you exceed this level, you can choose to deny further invocations, or allow any additional functional invocations to use the on-demand model. You can do this by setting the per-function concurrency limit. In the latter case, while these invocations exhibit a more typical Lambda start-up performance profile, you are not throttled or limited from running invocations at high levels of throughput.

Using CloudWatch Logs or the Monitoring tab for your function in the Lambda console, you can see metrics for the number of Provisioned Concurrency invocations, compared with the total. This can help identify when total load is above the amount of concurrency, and you can make changes accordingly. Alternatively you can use the Auto Scaling CLI commands to have this managed by the service, so that the amount of concurrency tracks more closely with actual usage.

Turning on Provisioned Concurrency

Configuring Provisioned Concurrency is straight-forward. In this post, I demonstrate how to do this in the AWS Management Console but you can also use the AWS CLI, AWS SDK, and AWS CloudFormation to modify these settings.

1. Go to the AWS Lambda console and then choose your existing Lambda function.

2. Settings must be applied to a published version or an alias. Go to the Actions drop-down and choose Publish new version.

Publish new version

3. Enter an optional description for your version and choose Publish.

3. Enter an optional description for your version

4. Go to the Actions drop-down and choose Create alias.

4. Go to the Actions drop-down and choose Create alias

5. Enter a name for the alias (for example, “Test”), select 1 from the Version drop-down, and choose Create.

Enter a name for the alias

6. Locate the Concurrency card and choose Add.

Locate the Concurrency card and choose Add

7. Select the Alias radio button for Qualifier Type, choose Test in the Alias drop-down, and enter 100 for the Provisioned concurrency. Choose Save.

Enter a name for the alias

8. The Provisioned Concurrency card in the Lambda console will then show the status In progress.

status In progress

After a few minutes, the initialization process is complete, and your function’s published alias can now be used with the Provisioned Concurrency feature.

Provisioned Concurrency ready

Since the feature is applied explicitly to a function alias, ensure that your invocation method is calling this alias, and not the $LATEST version. Provisioned Concurrency cannot be applied to the $LATEST version.

Comparing results for on-demand and Provisioned Concurrency

In this example, I use a simple Node.js Lambda function that simulates 5 seconds of activity before exiting. The handler contains the following code:

exports.handler = async (event) => {
    await doWork()

    return {
      "statusCode": 200
    }
}

function doWork() {
  return new Promise(resolve => {
    setTimeout(() => {
      resolve()
    }, 5000)
  })
}

I include multiple NPM packages to increase the function’s zip file to around 10 Mb, to approximate a reasonable package size for a typical production function. I also enable AWS X-Ray tracing on the Lambda function to collect and compare detailed performance statistics.

I add an API Gateway trigger to call the function for testing. This endpoint invokes the $LATEST version and has no Provisioned Concurrency settings applied, so it performs as a typical on-demand Lambda function. For testing, I use the Artillery NPM package to simulate user load. After the test is completed, the X-Ray report for this on-demand Lambda function shows an expected response distribution:

Response distribution

While most of the requests are completed near the expected 5-second execution time, there is a long tail where p95 and p99 performance times are slower. This is caused by the function is scaling up, and new concurrent invocations are slower to start due to execution environment initialization. The detailed distribution chart in X-Ray shows this more clearly, where these cold-starts are notable on the far right of the graph:

Cold-starts are notable on the far right of the graph

Finding the slowest-performing execution in this test, AWS X-Ray shows that the cold-start latency added approximately 650 ms to the overall performance:

Cold-start latency

This is slowest execution in the test, but typically the initialization process could take significantly longer in many production scenarios. This test shows how on-demand Lambda performance exhibits higher latencies at the p95 and p99 intervals when functions scale up.

For the comparison, I configure Provisioned Concurrency for this function using the same steps in the previous section. I also update the API Gateway integration to target the published version of the function, instead of the $LATEST version. Launching the same load test using Artillery, now the performance is dramatically different:

Response distribution with Provisioned Concurrency

There is no performance long-tail, and zooming into the detailed chart in X-Ray shows that there were no extended latencies caused by execution environment initialization:

Response time compared

Finding the fastest and slowest executions in this test, the fastest completed in 5007 ms, while the slowest finished in 5066 ms, representing a spread of 7-66 ms overhead in total execution time.

Fastest and slowest executions in this test

If you compare the results of each load test using the standard on-demand invocation model and Provisioned Concurrency, the combined response time distribution shows the impact:

Response times

The brown line is the on-demand test, showing the long-tail latency caused by both execution environment initialization and code initialization inherent in the scaling up of Lambda functions. The blue line is the Provisioned Concurrency test where the long-tail is eliminated completely, showing much more consistent function latency using this new feature.

Conclusion

AWS Lambda continues to make significant performance improvements for all Lambda users, and remains committed to improving execution times for the existing on-demand scaling model. This new feature provides an option for builders with the most demanding, latency-sensitive workloads to execute their functions with predictable start-up times at any scale.

In this post, I reviewed how to set this feature on an alias of a Lambda function. I compared the total execution times between on-demand and Provisioned Concurrency for the same function over load. The results show that the start-up latency variability is eliminated when Provisioned Concurrency is enabled.

You can also use provisioned concurrency today with AWS Partner tools, including configuring provisioned concurrency settings with the Serverless Framework, or viewing metrics with Datadog, EpsagonLumigoSignalFx, SumoLogic, and Thundra.

Provisioned Concurrency is available today in the following Regions: US East (Ohio), US East (N. Virginia), US West (N. California), US West (Oregon), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Canada (Central), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), and Europe (Stockholm), Middle East (Bahrain), and South America (Sao Paulo).

To learn more about Provisioned Concurrency, visit our documentation, or read the launch blog post.

Using Amazon RDS Proxy with AWS Lambda

Post Syndicated from George Mao original https://aws.amazon.com/blogs/compute/using-amazon-rds-proxy-with-aws-lambda/

The AWS Serverless platform allows you to build applications that automatically scale in response to demand. During periods of high volume, Amazon API Gateway and AWS Lambda scale automatically in response to incoming load.

Often developers must access data stored in relational databases from Lambda functions. But it can be challenging to ensure that your Lambda invocations do not overload your database with too many connections. The number of maximum concurrent connections for a relational database depends on how it is sized.

This is because each connection consumes memory and CPU resources on the database server. Lambda functions can scale to tens of thousands of concurrent connections, meaning your database needs more resources to maintain connections instead of executing queries.

See the architecture blog post “How to Design your serverless apps for massive scale” for more detail on scaling.

Serverless Architecture with RDS

Serverless Architecture with RDS

This design places high load on your backend relational database because Lambda can easily scale to tens of thousands of concurrent requests. In most cases, relational databases are not designed to accept the same number of concurrent connections.

Database proxy for Amazon RDS

Today, we’re excited to announce the preview for Amazon RDS Proxy. RDS Proxy acts as an intermediary between your application and an RDS database. RDS Proxy establishes and manages the necessary connection pools to your database so that your application creates fewer database connections.

You can use RDS Proxy for any application that makes SQL calls to your database. But in the context of serverless, we focus on how this improves the Lambda experience. The proxy handles all database traffic that normally flows from your Lambda functions directly to the database.

Your Lambda functions interact with RDS Proxy instead of your database instance. It handles the connection pooling necessary for scaling many simultaneous connections created by concurrent Lambda functions. This allows your Lambda applications to reuse existing connections, rather than creating new connections for every function invocation.

The RDS Proxy scales automatically so that your database instance needs less memory and CPU resources for connection management. It also uses warm connection pools to increase performance. With RDS Proxy, you no longer need code that handles cleaning up idle connections and managing connection pools. Your function code is cleaner, simpler, and easier to maintain.

Getting started

The RDS Database proxy is in preview, so there are a few things to keep in mind:

  • We currently support Amazon RDS MySQL or Aurora MySQL, running on MySQL versions 5.6 or 5.7
  • The preview is available in Asia Pacific (Tokyo), EU (Ireland), US East (Ohio), US East (N.Virginia), and US West (Oregon)
  • During the public preview, you should use the AWS Management Console to interact with RDS Proxy
  • Do not use this service for production workloads as you might encounter preview-related changes

Review the preview guide for a detailed description of the service

Prerequisites

Start with an existing database that is either Amazon RDS MySQL or Aurora MySQL. Then, store your database credentials as a secret in AWS Secrets Manager, and create an IAM Policy that allows RDS Proxy to read this secret.

To create the secret:

  1. Sign into AWS Secrets Manager and choose Store a new Secret.
  2. Choose Credentials for RDS Database.
  3. Enter the user name and password.
  4. Select the RDS Database this secret is valid for. Choose Next.

    Store a new secret

    Store a new secret

  5. Enter a Secret Name and choose Next.

    Save the secret

    Save the secret

  6. Accept all defaults and choose Store. Note the ARN assigned to this secret, as you need it later.

    Secret details

    Secret details

  7. Now create an IAM role that allows RDS Proxy to read this secret. RDS Proxy uses this secret to maintain a connection pool to your database. Go to your IAM console and create a new role. Add a policy that provides secretsmanager permissions to the secret you created in the previous step. For example:
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": [
            "secretsmanager:GetResourcePolicy",
            "secretsmanager:GetSecretValue",
            "secretsmanager:DescribeSecret",
            "secretsmanager:ListSecretVersionIds"
          ],
          "Resource": [
            "arn:aws:secretsmanager:us-east-2:[your-account-number]:secret:gmao-rds-secret-YZ2MMN"
          ]
        },
        {
          "Sid": "VisualEditor1",
          "Effect": "Allow",
          "Action": [
            "secretsmanager:GetRandomPassword",
            "secretsmanager:ListSecrets"
          ],
          "Resource": "*"
        }
      ]
    }
  8. Add the following Trust Policy to allow RDS to assume the role. Save the role and take note of the IAM Role ARN, as you need it later.
    {
     "Version": "2012-10-17",
     "Statement": [
      {
       "Sid": "",
       "Effect": "Allow",
       "Principal": {
        "Service": "rds.amazonaws.com"
       },
       "Action": "sts:AssumeRole"
      }
     ]
    }

Create and attach a proxy to a Lambda function

Next, use the Lambda console to Add a Database proxy to a Lambda function.

  1. Sign into the AWS Lambda console and open the Lambda function you would like to enable RDS Proxy.
  2. Scroll to the bottom of your Lambda configuration page and choose Add Database Proxy.

    Add database proxy

    Add database proxy

  3. Follow the Add database proxy wizard, and fill in the Proxy Identifier and select your RDS Database. Then choose the Secrets Manager secret and the IAM role you created earlier. RDS Proxy uses this secret to connect to your database. Choose Add.

    Configure database proxy

    Configure database proxy

  4. Wait a few minutes for the RDS Proxy to provision and the status updates to Available.

    Database proxy available

    Database proxy available

  5. Choose your proxy to view the details. Note the Proxy endpoint. You need this later in the Lambda function code.

    Available Proxy configuration

    Available Proxy configuration

Now the Lambda function has permission to use the configured RDS Proxy, and you are ready to connect to the proxy.

Using the proxy

Instead of connecting directly to the RDS instance, connect to the RDS proxy. To do this, you have two security options. You can use IAM authentication or you can use your native database credentials stored in Secrets Manager. IAM authentication is recommended  because it removes the need to embed or read credentials in your function code. For simplicity, this guide uses the database credentials created earlier in Secrets Manager.

You can use any Lambda-supported programming language. The example below uses Node.js:

let mysql = require('mysql');

let connection;
  
connection = mysql.createConnection({
  host   : process.env['endpoint'],
  user   : process.env['user'],
  password : process.env['password'],
  database : process.env['db']
});

exports.handler = async (event) => {

  console.log("Starting query ...");
  
  connection.connect(function(err) {
    if (err) {
     console.error('error connecting: ' + err.stack);
     return;
    }
    
    console.log('connected as id ' + connection.threadId);
  });

  // Do some work here
  
  connection.end(function(error, results) {
     // The connection is terminated now 
     console.log("Connection ended");
     return "success";
  });
};

You need to package the NodeJS MySQL client module with your function. I use Lambda environment variables to store the connection information. This is the best practice for database configuration settings so you can change these details without updating your code. The endpoint environment variable is the RDS Proxy Endpoint noted earlier. The user and password are the database credentials, and the db variable is the database schema name.

If you choose to authenticate with IAM, make sure that your Lambda execution role includes rds-db:connect permissions as outlined here. The Lambda console automatically does this on your behalf. This option allows you to retrieve a temporary token from IAM to authenticate to the database, instead of using native database credentials.

Conclusion

RDS Proxy helps you manage a large number of connections from Lambda to an RDS database by establishing a warm connection pool to the database. Your Lambda functions can scale to meet your needs and use the RDS Proxy to serve multiple concurrent application requests. This reduces the CPU and Memory requirements for your database, and eliminates the need for connection management logic in your code.

We look forward to your feedback during this preview!

New – Provisioned Concurrency for Lambda Functions

Post Syndicated from Danilo Poccia original https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/

It’s really true that time flies, especially when you don’t have to think about servers: AWS Lambda just turned 5 years old and the team is always looking for new ways to help customers build and run applications in an easier way.

As more mission critical applications move to serverless, customers need more control over the performance of their applications. Today we are launching Provisioned Concurrency, a feature that keeps functions initialized and hyper-ready to respond in double-digit milliseconds. This is ideal for implementing interactive services, such as web and mobile backends, latency-sensitive microservices, or synchronous APIs.

When you invoke a Lambda function, the invocation is routed to an execution environment to process the request. When a function has not been used for some time, when you need to process more concurrent invocations, or when you update a function, new execution environments are created. The creation of an execution environment takes care of installing the function code and starting the runtime. Depending on the size of your deployment package, and the initialization time of the runtime and of your code, this can introduce latency for the invocations that are routed to a new execution environment. This latency is usually referred to as a “cold start”. For most applications this additional latency is not a problem. For some applications, however, this latency may not be acceptable.

When you enable Provisioned Concurrency for a function, the Lambda service will initialize the requested number of execution environments so they can be ready to respond to invocations.

Configuring Provisioned Concurrency
I create two Lambda functions that use the same Java code and can be triggered by Amazon API Gateway. To simulate a production workload, these functions are repeating some mathematical computation 10 million times in the initialization phase and 200,000 times for each invocation. The computation is using java.Math.Random and conditions (if ...) to avoid compiler optimizations (such as “unlooping” the iterations). Each function has 1GB of memory and the size of the code is 1.7MB.

I want to enable Provisioned Concurrency only for one of the two functions, so that I can compare how they react to a similar workload. In the Lambda console, I select one the functions. In the configuration tab, I see the new Provisioned Concurrency settings.

I select Add configuration. Provisioned Concurrency can be enabled for a specific Lambda function version or alias (you can’t use $LATEST). You can have different settings for each version of a function. Using an alias, it is easier to enable these settings to the correct version of your function. In my case I select the alias live that I keep updated to the latest version using the AWS SAM AutoPublishAlias function preference. For the Provisioned Concurrency, I enter 500 and Save.

Now, the Provisioned Concurrency configuration is in progress. The execution environments are being prepared to serve concurrent incoming requests based on my input. During this time the function remains available and continues to serve traffic.

After a few minutes, the concurrency is ready. With these settings, up to 500 concurrent requests will find an execution environment ready to process them. If I go above that, the usual scaling of Lambda functions still applies.

To generate some load, I use an Amazon Elastic Compute Cloud (EC2) instance in the same region. To keep it simple, I use the ab tool bundled with the Apache HTTP Server to call the two API endpoints 10,000 times with a concurrency of 500. Since these are new functions, I expect that:

  • For the function with Provisioned Concurrency enabled and set to 500, my requests are managed by pre-initialized execution environments.
  • For the other function, that has Provisioned Concurrency disabled, about 500 execution environments need to be provisioned, adding some latency to the same amount of invocations, about 5% of the total.

One cool feature of the ab tool is that is reporting the percentage of the requests served within a certain time. That is a very good way to look at API latency, as described in this post on Serverless Latency by Tim Bray.

Here are the results for the function with Provisioned Concurrency disabled:

Percentage of the requests served within a certain time (ms)
50% 351
66% 359
75% 383
80% 396
90% 435
95% 1357
98% 1619
99% 1657
100% 1923 (longest request)

Looking at these numbers, I see that 50% the requests are served within 351ms, 66% of the requests within 359ms, and so on. It’s clear that something happens when I look at 95% or more of the requests: the time suddenly increases by about a second.

These are the results for the function with Provisioned Concurrency enabled:

Percentage of the requests served within a certain time (ms)
50% 352
66% 368
75% 382
80% 387
90% 400
95% 415
98% 447
99% 513
100% 593 (longest request)

Let’s compare those numbers in a graph.

As expected for my test workload, I see a big difference in the response time of the slowest 5% of the requests (between 95% and 100%), where the function with Provisioned Concurrency disabled shows the latency added by the creation of new execution environments and the (slow) initialization in my function code.

In general, the amount of latency added depends on the runtime you use, the size of your code, and the initialization required by your code to be ready for a first invocation. As a result, the added latency can be more, or less, than what I experienced here.

The number of invocations affected by this additional latency depends on how often the Lambda service needs to create new execution environments. Usually that happens when the number of concurrent invocations increases beyond what already provisioned, or when you deploy a new version of a function.

A small percentage of slow response times (generally referred to as tail latency) really makes a difference in end user experience. Over an extended period of time, most users are affected during some of their interactions. With Provisioned Concurrency enabled, user experience is much more stable.

Provisioned Concurrency is a Lambda feature and works with any trigger. For example, you can use it with WebSockets APIsGraphQL resolvers, or IoT Rules. This feature gives you more control when building serverless applications that require low latency, such as web and mobile apps, games, or any service that is part of a complex transaction.

Available Now
Provisioned Concurrency can be configured using the console, the AWS Command Line Interface (CLI), or AWS SDKs for new or existing Lambda functions, and is available today in the following AWS Regions: in US East (Ohio), US East (N. Virginia), US West (N. California), US West (Oregon), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Canada (Central), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), and Europe (Stockholm), Middle East (Bahrain), and South America (São Paulo).

You can use the AWS Serverless Application Model (SAM) and SAM CLI to test, deploy and manage serverless applications that use Provisioned Concurrency.

With Application Auto Scaling you can automate configuring the required concurrency for your functions. As policies, Target Tracking and Scheduled Scaling are supported. Using these policies, you can automatically increase the amount of concurrency during times of high demand and decrease it when the demand decreases.

You can also use Provisioned Concurrency today with AWS Partner tools, including configuring Provisioned Currency settings with the Serverless Framework and Terraform, or viewing metrics with Datadog, Epsagon, Lumigo, New Relic, SignalFx, SumoLogic, and Thundra.

You only pay for the amount of concurrency that you configure and for the period of time that you configure it. Pricing in US East (N. Virginia) is $0.015 per GB-hour for Provisioned Concurrency and $0.035 per GB-hour for Duration. The number of requests is charged at the same rate as normal functions. You can find more information in the Lambda pricing page.

This new feature enables developers to use Lambda for a variety of workloads that require highly consistent latency. Let me know what you are going to use it for!

Danilo

The Serverlist: Full Stack Serverless, Serverless Architecture Reference Guides, and more

Post Syndicated from Connor Peshek original https://blog.cloudflare.com/serverlist-10th-edition/

The Serverlist: Full Stack Serverless, Serverless Architecture Reference Guides, and more

Check out our tenth edition of The Serverlist below. Get the latest scoop on the serverless space, get your hands dirty with new developer tutorials, engage in conversations with other serverless developers, and find upcoming meetups and conferences to attend.

Sign up below to have The Serverlist sent directly to your mailbox.


Introducing Amazon EventBridge schema registry and discovery – In preview

Post Syndicated from Julian Wood original https://aws.amazon.com/blogs/compute/introducing-amazon-eventbridge-schema-registry-and-discovery-in-preview/

Today, AWS announces the preview of Amazon EventBridge schema registry and discovery. These are new developer tool features, which allow you to automatically find events and their structure, or schema, and store them in a shared central location. This makes it faster and easier to build event-driven applications. You can access the registry and generate code bindings for schemas directly in popular Integrated Development Environments (IDEs), including JetBrains IntelliJ and PyCharm, Microsoft Visual Studio Code, as well as through the Amazon EventBridge console and APIs.

What is a schema?

A schema represents the structure of an event, and commonly includes information such as the title and type of each piece of data. For example, in a review on a product website, a schema might include fields for the reviewer’s name, user id, and review description, and that the name is a text string, and the user id is an integer. The event schema is important for developers as it shows what data is contained in the event, and allows them to write code based on that data.

Event-driven architectures

Developers embracing event-driven architectures may use an event bus such as Amazon EventBridge. EventBridge allows application decoupling without needing to write point-to-point integrations between services. This decoupling increases developer independence, as they simply subscribe to the events they’re interested in, reducing dependencies on other teams to write integrations.

However, decoupling introduces a new set of challenges. Finding events and their schema is a manual process. Developers must coordinate with the team responsible for publishing an event, or look through documentation to find its schema, and then manually create an object for the event in order to use it in their code.

EventBridge schema registry solves these problems by introducing two capabilities, a schema registry and schema discovery.

Schema registry

A schema registry stores a collection of schemas. You can use schema registry to search for, find, and track different schemas used and generated by your applications. Schemas for all AWS sources supported in EventBridge are automatically visible in your schema registry. SaaS partner and custom schemas can be generated and added to the registry using the schema discovery feature.

Schema discovery

Schema discovery automates the process of finding schemas and adding them to your registry. When schema discovery is enabled for an EventBridge event bus, the schema of each event sent to the bus is automatically added to the registry. If the schema of an event changes, schema discovery automatically creates a new version in the registry. Once a schema is added to the registry, you can generate a code binding for the schema, either in the EventBridge console or directly in your IDE.

Generally, you only enable schema discovery in your development environments (AWS Free Tier includes 5 million ingested events). Schemas of any new events you create are automatically added to your registry to use when developing your application. If you need to audit all of the events going through your event bus, you can enable discovery on your production event bus, and pay $0.10 per million events ingested for any usage outside of the Free Tier.

Code bindings

Once a schema is added to the registry, you can download a code binding. This allows you to represent the event as a strongly typed object in your code, and take advantage of IDE features such as validation and auto-complete. Code bindings are available for Java, Python, or TypeScript programming languages. You can download bindings from the AWS Management Console, or directly from your IDE with the AWS Toolkit plugin for IntelliJ and VS Code.

If you use the AWS Serverless Application Model (SAM), you can now use the interactive sam init command to generate a serverless application with a schema as a trigger. This automatically adds a class file for the schema to your code, and generates a handler function that serializes the trigger event into an object. This makes it quicker to build serverless event-driven applications.

Event Schemas

Viewing the schema registry

You can view the schema registry in the Amazon EventBridge console and download code bindings.

  1. Choose Go to schema registry and then choose Schemas in the left-side navigation bar. You can view and search for built-in AWS schemas, as well as your discovered and custom schemas within the registry.
  2. For example, searching for a schema for AWS Step Functions, there is an existing schema for: aws.states.StepFunctionsExecutionStatusChange. The schema details as well as the JSON representation are visible.
  3. json schema

  4. To Download code bindings to use in your IDE, select a language option, download the .zip file, and manually import the schema into your IDE.

Automatically discovering schemas

Schema registry’s discovery capabilities allow you to generate schemas for your own events. EventBridge can ingest events from a number of SaaS vendors including, for example MongoDB.

In this example, I use an online shop, which stores product reviews in a MongoDB database. A MongoDB trigger is configured to send all new database entries to the EventBridge MongoDB partner event bus. A Lambda function is then triggered for all new events and calls Amazon Comprehend to do sentiment analysis on the new reviews. Any negative reviews generate a service desk ticket for further investigation.

  1. I have previously set up MongoDB Atlas to connect as a SaaS partner to EventBridge. I have configured a stitch trigger to send events to EventBridge for any updates to the database.
  2. To discover the schema, in the EventBridge console, I navigate to Events and Partner event sources.
  3. I select the MongoDB event source and choose Associate with event bus.
  4. I choose Associate.
  5. I Navigate to Event buses and choose the MongoDB Custom event bus and choose Start discovery. An event bus-managed rule is created automatically.
  6. custom event busses

  7. I write a new product review, creating a new database record, which triggers a new event on the event bus. You can simulate this using the AWS CLI. Replace the EventBusName with your partner event bus.
  8. aws events put-events --entries '[[{"Source": "mystore","DetailType": "Review Created","EventBusName":"aws.partner/mongodb.com/stitch.trigger/5ddf5c9476ff0ff8b0916763","Detail": "{\"star_rating\": 5,  \"description\": \"The size and length fit me well and the design is fun. I felt very secure wearing this tshirt. \",  \"helpful_count\": 34,  \"unhelpful_count\": 1,  \"pros\": [\"lightweight\",\"fits well\"  ],  \"cons\": [],  \"customer\": {\"name\": \"Julian Wood\",\"email\": \"[email protected]\",\"phone\": \"+1 604 123 1234\"  },  \"product\": {\"product_id\": 788032119674292922,\"title\": \"Encrypt Everything Tshirt\",\"sku\": \"encrypt-everything-tshirt\",\"inventory_id\": 23190823132,\"size\": \"medium\",\"taxable\": true,\"image_url\": \"https://img.mystore.test/encrypt-tshirt.jpg\",\"weight\": 200.0}}"}]'

  9. I navigate to Schema registry and schemas, choose Discovered schema registry, and can see the discovered schema from the new review event.
  10. Choosing the schema name, I can view the generated schema. Here is an excerpt

Schema discovery has automatically discovered the MongoDB schema from events passing through the event bus and added it to the registry.

Downloading code bindings directly into an IDE

Schema code bindings can be downloaded directly from the AWS Management Console as well as within an IDE, for example JetBrains IntelliJ.

  1. I have IntelliJ installed.
  2. I launch IntelliJ, navigate to File | Settings, and choose Plugins.
  3. On the Marketplace tab, in Search plugins, I enter AWS. When AWS Toolkit by Amazon Web Services is displayed, I select it and choose Install. Minimum version 1.9 is required.
  4. AWSToolkit

  5. I accept the third-party Plugins Privacy Note and choose Restart IDE.
  6. Once the IDE has restarted, I navigate to AWS Explorer at the bottom-left of the IDE to view available schemas.
  7. I choose Configure AWS connection.
  8. Configure AWS Connection

  9. I choose Credentials, my local AWS account profiles are automatically loaded from the AWS credentials file.
  10. I choose Schemas\discovered-schemas, right-click and then choose View Schema to see the schema.
  11. View Schema

  12. I can then use the code binding in a project, I navigate to File | New Project.
  13. A number of languages and frameworks are available to build an application. I want to build a serverless application and so choose AWS and AWS Serverless Application, which builds an AWS Serverless Application Model application project. I choose Next.
  14. Create SAM App

  15. I enter a Project name and file location, choosing java8 as the Runtime.
  16. I choose AWS SAM EventBridge App from Scratch (for any Event trigger from a Schema Registry) for Gradle.
  17. I select Credentials and Region
  18. Under Event Schema, I browse the available schema. At the end of the list, I find the discovered MongoDB schema. I select it and choose Finish.

SAM Peoject

Once the project is created, I can see the event schema imported into the SAM project.

I navigate to and open mongodb-app\HelloWorldFunction\src\main\java\helloworld\App.java to see the Lambda handler created with the event schema details. I can also use full IDE auto-complete with the schema.

Code Completion

I can then add the Amazon Comprehend sentiment analysis code using the schema into the Lambda function.
The SAM template.yaml in the project root directory specifies the Lambda function triggered by an EventBridge event (previously called CloudWatchEvent) using the discovered MongoDB schema.
I change the EventBusName to the correct partner event bus.

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: HelloWorldFunction
      Handler: helloworld.App::handleRequest
      Runtime: java8
      Environment: 
        Variables:
          PARAM1: VALUE
      Events:
        HelloWorld:
          Type: CloudWatchEvent 
          Properties:
            EventBusName:
                - aws.partner/mongodb.com/stitch.trigger/5ddf5c9476ff0ff8b0916763
            Pattern:
              source:
                - aws.partner/mongodb.com.test/stitch.trigger/5ddf5c9476ff0ff8b0916
              detail-type:
                - MongoDB Database Trigger for my_store.reviews

I can then deploy the SAM application.
I navigate to File | Deploy SAM Application .I choose Create Stack and enter a name. Select an S3 bucket and choose Deploy.

Deploy SAM

The SAM application is deployed. I can also automate the process using SAM CLI. Version 0.35.0 includes the schema features in the new interactive sam init command. This uses an AWS Quick Start Template to generate a handler function and add a class file for the schema to my code.

Pricing

Usage of the schema registry is free.

Schema discovery includes a free tier of 5M ingested events per month. In use cases where discovery is used in your development environment, your usage should stay within the free tier.

For additional usage outside of the free tier:

  • $0.10 per million events ingested for discovery.

All ingested events are measured in 8 KB chunks.

Availability

The EventBridge schema registry preview is available in the US East (Ohio), US West (Oregon), US East (Northern Virginia) Asia Pacific (Tokyo) Region, and Europe (Ireland) Regions. For details on EventBridge availability, please see the AWS Region table (https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/).

Conclusion

Amazon EventBridge schema registry and discovery helps developers take advantage of new schema capabilities to build exciting event-driven applications. All AWS sources supported in EventBridge are visible in your schema registry. SaaS partner and custom schemas can be added automatically using schema discovery. Code bindings, which allow you to represent the event as an object in your code, can be downloaded from the console or directly within IDEs.

Happy coding with event schemas!

A History of HTML Parsing at Cloudflare: Part 2

Post Syndicated from Andrew Galloni original https://blog.cloudflare.com/html-parsing-2/

A History of HTML Parsing at Cloudflare: Part 2

A History of HTML Parsing at Cloudflare: Part 2

The second blog post in the series on HTML rewriters picks up the story in 2017 after the launch of the Cloudflare edge compute platform Cloudflare Workers. It became clear that the developers using workers wanted the same HTML rewriting capabilities that we used internally, but accessible via a JavaScript API.

This blog post describes the building of a streaming HTML rewriter/parser with a CSS-selector based API in Rust. It is used as the back-end for the Cloudflare Workers HTMLRewriter. We have open-sourced the library (LOL HTML) as it can also be used as a stand-alone HTML rewriting/parsing library.

The major change compared to LazyHTML, the previous rewriter, is the dual-parser architecture required to overcome the additional performance overhead of wrapping/unwrapping each token when propagating tokens to the workers runtime. The remainder of the post describes a CSS selector matching engine inspired by a Virtual Machine approach to regular expression matching.

v2 : Give it to everyone and make it faster

In 2017, Cloudflare introduced an edge compute platform – Cloudflare Workers. It was no surprise that customers quickly required the same HTML rewriting capabilities that we were using internally. Our team was impressed with the platform and decided to migrate some of our features to Workers. The goal was to improve our developer experience working with modern JavaScript rather than statically linked NGINX modules implemented in C with a Lua API.

It is possible to rewrite HTML in Workers, though for that you needed a third party JavaScript package (such as Cheerio). These packages are not designed for HTML rewriting on the edge due to the latency, speed and memory considerations described in the previous post.

JavaScript is really fast but it still can’t always produce performance comparable to native code for some tasks – parsing being one of those. Customers typically needed to buffer the whole content of the page to do the rewriting resulting in considerable output latency and memory consumption that often exceeded the memory limits enforced by the Workers runtime.

We started to think about how we could reuse the technology in Workers. LazyHTML was a perfect fit in terms of parsing performance, but it had two issues:

  1. API ergonomics: LazyHTML produces a stream of HTML tokens. This is sufficient for our internal needs. However, for an average user, it is not as convenient as the jQuery-like API of Cheerio.
  2. Performance: Even though LazyHTML is tremendously fast, integration with the Workers runtime adds even more limitations. LazyHTML operates as a simple parse-modify-serialize pipeline, which means that it produces tokens for the whole content of the page. All of these tokens then have to be propagated to the Workers runtime and wrapped inside a JavaScript object and then unwrapped and fed back to LazyHTML for serialization. This is an extremely expensive operation which would nullify the performance benefit of LazyHTML.

A History of HTML Parsing at Cloudflare: Part 2
LazyHTML with V8

LOL HTML

We needed something new, designed with Workers requirements in mind, using a language with the native speed and safety guarantees (it’s incredibly easy to shoot yourself in the foot doing parsing). Rust was the obvious choice as it provides the native speed and the best guarantee of memory safety which minimises the attack surface of untrusted input. Wherever possible the Low Output Latency HTML rewriter (LOL HTML) uses all the previous optimizations developed for LazyHTML such as tag name hashing.

Dual-parser architecture

Most developers are familiar and prefer to use CSS selector-based APIs (as in Cheerio, jQuery or DOM itself) for HTML mutation tasks. We decided to base our API on CSS selectors as well. Although this meant additional implementation complexity, the decision created even more opportunities for parsing optimizations.

As selectors define the scope of the content that should be rewritten, we realised we can skip the content that is not in this scope and not produce tokens for it. This not only significantly speeds up the parsing itself, but also avoids the performance burden of the back and forth interactions with the JavaScript VM. As ever the best optimization is not to do something.

A History of HTML Parsing at Cloudflare: Part 2

Considering the tasks required, LOL HTML’s parser consists of two internal parsers:

  • Lexer – a regular full parser, that produces output for all types of content that it encounters;
  • Tag scanner – looks for start and end tags and skips parsing the rest of the content. The tag scanner parses only the tag name and feeds it to the selector matcher. The matcher will switch parser to the lexer if there was a match or additional information about the tag (such as attributes) are required for matching.

The parser switches back to the tag scanner as soon as input leaves the scope of all selector matches. The tag scanner may also sometimes switch the parser to the Lexer – if it requires additional tag information for the parsing feedback simulation.

A History of HTML Parsing at Cloudflare: Part 2
LOL HTML architecture

Having two different parser implementations for the same grammar will increase development costs and is error-prone due to implementation inconsistencies. We minimize these risks by implementing a small Rust macro-based DSL which is similar in spirit to Ragel. The DSL program describes Nondeterministic finite automaton states and actions associated with each state transition and matched input byte.

An example of a DSL state definition:

tag_name_state {
   whitespace => ( finish_tag_name?; --> before_attribute_name_state )
   b'/'       => ( finish_tag_name?; --> self_closing_start_tag_state )
   b'>'       => ( finish_tag_name?; emit_tag?; --> data_state )
   eof        => ( emit_raw_without_token_and_eof?; )
   _          => ( update_tag_name_hash; )
}

The DSL program gets expanded by the Rust compiler into not quite as beautiful, but extremely efficient Rust code.

We no longer need to reimplement the code that drives the parsing process for each of our parsers. All we need to do is to define different action implementations for each. In the case of the tag scanner, the majority of these actions are a no-op, so the Rust compiler does the NFA optimization job for us: it optimizes away state branches with no-op actions and even whole states if all of the branches have no-op actions. Now that’s cool.

Byte slice processing optimisations

Moving to a memory-safe language provided new challenges. Rust has great memory safety mechanisms, however sometimes they have a runtime performance cost.

The task of the parser is to scan through the input and find the boundaries of lexical units of the language – tokens and their internal parts. For example, an HTML start tag token consists of multiple parts: a byte slice of input that represents the tag name and multiple pairs of input slices that represent attributes and values:

struct StartTagToken<'i> {
   name: &'i [u8],
   attributes: Vec<(&'i [u8], &'i [u8])>,
   self_closing: bool
}

As Rust uses bound checks on memory access, construction of a token might be a relatively expensive operation. We need to be capable of constructing thousands of them in a fraction of second, so every CPU instruction counts.

Following the principle of doing as little as possible to improve performance we use a “token outline” representation of tokens: instead of having memory slices for token parts we use numeric ranges which are lazily transformed into a byte slice when required.

struct StartTagTokenOutline {
   name: Range<usize>,
   attributes: Vec<(Range<usize>, Range<usize>)>,
   self_closing: bool
}

As you might have noticed, with this approach we are no longer bound to the lifetime of the input chunk which turns out to be very useful. If a start tag is spread across multiple input chunks we can easily update the token that is currently in construction, as new chunks of input arrive by just adjusting integer indices. This allows us to avoid constructing a new token with slices from the new input memory region (it could be the input chunk itself or the internal parser’s buffer).

This time we can’t get away with avoiding the conversion of input character encoding; we expose a user-facing API that operates on JavaScript strings and input HTML can be of any encoding. Luckily, as we can still parse without decoding and only encode and decode within token bounds by a request (though we still can’t do that for UTF-16 encoding).

So, when a user requests an element’s tag name in the API, internally it is still represented as a byte slice in the character encoding of the input, but when provided to the user it gets dynamically decoded. The opposite process happens when a user sets a new tag name.

For selector matching we can still operate on the original encoding representation – because we know the input encoding ahead of time we preemptively convert values in a selector to the page’s character encoding, so comparisons can be done without decoding fields of each token.

As you can see, the new parser architecture along with all these optimizations produced great performance results:

A History of HTML Parsing at Cloudflare: Part 2
Average parsing time depending on the input size – lower is better

LOL HTML’s tag scanner is typically twice as fast as LazyHTML and the lexer has comparable performance, outperforming LazyHTML on bigger inputs. Both are a few times faster than the tokenizer from html5ever – another parser implemented in Rust used in the Mozilla’s Servo browser engine.

CSS selector matching VM

With an impressively fast parser on our hands we had only one thing missing – the CSS selector matcher. Initially we thought we could just use Servo’s CSS selector matching engine for this purpose. After a couple of days of experimentation it turned out that it is not quite suitable for our task.

It did not work well with our dual parser architecture. We first need to to match just a tag name from the tag scanner, and then, if we fail, query the lexer for the attributes. The selectors library wasn’t designed with this architecture in mind so we needed ugly hacks to bail out from matching in case of insufficient information. It was inefficient as we needed to start matching again after the bailout doing twice the work. There were other problems, such as the integration of lazy character decoding and integration of tag name comparison using tag name hashes.

Matching direction

The main problem encountered was the need to backtrack all the open elements for matching. Browsers match selectors from right to left and traverse all ancestors of an element. This StackOverflow has a good explanation of why they do it this way. We would need to store information about all open elements and their attributes – something that we can’t do while operating with tight memory constraints. This matching approach would be inefficient for our case – unlike browsers, we expect to have just a few selectors and a lot of elements. In this case it is much more efficient to match selectors from left to right.

And this is when we had a revelation. Consider the following CSS selector:

body > div.foo  img[alt] > div.foo ul

It can be split into individual components attributed to a particular element with hierarchical combinators in between:

body > div.foo img[alt] > div.foo  ul
---    ------- --------   -------  --

Each component is easy to match having a start tag token – it’s just a matter of comparison of token fields with values in the component. Let’s dive into abstract thinking and imagine that each such component is a character in the infinite alphabet of all possible components:

Selector componentCharacter
bodya
div.foob
img[alt]c
uld

Let’s rewrite our selector with selector components replaced by our imaginary characters:

a > b c > b d

Does this remind you of something?

A   `>` combinator can be considered a child element, or “immediately followed by”.

The ` ` (space) is a descendant element can be thought of as there might be zero or more elements in between.

There is a very well known abstraction to express these relations – regular expressions. The selector replacing combinators can be replaced with a regular expression syntax:

ab.*cb.*d

We transformed our CSS selector into a regular expression that can be executed on the sequence of start tag tokens. Note that not all CSS selectors can be converted to such a regular grammar and the input on which we match has some specifics, which we’ll discuss later. However, it was a good starting point: it allowed us to express a significant subset of selectors.

Implementing a Virtual Machine

Next, we started looking at non-backtracking algorithms for regular expressions. The virtual machine approach seemed suitable for our task as it was possible to have a non-backtracking implementation that was flexible enough to work around differences between real regular expression matching on strings and our abstraction.

VM-based regular expression matching is implemented as one of the engines in many regular expression libraries such as regexp2 and Rust’s regex. The basic idea is that instead of building an NFA or DFA for a regular expression it is instead converted into DSL assembly language with instructions later executed by the virtual machine – regular expressions are treated as programs that accept strings for matching.

Since the VM program is just a representation of NFA with ε-transitions it can exist in multiple states simultaneously during the execution, or, in other words, spawns multiple threads. The regular expression matches if one or more states succeed.

For example, consider the following VM instructions:

  • expect c – waits for next input character, aborts the thread if doesn’t equal to the instruction’s operand;
  • jmp L – jump to label ‘L’;
  • thread L1, L2 – spawns threads for labels L1 and L2, effectively splitting the execution;
  • match – succeed the thread with a match;

For example, using this instructions set regular expression “ab*c” can be translated into:

    expect a
L1: thread L2, L3
L2: expect b
    jmp L1
L3: expect c
    match

Let’s try to translate the regular expression ab.*cb.*d from the selector we saw earlier:

    expect a
    expect b
L1: thread L2, L3
L2: expect [any]
    jmp L1
L3: expect c
    expect b
L4: thread L5, L6
L5: expect [any]
    jmp L4
L6: expect d
    match

That looks complex! Though this assembly language is designed for regular expressions in general, and regular expressions can be much more complex than our case. For us the only kind of repetition that matters is “.*”. So, instead of expressing it with multiple instructions we can use just one called hereditary_jmp:

    expect a
    expect b
    hereditary_jmp L1
L1: expect c
    expect b
    hereditary_jmp L2
L2: expect d
    match

The instruction tells VM to memoize instruction’s label operand and unconditionally spawn a thread with a jump to this label on each input character.

There is one significant distinction between the string input of regular expressions and the input provided to our VM. The input can shrink!

A regular string is just a contiguous sequence of characters, whereas we operate on a sequence of open elements. As new tokens arrive this sequence can grow as well as shrink. Assume we represent <div> as ‘a’ character in our imaginary language, so having <div><div><div> input we can represent it as aaa, if the next token in the input is </div> then our “string” shrinks to aa.

You might think at this point that our abstraction doesn’t work and we should try something else. What we have as an input for our machine is a stack of open elements and we needed a stack-like structure to store our hereditrary_jmp instruction labels that VM had seen so far. So, why not store it on the open element stack? If we store the next instruction pointer on each of stack items on which the expect instruction was successfully executed, we’ll have a full snapshot of the VM state, so we can easily roll back to it if our stack shrinks.

With this implementation we don’t need to store anything except a tag name on the stack, and, considering that we can use the tag name hashing algorithm, it is just a 64-bit integer per open element. As an additional small optimization, to avoid traversing of the whole stack in search of active hereditary jumps on each new input we store an index of the first ancestor with a hereditary jump on each stack item.

For example, having the following selector “body > div span” we’ll have the following VM program (let’s get rid of labels and just use instruction indices instead):

0| expect <body>
1| expect <div>
2| hereditary_jmp 3
3| expect <span>
4| match

Having an input “<body><div><div><a>” we’ll have the following stack:

A History of HTML Parsing at Cloudflare: Part 2

Now, if the next token is a start tag <span> the VM will first try to execute the selectors program from the beginning and will fail on the first instruction. However, it will also look for any active hereditary jumps on the stack. We have one which jumps to the instructions at index 3. After jumping to this instruction the VM successfully produces a match. If we get yet another <span> start tag later it will much as well following the same steps which is exactly what we expect for the descendant selector.

If we then receive a sequence of “</span></span></div></a></div>” end tags our stack will contain only one item:

A History of HTML Parsing at Cloudflare: Part 2

which instructs VM to jump to instruction at index 1, effectively rolling back to matching the div component of the selector.

We mentioned earlier that we can bail out from the matching process if we only have a tag name from the tag scanner and we need to obtain more information by running the lexer? With a VM approach it is as easy as stopping the execution of the current instruction and resuming it later when we get the required information.

Duplicate selectors

As we need a separate program for each selector we need to match, how can we stop the same simple components doing the same job? The AST for our selector matching program is a radix tree-like structure whose edge labels are simple selector components and nodes are hierarchical combinators.
For example for the following selectors:

body > div > link[rel]
body > span
body > span a

we’ll get the following AST:

A History of HTML Parsing at Cloudflare: Part 2

If selectors have common prefixes we can match them just once for all these selectors. In the compilation process, we flatten this structure into a vector of instructions.

[not] JIT-compilation

For performance reasons compiled instructions are macro-instructions – they incorporate multiple basic VM instruction calls. This way the VM can execute only one macro instruction per input token. Each of the macro instructions compiled using the so-called “[not] JIT-compilation” (the same approach to the compilation is used in our other Rust project – wirefilter).

Internally the macro instruction contains expect and following jmp, hereditary_jmp and match basic instructions. In that sense macro-instructions resemble microcode making it easy to suspend execution of a macro instruction if we need to request attributes information from the lexer.

What’s next

It is obviously not the end of the road, but hopefully, we’ve got a bit closer to it. There are still multiple bits of functionality that need to be implemented and certainly, there is a space for more optimizations.

If you are interested in the topic don’t hesitate to join us in development of LazyHTML and LOL HTML at GitHub and, of course, we are always happy to see people passionate about technology here at Cloudflare, so don’t hesitate to contact us if you are too :).

A History of HTML Parsing at Cloudflare: Part 1

Post Syndicated from Andrew Galloni original https://blog.cloudflare.com/html-parsing-1/

A History of HTML Parsing at Cloudflare: Part 1

A History of HTML Parsing at Cloudflare: Part 1

To coincide with the launch of streaming HTML rewriting functionality for Cloudflare Workers we are open sourcing the Rust HTML rewriter (LOL  HTML) used to back the Workers HTMLRewriter API. We also thought it was about time to review the history of HTML rewriting at Cloudflare.

The first blog post will explain the basics of a streaming HTML rewriter and our particular requirements. We start around 8 years ago by describing the group of ‘ad-hoc’ parsers that were created with specific functionality such as to rewrite e-mail addresses or minify HTML. By 2016 the state machine defined in the HTML5 specification could be used to build a single spec-compliant HTML pluggable rewriter, to replace the existing collection of parsers. The source code for this rewriter is now public and available here: https://github.com/cloudflare/lazyhtml.

The second blog post will describe the next iteration of rewriter. With the launch of the edge compute platform Cloudflare Workers we came to realise that developers wanted the same HTML rewriting capabilities with a JavaScript API. The post describes the thoughts behind a low latency streaming HTML rewriter with a CSS-selector based API. We open-sourced the Rust library as it can also be used as a stand-alone HTML rewriting/parsing library.

What is a streaming HTML rewriter ?

A streaming HTML rewriter takes either a HTML string or byte stream input, parses it into tokens or any other structured intermediate representation (IR) – such as an Abstract Syntax Tree (AST). It then performs transformations on the tokens before converting back to HTML. This provides the ability to modify, extract or add to an existing HTML document as the bytes are being  processed. Compare this with a standard HTML tree parser which needs to retrieve the entire file to generate a full DOM tree. The tree-based rewriter will both take longer to deliver the first processed bytes and require significantly more memory.

A History of HTML Parsing at Cloudflare: Part 1
HTML rewriter

For example; consider you own a large site with a lot of historical content that you want to now serve over HTTPS. You will quickly run into the problem of resources (images, scripts, videos) being served over HTTP. This ‘mixed content’ opens a security hole and browsers will warn or block these resources. It can be difficult or even impossible to update every link on every page of a website. With a streaming HTML rewriter you can select the URI attribute of any HTML tag and change any HTTP links to HTTPS. We built this very feature Automatic HTTPS rewrites back in 2016 to solve mixed content issues for our customers.

The reader may already be wondering: “Isn’t this a solved problem, aren’t there many widely used open-source browsers out there with HTML parsers that can be used for this purpose?”. The reality is that writing code to run in 190+ PoPs around the world with a strict low latency requirement turns even seemingly trivial problems into complex engineering challenges.

The following blog posts will detail the journey of how starting with a simple idea of finding email addresses within an HTML page led to building an almost spec compliant HTML parser and then on to a CSS selector matching Virtual Machine. We learned a lot on this journey. I hope you find some of this as interesting as we did.

Rewriting at the edge

When rewriting content through Cloudflare we do not want to impact site performance. The balance in designing a streaming HTML rewriter is to minimise the pause in response byte flow by holding onto as little information as possible whilst retaining the ability to rewrite matching tokens.

The difference in requirements compared to an HTML parser used in a browser include:

Output latency

For browsers, the Document Object Model (DOM) is the end product of the parsing process but in our case we have to parse, rewrite and serialize back to HTML. In the case of Cloudflare’s reverse proxy any content processing on the edge server results in latency between the server and an eyeball. It is desirable to minimize the latency impact of HTML handling, which involves parsing, rewriting and serializing back to HTML. In all of these stages we want to be as fast as possible to minimize latency.

Parser throughput

Let’s assume that usually browsers rarely need to deal with HTML pages bigger than 1Mb in size and an average page load time is somewhere around 3s at best. HTML parsing is not the main bottleneck of the page loading process as the browser will be blocked on running scripts and loading other render-critical resources. We can roughly estimate that ~3Mbps is an acceptable throughput for browser’s HTML parser. At Cloudflare we have hundreds of megabytes of traffic per CPU, so we need a parser that is faster by an order of magnitude.

Memory limitations

As most users must realise, browsers have the luxury of being able to consume memory. For example, this simple HTML markup when opened in a browser will consume a significant chunk of your system memory before eventually killing a browser tab (and all this memory will be consumed by the parser) :

<script>
   document.write('<');
   while(true) {
      document.write('aaaaaaaaaaaaaaaaaaaaaaaa');
   }
</script>

Unfortunately, buffering of some fraction of the input is inevitable even for streaming HTML rewriting. Consider these 2 HTML snippets:

<div foo="bar" qux="qux">

<div foo="bar" qux="qux"

These seemingly similar fragments of HTML will be treated completely differently when encountered at the end of an HTML page. The first fragment will be parsed as a start tag and the second one will be ignored. By just seeing a `<` character followed by a tag name, the parser can’t determine if it has found a start tag or not. It needs to traverse the input in the search of the closing `>` to make a decision, buffering all content in between, so it can later be emitted to the consumer as a start tag token.

This requirement forces browsers to indefinitely buffer content before eventually giving up with the out-of-memory error.

In our case, we can’t afford to spend hundreds of megabytes of memory parsing a single HTML file (actual constraints are even tighter – even using a dozen kilobytes for each request would be unacceptable). We need to be much more sophisticated than other implementations in terms of memory usage and gracefully handle all the situations where provided memory capacity is insufficient to accomplish parsing.

v0 : “Ad-hoc parsers”

As usual with big projects, it all started pretty innocently.

Find and obfuscate an email

In 2010, Cloudflare decided to provide a feature that would stop popular email scrapers. The basic idea of this protection was to find and obfuscate emails on pages and later decode them back in the browser with injected JavaScript code. Sounds easy, right? You search for anything that looks like an email, encode it and then decode it with some JavaScript magic and present the result to the end-user.

However, even such a seemingly simple task already requires solving several issues. First of all, we need to define what an email is, and there is no simple answer. Even the infamous regex supposedly covering the entire RFC is, in fact, outdated and incomplete as the new RFC added lots of valid email constructions, including Unicode support. Let’s not go down that rabbit hole for now and instead focus on a higher-level issue: transforming streaming content.

Content from the network comes in packets, which have to be buffered and parsed as HTTP by our servers. You can’t predict how the content will be split, which means you always need to buffer some of it because content that is going to be replaced can be present in multiple input chunks.

Let’s say we decided to go with a simple regex like `[\w.][email protected][\w.]+`. If the content that comes through contains the email “[email protected]”, it might be split in the following chunks:

A History of HTML Parsing at Cloudflare: Part 1

In order to keep good Time To First Byte (TTFB) and consistent speed, we want to ensure that the preceding chunk is emitted as soon as we determine that it’s not interesting for replacement purposes.

The easiest way to do that is to transform our regex into a state machine, or a finite automata. While you could do that by hand, you will end up with hard-to-maintain and error-prone code. Instead, Ragel was chosen to transform regular expressions into efficient native state machine code. Ragel doesn’t try to take care of buffering or anything other than traversing the state machine. It provides a syntax that not only describes patterns, but can also associate custom actions (code in a host language) with any given state.

In our case we can pass through buffers until we match the beginning of an email. If we subsequently find out the pattern is not an email we can bail out from buffering as soon as the pattern stops matching. Otherwise, we can retrieve the matched email and replace it with new content.

To turn our pattern into a streaming parser we can remember the position of the potential start of an email and, unless it was already discarded or replaced by the end of the current input, store the unhandled part in a permanent buffer. Then, when a new chunk comes, we can process it separately, resuming from a state Ragel remembers itself, but then use both the buffered chunk and a new one to either emit or obfuscate.

Now that we have solved the problem of matching email patterns in text, we need to deal with the fact that they need to be obfuscated on pages. This is when the first hints of HTML “parsing” were introduced.

I’ve put “parsing” in quotes because, rather than implementing the whole parser, the email filter (as the module was called) didn’t attempt to replicate the whole HTML grammar, but rather added custom Ragel patterns just for skipping over comments and tags where emails should not be obfuscated.

This was a reasonable approach, especially back in 2010 – four years before the HTML5 specification, when all browsers had their own quirks handling of HTML. However, as you can imagine, this approach did not scale well. If you’re trying to work around quirks in other parsers, you start gaining more and more quirks in your own, and then work around these too. Simultaneously, new features started to be added, which also required modifying HTML on the fly (like automatic insertion of Google Analytics script), and an existing module seemed to be the best place for that. It grew to handle more and more tags, operations and syntactic edge cases.

Now let’s minify..

In 2011, Cloudflare decided to also add minification to allow customers to speed up their websites even if they had not employed minification themselves. For that, we decided to use an existing streaming minifier – jitify. It already had NGINX bindings, which made it a great candidate for integration into the existing pipeline.

Unfortunately, just like most other parsers from that time as well as ours described above, it had its own processing rules for HTML, JavaScript and CSS, which weren’t precise but rather tried to parse content on a best-effort basis. This led to us having two independent streaming parsers that were incompatible and could produce bugs either individually or only in combination.

v1 : “(Almost) HTML5 Spec compliant parser”

Over the years engineers kept adding new features to the ever-growing state machines, while fixing new bugs arising from imprecise syntax implementations, conflicts between various parsers, and problems in features themselves.

By 2016, it was time to get out of the multiple ad hoc parsers business and do things ‘the right way’.

The next section(s) will describe how we built our HTML5 compliant parser starting from the specification state machine. Using only this state machine it should have been straight-forward to build a parser. You may be aware that historically the parsing of HTML had not been entirely strict which meant to not break existing implementations the building of an actual DOM was required for parsing. This is not possible for a streaming rewriter so a simulator of the parser feedback was developed. In terms of performance, it is always better not to do something. We  then describe why the rewriter can be ‘lazy’ and not perform the expensive encoding and decoding of text when rewriting HTML. The surprisingly difficult problem of deciding if a response is HTML is then detailed.

HTML5

By 2016, HTML5 had defined precise syntax rules for parsing and compatibility with legacy content and custom browser implementations. It was already implemented by all browsers and many 3rd-party implementations.

The HTML5 parsing specification defines basic HTML syntax in the form of a state machine. We already had experience with Ragel for similar use cases, so there was no question about what to use for the new streaming parser. Despite the complexity of the grammar, the translation of the specification to Ragel syntax was straightforward. The code looks simpler than the formal description of the state machine, thanks to the ability to mix regex syntax with explicit transitions.

A History of HTML Parsing at Cloudflare: Part 1
A visualisation of a small fraction of the HTML state machine. Source: https://twitter.com/RReverser/status/715937136520916992

HTML5 parsing requires a ‘DOM’

However, HTML has a history. To not break existing implementations HTML5 is specified with  recovery procedures for incorrect tag nesting, ordering, unclosed tags, missing attributes and all the other possible quirks that used to work in older browsers.  In order to resolve these issues, the specification expects a tree builder to drive the lexer, essentially meaning you can’t correctly tokenize HTML (split into separate tags) without a DOM.

A History of HTML Parsing at Cloudflare: Part 1
HTML parsing flow as defined by the specification

For this reason, most parsers don’t even try to perform streaming parsing and instead take the input as a whole and produce a document tree as an output. This is not something we could do for streaming transformation without adding significant delays to page loading.

An existing HTML5 JavaScript parser – parse5 – had already implemented spec-compliant tree parsing using a streaming tokenizer and rewriter. To avoid having to create a full DOM the concept of a “parser feedback simulator” was introduced.

Tree builder feedback

As you can guess from the name, this is a module that aims to simulate a full parser’s feedback to the tokenizer, without actually building the whole DOM, but instead preserving only the required information and context necessary for correctly driving the state machine.

After rigorous testing and upstreaming a test runner to parse5, we found this technique to be suitable for the majority of even poorly written pages on the Internet, and employed it in LazyHTML.

A History of HTML Parsing at Cloudflare: Part 1
LazyHTML architecture

Avoiding decoding – everything is ASCII

Now that we had a streaming tokenizer working, we wanted to make sure that it was fast enough so that users didn’t notice any slowdowns to their pages as they go through the parser and transformations. Otherwise it would completely circumvent any optimisations we’d want to attempt on the fly.

It would not only cause a performance hit due to decoding and re-encoding any modified HTML content, but also significantly complicates our implementation due to multiple sources of potential encoding information  required to determine the character encoding, including sniffing of the first 1 KB of the content.

The “living” HTML Standard specification permits only encodings defined in the Encoding Standard. If we look carefully through those encodings, as well as a remark on Character encodings section of the HTML spec, we find that all of them are ASCII-compatible with the exception of UTF-16 and ISO-2022-JP.

This means that any ASCII text will be represented in such encodings exactly as it would be in ASCII, and any non-ASCII text will be represented by bytes outside of the ASCII range. This property allows us to safely tokenize, compare and even modify original HTML without decoding or even knowing which particular encoding it contains. It is possible as all the token boundaries in HTML grammar are represented by an ASCII character.

We need to detect UTF-16 by sniffing and either decode or skip such documents without modification. We chose the latter to avoid potential security-sensitive bugs which are common with UTF-16, and because the character encoding is seen in less than 0.1% of known character encodings luckily.

The only issue left with this approach is that in most places the HTML tokenization specification  requires you to replace U+0000 (NUL) characters with U+FFFD (replacement character) during parsing. Presumably, this was added as a security precaution against bugs in C implementations of old engines which could treat NUL character, encoded in ASCII / UTF-8 / … as a 0x00 byte, as the end of the string (yay, null-terminated strings…). It’s problematic for us because U+FFFD is outside of the ASCII range, and will be represented by different sequences of bytes in different encodings. We don’t know the encoding of the document, so this will lead to corruption of the output.

Luckily, we’re not in the same business as browser vendors, and don’t worry about NUL characters in strings as much – we use “fat pointer” string representation, in which the length of the string is determined not by the position of the NUL character, but stored along with the data pointer as an integer field:

typedef struct {
   const char *data;
   size_t length;
} lhtml_string_t;

Instead, we can quietly ignore these parts of the spec (sorry!), and keep U+0000 characters as-is and add them as such to tag, attribute names, and other strings, and later re-emit to the document. This is safe to do, because it doesn’t affect any state machine transitions, but merely preserves original 0x00 bytes and delegates their replacement to the parser in the end user’s browser.

Content type madness

We want to be lazy and minimise false positives. We only want to spend time parsing, decoding and rewriting actual HTML rather than breaking images or JSON. So the question is how do you decide if something is a HTML document. Can you just use the Content-Type for example ? A comment left in the source code best describes the reality.

/*
Dear future generations. I didn't like this hack either and hoped
we could do the right thing instead. Unfortunately, the Internet
was a bad and scary place at the moment of writing. If this
ever changes and websites become more standards compliant,
please do remove it just like I tried.
Many websites use PHP which sets Content-Type: text/html by
default. There is no error or warning if you don't provide own
one, so most websites don't bother to change it and serve
JSON API responses, private keys and binary data like images
with this default Content-Type, which we would happily try to
parse and transforms. This not only hurts performance, but also
easily breaks response data itself whenever some sequence inside
it happens to look like a valid HTML tag that we are interested
in. It gets even worse when JSON contains valid HTML inside of it
and we treat it as such, and append random scripts to the end
breaking APIs critical for popular web apps.
This hack attempts to mitigate the risk by ensuring that the
first significant character (ignoring whitespaces and BOM)
is actually `<` - which increases the chances that it's indeed HTML.
That way we can potentially skip some responses that otherwise
could be rendered by a browser as part of AJAX response, but this
is still better than the opposite situation.
*/

The reader might think that it’s a rare edge case, however, our observations show that almost 25% of the traffic served through Cloudflare with the “text/html” content type is unlikely to be HTML.

A History of HTML Parsing at Cloudflare: Part 1

The trouble doesn’t end there: it turns out that there is a considerable amount of XML content served with the “text/html” content type which can’t be always processed correctly when treated as HTML.

Over time bailouts for binary data, JSON, AMP and correctly identifying HTML fragments leads to the content sniffing logic which can be described by the following diagram:

A History of HTML Parsing at Cloudflare: Part 1

This is a good example of divergence between formal specifications and reality.

Tag name comparison optimisation

But just having fast parsing is not enough – we have functionality that consumes the output of the parser, rewrites it and feeds it back for the serialization. And all the memory and time constraints that we have for the parser are applicable for this code as well, as it is a part of the same content processing pipeline.

It’s a common requirement to compare parsed HTML tag names, e.g. to determine if the current tag should be rewritten or not. A naive implementation will use regular per-byte comparison which can require traversing the whole tag name. We were able to narrow this operation to a single integer comparison instruction in the majority of cases by using specially designed hashing algorithm.

The tag names of all standard HTML elements contain only alphabetical ASCII characters and digits from 1 to 6 (in numbered header tags, i.e. <h1> – <h6>). Comparison of tag names is case-insensitive, so we only need 26 characters to represent alphabetical characters. Using the same basic idea as arithmetic coding, we can represent each of the possible 32 characters of a  tag name using just 5 bits and, thus, fit up to floor(64 / 5) = 12 characters in a 64-bit integer which is enough for all the standard tag names and any other tag names that satisfy the same requirements! The great part is that we don’t even need to additionally traverse a tag name to hash it – we can do that as we parse the tag name consuming the input byte by byte.

However, there is one problem with this hashing algorithm and the culprit is not so obvious: to fit all 32 characters in 5 bits we need to use all possible bit combinations including 00000. This means that if the leading character of the tag name is represented with 00000 then we will not be able to differentiate between a varying number of consequent repetitions of this character.

For example, considering that ‘a’ is encoded as 00000 and ‘b’ as 00001 :

Tag nameBit representationEncoded value
ab00000 000011
aab00000 00000 000011

Luckily, we know that HTML grammar doesn’t allow the first character of a tag name to be anything except an ASCII alphabetical character, so reserving numbers from 0 to 5 (00000b-00101b) for digits and numbers from 6 to 31 (00110b – 11111b) for ASCII alphabetical characters solves the problem.

LazyHTML

After taking everything  mentioned  above into  consideration the LazyHTML (https://github.com/cloudflare/lazyhtml) library was created. It is a fast streaming HTML parser and serializer with a token based C-API derived from the HTML5 lexer written in Ragel. It provides a pluggable transformation pipeline to allow multiple transformation handlers to be chained together.

An example of a function that transforms `href` property of links:

// define static string to be used for replacements
static const lhtml_string_t REPLACEMENT = {
   .data = "[REPLACED]",
   .length = sizeof("[REPLACED]") - 1
};

static void token_handler(lhtml_token_t *token, void *extra /* this can be your state */) {
  if (token->type == LHTML_TOKEN_START_TAG) { // we're interested only in start tags
    const lhtml_token_starttag_t *tag = &token->start_tag;
    if (tag->type == LHTML_TAG_A) { // check whether tag is of type <a>
      const size_t n_attrs = tag->attributes.count;
      const lhtml_attribute_t *attrs = tag->attributes.items;
      for (size_t i = 0; i < n_attrs; i++) { // iterate over attributes
        const lhtml_attribute_t *attr = &attrs[i];
        if (lhtml_name_equals(attr->name, "href")) { // match the attribute name
          attr->value = REPLACEMENT; // set the attribute value
        }
      }
    }
  }
  lhtml_emit(token, extra); // pass transformed token(s) to next handler(s)
}

So, is it correct and how fast is  it?

It is HTML5 compliant as tested against the official test suites. As part of the work several contributions were sent to the specification itself for clarification / simplification of the spec language.

Unlike the previous parser(s), it didn’t bail out on any of the 2,382,625 documents from HTTP Archive, although 0.2% of documents exceeded expected bufferization limits as they were in fact JavaScript or RSS or other types of content incorrectly served with Content-Type: text/html, and since anything is valid HTML5, the parser tried to parse e.g. a<b; x=3; y=4 as incomplete tag with attributes. This is very rare (and goes to even lower amount of 0.03% when two error-prone advertisement networks are excluded from those results), but still needs to be accounted for and is a valid case for bailing out.

As for the benchmarks, In September 2016 using an example which transforms the HTML spec itself (7.9 MB HTML file) by replacing every <a href> (only that property only in those tags) to a static value. It was compared against the few existing and popular HTML parsers (only tokenization mode was used for the fair comparison, so that they don’t need to build AST and so on), and timings in milliseconds for 100 iterations are the following (lazy mode means that we’re using raw strings whenever possible, the other one serializes each token just for comparison):

A History of HTML Parsing at Cloudflare: Part 1

The results show that LazyHTML parser speeds are around an order of magnitude faster.

That concludes the first post in our series on HTML rewriters at Cloudflare. The next post describes how we built  a new streaming rewriter on top of the ideas of LazyHTML. The major update was to provide an easier to use CSS selector API.  It provides the back-end for the Cloudflare workers HTMLRewriter JavaScript API.

Introducing the HTMLRewriter API to Cloudflare Workers

Post Syndicated from Rita Kozlov original https://blog.cloudflare.com/introducing-htmlrewriter/

Introducing the HTMLRewriter API to Cloudflare Workers

Introducing the HTMLRewriter API to Cloudflare Workers

We are excited to announce that the HTMLRewriter API for Cloudflare Workers is now GA! You can get started today by checking out our documentation, or trying out our tutorial for localizing your site with the HTMLRewriter.

Want to know how it works under the hood? We are excited to tell you everything you wanted to know but were afraid to ask, about building a streaming HTML parser on the edge; read about it in part 1 (and stay tuned for part two coming tomorrow!).

Faster, more scalable applications at the edge

The HTMLRewriter can help solve two big problems web developers face today: making changes to the HTML, when they are hard to make at the server level, and making it possible for HTML to live on the edge, closer to the user — without sacrificing dynamic functionality.

Since the introduction of Workers, Workers have helped customers regain control where control either wasn’t provided, or very hard to obtain at the origin level. Just like Workers can help you set CORS headers at the middleware layer, between your users and the origin, the HTMLRewriter can assist with things like URL rewrites (see the example below!).

Back in January, we introduced the Cache API, giving you more control than ever over what you could store in our edge caches, and how. By helping users have closer control of the cache, users could push experiences that were previously only able to live at the origin server to the network edge. A great example of this is the ability to cache POST requests, where previously only GET requests were able to benefit from living on the edge.

Later this year, during Birthday Week, we announced Workers Sites, taking the idea of content on the edge one step further, and allowing you to fully deploy your static sites to the edge.

We were not the first to realize that the web could benefit from more content being accessible from the CDN level. The motivation behind the resurgence of static sites is to have a canonical version of HTML that can easily be cached by a CDN.

This has been great progress for static content on the web, but what about so much of the content that’s almost static, but not quite? Imagine an e-commerce website: the items being offered, and the promotions are all the same — except that pesky shopping cart on the top right. That small difference means the HTML can no longer be cached, and sent to the edge. That tiny little number of items in the cart requires you to travel all the way to your origin, and make a call to the database, for every user that visits your site. This time of year, around Black Friday, and Cyber Monday, this means you have to make sure that your origin, and database are able to scale to the maximum load of all the excited shoppers eager to buy presents for their loved ones.

Edge-side JAMstack with HTMLRewriter

One way to solve this problem of reducing origin load while retaining dynamic functionality is to move more of the logic to the client — this approach of making the HTML content static, leveraging CDN for caching, and relying on client-side API calls for dynamic content is known as the JAMstack (JavaScript, APIs, Markdown). The JAMstack is certainly a move in the right direction, trying to maximize content on the edge. We’re excited to embrace that approach even further, by allowing dynamic logic of the site to live on the edge as well.

In September, we introduced the HTMLRewriter API in beta to the Cloudflare Workers runtime — a streaming parser API to allow developers to develop fully dynamic applications on the edge. HTMLRewriter is a jQuery-like experience inside your Workers application, allowing you to manipulate the DOM using selectors and handlers. The same way jQuery revolutionized front-end development, we’re excited for the HTMLRewriter to change how we think about architecting dynamic applications.

There are a few advantages to rewriting HTML on the edge, rather than on the client. First, updating content client-side introduces a tradeoff between the performance and consistency of the application — that is, if you want to, update a page to a logged in version client side, the user will either be presented with the static version first, and witness the page update (resulting in an inconsistency known as a flicker), or the rendering will be blocked until the customization can be fully rendered. Having the DOM modifications made edge-side provides the benefits of a scalable application, without the degradation of user experience. The second problem is the inevitable client-side bloat. Most of the world today is connected to the internet on a mobile phone with significantly less powerful CPU, and on shaky last-mile connections where each additional API call risks never making it back to the client. With the HTMLRewriter within Cloudflare Workers, the API calls for dynamic content, (or even calls directly to Workers KV for user-specific data) can be made from the Worker instead, on a much more powerful machine, over much more resilient backbone connections.

Here’s what the developers at Happy Cog have to say about HTMLRewriter:

"At Happy Cog, we’re excited about the JAMstack and static site generators for our clients because of their speed and security, but in the past have often relied on additional browser-based JavaScript and internet-exposed backend services to customize user experiences based on a cookied logged-in state. With Cloudflare’s HTMLRewriter, that’s now changed. HTMLRewriter is a fundamentally new powerful capability for static sites. It lets us parse the request and customize each user’s experience without exposing any backend services, while continuing to use a JAMstack architecture. And it’s fast. Because HTMLRewriter streams responses directly to users, our code can make changes on-the-fly without incurring a giant TTFB speed penalty. HTMLRewriter is going to allow us to make architectural decisions that benefit our clients and their users — while avoiding a lot of the tradeoffs we’d usually have to consider."

Matt Weinberg, – President of Development and Technology, Happy Cog

HTMLRewriter in Action

Since most web developers are familiar with jQuery, we chose to keep the HTMLRewriter very similar, using selectors and handlers to modify content.

Below, we have a simple example of how to use the HTMLRewriter to rewrite the URLs in your application from myolddomain.com to mynewdomain.com:

async function handleRequest(req) {
  const res = await fetch(req)
  return rewriter.transform(res)
}
 
const rewriter = new HTMLRewriter()
  .on('a', new AttributeRewriter('href'))
  .on('img', new AttributeRewriter('src'))
 
class AttributeRewriter {
  constructor(attributeName) {
    this.attributeName = attributeName
  }
 
  element(element) {
    const attribute = element.getAttribute(this.attributeName)
    if (attribute) {
      element.setAttribute(
        this.attributeName,
        attribute.replace('myolddomain.com', 'mynewdomain.com')
      )
    }
  }
}
 
addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

The HTMLRewriter, which is instantiated once per Worker, is used here to select the a and img elements.

An element handler responds to any incoming element, when attached using the .on function of the HTMLRewriter instance.

Here, our AttributeRewriter will receive every element parsed by the HTMLRewriter instance. We will then use it to rewrite each attribute, href for the a element for example, to update the URL in the HTML.

Getting started

You can get started with the HTMLRewriter by signing up for Workers today. For a full guide on how to use the HTMLRewriter API, we suggest checking out our documentation.

How the the HTMLRewriter work?

How does the HTMLRewriter work under the hood? We’re excited to tell you about it, and have spared no details. Check out the first part of our two-blog post series to learn all about the not-so-brief history of rewriter HTML at Cloudflare, and stay tuned for part two tomorrow!

ICYMI: Serverless pre:Invent 2019

Post Syndicated from Eric Johnson original https://aws.amazon.com/blogs/compute/icymi-serverless-preinvent-2019/

With Contributions from Chris Munns – Sr Manager – Developer Advocacy – AWS Serverless

The last two weeks have been a frenzy of AWS service and feature launches, building up to AWS re:Invent 2019. As there has been a lot announced we thought we’d ship an ICYMI post summarizing the serverless service specific features that have been announced. We’ve also dropped in some service announcements from services that are commonly used in serverless application architectures or development.

AWS re:Invent

AWS re:Invent 2019

We also want you to know that we’ll be talking about many of these features (as well as those coming) in sessions at re:Invent.

Here’s what’s new!

AWS Lambda

On September 3, AWS Lambda started rolling out a major improvement to how AWS Lambda functions work with your Amazon VPC networks. This change brings both scale and performance improvements, and addresses several of the limitations of the previous networking model with VPCs.

On November 25, Lambda announced that the rollout of this new capability has completed in 6 additional regions including US East (Virginia) and US West (Oregon).

New VPC to VPC NAT for Lambda functions

New VPC to VPC NAT for Lambda functions

On November 18, Lambda announced three new runtime updates. Lambda now supports Node.js 12, Java 11, and Python 3.8. Each of these new runtimes has new language features and benefits so be sure to check out the linked release posts. These new runtimes are all based on an Amazon Linux 2 execution environment.

Lambda has released a number of controls for both stream and async based invocations:

  • For Lambda functions consuming events from Amazon Kinesis Data Streams or Amazon DynamoDB Streams, it’s now possible to limit the retry count, limit the age of records being retried, configure a failure destination, or split a batch to isolate a problem record. These capabilities will help you deal with potential “poison pill” records that would previously cause streams to pause in processing.
  • For asynchronous Lambda invocations, you can now set the maximum event age and retry attempts on the event. If either configured condition is met, the event can be routed to a dead letter queue (DLQ), Lambda destination, or it can be discarded.

In addition to the above controls, Lambda Destinations is a new feature that allows developers to designate an asynchronous target for Lambda function invocation results. You can set one destination for a success, and another for a failure. This unlocks really useful patterns for distributed event-based applications and can reduce code to send function results to a destination manually.

Lambda Destinations

Lambda Destinations

Lambda also now supports setting a Parallelization Factor, which allows you to set multiple Lambda invocations per shard for Amazon Kinesis Data Streams and Amazon DynamoDB Streams. This allows for faster processing without the need to increase your shard count, while still guaranteeing the order of records processed.

Lambda Parallelization Factor diagram

Lambda Parallelization Factor diagram

Lambda now supports Amazon SQS FIFO queues as an event source. FIFO queues guarantee the order of record processing compared to standard queues which are unordered. FIFO queues support messaging batching via a MessageGroupID attribute which allows for parallel Lambda consumers of a single FIFO queue. This allows for high throughput of record processing by Lambda.

Lambda now supports Environment Variables in AWS China (Beijing) Region, operated by Sinnet and the AWS China (Ningxia) Region, operated by NWCD.

Lastly, you can now view percentile statistics for the duration metric of your Lambda functions. Percentile statistics tell you the relative standing of a value in a dataset, and are useful when applied to metrics that exhibit large variances. They can help you understand the distribution of a metric, spot outliers, and find hard-to-spot situations that create a poor customer experience for a subset of your users.

AWS SAM CLI

AWS SAM CLI deploy command

AWS SAM CLI deploy command

The SAM CLI team simplified the bucket management and deployment process in the SAM CLI. You no longer need to manage a bucket for deployment artifacts – SAM CLI handles this for you. The deployment process has also been streamlined from multiple flagged commands to a single command, sam deploy.

AWS Step Functions

One of the powerful features of Step Functions is its ability to integrate directly with AWS services without you needing to write complicated application code. Step Functions has expanded its integration with Amazon SageMaker to simplify machine learning workflows, and added a new integration with Amazon EMR, making it faster to build and easier to monitor EMR big data processing workflows.

Step Functions step with EMR

Step Functions step with EMR

Step Functions now provides the ability to track state transition usage by integrating with AWS Budgets, allowing you to monitor and react to usage and spending trends on your AWS accounts.

You can now view CloudWatch Metrics for Step Functions at a one-minute frequency. This makes it easier to set up detailed monitoring for your workflows. You can use one-minute metrics to set up CloudWatch Alarms based on your Step Functions API usage, Lambda functions, service integrations, and execution details.

AWS Step Functions now supports higher throughput workflows, making it easier to coordinate applications with high event rates.

In US East (N. Virginia), US West (Oregon), and EU (Ireland), throughput has increased from 1,000 state transitions per second to 1,500 state transitions per second with bucket capacity of 5,000 state transitions. The default start rate for state machine executions has also increased from 200 per second to 300 per second, with bucket capacity of up to 1,300 starts in these regions.

In all other regions, throughput has increased from 400 state transitions per second to 500 state transitions per second with bucket capacity of 800 state transitions. The default start rate for AWS Step Functions state machine executions has also increased from 25 per second to 150 per second, with bucket capacity of up to 800 state machine executions.

Amazon SNS

Amazon SNS now supports the use of dead letter queues (DLQ) to help capture unhandled events. By enabling a DLQ, you can catch events that are not processed and re-submit them or analyze to locate processing issues.

Amazon CloudWatch

CloudWatch announced Amazon CloudWatch ServiceLens to provide a “single pane of glass” to observe health, performance, and availability of your application.

CloudWatch ServiceLens

CloudWatch ServiceLens

CloudWatch also announced a preview of a capability called Synthetics. CloudWatch Synthetics allows you to test your application endpoints and URLs using configurable scripts that mimic what a real customer would do. This enables the outside-in view of your customers’ experiences, and your service’s availability from their point of view.

On November 18, CloudWatch launched Embedded Metric Format to help you ingest complex high-cardinality application data in the form of logs and easily generate actionable metrics from them. You can publish these metrics from your Lambda function by using the PutLogEvents API or for Node.js or Python based applications using an open source library.

Lastly, CloudWatch announced a preview of Contributor Insights, a capability to identify who or what is impacting your system or application performance by identifying outliers or patterns in log data.

AWS X-Ray

X-Ray announced trace maps, which enable you to map the end to end path of a single request. Identifiers will show issues and how they affect other services in the request’s path. These can help you to identify and isolate service points that are causing degradation or failures.

X-Ray also announced support for Amazon CloudWatch Synthetics, currently in preview. X-Ray supports tracing canary scripts throughout the application providing metrics on performance or application issues.

X-Ray Service map with CloudWatch Synthetics

X-Ray Service map with CloudWatch Synthetics

Amazon DynamoDB

DynamoDB announced support for customer managed customer master keys (CMKs) to encrypt data in DynamoDB. This allows customers to bring your own key (BYOK) giving you full control over how you encrypt and manage the security of your DynamoDB data.

It is now possible to add global replicas to existing DynamoDB tables to provide enhanced availability across the globe.

Currently under preview, is another new DynamoDB capability to identify frequently accessed keys and database traffic trends. With this you can now more easily identify “hot keys” and understand usage of your DynamoDB tables.

CloudWatch Contributor Insights for DynamoDB

CloudWatch Contributor Insights for DynamoDB

Last but far from least for DynamoDB, is adaptive capacity, a feature which helps you handle imbalanced workloads by isolating frequently accessed items automatically and shifting data across partitions to rebalance them. This helps both reduce cost by enabling you to provision throughput for a more balanced out workload vs. over provisioning for uneven data access patterns.

AWS Serverless Application Repository

The AWS Serverless Application Repository (SAR) now offers Verified Author badges. These badges enable consumers to quickly and reliably know who you are. The badge will appear next to your name in the SAR and will deep-link to your GitHub profile.

SAR Verified developer badges

SAR Verified developer badges

AWS Code Services

AWS CodeCommit launched the ability for you to enforce rule workflows for pull requests, making it easier to ensure that code has pass through specific rule requirements. You can now create an approval rule specifically for a pull request, or create approval rule templates to be applied to all future pull requests in a repository.

AWS CodeBuild added beta support for test reporting. With test reporting, you can now view the detailed results, trends, and history for tests executed on CodeBuild for any framework that supports the JUnit XML or Cucumber JSON test format.

CodeBuild test trends

CodeBuild test trends

AWS Amplify and AWS AppSync

Instead of trying to summarize all the awesome things that our peers over in the Amplify and AppSync teams have done recently we’ll instead link you to their own recent summary: “A round up of the recent pre-re:Invent 2019 AWS Amplify Launches”.

AWS AppSync

AWS AppSync

Still looking for more?

We only covered a small bit of all the awesome new things that were recently announced. Keep your eyes peeled for more exciting announcements next week during re:Invent and for a future ICYMI Serverless Q4 roundup. We’ll also be kicking off a fresh series of Tech Talks in 2020 with new content helping to dive deeper on everything new coming out of AWS for serverless application developers.

Tracking the state of Lambda functions

Post Syndicated from Chris Munns original https://aws.amazon.com/blogs/compute/tracking-the-state-of-lambda-functions/

AWS Lambda functions often require resources from other AWS services in order to execute successfully, such as AWS Identity and Access Management (IAM) roles or Amazon Virtual Private Cloud (Amazon VPC) network interfaces. When you create or update a function, Lambda provisions the required resources on your behalf that enable your function to execute. In most cases, this process is very fast, and your function is ready to be invoked or modified immediately. However, there are situations where this action can take longer.

To better communicate the current “state” of your function when resources are being created or updated, AWS Lambda now includes several additional attributes in the function information returned by several Lambda API actions. This change does not impact the way that functions are invoked or how your code is
executed. In this post we cover the various states your Lambda function can be in, the conditions that lead to them, and how the Lambda service transitions the function through different states. You can also read more about this in the AWS Lambda documentation, Monitoring the State of a Function with the Lambda API.

All about function states

There are two main lifecycles that a Lambda function goes through, depending on if it is being created for the first time or existing and being updated. Before we go into the flows, let’s cover what the function states are:

Pending

Pending is the first state that all functions go through when they are created. During Pending Lambda attempts to create or configure any external resources. A function can only be invoked while in the Active state and so any invocations or other API actions that operate on the function will fail. Any automation built around invoking or updating functions immediately after creation must be modified to first see if the function has transitioned from Pending to Active.

Active

Active is reached after any configuration or provisioning of resources has completed during the initial creation of a function. Functions can only be invoked in the Active state.

During a function update, the state remains set to Active, but there is a separate attribute called LastUpdateStatus, which represents the state of an update of an active function. During an update, invocations execute the function’s previous code and configuration until it has successfully completed the update operation, after which the new code and configuration is used.

Failed

Failed is a state that represents something has gone wrong with either the configuration or provisioning of external resources.

Inactive

A function transitions to the Inactive state when it has been idle long enough for the Lambda service to reclaim the external resources that were configured for it. When you invoke a function that is in the Inactive state, it fails and sets the function to the Pending state until those resources are recreated. If the resources fail to be recreated, the function is set back to the Inactive state.

For all states, there are two other attributes that are set: StateReason and StateReasonCode. Both exist for troubleshooting issues and provide more information about the current state.

LastUpdateStatus

LastUpdateStatus represents a child lifecycle of the main function states lifecycle. It only becomes relevant during an update and signifies the changes a function goes through. There are three statuses associated with this:

InProgress

InProgress signifies that an update is happening on an existing function. While a function is in the InProgress status, invocations go to the function’s previous code and configuration.

Successful

Successful signifies that the update has completed. Once a function has been updated, this stays set until a further update.

Failed

Failed signifies that the function update has failed. The change is aborted and the function’s previous code and configuration remains in the Active state and available.

For all LastUpdateStatus values there are two other attributes that are set: LastUpdateStatusReason and LastUpdateStatusReasonCode. These exist to help troubleshoot any issues with an update.

Function State Lifecycles

The transition of a function from one state to another is completely dependent on the actions taken against it. There is no way for you to manually move a function between states.

Create function state lifecycle

Create function state lifecycle

For all functions the primary state lifecycle looks like this:

  • On create, a function enters the Pending state
  • Once any required resources are successfully created, it transitions into the Active state
    • If resource or function create fails for any reason then it transitions to the Failed state
  • If a function remains idle for several weeks it will enter the Inactive state
  • On the first invoke after going Inactive, a function re-enters the Pending state
    • Success sets the state to Active again
    • Failure sets the state back to Inactive
  • Successfully updating an Inactive function also sets the state back to Active again

For updating functions, the lifecycle looks like this:

Update function state lifecycle

Update function state lifecycle

NOTE: Functions can only be updated if they are in the Active or Inactive state. Update commands issued against functions that aren’t in the Active or Inactive state will fail before execution.

  • On update, a function’s LastUpdateStatus is set to Pending
    • Success sets the LastUpdateStatus to Successful
    • Failure sets the LastUpdateStatus to Failed
  • A function that is Inactive will go back to Active after a successful update.

Accessing function state information

You can query the current state of the function using the latest AWS SDKs and AWS CLI (version 1.16.291 or greater). To make it easy for you to write the code to check the state of a function we’ve added a new set of Waiters to the AWS SDKs. Waiters are a feature of certain AWS SDKs that can be used when AWS services have asynchronous operations, that you would need to poll for a change in their state or status. Using waiters removes the need to write complicated polling code and make it easy to check the state of a Lambda function. You can find out more about these in the documentation for each SDK: Python, Node.js, Java, Ruby, Go, and PHP.

What’s next

Starting today, all functions today will show an Active state only. You will not see a function transition from Pending at all.

Our first feature leveraging the function states lifecycle, will be a change to the recently announced improved VPC networking for AWS Lambda functions. As stated in the announcement post, Lambda precreates the ENIs required for your function to connect to your VPCs, which can take 60 to 90 seconds to complete. We will be changing this process slightly, by creating the required ENI resources while the function is placed in a Pending state and transitioning to Active after that process is completed. Remember, any invocations or other API actions that operate on the function will fail during the time before the function is Active. We will roll this behavior out for VPC configured functions in a phased manner, and will provide further details about timelines, and approaches to test before it is 100% live, in an upcoming post.

Until then, we encourage you to update the tooling you use to deploy and manage Lambda-based applications to confirm that your function is in the Active state before trying to invoke or perform other actions against them.  If you use AWS tools like AWS CloudFormation or AWS SAM, or tools from Lambda’s partners to deploy and manage your functions, there is no further action required.