Post Syndicated from Rob Sutter original https://aws.amazon.com/blogs/compute/icymi-serverless-q4-2019/
Welcome to the eighth edition of the AWS Serverless ICYMI (in case you missed it) quarterly recap. Every quarter, we share the most recent product launches, feature enhancements, blog posts, webinars, Twitch live streams, and other interesting things that you might have missed!
In case you missed our last ICYMI, checkout what happened last quarter here.
re:Invent 2019 dominated the fourth quarter at AWS. The serverless team presented a number of talks, workshops, and builder sessions to help customers increase their skills and deliver value more rapidly to their own customers.
Serverless talks from re:Invent 2019
We presented dozens of sessions showing how customers can improve their architecture and agility with serverless. Here are some of the most popular.
- Building event-driven architectures with Amazon EventBridge (API320)
- Serverless architectural patterns and best practices (ARC307-R3)
- Build predictive models using Amazon SageMaker (DEM148)
- Introducing The Amazon Builders’ Library (DOP328)
- I didn’t know API Gateway did that (SVS212-R2)
- Building microservices with AWS Lambda (SVS343-R1)
- Optimizing your serverless applications (SVS401-R1)
- Building APIs from front to back (SVS402-R2)
- A serverless journey: AWS Lambda under the hood (SVS405-R1)
You can also find decks for many of the serverless presentations and other re:Invent presentations on our AWS Events Content.
For developers needing greater control over performance of their serverless applications at any scale, AWS Lambda announced Provisioned Concurrency at re:Invent. This feature enables Lambda functions to execute with consistent start-up latency making them ideal for building latency sensitive applications.
As shown in the below graph, provisioned concurrency reduces tail latency, directly impacting response times and providing a more responsive end user experience.
Lambda rolled out enhanced VPC networking to 14 additional Regions around the world. This change brings dramatic improvements to startup performance for Lambda functions running in VPCs due to more efficient usage of elastic network interfaces.
Lambda now supports three additional runtimes: Node.js 12, Java 11, and Python 3.8. Each of these new runtimes has new version-specific features and benefits, which are covered in the linked release posts. Like the Node.js 10 runtime, these new runtimes are all based on an Amazon Linux 2 execution environment.
Lambda released a number of controls for both stream and async-based invocations:
- You can now configure error handling for Lambda functions consuming events from Amazon Kinesis Data Streams or Amazon DynamoDB Streams. It’s now possible to limit the retry count, limit the age of records being retried, configure a failure destination, or split a batch to isolate a problem record. These capabilities help you deal with potential “poison pill” records that would previously cause streams to pause in processing.
- For asynchronous Lambda invocations, you can now set the maximum event age and retry attempts on the event. If either configured condition is met, the event can be routed to a dead letter queue (DLQ), Lambda destination, or it can be discarded.
AWS Lambda Destinations is a new feature that allows developers to designate an asynchronous target for Lambda function invocation results. You can set separate destinations for success and failure. This unlocks new patterns for distributed event-based applications and can replace custom code previously used to manage routing results.
Lambda also now supports setting a Parallelization Factor, which allows you to set multiple Lambda invocations per shard for Kinesis Data Streams and DynamoDB Streams. This enables faster processing without the need to increase your shard count, while still guaranteeing the order of records processed.
Lambda introduced Amazon SQS FIFO queues as an event source. “First in, first out” (FIFO) queues guarantee the order of record processing, unlike standard queues. FIFO queues support messaging batching via a MessageGroupID attribute that supports parallel Lambda consumers of a single FIFO queue, enabling high throughput of record processing by Lambda.
Lambda now supports Environment Variables in the AWS China (Beijing) Region and the AWS China (Ningxia) Region.
You can now view percentile statistics for the duration metric of your Lambda functions. Percentile statistics show the relative standing of a value in a dataset, and are useful when applied to metrics that exhibit large variances. They can help you understand the distribution of a metric, discover outliers, and find hard-to-spot situations that affect customer experience for a subset of your users.
Amazon API Gateway
Amazon API Gateway announced the preview of HTTP APIs. In addition to significant performance improvements, most customers see an average cost savings of 70% when compared with API Gateway REST APIs. With HTTP APIs, you can create an API in four simple steps. Once the API is created, additional configuration for CORS and JWT authorizers can be added.
AWS SAM CLI
The AWS SAM CLI team simplified the bucket management and deployment process in the SAM CLI. You no longer need to manage a bucket for deployment artifacts – SAM CLI handles this for you. The deployment process has also been streamlined from multiple flagged commands to a single command, sam deploy.
AWS Step Functions
One powerful feature of AWS Step Functions is its ability to integrate directly with AWS services without you needing to write complicated application code. In Q4, Step Functions expanded its integration with Amazon SageMaker to simplify machine learning workflows. Step Functions also added a new integration with Amazon EMR, making EMR big data processing workflows faster to build and easier to monitor.
Step Functions now provides the ability to track state transition usage by integrating with AWS Budgets, allowing you to monitor trends and react to usage on your AWS account.
You can now view CloudWatch Metrics for Step Functions at a one-minute frequency. This makes it easier to set up detailed monitoring for your workflows. You can use one-minute metrics to set up CloudWatch Alarms based on your Step Functions API usage, Lambda functions, service integrations, and execution details.
Step Functions now supports higher throughput workflows, making it easier to coordinate applications with high event rates. This increases the limits to 1,500 state transitions per second and a default start rate of 300 state machine executions per second in US East (N. Virginia), US West (Oregon), and Europe (Ireland). Click the above link to learn more about the limit increases in other Regions.
Step Functions released AWS Step Functions Express Workflows. With the ability to support event rates greater than 100,000 per second, this feature is designed for high-performance workloads at a reduced cost.
Amazon EventBridge announced the preview of the Amazon EventBridge schema registry and discovery service. This service allows developers to automate discovery and cataloging event schemas for use in their applications. Additionally, once a schema is stored in the registry, you can generate and download a code binding that represents the schema as an object in your code.
Amazon SNS now supports the use of dead letter queues (DLQ) to help capture unhandled events. By enabling a DLQ, you can catch events that are not processed and re-submit them or analyze to locate processing issues.
CloudWatch also announced a preview of a capability called Synthetics. CloudWatch Synthetics allows you to test your application endpoints and URLs using configurable scripts that mimic what a real customer would do. This enables the outside-in view of your customers’ experiences, and your service’s availability from their point of view.
CloudWatch introduced Embedded Metric Format, which helps you ingest complex high-cardinality application data as logs and easily generate actionable metrics. You can publish these metrics from your Lambda function by using the PutLogEvents API or using an open source library for Node.js or Python applications.
Finally, CloudWatch announced a preview of Contributor Insights, a capability to identify who or what is impacting your system or application performance by identifying outliers or patterns in log data.
AWS X-Ray announced trace maps, which enable you to map the end-to-end path of a single request. Identifiers show issues and how they affect other services in the request’s path. These can help you to identify and isolate service points that are causing degradation or failures.
X-Ray also announced support for Amazon CloudWatch Synthetics, currently in preview. CloudWatch Synthetics on X-Ray support tracing canary scripts throughout the application, providing metrics on performance or application issues.
Amazon DynamoDB announced support for customer-managed customer master keys (CMKs) to encrypt data in DynamoDB. This allows customers to bring your own key (BYOK) giving you full control over how you encrypt and manage the security of your DynamoDB data.
It is now possible to add global replicas to existing DynamoDB tables to provide enhanced availability across the globe.
Another new DynamoDB capability to identify frequently accessed keys and database traffic trends is currently in preview. With this, you can now more easily identify “hot keys” and understand usage of your DynamoDB tables.
DynamoDB also released adaptive capacity. Adaptive capacity helps you handle imbalanced workloads by automatically isolating frequently accessed items and shifting data across partitions to rebalance them. This helps reduce cost by enabling you to provision throughput for a more balanced workload instead of over provisioning for uneven data access patterns.
The RDS Proxy maintains a pool of established connections to your RDS database instances. This pool enables you to support a large number of application connections so your application can scale without compromising performance. It also increases security by enabling IAM authentication for database access and enabling you to centrally manage database credentials using AWS Secrets Manager.
AWS Serverless Application Repository
The AWS Serverless Application Repository (SAR) now offers Verified Author badges. These badges enable consumers to quickly and reliably know who you are. The badge appears next to your name in the SAR and links to your GitHub profile.
AWS Developer Tools
AWS CodeCommit launched the ability for you to enforce rule workflows for pull requests, making it easier to ensure that code has pass through specific rule requirements. You can now create an approval rule specifically for a pull request, or create approval rule templates to be applied to all future pull requests in a repository.
AWS CodeBuild added beta support for test reporting. With test reporting, you can now view the detailed results, trends, and history for tests executed on CodeBuild for any framework that supports the JUnit XML or Cucumber JSON test format.
AWS announced a preview of Amazon CodeGuru at re:Invent 2019. CodeGuru is a machine learning based service that makes code reviews more effective and aids developers in writing code that is more secure, performant, and consistent.
AWS Amplify and AWS AppSync
The Amplify team has also improved offline data access and synchronization by announcing Amplify DataStore. Developers can now create applications that allow users to continue to access and modify data, without an internet connection. Upon connection, the data synchronizes transparently with the cloud.
For a summary of Amplify and AppSync announcements before re:Invent, read: “A round up of the recent pre-re:Invent 2019 AWS Amplify Launches”.
Q4 serverless content
- Improving the Getting Started experience with AWS Lambda
- Analyzing and visualizing AWS Global Accelerator flow logs using Amazon Athena and Amazon QuickSight
- Generating dynamic error responses in Amazon CloudFront with [email protected]
- Update: Issue affecting HashiCorp Terraform resource deletions after the VPC Improvements to AWS Lambda
- Implementing priority queueing with Amazon DynamoDB
- Performing Route 53 health checks on private resources in a VPC with AWS Lambda and Amazon CloudWatch
- Designing durable serverless apps with DLQs for Amazon SNS, Amazon SQS, AWS Lambda
- Building a serverless weather bot with two-way SMS, AWS SAM, and AWS Lambda
- Automating Zendesk With Amazon EventBridge and AWS Step Functions
- Tracking the state of AWS Lambda functions
- Handling [email protected] Part 1
- Handling [email protected] Part 2
- New – Using Step Functions to Orchestrate Amazon EMR Workloads
- Simplifying Serverless CI/CD with CircleCI and the AWS Serverless Application Model
- Understanding asynchronous messaging for microservices
- Prepare data for model-training and invoke machine learning models with Amazon Athena
- Decoupled Serverless Scheduler to Run HPC Applications at Scale on EC2
- EventBridge Support in Amazon Elastic Container Registry
- ICYMI: Serverless pre:Invent 2019
- Coming soon: Updated Lambda states lifecycle for VPC networking
- Integrating B2B using event notifications with Amazon SNS
- ICYMI: Serverless pre:Invent 2019
- ICYMI: Serverless re:Invent re:Cap 2019
- Check out The Amazon Builders’ Library – This is How We Do It!
We hold several AWS Online Tech Talks covering serverless tech talks throughout the year. These are listed in the Serverless section of the AWS Online Tech Talks page.
Here are the ones from Q4:
- Deep Dive on Amazon EventBridge
- Advanced Serverless Orchestration with AWS Step Functions
- Mocking and Testing Serverless APIs with AWS Amplify
- How to Build Highly Scalable Serverless Applications with Amazon Aurora Serverless
- Managing Serverless Applications with SAM Templates
- Using Relational Databases with AWS Lambda – Easy Connection Pooling
- Automate Code Reviews and Application Performance Recommendations with Amazon CodeGuru
There are also a number of other helpful video series covering Serverless available on the AWS Twitch Channel.
AWS Serverless Heroes
We are excited to welcome some new AWS Serverless Heroes to help grow the serverless community. We look forward to some amazing content to help you with your serverless journey.
- Brian LeRoux, co-founder and CTO of Begin.com
- Farrah Campbell, Ecosystems Director at Stackery
- Sebastian Müller, Senior Cloud Consultant at superluminar
- Special mention to Alex DeBrie, Engineering Manager at Serverless, Inc. – named as a Data Hero
AWS Serverless Application Repository (SAR) Apps
In this edition of ICYMI, we are introducing a section devoted to SAR apps written by the AWS Serverless Developer Advocacy team. You can run these applications and review their source code to learn more about serverless and to see examples of suggested practices.
Still looking for more?
The Serverless landing page has much more information. The Lambda resources page contains case studies, webinars, whitepapers, customer stories, reference architectures, and even more Getting Started tutorials. We’re also kicking off a fresh series of Tech Talks in 2020 with new content providing greater detail on everything new coming out of AWS for serverless application developers.
Throughout 2020, the AWS Serverless Developer Advocates are crossing the globe to tell you more about serverless, and to hear more about what you need. Follow this blog to keep up on new launches and announcements, best practices, and examples of serverless applications in action.
You can also follow all of us on Twitter to see latest news, follow conversations, and interact with the team.