Post Syndicated from Eric Johnson original https://aws.amazon.com/blogs/compute/working-with-aws-lambda-and-lambda-layers-in-aws-sam/

The introduction of serverless technology has enabled developers to shed the burden of managing infrastructure and concentrate on their application code. AWS Lambda has taken on that management by providing isolated, event-driven compute environments for the execution of application code. To use a Lambda function, a developer just needs to package their code and any dependencies into a zip file and upload that file to AWS. However, as serverless applications get larger and more functions are required for those applications, there is a need for the ability to share code across multiple functions within the application.

To meet this need, AWS released Lambda layers, providing a mechanism to externally package dependencies that can be shared across multiple Lambda functions. Lambda layers reduces lines of code and size of application artifacts and simplifies dependency management. Along with the release of Lambda layers, AWS also released support for layers in the AWS Serverless Application Model (SAM) and the AWS SAM command line interface (CLI). SAM is a template specification that enables developers to define a serverless application in clean and simple syntax. The SAM CLI is a command line tool that operates on SAM templates and application code. SAM can now define Lambda layers with the AWS::Serverless::LayerVersion type. The SAM CLI can build and test your layers locally as well as package, deploy, and publish your layers for public consumption.

How layers work

To understand how SAM CLI supports layers, you need to understand how layers work on AWS. When a Lambda function configured with a Lambda layer is executed, AWS downloads any specified layers and extracts them to the /opt directory on the function execution environment. Each runtime then looks for a language-specific folder under the /opt directory.

Lambda layers can come from multiple sources. You can create and upload your own layers for sharing, you can implement an AWS managed layer such as SciPi, or you can grab a third-party layer from an APN Partner or another trusted developer. The following image shows how layers work with multiple sources.

How layers work in the AWS SAM CLI

To support Lambda layers, SAM CLI replicates the AWS layer process locally by downloading all associated layers and caching them on your development machine. This happens the first time you run sam local invoke or the first time you execute your Lambda functions using sam local start-lambda or sam local start-api.

Two specific flags in SAM CLI are helpful when you’re working with Lambda layers locally. To specify where the layer cache should be located, pass the –layer-cache-basedir flag, followed by your desired cache directory. To force SAM CLI to rebuild the layer cache, pass the –force-image-build flag.

Time for some code

Now you’re going to create a simple application that does some temperature conversions using a simple library named temp-units-conv. After the app is running, you move the dependencies to Lambda layers using SAM. Finally, you add a layer managed by an AWS Partner Network Partner, Epsagon, to enhance the monitoring of the Lambda function.

Creating a serverless application

To create a serverless application, use the SAM CLI. If you don’t have SAM CLI installed, see Installing the AWS SAM CLI in the AWS Serverless Application Model Developer Guide.

1. To initialized a new application, run the following command.

   $ sam init -r nodejs8.10

   This creates a simple node application under the directory sam-app that looks like this.

   $ tree sam-app
   sam-app
   ├── README.md
   ├── hello-world
   │   ├── app.js
   │   ├── package.json
   │   └── tests
   └── template.yaml

   The template.yaml file is a SAM template describing the infrastructure for the application, and the app.js file contains the application code.

2. To install the dependencies for the application, run the following command from within the sam-app/hello-world directory.

   $ npm install temp-units-conv

3. The application is going to perform temperature scale conversions for Celsius, Fahrenheit, and Kelvin using the following code. In a code editor, open the file sam-app/hello-world/app.js and replace its contents with the following.

   const tuc = require('temp-units-conv');
   let response;
   
   const scales = {
       c: "celsius",
       f: "fahrenheit",
       k: "kelvin"
   }
   
   exports.lambdaHandler = async (event) => {
       let conversion = event.pathParameters.conversion
       let originalValue = event.pathParameters.value
       let answer = tuc[conversion](originalValue)
       try {
           response = {
               'statusCode': 200,
               'body': JSON.stringify({
                   source: scales[conversion[0]],
                   target: scales[conversion[2]],
                   original: originalValue,
                   answer: answer
               })
           }
       } catch (err) {
           console.log(err);
           return err;
       }
   
       return response
   };

4. Update the SAM template. Open the sam-app/template.yaml file. Replace the contents with the following. This is a YAML file, so spacing and indentation is important.

   AWSTemplateFormatVersion: '2010-09-09'
   Transform: AWS::Serverless-2016-10-31
   Description: sam app
   Globals:
       Function:
           Timeout: 3
           Runtime: nodejs8.10
   
   Resources:
       TempConversionFunction:
           Type: AWS::Serverless::Function 
           Properties:
               CodeUri: hello-world/
               Handler: app.lambdaHandler
               Events:
                   HelloWorld:
                       Type: Api
                       Properties:
                           Path: /{conversion}/{value}
                           Method: get

   This change dropped out some comments and output parameters and updated the function resource to TempConversionFunction. The primary change is the Path: /{conversion}/{value} line. This enables you to use path mapping for our conversion type and value.

5. Okay, now you have a simple app that does temperature conversions. Time to spin it up and make sure that it works. In the sam-app directory, run the following command.

   $ sam local start-api

   

6. Using curl or your browser, navigate to the address output by the previous command with a conversion and value attached. For reference, c = Celsius, f = Fahrenheit, and k = Kelvin. Use the pattern c2f/ followed by the temperature that you want to convert.

   $ curl http://127.0.0.1:3000/f2c/45
   {"source":"fahrenheit","target":"celsius","original":"45","answer":7.222222222222222}

Deploying the application

Now that you have a working application that you have tested locally, deploy it to AWS. This enables the application to run on Lambda, providing a public endpoint that you can share with others.

1. Create a resource bucket. The resource bucket gives you a place to upload the application so that AWS CloudFormation can access it when you run the deploy process. Run the following command to create a resource bucket.

   $ aws s3api create-bucket –bucket <your unique bucket name>

2. Use SAM to package the application. From the sam-app directory, run the following command.

   $ sam package --template-file template.yaml --s3-bucket <your bucket> --output-template-file out.yaml

3. Now you can use SAM to deploy the application. Run the following command from the sam-app folder.

   $ sam deploy --template-file ./out.yaml --stack-name <your stack name> --capabilities CAPABILITY_IAM

Sign in to the AWS Management Console. Navigate to the Lambda console to find your function.

Now that the application is deployed, you can access it via the API endpoint. In the Lambda console, click on the API Gateway option and scroll down. You will find a link to your API Gateway endpoint.

Using that value, you can test the live application. Your endpoint will be different from the one in the following image.

Let’s take a moment to talk through the structure of our new application. Because you installed temp-units-conv, there is a dependency folder named sam-app`hello-world/node_modules that you need to include when you upload the application.

$ tree sam-app
sam-app
├── README.md
├── hello-world
│   ├── app.js
│   ├── node_modules
│   ├── package-lock.json
│   ├── package.json
│   └── tests
└── template.yaml

Because you’re a node user, you can use something like webpack to minimize your uploads. However, this requires a processing step to pack your code, and it still forces you to upload unchanging, static code on every update. To simplify this, create a layer to separate the dependencies from the application code.

Creating a layer

To create and manage the dependency layer, you need to update your directory structure a bit.

$ tree sam-app
sam-app
├── README.md
├── dependencies
│   └── nodejs
│       └── package.json
├── hello-world
│   ├── app.js
│   └── tests
│       └── unit
└── template.yaml

In the root, create a new directory named dependencies. Under that directory, create a second directory named nodejs. This is the structure required for layers to be injected into a Lambda function. Next, move the package.json file from the hello-world directory to the dependencies/nodejs directory. Finally, clean up the hello-world directory by deleting the node_modules folder and the package-lock.json file.

Before you gather your dependencies, edit the sam-app/dependencies/nodejs/pakage.json file. Replace the entire contents with the following.

{
  "dependencies": {
    "temp-units-conv": "^1.0.2"
  }
}

Now that you have the package file cleaned up, install the required packages into the dependencies directory. From the sam-app/dependencies/nodejs directory, run the following command.

$ npm install

You now have a node_modules directory under the nodejs directory. With this in place, you have everything in place to create your first layer using SAM.

The next step is to update the AWS SAM template. Replace the contents of your sam-app/template.yaml file with the following.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: sam app
Globals:
    Function:
        Timeout: 3
        Runtime: nodejs8.10

Resources:
    TempConversionFunction:
        Type: AWS::Serverless::Function 
        Properties:
            CodeUri: hello-world/
            Handler: app.lambdaHandler
            Layers:
              - !Ref TempConversionDepLayer
            Events:
                HelloWorld:
                    Type: Api
                    Properties:
                        Path: /{conversion}/{value}
                        Method: get

    TempConversionDepLayer:
        Type: AWS::Serverless::LayerVersion
        Properties:
            LayerName: sam-app-dependencies
            Description: Dependencies for sam app [temp-units-conv]
            ContentUri: dependencies/
            CompatibleRuntimes:
              - nodejs6.10
              - nodejs8.10
            LicenseInfo: 'MIT'
            RetentionPolicy: Retain

There are two changes to the template. The first is a new resource named TempConversionDepLayer, which defines the new layer and points to the dependencies folder as the code source for the layer. The second is the addition of the Layers parameter in the TempConversionFunction resource. The single layer entry references the layer that is being created in the template file.

With that final change, you have separated the application code from the dependencies. Try out the application and see if it still works. From the sam-app directory, run the following command.

$ sam local start-api

If all went well, you can open your browser back up and try another conversion.

One other thing to note here is that you didn’t have to change our application code at all. As far as the application is concerned, nothing has changed. However, under the hood, SAM CLI is doing a bit of magic. SAM CLI is creating an image of the layer and caching it locally. It then makes that layer available in the /opt directory on the container being used to execute the Lambda function locally.

Using layers from APN Partners and third parties

So far, you have used a layer of your own creation. Now you’re going to branch out and add a managed layer by an APN Partner, Epsagon, who provides a tool to help with monitoring and troubleshooting your serverless applications. If you want to try this demo, you can sign up for a free trial on their website. After you create an account, you need to get the Epsagon token from the Settings page of your dashboard.

1. Add Epsagon layer reference. Edit the sam-app/template.yaml file. Update the Layers section of the TempConversionFunction resource to the following.

   Layers:
     - !Ref TempConversionDepLayer
     - arn:aws:lambda:us-east-1:066549572091:layer:epsagon-node-layer:1
   

   Note: This demo uses us-east-1 for the AWS Region. If you plan to deploy your Lambda function to a different Region, update the Epsagon LayerVersion Amazon Resource Name (ARN) accordingly. For more information, see the Epsagon blog post on layers.

2. To use the Epsagon library in our code, you need to add or modify nine lines of code. You reference and initialize the library, wrap the handler with the Epsagon library, and modify the output. Open the sam-app/hello-world/app.js file and replace the entire contents with the following. The changes are highlighted. Be sure to update 1122334455 with your token from Epsagon.

   const tuc = require('temp-units-conv');
   const epsagon = require('epsagon');
   epsagon.init({
       token: '1122334455',
       appName: 'layer-demo-app',
       metadataOnly: false, // Optional, send more trace data
   });
   
   let response;
   
   const scales = {
       c: "celsius",
       f: "fahrenheit",
       k: "kelvin"
   }
   
   exports.lambdaHandler = epsagon.lambdaWrapper((event, context, callback) => {
       let conversion = event.pathParameters.conversion
       let originalValue = event.pathParameters.value
       let answer = tuc[conversion](originalValue)
       try {
           response = {
               'statusCode': 200,
               'body': JSON.stringify({
                   source: scales[conversion[0]],
                   target: scales[conversion[2]],
                   original: originalValue,
                   answer: answer
               })
           }
       } catch (err) {
           console.log(err);
           return err;
       }
   
       callback(null, response)
   });

Test the change to make sure that everything still works. From the sam-app directory, run the following command.

$ sam cli start-api

Use curl to test your code.

$ curl http://127.0.0.1:3000/k2c/100

Your answer should be the following.

{"source":"kelvin","target":"celsius","original":"100","answer":-173.14999999999998}

Your Epsagon dashboard should display traces from your Lambda function, as shown in the following image.

Deploying the application with layers

Now that you have a functioning application that uses Lambda layers, you can package and deploy it.

1. To package the application, run the following command.

   $ sam package --template-file template.yaml --s3-bucket <your bucket> --output-template-file out.yaml

2. To deploy the application, run the following command.

   $ sam deploy --template-file ./out.yaml --stack-name <your stack name> --capabilities CAPABILITY_IAM

The Lambda console for your function has updated, as shown in the following image.

Also, the dependency code isn’t in your code environment in the Lambda function.

There you have it! You just deployed your Lambda function and your dependencies layer for that function. It’s important to note that you did not publish the Epsagon layer. You just told AWS to grab their layer and extract it in to your function’s execution environment. The following image shows the flow of this process.

Options for managing layers

You have several options for managing your layers through AWS SAM.

First, following the pattern you just walked through releases a new version of your layer each time you deploy your application. If you remember, one of the advantages of using layers is not having to upload the dependencies each time. One option to avoid this is to keep your dependencies in a separate template and deploy them only when the dependencies have changed.

Second, this pattern always uses the latest build of dependencies. If for some reason you want to get a specific version of your dependencies, you can. After you deploy the application for the first time, you can run the following command.

$ aws lambda list-layer-versions --layer-name sam-app-depedencies

You should see a response like the following.

{
    "LayerVersions": [
        {
            "LayerVersionArn": "arn:aws:lambda:us-east-1:5555555:layer:sam-app-dependencies:1",
            "Version": 1,
            "Description": "Dependencies for sam app",
            "CreatedDate": "2019-01-08T18:04:51.833+0000",
            "CompatibleRuntimes": [
                "nodejs6.10",
                "nodejs8.10"
            ],
            "LicenseInfo": "MIT"
        }
    ]
}

The critical information here is the LayerArnVersion. Returning to the sam-app/template.yaml file, you can change the Layers section of the TempConversionFunction resource to use this version.

Layers:
  - arn:aws:lambda:us-east-1:5555555:layer:sam-app-dependencies:1
  - arn:aws:lambda:us-east-1:066549572091:layer:epsagon-node-layer:1

Conclusion

This blog post demonstrates how AWS SAM can manage Lambda layers via AWS SAM templates. It also demonstrates how the AWS SAM CLI creates a local development environment that provides layer support without any changes in the application.

Developers are often taught to think of code in an object-oriented manner and to code in a DRY way (don’t repeat yourself). For developers of serverless applications, these practices remain true. As serverless applications grow in size and require more Lambda functions, using Lambda layers provides an efficient mechanism to reuse code and libraries. Layers also reduce the size of your upload packages, making iterations faster.

We’re excited to see what you do with layers and to hear how AWS SAM is helping you. As always, we welcome your feedback.

Now go code!