Post Syndicated from Dan Blanco original https://aws.amazon.com/blogs/devops/exploring-fnforeach-and-fnfindinmap-enhancements-in-aws-cloudformation/
AWS CloudFormation, an Infrastructure as Code (IaC) service that lets you model, provision, and manage AWS and third-party resources, recently released a new language transform that enhances the core CloudFormation language. Today, we’ll be covering two more enhancements we’ve added since our initial release: Fn::FindInMap enhancements and a new looping function – Fn::ForEach.
These new language extensions are the result of open discussions with the larger CloudFormation community via our Request For Comments (RFC) proposals for new language features at our Language Discussion GitHub repository. We want to collaborate with the community to better align features and incorporate early feedback into the development cycle to meet the community’s needs. We invite you to participate in new RFCs to help shape the future of the CloudFormation language.
In this post, I’ll dive deeper into the new enhancements for Fn::FindInMap as well as explore the new Fn::ForEach looping mechanism and provide some examples.
Prerequisites
To use these new language features, you must add AWS::LanguageExtensions to the transform section of your template.
---
AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::LanguageExtensions'
If you have a list of transforms, then we recommend having AWS managed transforms at the end, and AWS::LanguageExtensions must be listed before AWS::Serverless.
---
AWSTemplateFormatVersion: 2010-09-09
Transform:
- 'AWS::LanguageExtensions'
- 'AWS::Serverless-2016-10-31'
This transform will cover all of the existing and future language extensions.
FindInMap enhancements
We have updated the language extension transform for CloudFormation to support Fn::FindInMap enhancements, that extend the existing functionality of the Fn::FindInMap intrinsic function so that now you can:
- use an optional, default value in Fn::FindInMap parameters, if a given key in a
Mappings section is not found, and
- use a number of additional intrinsic functions in the parameters of
Fn::FindInMap; for more information, see Supported functions.
Let’s see an example use case where Fn::FindInMap enhancements can help you simplify the business logic of your template, and make it more readable and easier to maintain. Let’s suppose you create a CloudFormation template that describes an Amazon Elastic Compute Cloud (Amazon EC2) instance, and you need to use smaller EC2 instance types for pre-production environments, and a larger EC2 instance type for production for cost savings. In this example, you choose a t2.micro instance type for the dev environment, t2.medium for the qa environment, and t2.large for the prod environment, that you start to describe as follows:
---
AWSTemplateFormatVersion: "2010-09-09"
Description: 'Sample template that describes usage for `Fn::FindInMap` enhancements'
Parameters:
Environment:
Description: Lifecycle environment.
Type: String
AllowedValues:
- sandbox
- dev
- qa
- prod
Default: dev
LatestAmiId:
Description: Region-specific image to use.
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id> Default: /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2
Mappings:
LifecycleEnvToInstanceType:
dev:
InstanceType: t2.micro
qa:
InstanceType: t2.medium
prod:
InstanceType: t2.large
You described instance types for each of your 3 lifecycle environments in the Mappings section, and you engineered your template to read environment names as input data from Environment in the Parameters section. Looking closer at Environment, you define another allowed value: sandbox, that in this example is an environment for developers to use for prototype testing only: you choose not to include this environment in the mapping you created, with the intent to do the same for any other non-formal environment (for example, a contributor’s personal environment). Next, powered by the new enhancements toFn::FindInMap, you assign a default value for environment names that are different than dev, qa, and prod; this way, the only change you’ll need to make in this context is a new value(s) to AllowedValues in Environment. You describe this business logic in your template, to which you add the sample code shown next:
Transform: AWS::LanguageExtensions
Resources: Ec2Instance: Type: AWS::EC2::Instance Properties: ImageId: !Ref 'LatestAmiId' InstanceType: !FindInMap - LifecycleEnvToInstanceType - !Ref 'Environment' - InstanceType - DefaultValue: t2.micro Tags: - Key: test Value: test
In the snippet above, you have declared the AWS::LanguageExtensions transform, and described your configuration for an EC2 instance in the Resources section. For InstanceType, you chose to use Fn::FindInMap enhancements, and pass DefaultValue as an additional parameter with t2.micro as its value. When the user uses this template to create a stack, and chooses sandbox for Environment, the !Ref 'Environment' reference to the value for Environment will evaluate to sandbox, which is not present in the mapping you created: in this case, t2.micro will be used as a value for InstanceType.
These new enhancements also allow you to use more intrinsic functions inside of Fn::FindInMap. Let’s say you have received requirements to use -env as a suffix to environment names. You choose to make a minimal set of changes to your template, and start with the Parameters section as follows:
Parameters:
Environment:
Description: Lifecycle environment.
Type: String
AllowedValues:
- sandbox-env - dev-env - qa-env - prod-env Default: dev-env
Next, instead of modifying all of your keys the Mappings section, you choose to only change the second parameter to Fn::FindInMap enhancements as follows: first, you use the Fn::Split intrinsic function to split the user-selected environment value string (for example, dev-env) into a list of values using the ‘-‘ character as a delimiter, and next you use the Fn::Select intrinsic function to choose the first element (that is, 0) of that list:
Resources:
Ec2Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref 'LatestAmiId'
InstanceType: !FindInMap
- LifecycleEnvToInstanceType - !Select - 0 - !Split - '-' - !Ref 'Environment' - InstanceType - DefaultValue: t2.micro Tags:
- Key: test Value: test
With the updated code above, if the user selects the dev-env value for Environment, Fn::FindInMap enhancements will use dev as the second parameter when looking up values in the Mappings section.
Fn::ForEach intrinsic function
Another enhancement to the language extensions is the addition of native looping inside of CloudFormation with Fn::ForEach. Imagine you have a situation where you need three EC2 instances that look exactly the same. Currently, you would have to copy and paste each instance as a separate resource with CloudFormation:
---
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::LanguageExtensions
Resources:
FirstInstance:
Type: AWS::EC2::Instance
Properties:
SecondInstance:
Type: AWS::EC2::Instance
Properties:
ThirdInstance:
Type: AWS::EC2::Instance
Properties:
If you encounter the need to update one property (the AMI ID, for example), you will have to update all three separately. While this is trivially easy for our example, templates that extend into the hundreds of resources quickly becomes difficult to maintain.
With the Fn::ForEach language extension, we’re able to group all of these items together into an easy-to-manage snippet:
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::LanguageExtensions
Resources:
Fn::ForEach::Instances:
- InstanceLogicalId
- [FirstInstance, SecondInstance, ThirdInstance]
- ${InstanceLogicalId}:
Type: AWS::EC2::Instance
Properties:
# ..removed for brevity..
This results in the following output YAML, which is identical to our previous example:
---
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::LanguageExtensions
Resources:
FirstInstance:
Type: AWS::EC2::Instance
Properties:
SecondInstance:
Type: AWS::EC2::Instance
Properties:
ThirdInstance:
Type: AWS::EC2::Instance
Properties:
To break down the syntax, the Fn::ForEach function requires:
- A Logical ID for the looping function directly following the Fn::ForEach call. In our case, we named it
Instances
- The variable name we’ll be referencing in our snippet below
- The collection of strings we’ll be iterating over. You can write these inline, or pass them as parameters or mappings.
- A section of the template we’ll be iterating over using the variable name above. This is standard CloudFormation JSON/YAML.
These must be listed in an array immediately following the Fn::ForEach intrinsic function and in this exact order.
We can use our key to reference values found elsewhere in the template. This adds additional flexibility when combined with the aforementioned FindInMap enhancements. Imagine a similar scenario where each instance needs a specific instance type, dictated by which instance it is and which environment we’re in. As described before, we would add our parameters and mappings to our template:
---
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::LanguageExtensions
Parameters:
Environment:
Description: Lifecycle environment.
Type: String
AllowedValues:
- sandbox
- dev
- qa
- prod
Default: dev
LatestAmiId:
Description: Region-specific image to use.
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id> Default: /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2
Mappings:
dev:
FirstInstance:
InstanceType: t2.micro
SecondInstance:
InstanceType: t2.micro
ThirdInstance:
InstanceType: t2.micro
qa:
FirstInstance:
InstanceType: t2.medium
SecondInstance:
InstanceType: t2.medium
ThirdInstance:
InstanceType: t2.large
prod:
FirstInstance:
InstanceType: t2.large
SecondInstance:
InstanceType: t2.xlarge
ThirdInstance:
InstanceType: t2.2xlarge
Given this configuration, we have different environment values as Parameters and a Mapping section that details our sizing requirements for our instance. With this, we can then use our new Fn::ForEach functionality and FindInMap enhancements:
Resources:
Fn::ForEach::Instances:
- InstanceLogicalId - [FirstInstance, SecondInstance, ThirdInstance] - ${InstanceLogicalId}: Type: AWS::EC2::Instance
Properties:
ImageId: !Ref LatestAmiId
InstanceType: !FindInMap
- !Ref Environment - !Ref InstanceLogicalId - InstanceType - DefaultValue: t2.micro
This results in the following output:
Resources:
FirstInstance:
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref LatestAmiId
InstanceType: !FindInMap
- !Ref Environment
- FirstInstance
- InstanceType
- DefaultValue: t2.micro
SecondInstance:
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref LatestAmiId
InstanceType: !FindInMap
- !Ref Environment
- SecondInstance
- InstanceType
- DefaultValue: t2.micro
ThirdInstance:
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref LatestAmiId
InstanceType: !FindInMap
- !Ref Environment
- ThirdInstance
- InstanceType
- DefaultValue: t2.micro
This looping feature can be used to create more than just resources – say we want to reference outputs from these EC2 instances as we create them. We can modify our above template to add an Output section and iterate over it in the same way, as well as exporting the instance ID. We can even express more than one Output per iteration. We’ll also move the instances list to a parameter for increased clarity.
Parameters:
InstancesToManage:
Type: CommaDelimitedList
Description: Instances to be managed
Default: FirstInstance,SecondInstance,ThirdInstance
Outputs:
Fn::ForEach::InstanceOutputs:
- InstanceLogicalId
- !Ref InstancesToManage
- "${InstanceLogicalId}Id":
Export:
Name: !Sub ${AWS::AccountId}-${InstanceLogicalId}Id
Value: !Ref
Ref: InstanceLogicalId
"${InstanceLogicalId}AvailabilityZone":
Value:
Fn::GetAtt:
- !Ref InstanceLogicalId
- AvailabilityZone
This outputs to:
Outputs:
FirstInstanceId:
Export:
Name: !Sub ${AWS::AccountId}-FirstInstanceId
Value: !Ref FirstInstance
FirstInstanceAvailabilityZone:
Value:
Fn::GetAtt:
- FirstInstance
- AvailabilityZone
SecondInstanceId:
Export:
Name: !Sub ${AWS::AccountId}-SecondInstanceId
Value: !Ref SecondInstance
SecondInstanceAvailabilityZone:
Value:
Fn::GetAtt:
- SecondInstance
- AvailabilityZone
ThirdInstanceId:
Export:
Name: !Sub ${AWS::AccountId}-ThirdInstanceId
Value: !Ref ThirdInstance
ThirdInstanceAvailabilityZone:
Value:
Fn::GetAtt:
- ThirdInstance
- AvailabilityZone
In this snippet, we iterated over our collection and created multiple outputs. For each output, we concatenated our key with some other string. In this case, both Id and AvailabilityZone were concatenated with the key to create a unique output name based on the stack name and the logical ID of the resource.
Finally, loops can be nested inside other loops. Combined with the ability to concatenate values and do lookups inside of the Mapping section, we’re able to significantly simplify complex CloudFormation templates. Imagine an example where we are tasked with creating a Virtual Private Cloud (VPC) with three private subnets and three public subnets. This is a common configuration our customers have and we can configure it simply with looping.
---
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::LanguageExtensions
Parameters:
AvailabilityTypes:
Type: CommaDelimitedList
Description: Types of subnets availability - public, private, or both
AllowedValues:
- Public
- Private
Default: Public,Private
Mappings:
SubnetOne:
Public:
Cidr: 10.215.0.0/24
Private:
Cidr: 10.215.1.0/24
SubnetTwo:
Public:
Cidr: 10.215.2.0/24
Private:
Cidr: 10.215.3.0/24
SubnetThree:
Public:
Cidr: 10.215.4.0/24
Private:
Cidr: 10.215.5.0/24
Resources:
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.215.0.0/16
EnableDnsSupport: true
EnableDnsHostnames: true
Fn::ForEach::Subnets:
- SubnetIdentifier
- - SubnetOne
- SubnetTwo
- SubnetThree
- Fn::ForEach::SubnetAvailabilityType:
- AvailabilityType
- !Ref AvailabilityTypes
- "${SubnetIdentifier}${AvailabilityType}":
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
CidrBlock: !FindInMap
- !Ref SubnetIdentifier
- !Ref AvailabilityType
- Cidr
which outputs the following resource section:
Resources:
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref VpcCidr
EnableDnsSupport: true
EnableDnsHostnames: true
SubnetOnePublic:
Properties:
VpcId: !Ref VPC
CidrBlock: !FindInMap
- SubnetOne - Public - Cidr
SubnetOnePrivate:
Properties:
VpcId: !Ref VPC
CidrBlock: !FindInMap
- SubnetOne - Private - Cidr
SubnetTwoPublic:
Properties:
VpcId: !Ref VPC
CidrBlock: !FindInMap
- SubnetTwo - Public - Cidr
SubnetTwoPrivate:
Properties:
VpcId: !Ref VPC
CidrBlock: !FindInMap
- SubnetTwo - Private - Cidr
SubnetThreePublic:
Properties:
VpcId: !Ref VPC
CidrBlock: !FindInMap
- SubnetThree - Public - Cidr
SubnetThreePrivate:
Properties:
VpcId: !Ref VPC
CidrBlock: !FindInMap
- SubnetThree - Private - Cidr
Combining everything we’ve learned so far, this created six subnets total, three public, three private, and attached them to the respective VPC with a relevant CIDR block.
We’re excited to share this functionality with our community, and we invite you to share feedback on future enhancements to the looping functionality here. A few enhancements we’re discussing are:
- Iterating over a key/value pair
- Iterating over a list of lists
- Support in other template sections
- And more!
Please head over and let us know what you think!
Conclusion
In this post, we walked through the new CloudFormation additions to the language extensions transform, how to enable them in your templates, and how to engage in future language extensions via our open language discussion repository. Leave us your feedback at our Language Discussion GitHub repository to help shape the future of the CloudFormation language. We look forward to hearing from you!
About the Author:
