Tag Archives: AWS Proton

New – AWS Proton Supports Terraform and Git Repositories to Manage Templates

Post Syndicated from Marcia Villalba original https://aws.amazon.com/blogs/aws/new-aws-proton-supports-terraform-and-git-repositories-to-manage-templates/

Today we are announcing the launch of two features for AWS Proton. First, the most requested one in the AWS Proton open roadmap, to define and provision infrastructure using Terraform. Second, the capability to manage AWS Proton templates directly from Git repositories.

AWS Proton is a fully managed application delivery service for containers and serverless applications, announced during reinvent 2020. AWS Proton aims to help infrastructure teams automate and manage their infrastructure without impacting developer productivity. It allows developers to get the templates they need to deliver their applications without the need to involve the platform team.

When using AWS Proton, the infrastructure team needs to define the environment and the service templates. Learn more about the templates.

Template Sync
This new feature in AWS Proton enables the platform team to push, update, and publish templates directly from their Git repositories. Now when you create a new service or environment template, you can specify a remote Git repository containing the templates. AWS Proton will automatically sync those templates and make them available for use. When there are changes to the Git repository, AWS Proton will take care of the updates.

Create enviroment template

One important advantage of using repositories and syncing the templates is that it simplifies the process of the administrators for uploading, updating, and registering the templates. This process, when done manually, can be error-prone and inconvenient. Now you can automate the process of authoring and updating the templates. Also, you can add more validations using pull requests and track the changes to the templates.

Template sync allows collaboration between the platform team and the developers. By having all the templates in a Git repository, all the collaboration tooling available in platforms like GitHub becomes available to everybody. Now developers can see all the templates, and when they want to improve them, they can just create a pull request with the changes. In addition, tools like bug trackers and features requests can be used to manage the templates.

Configuring the Repository Link
To get started using template sync, you need to give AWS Proton permissions to access your repositories. For that, you need to create a link between AWS Proton and your repository.

To do this, first create a new source connection for your GitHub account. Then you need to create a new repository link from the AWS Proton. Go to the Repositories option in the side bar. Then in the Link new repository screen, use the GitHub connection that you just created and specify a repository name.

Create new link repository

AWS Proton supports Terraform
Until now, AWS CloudFormation was the only infrastructure as code (IaC) engine available in AWS Proton. Now you can define service and environment templates based on infrastructure defined using Terraform and through a pull-request-based mechanism, use Terraform to provision and keep your infrastructure updated.

Platforms teams author their IaC templates in HCL, the Terraform language, and then provision the infrastructure using Terraform Open Source. AWS Proton renders the ready-to-provision Terraform module and makes a pull request to your infrastructure repository, from where you can plan and apply the changes.

This operation is asynchronous, as AWS Proton is not the one managing the provision of infrastructure. Therefore it is important that in the process of provisioning the infrastructure, there is a step that notifies AWS Proton of the status of the deployment.

I want to show you a demo on how you can set up an environment using Terraform. For that, you will use GitHub actions to provision the Terraform infrastructure in your AWS account.

To get started with Terraform templates, first, configure the repository link as it was described before. Then you need to create a new role to give permissions to GitHub actions to perform some activities in your AWS account. You can find the AWS CloudFormation template for this role here.

Create an empty GitHub repository and create a folder .github/workflows/. Create a file called terraform.yml. In that file, you need to define the GitHub actions to plan and apply the infrastructure changes. Copy the template from the terraform example file.

This template configures your AWS credentials, configures Terraform, plans the whole infrastructure, and applies the changes in the infrastructure using Terraform, and then notifies AWS Proton on the status of this process.

In addition, you need to modify the file env_config.json, which is located inside that folder. In that file, you need to add the configuration for the environment you plan to create. You can append new environments to the JSON file. In the example, the environment is called tf-test. The role is the role you created previously, and the region is the region where you want to deploy this infrastructure. Look at the example file.

{
    “tf-test”: {
        “role”: “arn:aws:iam::123456789:role/TerraformGitHubActionsRole”,
        “region”: “us-west-2”
    }
}

For this example, you upload the Terraform project to Amazon S3. See an example of a Terraform project.

Now it is time to create a new environment template in AWS Proton. You can follow the instructions in the console.

When your environment template is ready, create a new environment using the template you just created. When configuring the environment, select Provision through pull request and then configure the repository with the correct parameters.

Configure new enviromentNow, in the Environment details, you can see the Deployment status to be In progress. This will stay like this until the GitHub action finishes.

Environment details

If you go to your repository, you should see a new pull request. Next to the pull request name, you will see a red cross, yellow dot, or green check. That icon depends on the status of the GitHub action. If you have a yellow dot, wait for it to turn red or green. If there is an error, you need to see what is going on inside the logs of the GitHub action.

If you see a green check on the pull request, it means that the GitHub actions has completed, and the pull request can be merged. After the pull request is merged, the infrastructure is provisioned. Go back to the Environment Details page. After a while, and once your infrastructure is provisioned, which can take some minutes depending on your template, you should see that the Deployment Status is Successful.

Github pull request

By the end of this demo, you have provisioned your infrastructure using AWS Proton to handle the environment templates and GitHub actions, and Terraform Open Source to provision the infrastructure in your AWS account.

Availability
Terraform support is available in public preview mode.

These new features are available in the regions where AWS Proton is available: US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Tokyo), and Europe (Ireland).

To learn more about these features, visit the AWS Proton service page.

Marcia

Simplifying Multi-account CI/CD Deployments using AWS Proton

Post Syndicated from Marvin Fernandes original https://aws.amazon.com/blogs/architecture/simplifying-multi-account-ci-cd-deployments-using-aws-proton/

Many large enterprises, startups, and public sector entities maintain different deployment environments within multiple Amazon Web Services (AWS) accounts to securely develop, test, and deploy their applications. Maintaining separate AWS accounts for different deployment stages is a standard practice for organizations. It helps developers limit the blast radius in case of failure when deploying updates to an application, and provides for more resilient and distributed systems.

Typically, the team that owns and maintains these environments (the platform team) is segregated from the development team. A platform team performs critical activities. These can include setting infrastructure and governance standards, keeping patch levels up to date, and maintaining security and monitoring standards. Development teams are responsible for writing the code, performing appropriate testing, and pushing code to repositories to initiate deployments. The development teams are focused more on delivering their application and less on the infrastructure and networking that ties them together. The segregation of duties and use of multi-account environments are effective from a regulatory and development standpoint. But monitoring, maintaining, and enabling the safe release to these environments can be cumbersome and error prone.

In this blog, you will see how to simplify multi-account deployments in an environment that is segregated between platform and development teams. We will show how you can use one consistent and standardized continuous delivery pipeline with AWS Proton.

Challenges with multi-account deployment

For platform teams, maintaining these large environments at different stages in the development lifecycle and within separate AWS accounts can be tedious. The platform teams must ensure that certain security and regulatory requirements (like networking or encryption standards) are implemented in each separate account and environment. When working in a multi-account structure, AWS Identity and Access Management (IAM) permissions and cross-account access management can be a challenge for many account administrators. Many organizations rely on specific monitoring metrics and tagging strategies to perform basic functions. The platform team is responsible for enforcing these processes and implementing these details repeatedly across multiple accounts. This is a pain point for many infrastructure administrators or platform teams.

Platform teams are also responsible for ensuring a safe and secure application deployment pipeline. To do this, they isolate deployment and production environments from one another limiting the blast radius in case of failure. Platform teams enforce the principle of least privilege on each account, and implement proper testing and monitoring standards across the deployment pipeline.

Instead of focusing on the application and code, many developers face challenges complying with these rigorous security and infrastructure standards. This results in limited access to resources for developers. Delays come with reliance on administrators to deploy application code into production. This can lead to lags in deployment of updated code.

Deployment using AWS Proton

The ownership for infrastructure lies with the platform teams. They set the standards for security, code deployment, monitoring, and even networking. AWS Proton is an infrastructure provisioning and deployment service for serverless and container-based applications. Using AWS Proton, the platform team can provide their developers with a highly customized and catered “platform as a service” experience. This allows developers to focus their energy on building the best application, rather than spending time on orchestration tools. Platform teams can similarly focus on building the best platform for that application.

With AWS Proton, developers use predefined templates. With only a few input parameters, infrastructure can be provisioned and code deployed in an effective pipeline. This way you can get your application running and updated more quickly, see Figure 1.

Figure 1. Platform and development team roles when using AWS Proton

Figure 1. Platform and development team roles when using AWS Proton

AWS Proton allows you to deploy any serverless or container-based application across multiple accounts. You can define infrastructure standards and effective continuous delivery pipelines for your organization. Proton breaks down the infrastructure into environment and service (“infrastructure as code” templates).

In Figure 2, platform teams provide a service template of a secure environment to host a microservices application on Amazon Elastic Container Service (Amazon ECS) and AWS Fargate. The environment template contains infrastructure that is shared across services. This includes the networking configuration: Amazon Virtual Private Cloud (VPC), subnets, route tables, Internet Gateway, security groups, and ECS cluster definition for the Fargate service.

The service template provides details of the service. It includes the container task definitions, monitoring and logging definitions, and an effective continuous delivery pipeline. Using the environment and service template definitions, development teams can define the microservices that are running on Amazon ECS. They can deploy their code following the continuous integration and continuous delivery (CI/CD) pipeline.

Figure 2. Platform teams provision environment and service infrastructure as code templates in AWS Proton management account

Figure 2. Platform teams provision environment and service infrastructure as code templates in AWS Proton management account

Multi-account CI/CD deployment

For Figures 3 and 4, we used publicly available templates and created three separate AWS accounts: the AWS Proton management account, development account, and production environment accounts. Additional accounts may be added based on your use case and security requirements. As shown in Figure 3, the AWS Proton service account contains the environment, service, and pipeline templates. It also provides the connection to other accounts within the organization. The development and production accounts follow the structure of a development pipeline for a typical organization.

AWS Proton alleviates complicated cross-account policies by using a secure “environment account connection” feature. With environment account connections, platform administrators can give AWS Proton permissions to provision infrastructure in other accounts. They create an IAM role and specify a set of permissions in the target account. This enables Proton to assume the role from the management account to build resources in the target accounts.

AWS Key Management Service (KMS) policies can also be hard to manage in multi-account deployments. Proton reduces managing cross-account KMS permissions. In an AWS Proton management account, you can build a pipeline using a single artifact repository. You can also extend the pipeline to additional accounts from a single source of truth. This feature can be helpful when accounts are located in different Regions, due to regulatory requirements for example.

Figure 3. AWS Proton uses cross-account policies and provisions infrastructure in development and production accounts with environment connection feature

Figure 3. AWS Proton uses cross-account policies and provisions infrastructure in development and production accounts with environment connection feature

Once the environment and service templates are defined in the AWS Proton management account, the developer selects the templates. Proton then provisions the infrastructure, and the continuous delivery pipeline that will deploy the services to each separate account.

Developers commit code to a repository, and the pipeline is responsible for deploying to the different deployment stages. You don’t have to worry about any of the environment connection workflows. Proton allows platform teams to provide a single pipeline definition to deploy the code into multiple different accounts without any additional account level information. This standardizes the deployment process and implements effective testing and staging policies across the organization.

Platform teams can also inject manual approvals into the pipeline so they can control when a release is deployed. Developers can define tests that initiate after a deployment to ensure the validity of releases before moving to a production environment. This simplifies application code deployment in an AWS multi-account environment and allows updates to be deployed more quickly into production. The resulting deployed infrastructure is shown in Figure 4.

Figure 4. AWS Proton deploys service into multi-account environment through standardized continuous delivery pipeline

Figure 4. AWS Proton deploys service into multi-account environment through standardized continuous delivery pipeline

Conclusion

In this blog, we have outlined how using AWS Proton can simplify handling multi-account deployments using one consistent and standardized continuous delivery pipeline. AWS Proton addresses multiple challenges in the segregation of duties between developers and platform teams. By having one uniform resource for all these accounts and environments, developers can develop and deploy applications faster, while still complying with infrastructure and security standards.

For further reading:

Getting started with Proton
Identity and Access Management for AWS Proton
Proton administrative guide