Post Syndicated from Matt Laver original https://aws.amazon.com/blogs/devops/aws-codebuild-managed-self-hosted-github-action-runners/
AWS CodeBuild now supports managed self-hosted GitHub Action runners, allowing you to build powerful CI/CD capabilities right beside your code and quickly implement a build, test and deploy pipeline. Last year AWS announced that customers can define their GitHub Actions steps within any phase of a CodeBuild buildspec file but with a self-hosted runner, jobs execute from GitHub Actions on GitHub.com to a system you deploy and manage.
With the recent announcement that AWS CodeBuild now supports managed GitHub Action runners, AWS can take care of managing the hosting of GitHub Action self-hosted runners within CodeBuild allowing teams to run their GitHub Actions workflow jobs natively within AWS.
For customers managing their self-hosted runners on their own infrastructure, CodeBuild can now provide a secure, scalable and lower latency solution. In addition, CodeBuild managed self-hosted GitHub Action runners bring features, such as:
- Reserved capacity to compile your software within your Amazon VPC and access resources such as Amazon Relational Database Service, Amazon ElastiCache, or any service endpoints that are only reachable from within a specific VPC.
- Available compute platforms including AWS Lambda, Windows, Linux, Linux GPU-enhanced and AWS Graviton Processors (Arm-based instances).
With the compute options available, customers can now run tests on hardware and operating system combinations that closely match production and reduce manual operational tasks by shifting the management of the runners to AWS.
In this blog, I will explore how AWS managed GitHub Action self-hosted runners work by building and deploying an application to AWS using GitHub Actions.
Architecture overview
The architecture of what I’ll be building can be seen below:
Figure 1. Architecture diagram of AWS CodeBuild running managed Self-Hosted GitHub Actions Runners
The architecture above shows how a developer pushes code changes to GitHub. This triggers CodeBuild to detect the update. CodeBuild then runs the defined GitHub Action Workflow, which builds and deploys it to AWS Lambda.
Step 1. Build a AWS Lambda Function
I’ll start with a simple application to demonstrate how to build and deploy an application on AWS via a Managed Self-Hosted GitHub Actions runner. We’ve written before about why AWS is the best place to run Rust, Amazon CTO Werner Vogels has been an outspoken advocate for exploring energy-efficient programming languages like Rust and AWS have great guides on using Rust to build on AWS such as:
Cargo Lambda is one of the simplest ways to run, build and deploy Rust lambda functions on AWS, I’ll start with the Getting Started guide:
- Navigate to GitHub.com and create a new GitHub repository
Figure 2 Create a new GitHub repository
- Clone the repository locally:
git clone https://github.com/{{user-name}}/rust-api-demo.git - From the above cloned repository, install Cargo Lambda:For macOS & Linux:
brew tap cargo-lambda/cargo-lambda brew install cargo-lambdaWindows users can follow the guide to see all the ways that you can install Cargo Lambda in your system.
- Use Cargo lambda to create a new project
cargo lambda new new-lambda-project && cd new-lambda-project
It’s now possible to explore the project, in this case I am using JetBrains RustRover with Amazon Q Developer installed to increase my productivity while working on the application:
Figure 3. JetBrains RustRover with Amazon Q Developer
Amazon Q Developer is available on a free tier and provides real-time code suggestions as well as advanced suggestions such as in-built chat to reason with the code we’re working on.
- Add, Commit & Push the code to your GitHub account:
git add . git commit -m “Initial Commit” git push origin
Step 2. Create AWS CodeBuild Project
The AWS Documentation outlines how to set up self-hosted GitHub Actions runners in AWS CodeBuild, the key here is to setup a GitHub webhook event of event type WORKFLOW_JOB_QUEUED so that CodeBuild will only process GitHub Actions workflow jobs.
I will create a new CodeBuild project as per the documentation to connect CodeBuild to our GitHub repository and correctly configure a webhook to trigger the GitHub Actions.
- Open the AWS CodeBuild console
- Create a build project.
- In Source:
- For Source provider, choose GitHub.
- For Repository, choose Repository in my GitHub account.
- For Repository URL, enter https://github.com/user-name/repository-name.
- In Primary source webhook events:
- For Webhook – optional, select Rebuild every time a code change is pushed to this repository.
- For Event type, select WORKFLOW_JOB_QUEUED. Once this is enabled, builds will only be triggered by GitHub Actions workflow jobs events.
Figure 4. WORKFLOW_JOB_QUEUED Event Type
- In Environment:
- Choose a supported Environment image and Compute. Note that you have the option to override the image and instance settings by using a label in your GitHub Actions workflow YAML.
- In Buildspec:
- Note that your Buildspec will be ignored. Instead, CodeBuild will override it to use commands that will setup the self-hosted runner. This project’s primary responsibility is to set up a self-hosted runner in CodeBuild to run GitHub Actions workflow jobs.
- In Source:
- Continue with the remaining default options and select Create build project.
CodeBuild Service Role Permissions
In order for the CodeBuild service role to be able to successfully create and deploy a Lambda function, the service role will require the necessary permissions. The Service role can be seen when editing the CodeBuild project:
Figure 5. CodeBuild Service Role Permissions
The required Lambda permissions are documented in the Cargo Lambda documentation:
- lambda:GetFunction
- lambda:CreateFunction
- lambda:UpdateFunctionCode
In addition, there are also IAM permissions required:
- iam:CreateRole
- iam:AttachRolePolicy
- iam:UpdateAssumeRolePolicy
- iam:PassRole
Add the required permissions to the service role for the CodeBuild project:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:CreateRole",
"iam:AttachRolePolicy",
"iam:UpdateAssumeRolePolicy",
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::{AWS:Account}:role/AWSLambdaBasicExecutionRole",
"arn:aws:iam::{AWS:Account}:role/cargo-lambda-role*"
]
},
{
"Effect": "Allow",
"Action": [
"lambda:CreateFunction",
"lambda:UpdateFunctionCode",
"lambda:GetFunction"
],
"Resource": "arn:aws:lambda::{AWS:Account}:function:{function-name}"
}
]
}
Note that I do not need to manage IAM permissions outside of our AWS Account, for example GitHub does not need to know about our AWS permissions.
Step 3. Create a GitHub Action Workflow
GitHub Actions is a continuous integration and continuous deliver (CI/CD) platform that provides automation through building, testing and deploying applications. In this section we will create a GitHub Action Workflow to build and deploy our Lambda.
- Navigate back to our GitHub project create a workflow within the .github/workflows directory, the Simple Workflow is a good starting point:
Figure 6. Create a Simple Workflow
- Update the Job to include the tooling required to build our Rust Lambda function, the details can be found in the GitHub Actions section. Our workflow file should now look like this:
name: rust-api-demo-cicd
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
env:
CARGO_TERM_COLOR: always
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Install Zig toolchain
uses: korandoru/setup-zig@v1
with:
zig-version: 0.10.0
- name: Install Cargo Lambda
uses: jaxxstorm/[email protected]
with:
repo: cargo-lambda/cargo-lambda
tag: v0.14.0
platform: linux
arch: x86_64
# Add your build steps below
The above GitHub Actions Workflow currently runs on GitHub; However, I now want to make two further changes:
- Define an AWS CodeBuild runner
- Define Build and Deploy Lambda steps
Define an AWS CodeBuild runner
A GitHub Actions workflow is made up of one or more jobs, each job runs in a runner environment specified by runs-on. The value for runs-on to specify CodeBuild as a runner takes the format:
runs-on: codebuild-<CodeBuildProjectName>-${{ github.run_id }}-${{ github.run_attempt }}I will update the <CodeBuildProjectName> to the CodeBuild project name that was entered in Step2, e.g. “GitHubActionsDemo”.
When configuring CodeBuild as a runner environment, BuildSpecs are ignored. In order to define the specification of our build environments it is possible to pass in variables including:
- EC2 compute builds: Image, image version, instance size
- Lambda compute builds: environment type, runtime version, instance size
For further details, see the action runner guide.
Define Build and Deploy Lambda steps
The last change is to add steps to check out our code onto the runner and then build and deploy using cargo lambda:
- name: Build Rust API
uses: actions/checkout@v4
- run: cargo lambda build --release
- run: cargo lambda deploy
The final workflow looks like this:
name: rust-api-demo-cicd
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
env:
CARGO_TERM_COLOR: always
jobs:
build:
runs-on: codebuild-GitHubActionsDemo-${{ github.run_id }}-${{ github.run_attempt }}
steps:
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Install Zig toolchain
uses: korandoru/setup-zig@v1
with:
zig-version: 0.10.0
- name: Install Cargo Lambda
uses: jaxxstorm/[email protected]
with:
repo: cargo-lambda/cargo-lambda
tag: v0.14.0
platform: linux
arch: x86_64
# Add your build steps below
- name: Build Rust API
uses: actions/checkout@v4
- run: cargo lambda build --release
- run: cargo lambda deploy
When I commit changes to the workflow to the main branch it will trigger the GitHub Action.
Step 4. Testing our GitHub Action Workflow.
The GitHub Action is currently triggered on all push and pull requests to main branch:
Figure 7. Trigger a build
Note that GitHub is where the CI/CD process is being driven, the build logs are available in GitHub as the job is running:
Figure 8. GitHub Action Logs
As the build progresses through the deployment step, the details of the Lambda function deployed are shown:
Figure 9. Deployment ARN
Navigating back to the AWS Console, the deployed Lambda Function can be seen:
Figure 10. Lambda Deployed
And finally, opening the CodeBuild console, it’s possible to observe the status of the Managed GitHub Actions Runner, the build number and also the duration:
Figure 11. Lambda Deployed via Managed Self-Hosted GitHub Action runner
Clean Up
To avoid incurring future charges:
- Delete the Lambda created via the deployment in Step 4.
- Delete the CodeBuild Project created in Step 2.
Conclusion
As I’ve shown in this blog, setting up GitHub Actions Workflows that run on AWS is now even easier to allow CodeBuild projects to receive GitHub Actions workflow job events and run them on CodeBuild ephemeral hosts. AWS customers can take advantage of natively integrating with AWS and providing security and convenience through features such as defining service role permissions with AWS IAM or passing credentials as environment variables to build jobs with AWS Secrets Manager.
Being able to use CodeBuild’s reserved capacity allows you to provision a fleet of CodeBuild hosts that persist your build environment. These hosts remain available to receive subsequent build requests, which reduces build start-up latencies but also make it possible to compile your software within your VPC and access resources such as Amazon Relational Database Service, Amazon ElastiCache, or any service endpoints that are only reachable from within a specific VPC.
CodeBuild-hosted GitHub Actions runners are supported in all CodeBuild regions and customers managing their CI/CD processes via GitHub Actions can use the compute platforms CodeBuild offers, including Lambda, Windows, Linux, Linux GPU-enhanced and Arm-based instances powered by AWS Graviton Processors.
Read more in our documentation for GitHub Action runner in AWS CodeBuild.
GitHub for iOS
Let’s hear more from 
Detects when IssueOps commands are used on a pull request. 
Configurable: choose your command syntax, environment, noop trigger, base branch, reaction, and more.
Respects your branch protection settings configured for the repository.
Comments and reacts to your IssueOps commands.
Triggers GitHub deployments for you with simple configuration.
Deploy locks to prevent multiple deployments from clashing.
Configurable environment targets.
Share company updates to GitHub’s intranet
Create weekly reports on program status updates
Turn weekly team photos into GIFs and upload to README
or merge pull requests whilst lounging on the couch. 
