Tag Archives: Compute

Implementing network traffic inspection on AWS Outposts rack

Post Syndicated from Art Baudo original https://aws.amazon.com/blogs/compute/implementing-network-traffic-inspection-on-aws-outposts-rack-2/

This post is written by Arun Kumar N C, Technical Account Manager; Debapriyo Jogi, Technical Account Manager; and Ashish Nagaraj, Cloud Support Engineer 2

Organizations are increasingly adopting hybrid cloud architectures that combine the scalability of cloud computing with the control and compliance benefits of on-premises infrastructure. AWS Outposts extends AWS infrastructure, AWS services, APIs, and tools to on-premises locations for workloads that require low latency, local data processing, or data residency. Outposts comes in a variety of form factors, from 42U Outposts racks to 1U and 2U Outposts servers. This post will focus on implementing network traffic inspection on Outposts rack.

Comprehensive security is critical for organizations deploying production workloads on Outposts. Network traffic inspection serves as a crucial security control, protecting against threats while enabling secure communication between different network segments. This post provides guidance on how to implement effective network traffic inspection across your hybrid cloud infrastructure using Outposts rack.

Overview

In the coming sections we will cover strategies for network traffic inspection on Outposts rack, focusing on outbound internet access and communication with on-premises networks. We explore AWS native services and third-party tools, offering a comprehensive overview of your options. We will cover architectural patterns, implementation guides, and best practices to help build a strong security posture for your hybrid cloud environment.

Securing internet-facing applications

Securing internet-facing applications on Outposts requires a robust, multi-layered approach for high availability and comprehensive security. We will explore two key architectural patterns that ensure enterprise-grade security for your workloads below.

Amazon CloudFront with AWS WAF integration

This architecture uses multiple AWS services including AWS Shield and AWS WAF for multi-layered security, Amazon CloudFront for global content delivery, and an Application Load Balancer (ALB) on Outposts for on-premises traffic management. Applications are deployed on Outposts, with CloudFront as the content delivery network. AWS WAF rules on CloudFront protect against web exploits, while the ALB distributes requests to application instances within Outposts.

This diagram illustrates AWS CloudFront with WAF integration connecting AWS cloud services to a customer data center through the internet. The setup includes CloudFront protected by WAF and Shield, EC2 instances in a VPC, and an Outpost deployment in the customer data center with ALB and EC2 instances.

Figure 1 – Amazon CloudFront with AWS WAF integration

  1. User sends a request via web browser or mobile app to access the application.
  2. The request is received by the CloudFront in AWS Edge Location, performing content-based routing.
  3. CloudFront integrates with AWS WAF to filter web traffic and block common attack patterns.
  4. ALB routes it to the appropriate targets.
  5. The application on Outposts processes the request and generates a response.

This flow ensures secure and efficient handling of user requests using both cloud and on-premises resources.

ALB with AWS WAF


This architecture offers more control over traffic routing while using AWS WAF for security. Applications are deployed on Outposts, but the ALB is in the parent Region, as AWS WAF cannot be associated with Outposts ALBs. The regional ALB handles incoming traffic, with AWS WAF providing firewall capabilities. After passing through AWS WAF, traffic is routed to Outposts applications. This configuration allows advanced WAF features but may introduce latency, as traffic must first reach the regional ALB. This trade-off between security and latency should be considered based on application needs.

Note: A critical dependency exists on the service link connection, as application traffic routing relies on the regional ALB. Service link failures will disrupt workload operations, making connection resilience essential for this architecture.

Figure 2 – ALB with AWS WAF

  1. User sends a request via web browser or mobile app for a webpage, API call, or service.
  2. The ALB in the AWS Region receives the request and performs Layer 7 content-based routing.
  3. ALB integrates with AWS WAF for security inspection.
  4. If the request passes, ALB routes it to the appropriate target in Outposts, selecting a specific instance or service.
  5. The application on Outposts processes the request, generates a response, and returns it.
  6. The response travels back through Outposts ALB to the regional ALB, which forwards it to the user’s browser or app.

Inspection between the Outpost subnet and regional subnet

Network traffic inspection between the Outpost and regional subnets is vital for security in hybrid cloud deployments. It makes sure traffic between Outposts and the parent Region complies with security policies and requirements. Two main architectural approaches exist for implementing this inspection:

  1. Using a third-party firewall in the Outpost subnet.
  2. Using AWS Network Firewall in an AWS Region.

Both approaches support various connectivity (service link) options between Outposts and the Region, including AWS Direct Connect.

Using third-party firewall in the Outpost subnet

This architecture uses a third-party firewall in the Outposts subnet, routing all traffic between the Outposts and regio0nal subnets through it. This setup enables local traffic inspection, reducing latency while enforcing security policies before traffic leaves the Outposts.

This diagram illustrates AWS Region and customer data center connected via service-linked VPN. Outpost deployment includes third-party firewall EC2 instance and target EC2 instances in Outpost subnet.

Figure 3 – Third-party firewall in the Outpost subnet

Traffic can originate from either Outposts or AWS regional subnet.

  1. Traffic originating from the Outpost to AWS Region:

a. Traffic is sent to the third-party firewall in the Outpost.
b. The firewall inspects the traffic and applies security policies.
c. If allowed, the firewall forwards traffic to the Region.
d. Traffic travels via service link connectivity (Direct Connect or public internet) to the regional subnet.

  1. Traffic originating from AWS Region to the Outpost:

a. Traffic originates in the regional subnet.
b. Traffic travels via service link connectivity (Direct Connect or public internet).
c. Upon reaching the Outpost, the traffic is sent to the third-party firewall.
d. The firewall inspects packets and applies security policies.
e. If allowed, the firewall forwards traffic to the Outpost subnet destination.

Using AWS Network Firewall in an AWS Region

In this architecture, a Network Firewall is deployed in the regional VPC, routing all traffic between the Outpost and regional subnets through it. This centralized approach ensures consistent policy enforcement with AWS native tools. The firewall inspects all traffic between Outposts and the AWS infrastructure in the Region.

This diagram illustrates AWS Network Firewall in a Region connected to customer data center via service-linked VPN. Includes VPC with Network Firewall, and EC2 in Outpost routing the traffic through the AWS Network Firewall endpoint.

Figure 4 – AWS Network Firewall in an AWS Region

Traffic can originate from either the Outposts subnet or AWS regional subnet.

All traffic is routed to the Network Firewall in the AWS Region.

  1. The firewall applies configured rules, including:
  • Custom rules for specific security needs.
  • Managed AWS rule groups for common threats.
  • Third-party rule groups for specialized protection.
  1. If traffic passes all rules, it is forwarded to its destination (Outpost or Region).
  2. Return traffic follows the same path, all traffic is inspected by the Network Firewall.

Inspection between on-premises and Outposts through Local Gateway

Network traffic inspection between on-premises networks and Outposts via Local Gateway (LGW) is essential for securing hybrid environments. It helps you make sure safe communication is happening between Outposts workloads and on-premises infrastructure.
Two primary architectural approaches are available explained below. The choice depends on infrastructure, security needs, and operational preferences.

Using third-party firewall on Outposts

For more details on implementing network traffic inspection between on-premises networks and Outposts via LGW, refer to Implementing network traffic inspection on AWS Outposts rack.

This post expands on the preceding blog by offering detailed guidance on architectural options and traffic flows for inspecting network traffic between on-premises environments and Outposts via LGW.

Using your on-premises router/firewall

This approach uses the existing firewall capabilities of your on-premises router/firewall. The network is configured to route all traffic between the on-premises environment and Outposts through this router/firewall. The LGW on your Outpost connects directly to your router/firewall, which handles the firewall functions. This setup uses the on-premises security infrastructure and policies, ensuring continuity in security management while integrating Outposts into the broader network security strategy.

Traffic flow:

  1. Traffic originates from on-premises network
  2. Passes through your router with the firewall
  3. Router inspects the traffic
  4. If allowed, traffic is sent to Outposts through the LGW
  5. Outbound inspection to the internet from Outposts instances

Outbound inspection to the internet from Outposts instances

Outbound internet traffic inspection for Outposts instances is useful for security and controlling access to external resources. Three architectural approaches are available for implementing this inspection, which are discussed in the following sections.

Using Customer-Owned IP (CoIP) with on-premises firewall

In this architecture, Outposts instances are assigned Customer-Owned IP (CoIP) addresses, with all outbound internet traffic routed through the on-premises network and firewall. The LGW connects the Outposts environment to the on-premises network. This setup enables organizations to leverage existing on-premises security and internet connectivity while ensuring consistent IP addressing across their hybrid environment.

This diagram illustrates Customer-Owned IP (CoIP) implementation in a customer data center, where Outpost EC2 instances use CoIP addresses, routing through LGW to an on-premises firewall for inspection.

Figure 5 – Customer-Owned IP (CoIP) with on-premises firewall

  1. An Outposts instance with a CoIP address initiates outbound internet traffic.
  2. The traffic is routed to the LGW on the Outpost.
  3. The LGW forwards the traffic to the on-premises network.
  4. The traffic reaches the on-premises firewall and inspects the traffic, applying security policies and rules.
  5. If allowed, the firewall forwards the traffic to the internet through the on-premises connection.
  6. Return traffic follows the reverse path, being inspected by the firewall before reaching the Outposts instance.

Using CoIP with third-party firewalls on Outposts

Using this configuration, you would assign a CoIP addresses to your Outposts instances and deploy a third-party firewall appliance directly on the Outposts rack. Outbound internet traffic from these instances is routed through the local firewall running on EC2 before reaching the internet via the LGW. This approach ensures local traffic inspection while preserving the advantages of CoIP addressing, enabling seamless integration with existing IP management systems.

This diagram illustrates third-party firewalls on Outposts as EC2 instances. Customer data center contains Outpost subnet with EC2 instances using CoIP, connected to LGW. Traffic routes through Customer Edge Router/Firewall before reaching Internet

Figure 6 – CoIP with third-party firewalls on Outposts

  • An Outposts instance with a CoIP address initiates outbound internet traffic.
  • The traffic is routed to the third-party firewall deployed on the Outpost.
  • The firewall performs deep packet inspection, applying security policies and rules.
  • If allowed, the firewall forwards the traffic to the LGW.
  • The LGW sends the traffic to the internet through the on-premises connection.
  • Return traffic follows the reverse path, being inspected by the firewall before reaching the Outposts instance.

Using Internet Gateway (IGW) with Network Firewall in the Region

This architecture provides secure outbound internet access for Outposts workloads by using services in the parent Region. The VPC extends to include the Outposts rack, with internet-bound traffic routed via the service link to the AWS Region. In the Region, the Network Firewall inspects the traffic before forwarding it to the Internet Gateway (IGW) for internet access.

Traffic flow:

  1. Traffic is sent to the parent Region via the service link.
  2. In the Region, traffic is routed to the Network Firewall.
  3. The Network Firewall inspects the traffic and applies rules.
  4. If allowed, traffic is forwarded to the IGW via the NAT Gateway.
  5. The IGW sends the traffic to the internet.
  6. Return traffic follows the reverse path, inspected before reaching Outposts.

Conclusion

Implementing effective network traffic inspection for AWS Outposts requires a strategic approach balancing security, efficiency, and architectural complexity. We’ve explored multiple architectural patterns for implementing network traffic inspection with Outposts rack.

Reach out to your AWS account team or AWS support to learn more about inspection in Outpost.

Migrating your on-premises workloads to AWS Outposts Rack

Post Syndicated from Art Baudo original https://aws.amazon.com/blogs/compute/migrating-your-on-premises-workloads-to-aws-outposts-rack-2/

This post is written by Craig Warburton, Senior Solutions Architect, Hybrid; Sedji Gaouaou, Senior Solutions Architect, Hybrid; and Brian Daugherty, Principal Solutions Architect, Hybrid.

Migrating workloads to AWS Outposts Rack offers you the opportunity to gain the benefits of cloud computing while keeping your data and applications on premises.

For organizations with strict data residency requirements, by deploying AWS infrastructure and services on premises, you can keep sensitive data and mission-critical applications within your own data centers or facilities, helping ensure compliance with data sovereignty laws and regulatory frameworks.

On the other hand, if your organization does not have stringent data residency requirements, you may opt for a hybrid approach, using both Outposts Rack and the AWS Regions. With this flexibility, you can process and store data in the most appropriate location based on factors such as latency, cost optimization, and application requirements.

In this post, we cover options to migrate your workloads to an Outposts Rack, taking into account your specific data residency requirements. We explore strategies, tools, and best practices to enable a successful migration tailored to your organization’s needs.

Overview

AWS has several services to help you migrate and rehost workloads, including AWS Migration Hub, AWS Application Migration Service, AWS Elastic Disaster Recovery. Alternatively, you can use backup and recovery solutions provided by AWS partners.

At AWS, we use the 7 Rs framework to help organizations evaluate and choose the appropriate migration strategy for moving applications and workloads to the AWS Cloud. The 7 Rs represent:

  1. Rehosting (rehost or lift and shift)
  2. Replatforming (lift, tinker, and shift)
  3. Repurchasing (republish or re-vendor)
  4. Refactoring (re-architecting)
  5. Retiring
  6. Retaining (revisit)
  7. Relocating (remigrate).

This post focuses on rehosting and the services available to help rehost on-premises applications to Outposts Rack.

Before getting started with any migration, AWS recommends a three-phase approach to migrating workloads to the cloud (AWS Region or Outposts Rack). The three phases are assess, mobilize, and migrate and modernize.

Figure 1: Diagram showing the three migration phases of assess, mobilize, and migrate and modernize

Figure 1: Diagram showing the three migration phases of assess, mobilize, and migrate and modernize

This post describes the steps that you can take in the migrate and modernize phase. However, the assess and mobilize phases are also critical to allow you to understand what applications are migrated, the dependencies between them, and the planning associated with how and when migration occurs.

Workload migration to Outposts Rack: With staging environment in a Region

After deploying an Outposts Rack to your desired on-premises location, you can perform migrations of on-premises systems and virtual machines using either Application Migration Service and AMI creation or third-party backup and recovery services. Both scenarios are described in the following sections.

Scenario 1: Using Application Migration Service with AMI creation

Application Migration Service is able to lift and shift a large number of physical or virtual servers without compatibility issues, performance disruption, or long cutover windows.

In this scenario, at least one Outposts Rack is deployed on premises with the following prerequisites:

  • An AWS Replication Agent installed on each source server
  • At least one Outposts Rack installed and activated
  • VPC in an AWS Region
  • Staging subnet for staging migrated instances
  • Cutover subnet to validating migrated instances
  • Extended VPC spanning Region to the Outposts Rack
  • Migrated resources subnet where instances will be deployed from AMIs

The following diagram shows the solution architecture including the prerequisites and the on-premises servers that will be migrated to the Outposts Rack.

Figure 2: Architecture diagram showing migration with Application Migration Service

Figure 2: Architecture diagram showing migration with Application Migration Service

Step 1: Outposts Rack configuration

You can work with AWS specialists to size your Outposts for your workload and application requirements. In this scenario, you don’t need additional Outposts Rack capacity for migration because the staging area will be deployed in the Region (see 1 in Figure 2).

Step 2: Prepare Application Migration service

Set up Application Migration Service from the console in the Region to which your Outposts Rack is anchored. If this is your first setup, then choose Get started on the Application Migration Service console. When creating the replication settings template, ensure that your staging area is using subnets in the anchor Region (see 2 in Figure 2).

Step 3: Install the AWS Replication Agent to the source servers or machines

For large migrations, source servers may have a wide variety of operating system versions and may be distributed across multiple data centers. Application Migration Service offers the MGN connector, a feature that allows you to automate running commands on your source environment. Finally, ensure that communication is possible between the agent and Application Migration Service (see 3 in Figure 2).

In the following image, there is an example of deploying the AWS Replication Agent providing the necessary parameters (AWS Region, AWS access key and AWS secret access key).

Figure 2: Architecture diagram showing migration with Application Migration Service

When the AWS Replication Agent is installed, the server is added to the Application Migration Service console. Next, it undergoes the initial syncronization process, which is completed when showing the Ready for testing lifecycle state in the Application Migration Service console.

Step 4: Configure launch settings

Prior to testing or cutting over an instance, you must configure the launch settings by creating Amazon Elastic Compute Cloud (Amazon EC2) launch templates, ensuring that your cutover subnet is selected and that you choose an available instance type (see 4 in Figure 2). The instance type right-sizing feature allows AWS Application Migration Service to launch a test or cutover instance type that best matches the hardware configuration of the source server, by selecting the Basic option, AWS Application Migration Service will launch a test or cutover AWS instance type that best matches the OS, CPU, and RAM of your source server.

Step 5: Install AWS Systems Manager Agent on your cutover instances. When the launch settings are defined, you must activate the post-launch actions for either a specific server or all the servers. You must leave the Install the Systems Manager agent and allow executing actions on launched servers option toggled on in order for post-launch actions to work. Untoggling the option would disallow Application Migration Service to install the AWS Systems Manager Agent on your servers, and post-launch actions would no longer be executed (see 5 in Figure 2).

Figure 3: Post-launch actions on the Application Migration Service console

Figure 3: Post-launch actions on the Application Migration Service console

Step 6: Testing and cutover in Region

When you have configured the launch settings for each source server, you are ready to launch the servers as test instances. Best practice is to test instances before cutover.

Figure 4: Application Migration Service console ready to launch test instances

Figure 4: Application Migration Service console ready to launch test instances

Finally, after completing the testing of all the source servers, you are ready for cutover (see 6 on Figure 2). Prior to launching cutover instances, check that the source servers are listed as Ready for cutover under Migration lifecycle and Healthy under Data replication status.

Figure 5: Application Migration Console ready for cutover

Figure 5: Application Migration Console ready for cutover

To launch the cutover instances, choose the instances you want to cutover and then choose Launch cutover instances under Cutover (see Figure 5). The Application Migration Service console indicates Cutover finalized when the cutover has completed successfully the chosen source servers’ Migration lifecycle column shows the Cutover complete status, the Data replication status column shows Disconnected, and the Next step column shows Mark as archived. The source servers have now been successfully migrated into AWS. You can now archive your source servers that have launched cutover instances.

Step 7: Create a Migration AMI

After migrating all your workloads in the region where the Outposts is anchored to, create Amazon Machine Images (AMI). When you create an AMI from an instance, Amazon EC2 powers down the instance before creating the AMI to make sure that everything on the instance is stopped and in a consistent state during the creation process. If you are confident that your instance is in a consistent state appropriate for AMI creation, you can tell Amazon EC2 not to power down and reboot the instance.

This step can be automated using an existing Post Launch Action.

Step 8: Launch instances on AWS Outposts

The final part is to launch your created AMIs to your Outposts. To identify the EC2 instances configured on your Outpost you can use the following AWS Command Line Interface (AWS CLI):

Outposts get-outpost-instance-types \

–outpost-id op-abcdefgh123456789

The output of this command lists the instance types and sizes configured on your Outpost:

InstanceTypes:

– InstanceType: c5.xlarge

– InstanceType: c5.4xlarge

– InstanceType: r5.2xlarge

– InstanceType: r5.4xlarge

With knowledge of the instance types configured, you can now determine how many of each are available. For example, the following AWS CLI command, which is run on the account that owns the Outpost, lists the number of c5.xlarge instances available for use:

aws cloudwatch get-metric-statistics \

–namespace AWS/Outposts \

–metric-name AvailableInstanceType_Count \

–statistics Average –period 3600 \

–start-time $(date -u -Iminutes -d ‘-1hour’) \

–end-time $(date -u -Iminutes) \

–dimensions \

Name=OutpostId,Value=op-abcdefgh123456789 \

Name=InstanceType,Value=c5.xlarge

This command returns:

Datapoints:

– Average: 10.0

Timestamp: ‘2024-04-10T10:39:00+00:00’

Unit: Count

Label: AvailableInstanceType_Count

The output indicates that there were (on average) 10 c5.xlarge instances available in the specified time period (one hour). Using the same command for the other instance types, you discover that there are also 20 c5.4xlarge, 10 r5.2xlarge, and 6 r5.4xlarge available for use in completing the necessary EC2 launch templates.

Scenario 2: Using partner backup and replication solutions

You may already be using a third-party or AWS Partner solution to create on-premises backups of bare-metal or virtualized systems. These solutions often use local disk-arrays or object stores to create tiered backups of systems covering restore-points going back years, days, or just a few hours or minutes.

These solutions may also have inherent capabilities to restore from these backups directly to the AWS. This enables migration of on-premises systems to EC2 instances deployed to Outposts Rack.

In the scenario illustrated in Figure 6, the partner backup and replication service (BR) creates backups (see 1 in Figure 6) of virtual machines to on-premises disk or object storage repositories. Using the service’s AWS integration, virtual machines can be restored (see 2 in Figure 6) to an EC2 instance deployed on Outposts Rack, which is also on-premises. The restoration may follow a process that uses helper instances and volumes (see 3 in Figure 6) during intermediate steps to create Amazon Elastic Block Store (Amazon EBS) snapshots (see 4 in Figure 6) and then AMIs of the systems being migrated (see 5 in Figure 6), which are ultimately deployed (see 6 in Figure 6) to Outposts Rack.

Figure 6: Architecture diagram of the partner backup and replication scenario

Figure 6: Architecture diagram of the partner backup and replication scenario

When deploying an AMI created from a restored instance you must specify the target VPC and subnet. These should be the VPC being extended to the Outpost and a subnet that has been created in that VPC on the Outpost. You also need to specify an EC2 instance type that is available on the Outpost, which can be discovered using the process described in the previous section.

Workload migration to Outposts Rack using AWS Elastic Disaster Recovery (DRS)

Data residency can be a critical consideration for organizations that collect and store sensitive information, such as personally identifiable information (PII), financial data, or medical records. AWS Elastic Disaster Recovery, supported on Outposts Rack, helps enable seamless replication of on-premises data to Outposts Rack and addresses data residency concerns by keeping data within your on-premises environment, using Amazon EBS and Amazon S3 on Outposts.

In this scenario, an Outpost Rack is deployed on-premises with the following prerequisites:

  • At least one Outposts Rack installed and activated
  • The Outposts Rack must be in Direct VPC Routing (DVR) mode
  • VPC extended to the Outposts Rack containing subnets for staging and target resources
  • Amazon S3 on Outposts (necessary for all Elastic Disaster Recovery replication destinations)
  • An AWS Replication Agent installed on each source server

The following diagram shows the solution architecture and includes the on-premises servers that are migrated from the local network to the Outposts Rack. It also includes the staging VPC used to deploy the replication servers on Outposts Rack, Amazon S3 on Outposts to store the local Amazon EBS snapshots, and the target VPC extended to Outposts Rack.

Figure 7: Architecture diagram for workflow migration to Outposts Rack

Figure 7: Architecture diagram for workflow migration to Outposts Rack

Step 1: Outposts Rack configuration

To use Elastic Disaster Recovery on Outposts Rack, you need to configure both Amazon EBS and Amazon S3 on Outposts to support continuous replication and point-in-time recovery for your workload needs (see 1 in Figure 7). Specifically, you need to size the Amazon EBS and Amazon S3 on Outposts capacity according to your workload capacity requirements and application interdependencies. To do this, you can define dependency groups: each dependency group is a collection of applications and their underlying infrastructure with technical or non-technical dependencies. A 2:1 ratio is recommended for the EBS volumes to be used for near-continuous replication, and a 1:1 ratio is recommended for the Amazon S3 on Outposts ratio for EBS snapshots. For example, to migrate 40 TB of workloads, you need to plan for 80 TB of EBS volumes and 40 TB of Amazon S3 on Outposts capacity.

Step 2: Extend VPC to your Outposts Rack

When your Outpost has been provisioned and is available, extend the necessary Amazon Virtual Private Cloud (Amazon VPC) connection to the Outpost from the Region by creating the desired staging and target subnets (see 2 in Figure 7).

Step 3: Prepare Elastic Disaster Recovery service

Prepare the Elastic Disaster Recovery service from the Console to set the default replication and launch settings. When defining these settings, make sure that the Outposts resources available are chosen for staging and target subnets and instance and storage type (see 3 in Figure 7).

Step 4: Install the AWS Replication Agent to the source servers or machines

The next phase is to install the AWS Replication Agent to the source servers and to make sure that communication is possible between the AWS Replication Agent and your Outposts replication subnet through the Outposts local gateway, which makes sure that replication traffic uses the local network (see 4 in Figure 7).

Step 5: Continuous block-level replication

Staging area resources are automatically created and managed by Elastic Disaster Recovery. When the AWS Replication Agent has been deployed, continuous block-level replication (compressed and encrypted in transit) occurs (see 5 in Figure 7) over the local network.

Step 6: Launch Outposts Rack resources

Finally, migrated instances can now be launched using Outposts Rack resources based on the launch settings defined previously (see 6 in Figure 7).

Conclusion

In this post, you have learned how to migrate your workloads from your on-premises environment to AWS Outposts Rack based on your specific data residency requirements. When you have the flexibility of using AWS Regional services, AWS migration services or partner solutions can be used with infrastructure already in place. If your data must stay on-premises, then using AWS Elastic Disaster Recovery allows you to migrate your data without using Regional services, allowing you to migrate to Outposts Rack without your data leaving the boundary of a certain geographic location.

To learn more about an end-to-end migration and modernization journey, visit the AWS Migration Hub.

Implementing a serverless architecture to detect absence of Guardrails in Amazon Bedrock inference API calls

Post Syndicated from Art Baudo original https://aws.amazon.com/blogs/compute/implementing-a-serverless-architecture-to-detect-absence-of-guardrails-in-amazon-bedrock-inference-api-calls/

This post is written by Sayan Chakraborty, Senior Solutions Architect, AWS

Implementing a serverless architecture to detect absence of Guardrails in Amazon Bedrock inference API calls

In today’s rapidly evolving artificial intelligence (AI) landscape, organizations are increasingly harnessing the power of foundation models through Amazon Bedrock to build sophisticated generative AI applications. Although this technology opens up exciting possibilities, it also brings forth important considerations around responsible AI implementation and content safety.

Amazon Bedrock Guardrails serve as a crucial safeguard, helping organizations filter out harmful content, prevent prompt injection attacks (LLM01:2025 from OWASP Top 10 for generative AI), and maintain ethical AI practices. These configurable safeguards are essential for enterprises committed to responsible AI development, especially when scaling their applications across various use cases.

However, there’s a critical consideration: although Guardrails are powerful, they’re optional by default in Amazon Bedrock inference API calls. For organizations that mandate the use of Guardrails as part of their responsible AI strategy, a solution is needed to make sure of consistent implementation across all API requests.

In this post, we explore how to build a serverless architecture that automatically detects when Guardrails are absent in Amazon Bedrock inference API calls. We demonstrate how enterprises can implement automated monitoring and alerting systems to maintain compliance with their AI safety standards, making sure that Guardrails are properly implemented wherever needed. This solution is particularly valuable for organizations prioritizing secure and responsible AI deployment at scale.

Prerequisites

Before proceeding with the implementation, make sure that you do the following:

1.Create an AWS account if you do not already have one, and log in. The AWS Identity and Access Management (IAM) user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.

2.Have AWS Command Line Interface (AWS CLI) installed and configured.

3.Have Git Installed.

Architecture

The following diagram shows an event-driven architecture of this solution.

Figure 1: Solution architecture diagram

Figure 1: Solution architecture diagram

Amazon Bedrock supports model invocation logging. When enabled, it collects the full request data, response data, and metadata associated with all model invocation calls performed in your AWS account. Logging can be configured to send the logs to supported destinations such as Amazon CloudWatch Logs and Amazon S3. This solution uses an S3 bucket to collect these logs. Note that this solution supports the below Amazon Bedrock inference APIs:

As logs get stored in the S3 bucket, an Amazon S3 event notification is generated to an Amazon EventBridge event bus. A rule that matches “Object Created” events from Amazon S3 routes these events to an AWS Step Functions state machine, which defines the orchestration logic to inspect the model invocation logs for missing Guardrails, and sends out an alert to a monitored email address when applicable.

Walkthrough of the orchestration

As mentioned previously, the Step Functions state machine is the orchestration engine that performs the business logic for this solution, as events are received from new logs created in the S3 bucket. When opened in the Workflow studio in the Step Functions console, you should observe the following diagram.

Figure 1: Step Functions state machine diagram as seen in workflow studio

Figure 2: Step Functions state machine diagram as seen in workflow studio

1.The first step in the state machine is to call an AWS Lambda function to get the logs from the S3 bucket using the bucket name and object key supplied in the event object received from EventBridge.

2.If the log shows that the Amazon Bedrock API invocation was successful, then the state machine collects the output object of the API response from the log that is needed for further evaluation.

3.The next step is to check if Amazon Bedrock Guardrails was used. This is done by looking for specific objects in the Amazon Bedrock API output that was captured from the logs.

4.If a Guardrail was detected, then the flow completes successfully, and no further action is needed.

5.If a Guardrail was not detected, then the next step in the state machine collects a few pieces of information from the log file that is necessary to record the transaction and adds the transaction date. Then, the transaction is logged in to the transactions table in Amazon DynamoDB.

6.A user or a role may be making a lot of API calls to Amazon Bedrock each day. Therefore, the solution implements a mechanism to prevent the monitored email address from being swamped by emails reporting the same user or role more than once each day. This is done in parallel to Step 5, where the flow checks if the principal’s identity (user ID/IAM role) is recorded as notified in the current date, by querying the notifications table in DynamoDB. If no results were found, meaning that a notification hasn’t been sent yet, then an email is sent out to a monitored email address through an Amazon Simple Notification Service (Amazon SNS) topic. Furthermore, an item is inserted into the notifications table in DynamoDB to prevent sending more notifications on the same day for the same principal.

Solution deployment

For deployment instructions, follow along in the GitHub repo or use this post. An AWS CloudFormation template is provided to deploy the solution.

1.Create an S3 bucket to store the model invocation logs from Amazon Bedrock. Under bucket Properties, turn the EventBridge notifications to On. This enables Amazon S3 to send an event notification to the EventBridge default event bus whenever a log file is created in the bucket by Amazon Bedrock.

2.Go to the Amazon Bedrock console and enable Model invocation logging under Bedrock Configuration > Settings, from the left navigation pane. Specify the bucket created in Step 1 under S3 location.

Figure 2: Amazon Bedrock settings for Model invocation logging

Figure 3: Amazon Bedrock settings for Model invocation logging

3. Create two more S3 buckets: one that is used by the Step Functions state machine to store Bedrock model invocation errors detected from the log, and the other that stores the Lambda function code for this solution. Inside the latter bucket, create a Folder called code (or any other preferred name) and upload the ZIP archive under the lambda-code folder of this repository, into that Amazon S3 folder. Note the names for these two S3 buckets and the Amazon S3 object key for the Lambda ZIP file. These must be specified as input parameters to the CloudFormation template.

4. From the CloudFormation console or using CLI, create a stack using the template provided in this repository called bedrock-guardrails-detection-template.yaml. For inputs, specify the BedrockLogsBucket (from Step 1), BedrockLogsErrorBucket (from Step 3), LambdaFunctionCodeBucket (from Step 3), LambdaFunctionCodeBucketKey (S3 object key for the ZIP file uploaded in Step 3, for example code/get-bedrock-logs-from-s3.py.zip), and NotificationEmailAddress (email address to subscribe to the SNS topic). It may take a few minutes to complete deployment of the CloudFormation stack.

5. When deployment is complete, access the email inbox for the email address specified during the CloudFormation stack deployment, and confirm the subscription using the email sent from the Amazon SNS topic. The email should be titled: AWS Notification – Subscription Confirmation. Choose the Confirm subscription link inside the email to complete the subscription process. The email account is now ready to receive notifications from this solution.

Scaling to multiple AWS accounts

The architecture discussed previously shows how Guardrails can be detected from within the same AWS account where Amazon Bedrock APIs are invoked. However, in most production environments, there are multiple AWS accounts where independent teams may be deploying their own generative AI workloads using Amazon Bedrock in their own accounts. To collect model invocation logs from all those accounts, EventBridge can be configured to send events from event buses in separate source workload accounts to a central event bus deployed in a central destination governance account. This central event bus can have a rule to route events to the Step Functions state machine deployed in that central governance account. The deployment model looks like the following diagram.

To learn more about sending and receiving events between AWS accounts in EventBridge, refer to the documentation.

Figure 3: Cross-account guardrail detection solution

Figure 4: Cross-account guardrail detection solution

Further considerations and clean up

Amazon Bedrock model invocation logging captures requests and responses from model invocations and stores the logs in the destination of your choosing. In this sample it is in an S3 bucket that you create. The following are some more security considerations.

1.To protect information, you may choose to use to encrypt the contents using server-side encryption with AWS KMS keys (SSE-KMS) on the S3 bucket, and specify a customer managed encryption key. More details are in this Amazon Bedrock user guide.

2.Perform regular cleanup of the model invocation logs bucket using an Amazon S3 lifecycle configuration rule as mentioned in this post.

To avoid ongoing charges, clean up your environment by following these steps to delete the resources you created by following this post, if they are no longer needed:

1.Delete the stack:
aws cloudformation delete-stack –stack-name STACK_NAME

2.Confirm the stack has been deleted:
aws cloudformation list-stacks –query “StackSummaries[?contains(StackName,’STACK_NAME’)].StackStatus”

3.Empty contents of the S3 buckets created manually as a prerequisite to deploying the CloudFormation stack and delete the buckets.

4.Turn off model invocation logging from under Settings in the Amazon Bedrock console, if it’s not desired any longer.

Conclusion

This post discussed implementing a serverless event-driven architecture to detect the absence of Guardrails in Amazon Bedrock inference API calls. As organizations increasingly use foundation models through Amazon Bedrock for generative AI applications, making sure of responsible AI implementation becomes crucial.

The solution presents an event-driven architecture that automatically detects when Guardrails are missing in API calls. It uses the Amazon Bedrock model invocation logging, storing logs in an Amazon S3 bucket. When new logs are created, an Amazon S3 event notification triggers an Amazon EventBridge event bus, which routes events to an AWS Step Functions state machine. Then, the state machine inspects the logs for missing Guardrails and sends alerts through Amazon SNS to a monitored email address.

The architecture includes features to prevent notification flooding and can scale across multiple AWS accounts. The post provides detailed deployment instructions using AWS CloudFormation and includes security considerations and cleanup procedures. With this solution you can help your organization maintain compliance with AI safety standards while scaling generative AI applications.

Efficiently manage Amazon EC2 On-Demand Capacity Reservations (ODCRs) with split, move, and modify

Post Syndicated from Art Baudo original https://aws.amazon.com/blogs/compute/efficiently-manage-amazon-ec2-on-demand-capacity-reservations-odcrs-with-split-move-and-modify/

This post is written by Ninad Joshi, Senior Solutions Architect, Ballu Singh, Principal Solutions Architect, and Ankush Goyal, Enterprise Support Lead AWS.

Introduction

In today’s cloud-first world, managing compute capacity efficiently while making sure of application availability is crucial for your business. Amazon EC2 On-Demand Capacity Reservations (ODCR) is a valuable tool for organizations looking to manage their reservations, but managing reservations across multiple teams and accounts is challenging. Recently, AWS introduced new capabilities – split, move, and modify – that improve how organizations can manage their Capacity Reservations. In this post, we explore how these features can transform your operations.

Common ODCR management challenges

As a consumer of ODCR, you might face several challenges managing your Capacity Reservations. These challenges include but are not limited to the following:

  • Underused reserved capacity in some accounts
  • Inability to redistribute excess capacity efficiently
  • Difficulty in managing existing capacity across multiple AWS accounts
  • Difficulty in modifying reservation attributes post-creation

With multiple development teams and various projects running simultaneously, you might struggle with efficient capacity allocation. You might also find yourself dealing with situations where one team has excess capacity while another desperately needs it.

Use case 1: Redistributing capacity across teams

The unused capacity dilemma

Consider a scenario where your machine learning (ML) team has an ODCR for ten c5.2xlarge instances, but they’re only using five. Meanwhile, your Analytics team urgently needs three Amazon Elastic Compute Cloud (Amazon EC2) instances of the same type for a new project. Previously, your Analytics team would have had to create a new reservation, leading to unnecessary overhead of managing their own Capacity Reservation. Meanwhile, the five unused capacity slots of the ODCR owned by your ML team results in unnecessary costs.

Split capability to the rescue

Using the new split capability, you can now divide the existing ODCR (see ODCR-1 in the following figure), which has a total capacity of ten EC2 instances, and create a new ODCR with three of the unused capacity.

Before split, ODCR-1 with original total and unused capacity

Figure 1: Before split, ODCR-1 with original total and unused capacity

This results in the creation of two ODCRs:

  1. Original ODCR: total capacity of seven instances for the ML team
  2. New ODCR: three instances for the Analytics team

The following figure illustrates the split result:After split, ODCR-1 with updated total and unused capacity, and newly created ODCR-2

Figure 2: After split, ODCR-1 with updated total and unused capacity, and newly created ODCR-2

Sharing across accounts

The split operation creates the new ODCR in the same AWS account. If your teams operate under the same AWS account, then the split operation is direct without any further steps. However, if your teams use different AWS accounts, then you would need to use AWS Resource Access Manager (AWS RAM) to share the newly created ODCR after the split operation. This enables cross-account capacity management while maintaining centralized control.

Refer to the AWS Documentation for more information on pre-requisites and considerations when splitting off capacity from one reservation to a new one.

Refer to the API and CLI documentation for further information on the split capability such as parameters, exceptions, and limits.

Use case 2: moving capacity between reservations

Scaling for growth

After a few days, when your Analytics team needs one more capacity to launch an instance for their expanding project, you need to add more capacity to ODCR-2.

Move capability to the rescue

Instead of creating a new ODCR for this purpose, you can move one of the unused slots from ODCR-1 to ODCR-2. This flexibility saves you multiple steps involved in reserving new capacity, removes any disruptions to running existing workloads, and helps with simpler ODCR management. This rebalancing makes sure of optimal resource usage without further procurement.

Before move, ODCR-1 with unused capacity and ODCR-2 with current capacity

Figure 3: Before move, ODCR-1 with unused capacity and ODCR-2 with current capacity

After move, ODCR-1 with reduced capacity and ODCR-2 with additional capacity

Figure 4: After move, ODCR-1 with reduced capacity and ODCR-2 with additional capacity

Refer to the AWS Documentation for more information on pre-requisites and considerations when moving capacity from one reservation to another one.

Refer to the API and CLI documentation for further information on the move capability such as parameters, exceptions, and limits.

Use case 3: adjusting reservation attributes for changing workload patterns

Dynamic workload requirements

When your data processing workload patterns change significantly, you must adapt. Initially, you might have set up your ODCR with specific instance matching criteria, making it a targeted reservation for predictable workloads. However, as you introduce more dynamic, impromptu analysis projects, you need more flexibility in how instances can be launched against your reservation.

Modify feature to the rescue

Using the modify capability, you can now change the reservation’s attributes without creating a new reservation or disrupting running workloads. You can modify your ODCR by:

  • Changing instance quantity
  • Changing instance eligibility from Targeted to Open
  • Adjusting the reservation’s end date to align with your project timeline

This modification allows you to:

  • Launch new instances more flexibly without strict instance eligibility
  • Improve the usage of reserved capacity across different projects
  • Maintain cost optimization while adapting to changing business needs

The modify feature provides this flexibility while making sure that your existing workloads continue running uninterrupted, making it an invaluable tool for dynamic environments. See the following figures for an example where the instance quantity of ODCR-2 is modified from four to six:

Before modify, ODCR-2 with total capacity of four and instance eligibility of targeted

Figure 5: Before modify, ODCR-2 with total capacity of four and instance eligibility of targeted

After modify, ODCR-2 with new total capacity of six and instance eligibility of open

Figure 6: After modify, ODCR-2 with new total capacity of six and instance eligibility of open

Increasing ODCR size or creating a new one is subject to capacity availability in Amazon EC2 on-demand availability. Therefore, if unused capacity is available in an existing ODCR, then moving/splitting that could be a better option than modifying an ODCR.

Refer to the AWS Documentation for more information on pre-requisites and considerations when modifying Capacity Reservations.

Refer to the API and CLI documentation for further information on the modify capability such as parameters, exceptions, and limits.

Special considerations for split capacity

In the preceding sections, we saw how you can use the split capability to detach excess unused capacity to create an ODCR for another team. However, you can also use this capability to split used capacity to create new ODCRs. This capability is particularly helpful when you want to split partially used ODCRs to create a new one for easier tracking and management. Along with the considerations for splitting unused/excess capacity, the following considerations apply for splitting used capacity:

  1. The used capacity can only be split for an ODCR with open instance eligibility that isn’t shared with any account.
  2. The instances running inside the reservation are of open eligibility (in other words they are not targeting the reservation).
  3. When you split the used capacity, the eligible instances are randomly selected. You cannot specify which running instances are split. If a sufficient number of eligible instances aren’t found to fulfill the split quantity, then the split operation fails. When you specify the quantity of instances to be split, by default any unused capacity is moved first, followed by any eligible running instances (the used capacity in your reservation).

In the next section we different scenarios where you can or can’t use split capability.

Scenario 1: managing internal ODCRs (Capacity Reservation not shared with any other AWS account)

For your internal projects, when managing ODCRs that aren’t shared with external partners (other AWS accounts) and all have open instance eligibility, consider this example with ODCR-1:

  • Total capacity of ten c5.2xlarge instances, all with open instance eligibility
  • Eight instances currently in use by your ML team
  • Two unused instances

Before split, ODCR-1 with total capacity of 10 and 2 unused instances

Figure 7: Before split, ODCR-1 with total capacity of 10 and 2 unused instances

This ODCR isn’t shared with any external AWS accounts, thus you have maximum flexibility in splitting the reservation. You can split up to nine instances into a new reservation (total capacity minus one), regardless of how many instances are currently in use. In this scenario, you can share used as well as unused capacity. This gives you significant freedom in restructuring the capacity allocation for your internal teams.

After split, ODCR-1 remains with total capacity of one, and ODCR-2 with total capacity of nine with two unused capacities

Figure 8: After split, ODCR-1 remains with total capacity of one, and ODCR-2 with total capacity of nine with two unused capacities

Scenario 2: managing shared ODCRs with partners (Capacity Reservation shared with other AWS account)

When you need to share your ODCR with a partner’s AWS account, consider this scenario where ODCR-1 has:

  • Total capacity of ten c5.2xlarge instances
  • Eight instances in use by both your team and your partner’s team
  • Two unused instances

Before split, ODCR-1 shared with another AWS account

Figure 9: Before split, ODCR-1 shared with another AWS account
In this case, your options are more limited. ODCR-1 is shared with your partner’s AWS account, thus you can only split the unused capacity (maximum of two instances). After split, the newly created ODCR (ODCR-2) remains in your AWS account and isn’t shared with any other AWS account. This restriction helps prevent any disruption to your partner’s running workloads while still allowing for some flexibility in capacity management.

After split, ODCR-1 remains shared with another AWS account, and newly created ODCR-2 isn’t shared

Figure 10: After split, ODCR-1 remains shared with another AWS account, and newly created ODCR-2 isn’t shared

These scenarios demonstrate important factors about capacity management in both internal and partner-shared environments. You should carefully consider the sharing status of ODCRs before planning any splits or modifications, making sure of smooth operations for both your teams and your partners.

Special considerations for move capability

The move capability enables you to redistribute available (or excess) capacity between ODCRs. However, in certain cases, you can also use this capability to move used instances between ODCRs. This capability is particularly helpful if you want to merge partially used ODCRs into one for easier tracking and management. Along with the considerations for moving unused capacity, the following considerations apply for moving used capacity:

  1. Both source and destination ODCR are of open instance eligibility and in active state.
  2. The instances running inside the reservation are of open eligibility (in other words they are not targeting the reservation).
  3. Both source and destination ODCRs are owned by the same account.
  4. The source and destination ODCRs can be shared, but with the same list of accounts when moving used portion. This sharing to same accounts condition doesn’t apply to the unused portion of the ODCR.

When you specify the quantity of instances to be moved, by default any unused capacity is moved first, followed by any eligible running instances (the used capacity in your reservation).

In the next sections, we review where you can or can’t use this capability.

Scenario 1: source and destination ODCRs not shared with other account(s) (Team Transfers)

When managing capacity between your internal teams using the same AWS account (Account-A), you find the process clear. For example, when consolidating the ML team’s resources:

  • ODCR-1 (ML Team A): had ten capacities total (all with open eligibility), with eight in use and two unused.
  • ODCR-2 (ML Team B): had five capacities (all with open eligibility), all in use.

Before move, ODCR-1 and ODCR-2 both in the same AWS account, unshared

Figure 11: Before move, ODCR-1 and ODCR-2 both in the same AWS account, unshared

Both ODCRs belonged to the same account and weren’t shared externally, and the ODCRs have open instance eligibility. Therefore, you could freely move all ten instances from ODCR-1 to ODCR-2, creating a unified pool of 15 instances for the consolidated DevOps team.

After moving capacity from ODCR-1, ODCR-2 has combined total capacity of 15 with 2 unused

Figure 12: After moving capacity from ODCR-1, ODCR-2 has combined total capacity of 15 with 2 unused

Scenario 2: source and destination ODCRs shared with the same account(s) (External Partner Collaboration)

If your ML team (ODCR-1) collaborates with an external AI research partner (Account-B), your setup might look like the following:

  • ODCR-1: ten instances (eight used, two unused), all with open instance eligibility, shared with the research partner through AWS RAM.
  • ODCR-2: Five instances (all used), all with open instance eligibility, for internal Analytics team.

Before move, ODCR-1 and ODCR-2 both in the same AWS account, with ODCR-1 shared with other AWS account

Figure 13: Before move, ODCR-1 and ODCR-2 both in the same AWS account, with ODCR-1 shared with other AWS account

When your Analytics team needs more capacity, you can only move the two unused instances from ODCR-1 to ODCR-2, as the other eight are actively used in the partner collaboration.

Since ODCR-1 is shared with other AWS account, only unused capacity is moved to ODCR-2

Figure 14: Since ODCR-1 is shared with other AWS account, only unused capacity is moved to ODCR-2

Scenario 3: source and destination ODCRs shared with different account(s) (Multi-Partner Projects)

In this scenario involving managing capacity across different partner engagements:

  • ODCR-1: Ten instances (eight used, two unused), shared with a database partner (Account-B).
  • ODCR-2: Five instances (all used), shared with a security partner (Account-C).

ODCR-1 and ODCR-2 are shared with different AWS account

Figure 15: ODCR-1 and ODCR-2 are shared with different AWS account

Due to the different partner arrangements, in other words ODCRs shared with another accounts, you can only move the two unused capacities from ODCR-1 to ODCR-2. This makes sure that there is no disruption to database partner workloads.

Only unused capacity moved to ODCR-2 due to shared capacity reservations

Figure 16: Only unused capacity moved to ODCR-2 due to shared capacity reservations

These scenarios teach valuable lessons about capacity management in multi-account environments. You can develop a comprehensive sharing strategy that balances flexibility with partner commitments, enabling you to optimize your resource usage while maintaining strong partner relationships.

Conclusion

The new ODCR features of AWS –a split, move, and modify – represent a significant advancement in cloud capacity management. For your organization, these features transform how you handle compute resources, enabling more efficient operations and cost management. The ability to dynamically adjust and share Capacity Reservations provides the flexibility you need while maintaining the stability necessary for your critical workloads.

As cloud infrastructure continues to evolve, these features demonstrate the AWS commitment to addressing real-world challenges that you face when managing complex cloud environments. If you’re looking to optimize your AWS infrastructure, then these new ODCR capabilities offer powerful tools for better capacity management and resource usage.

To enhance your understanding of these capabilities, we’ve created a GitHub repository containing APIs for implementation purposes. For more details, refer to the updated Capacity Reservations documentation. If you have any questions or feedback, feel free to share them in the comments section or contact AWS Support.

Architecting for seamless on-premises connectivity with AWS Outposts servers

Post Syndicated from aostan original https://aws.amazon.com/blogs/compute/architecting-for-seamless-on-premises-connectivity-with-aws-outposts-servers/

This post is written by Mark Nguyen, Principal Solutions Architect, AWS and Ryan Fillis, Solutions Architect, AWS.

AWS Outposts brings native AWS services, infrastructure, and operating models to virtually any data center, co-location space, or on-premises facility. Deploying Outposts servers in your environment necessitates additional considerations regarding local network connectivity and Amazon Elastic Compute Cloud (Amazon EC2) instance networking. This post demonstrates the scalability of Outposts servers through automation and the deployment of Amazon EC2 network interfaces. This reduces the number of manual steps required to configure an Outposts server.

This post details physically connecting your servers to your Local Area Network (LAN) and the networking options available for EC2 instances running on Outposts. We cover the physical cabling options, virtual networking components such as VPCs and subnets, and walkthrough an example setup for an EC2 instance with a user-data script to route traffic locally over your on-premises network.

This post assumes that you have some familiarity with Outposts servers. If you would like a general refresher, observe What is AWS Outposts. For more information about how to provision your Outposts server, see Installing an AWS Outposts server.

Basic Amazon EC2 networking using a single interface

When launching an EC2 instance on an Outposts server a single interface is created for network connectivity. This default setting, depicted in the following diagram, is the most direct method for your instance to communicate externally.

Figure 1 Simple network connectivity on an Outposts server

Figure 1: Simple network connectivity on an Outposts server

When deploying an EC2 instance to an Outposts server, there are certain differences in using the default Elastic Network Interface (ENI) as compared to deploying in an AWS Region. Understanding these differences is critical before modifying the network configuration, which you do in the next step.

ENI differentiators between Outposts servers and the Region:

  • Primary interface: The primary interface is an ENI. This ENI is associated to a subnet within a VPC. This VPC is extended from the Region to the Outposts server.
  • IP address configuration: The primary network interface within the guest operating system (OS) of the EC2 instance must be configured to obtain an IP address through DHCP. The assigned IP address is from the IP address range of the VPC subnet associated with the Outposts server.
  • Security group: A security group is associated with the ENI. This security group falls within the VPC that is extended from the Region. The user must apply appropriate access control rules to permit access to the EC2 instance. You may reuse security groups that already exist within the VPC.
  • Outbound traffic: By default, an EC2 instance uses its ENI to direct outbound traffic toward the VPC subnet. Traffic flows according to the routing table associated with the Outposts server’s VPC subnet.
  • Inbound traffic: If you’re only using an ENI, then traffic destined to EC2 instances on Outposts servers must traverse through the service link. In the preceding diagram, the user communicates with the EC2 instance over the internet. Traffic from the internet reaches the Region through the Internet Gateway of the VPC. Then, the VPC forwards the traffic to the appropriate subnet of the Outposts server (through the service link) and reaches the EC2 instance. The user must configure the necessary VPC components (Internet Gateway and associated routing table entries) for internet connectivity.
  • Local network connectivity: There is no local network connectivity using the ENI. For local network connectivity, see the next section where we discuss the Outposts server Local Network Interface (LNI).

Local network connectivity for EC2 instances

Outposts servers allow you to communicate through the Local Network Interface (LNI) in addition to the ENI. The LNI is a logical networking component that connects the Amazon EC2 instances in your Outposts subnet to your on-premises network.

The Outposts server EC2 instance local communications characteristics:

  • Local network traffic needs the use of an LNI.
  • The subnets on Outposts servers must be enabled for LNIs. This is done by entering the following command:

aws ec2 modify-subnet-attribute \

--subnet-id subnet-1a2b3c4d \

--enable-lni-at-device-index 1

  • IP address assignment for the LNI can be DHCP or static.
  • You can’t apply VPC security groups to the LNI. To control traffic on the LNI, you can use an OS based firewall, external on-premises firewall, or other security devices.
  • Amazon CloudWatch metrics are produced for each LNI.
  • Outposts servers don’t tag VLAN traffic. If VLAN tags are needed, then the network interface settings inside the guest OS must apply the VLAN tags. Multiple VLAN interfaces can exist within the same LNI (in this case you would be using the LNI as a VLAN trunk).
  • Local traffic bandwidth performance depends on the instance type. The larger the instance type, the higher performance the throughput of the LNI. The maximum throughput is 10 Gbps.
  • EC2 instances that communicate locally always have at least two interfaces: one ENI and one or more LNIs. Therefore, the instance OS’s routing table must be configured based on the desired traffic behavior.

Example configuration: Local traffic for EC2 instance on Outposts server

Figure 2 Example scenario topology

Figure 2: Example scenario topology

In the example scenario, we want to launch an Amazon Linux 2023 instance and route all default traffic through the local network. Eth0 is the primary interface (ENI) and is used for traffic towards the Region. Eth1 is the LNI and is used for all other traffic. A user-data script is used to make the necessary routing changes at launch.

Here is a sample user-data script. These commands run as root so there is no need to prepend each command with sudo.

User data script (my_userdata.txt):

#!/bin/bash 
route add -net 172.31.0.0/16 gw 172.31.239.1 
route del default gw 172.31.239.1 
cp -RL /run/systemd/network/* /etc/systemd/network/ 
echo -e '\n[Route]\nDestination=172.31.0.0/16\nGateway=172.31.239.1\nGatewayOnLink=yes' >> /etc/systemd/network/70-ens5.network 
sed -i -e 's/UseGateway=true/UseGateway=false/g' /etc/systemd/network/70- ens5.network.d/eni.conf

We can break down this script to observe the intent of each command:

route add -net 172.31.0.0/16 gw 172.31.239.1 
route del default gw 172.31.239.1

When an instance is launched on Outposts server, the instance automatically has a default route that points toward the VPC through the ENI. In the example scenario, the desired configuration is to have all default traffic go through the LNI toward our on-premises LAN, not through the ENI. To accomplish this routing behavior for the ENI, we have to add a route toward the VPC and remove the default route. The first line adds a route through the VPC (172.31.0.0/16), using 172.31.239.1 as the gateway. The second line removes the default route that uses 172.31.239.1 (via the ENI) as the gateway.

Traffic not destined for the VPC routes through the LNI. This includes all local traffic and internet-bound traffic. The local network’s DHCP server provides a default-gateway in its DHCP lease. Therefore, there is already a default route assigned to the LNI. This steers any traffic without a more specific route, including internet traffic, toward the LNI.

Next, the user-data script makes the network settings persistent after reboot. The procedure varies depending on your OS. In the case of Amazon Linux 2023, it uses systemd-networkd.

cp -RL /run/systemd/network/* /etc/systemd/network/

This command copies the configuration files from the /run/systemd/network/ folder to /etc/systemd/network/. The configuration files in the /etc/systemd/network/ folder override the default settings and load during boot. The next is step is to modify the newly copied network configuration files.

echo -e '\n[Route] \nDestination=172.31.0.0/16 \nGateway=172.31.239.1 \nGatewayOnLink=yes' >> /etc/systemd/network/70-ens5.network

In this case the ENI is ens5. This line appends the static route section to the 70-ens5.network configuration file. This makes the static route added earlier in the script (route add -net 172.31.0.0/16 gw 172.31.239.1) persistent across reboots.

sed -i -e 's/UseGateway=true/UseGateway=false/g' /etc/systemd/network/70- ens5.network.d/eni.conf

Next, the user-script edits the configuration file, eni.conf, such that the default route isn’t used for the ENI at bootup. This is accomplished using sed to search and replace true with false for the UseGateway parameter.

Launching an instance with ENI and LNI

Now that the user-data script has been created, use the AWS Command Line Interface (AWS CLI) to launch an EC2 instance:

aws ec2 run-instances \
--image-id ami-051f8a213df8bc089 \
--count 1 \
--instance-type c6id.xlarge \
--key-name my_key \
--user-data file://my_userdata.txt \
--network-interfaces '[ \
  { "DeviceIndex":0, "SubnetId":"subnet-0ca6abe6b34adfcce", "Groups": ["sg-0a9f8c2200c0a56f1"] }, \
  { "DeviceIndex":1, "SubnetId":"subnet-0ca6abe6b34adfcce", "Groups": ["sg-0a9f8c2200c0a56f1"] }]' \
--tag-specifications '[{ "ResourceType":"instance","Tags":[ \
  { "Key":"Name", "Value":"server1" } ] }]'

We can break down the parameters used in the preceding command:

--image-id ami-051f8a213df8bc089 \

This specifies the Amazon Machine Image (AMI) ID. ami-051f8a213df8bc089 is the AMI ID for Amazon Linux 2023 in us-east-1.

--count 1 \

This specifies how many EC2 instances to launch. You can launch multiple at the same time.

--instance-type c6id.xlarge

This specifies the instance type. By default, Outposts 2U servers are slotted with the c6id.8xlarge instance type and Outposts 1U servers are slotted with the c6gd.8xlarge instance type. You can adjust the slotting assignment during the ordering process or you can change the slotting assignment later by using the Self-service Capacity Management feature for AWS Outposts.

--key-name my_key

This specifies the public RSA key that is added to your EC2 instance. This key must already be defined in the same Region of your AWS account.

--user-data file://my_userdata.txt

This specifies the filename that contains your user-data script (that was created previously).

{ "DeviceIndex":0, "SubnetId":"subnet-0ca6abe6b34adfcce", "Groups": ["sg-0a9f8c2200c0a56f1"] }, \

{ "DeviceIndex":1, "SubnetId":"subnet-0ca6abe6b34adfcce", "Groups": ["sg-0a9f8c2200c0a56f1"] }]' \

This specifies the network interface configuration. By default, a single network interface, the ENI, is created. This example calls for a second interface for the LNI. DeviceIndex:0 is for the ENI and doesn’t change. DeviceIndex:1 is for the LNI, which we defined when we enabled LNI for the subnet (--enable-lni-at-device-index 1). The SubnetId refers to the subnet that was created on the Outposts server. If you want to deploy to a different Outposts server, then change the SubnetId. Groups refer to the security group that you would like assigned to the ENI. Security groups aren’t supported for the LNI, thus the security group specified for DeviceIndex:1 is only to comply with the command syntax check. A security group will not be applied to the LNI.

--tag-specifications '[{ "ResourceType":"instance","Tags":[ \

{ "Key":"Name", "Value":"server1" } ] }]'

This assigns a name to the EC2 instance, which in this case is server1.

Conclusion

AWS Outposts servers allow you to run native AWS services on-premises by providing local compute. This supports workloads with low latency and data residency requirements through on-premises processing.

Although Outposts servers integrate seamlessly with the AWS cloud, there are some unique networking considerations when deploying in your data center environment. Amazon EC2 instances on the Outposts server can route traffic over the AWS global network, but you can also enable Local Network Interfaces (LNIs) to directly access your on-premises networks.

In this post we’ve demonstrated using user-data scripts during instance launch to automate hybrid cloud networking flows tailored to your requirements. With proper planning, you can use the benefits of consistent AWS services and tooling while maintaining connectivity to your existing on-premises infrastructure.

Ready to get started with hybrid cloud networking on Outposts servers? Check out the Outposts server documentation and best practices guide to begin planning your on-premises deployment.

Streamlining AMI creation with EC2 Image Builder components in AWS Marketplace

Post Syndicated from Art Baudo original https://aws.amazon.com/blogs/compute/streamliningamicreationwith-ec2imagebuilder/

This post is written by Smriti Ohri, Senior Product Manager, EC2 and Omar Chehab, Senior Product Manager, AWS Marketplace.

At re:Invent 2024, Amazon Web Services (AWS) announced the availability of third-party EC2 Image Builder components in AWS Marketplace. EC2 Image Builder is a fully managed service that streamlines the customization, testing, distribution, and lifecycle management of images. You can use this new feature to procure third-party components from AWS Marketplace directly on the EC2 Image Builder console and in the AWS Marketplace website. You can add multiple of these components to create your golden images.

A golden image is a customized and pre-configured Amazon Machine Image (AMI) needed for launching Amazon Elastic Compute Cloud (Amazon EC2) instances. It includes a standardized set of software, configurations, and security settings that meet an organization’s specific requirements, promoting consistency and efficiency across all EC2 instances.

EC2 Image Builder provides Amazon managed components, and you can build your own components that help when building custom images. However, you may need third-party software to build your golden images. Procuring this software can be time-consuming and necessitates custom setup. This integration aims to address these challenges by providing the ability to add third-party software from AWS Marketplace directly while creating golden images using EC2 Image Builder. While creating the image, you can customize your image recipe to use the latest version of components published in AWS Marketplace and make sure that you always remain up to date.

This post shows you how to find, subscribe to, and incorporate components from AWS Marketplace using the EC2 Image Builder console.

Prerequisites

You must have access to subscribe to a product in AWS Marketplace. Check AWS Marketplace subscription permissions.

Solution overview

Three high-level steps are involved in using the third-party component from AWS Marketplace in EC2 Image Builder:

  1. Discover and subscribe to the third-party component on the EC2 Image Builder console.
  2. Build the golden image with the third-party component.
  3. Launch the EC2 instance using the golden image.

Solution walkthrough: Streamlining AMI creation with EC2 Image builder components in AWS Marketplace

To perform the solution, go through the steps in the following sections.

Discover and subscribe to a component by Cribl

To discover and subscribe to the component, follow these steps:

  1. On the EC2 Image Builder console, in the navigation pane, choose Discover products. On the Components tab, you can view the list of available AWS Marketplace image products and the associated components. As shown in the following screenshot, choose View subscription options, which shows the different pricing offered.

 Figure 1: Discover components on EC2 Image Builder console

 Figure 1: Discover components on EC2 Image Builder console

  1. To subscribe to the product, from the dropdown menu choose the available offers and choose Subscribe, as shown in the following screenshot. You can now start using the associated component in your image recipe.

Figure 2: Subscribe to the product that has the component

Figure 2: Subscribe to the product that has the component

Build the golden image with the third-party component

To use the component, you can either subscribe to it first, or you can create the pipeline and subscribe to the component later based on your preference. For this walkthrough, I already subscribed to the component. The following section shows how to create a pipeline to build a custom AMI using the component to which I subscribed. You can follow a similar process to install other components to create your golden AMIs. The high-level steps are:

  1. Create the recipe.
  2. Create the pipeline.

To create the recipe, follow these steps:

  1. On the EC2 Image Builder console, choose Image recipes and Create image recipe. A recipe has a base image and the components that you want to install on it.

For this example, Amazon Linux was chosen as the base image operating system and “Amazon Linux 2023 x86” as the image name.

  1. In the Build components section, choose Add build components and, from the dropdown, choose AWS Marketplace. Search for the component to which you subscribed and choose Add to recipe, as shown in the following screenshot.

You can choose to use the latest version or a specific version of the component. For this walkthrough, the latest available version was selected.

Figure 3: Create recipe and add components from AWS Marketplace

Figure 3: Create recipe and add components from AWS Marketplace

To create the pipeline, an automation configuration (where you define the infrastructure configuration), image workflows, and distribution configuration, follow these steps:

  1. On the EC2 Image Builder console, choose Image pipelines and Create image pipeline. Provide the name of the pipeline and choose a Build schedule. You can also enable scanning, which scans your AMIs for Common Vulnerabilities and Exposures (CVEs) using Amazon Inspector.

For more information, refer to Amazon Inspector integration in Image Builder in the EC2 Image Builder User Guide. For this example, image scanning is enabled and the option to manually trigger the pipeline was selected.

Figure 4: Create the pipeline with the recipe and other configurations

Figure 4: Create the pipeline with the recipe and other configurations

  1. Choose the recipe you created with third-party components from AWS Marketplace.
  2. Choose the image workflows for the image creation process and define infrastructure configurations for creating the image.

You can choose Dedicated Host, Dedicated Instance, or Shared Tenancy. By default, it uses Shared Tenancy. For this example, the default configuration was selected. I chose the c5.large instance type since that is the supported instance type for this component.

Figure 5: Select the supported instance type in the infrastructure configurations

Figure 5: Select the supported instance type in the infrastructure configurations

  1. Provide the distribution configuration details to share or copy the output image to other accounts and in other AWS Regions.

To allow these accounts to use any component from AWS Marketplace, you must share license entitlements with these accounts using AWS License Manager. Instructions for sharing license entitlements are outside the scope of this post. To learn more, refer to Associating licenses with AMI based products using AWS License Manager.

  1. Choose the pipeline that you created and choose Run pipeline. After a while, the image is created and ready to use.

Run the EC2 instance using the golden image

Create an EC2 instance with the output golden image. You can also view the product code stamped on the AMIs, as shown in the following figure.

 

Figure 6: View the output image to check the product code

Conclusion

This feature helps you save time and automate the process of using the latest versions of the software. With this integration, you get a diverse set of software components from verified sellers in AWS Marketplace to address the monitoring, security, governance, and compliance needs of your organization. You can learn more about these components in the documentation. Visit AWS Marketplace to view all supported EC2 Image Builder components.

If you’re an AWS Partner, then you can publish your software as components in AWS Marketplace to cater to your customers. To learn more about onboarding your software to AWS Marketplace, visit this blog post. You can reach out to [email protected] if you have questions about this new feature or the publishing process.

Start building your custom AMIs using components from Marketplace today.

Dynamically reconfigure your AWS Outposts capacity using Capacity Tasks

Post Syndicated from aostan original https://aws.amazon.com/blogs/compute/dynamically-reconfigure-your-aws-outposts-capacity-using-capacity-tasks/

This post is written by Brianna Rosentrater, Hybrid Edge Specialist SA and Adam Duffield, Senior Technical Account Manager.

AWS Outposts extends AWS infrastructure, AWS services, APIs, and tools to on-premises locations for workloads that require low latency, local data processing, or data residency. Outposts comes in a variety form factors, from 42U Outposts racks to 1U and 2U Outposts servers. Outposts now supports self-service capacity management, making it easy for you to view and manage compute capacity on your Outposts. A default capacity configuration for each new Outpost is determined during the ordering process. This default configuration can subsequently be modified to create a range of instance sizes and quantities to meet your changing business needs. To do so, you create a capacity task, specify the instance sizes and quantity, and run the capacity task to implement the changes. This post focuses on how to use capacity tasks to perform multi-host reconfigurations and view existing capacity configurations.

Overview

Amazon Elastic Compute Cloud (Amazon EC2) capacity on an Outpost is determined by the total volume of compute capacity within the Outpost when ordered. Outposts can also be scaled up or out as needed during your commitment term. For further details on Outpost capacity planning including best practices, refer to the Capacity Planning – AWS Outposts High Availability Design and Architecture whitepaper. We recommend planning spare capacity for N+M host availability when making modifications to your Outpost capacity configuration if your workloads need to be highly available. To calculate, take the number of hosts (N) you need to run all your workloads, and then add (M) additional hosts to meet your requirements for server availability during failure and maintenance events.

Viewing existing Outposts capacity configuration

Outposts users now have visibility into capacity configurations at both an instance family and host level. This gives greater insight into capacity usage and instance placements. Within the Outposts console, after choosing the Outpost ID on which you want to view the capacity configuration, two new views have been provided: the Instance view and the Rack view.

Instance view

The Instance view provides a granular breakdown of the currently deployed instances on the Outpost along with an overall view of instance family capacity pools and their usage, as shown in the preceding figure. The Instances section gives detailed information around the deployed instances, their associated instance ID, instance size, AWS managed service (if applicable), and asset ID of where the instance is running.

Figure 1 - Outposts Instance View

Figure 1 – Outposts Instance View

The Instance capacity distribution summary displays how the various instance sizes are allocated within each instance family, as shown in the following figure. Each host of the same instance family contributes its capacity to the overall pool, which is represented in this section as a percentage rather than number of instance slots. This shows the configured capacity, but it doesn’t reflect any level of usage.

Figure 2 - Outposts Instance Capacity Distribution Summary

Figure 2 – Outposts Instance Capacity Distribution Summary

The Instance capacity distribution details section, shown in the following figure, provides a more detailed breakdown of each instance family capacity pool. This section provides a view of the total available instance capacity, the number of used instances, and the number of instances that are unavailable you at that time (such as when a hardware failure occurs).

Figure 3 - Instance Capacity Distribution Details

Figure 3 – Instance Capacity Distribution Details

Rack view

The Rack view tab provides a more granular view of the overall configuration of each host on a per rack basis, as shown in the following figure. It allows you to analyze the spread and usage of the instance size allocations across each host (asset) and, when choosing the show instance details button, provides the instance ID of each used slot. Using the search box, you can filter by Instance Family or Instance Size to provide a more concise view. If you’re using Outposts server, the Rack view tab will show the capacity configuration of your server.

Figure 4 - Rack View

Figure 4 – Rack View

Obtaining a view of current configuration

Alongside these views, two buttons are available on each of the pages. The Export JSON button gives the ability to download a JSON formatted copy of the current configuration for an Outpost. This is especially useful if you’re looking to record current state, or wanting to use the JSON upload option when submitting a new capacity task. The JSON file structure provides the overall configuration of each capacity pool. However, it doesn’t provide any details in terms of usage. The second button, Modify Instance Capacity, provides a shortcut to creating a capacity task.

This level of capacity visibility is also now available through AWS Command Line Interface (AWS CLI)/Outposts API calls, which some may prefer over the console. For example, the list-assets CLI command can be used to obtain a breakdown of the capacity configuration of each Outpost host:

aws outposts list-assets --outpost-identifier outpost-arn

Figure 5 - list-assets CLI command sample output

Figure 5 – list-assets CLI command sample output

If you want to obtain details of running instances on an Outpost, such as the instance size, the asset ID on which an instance is running, and the related AWS service name (if relevant), then the list-asset-instances CLI command can be used:

aws outposts list-asset-instances --outpost-identifier outpost-arn

Figure 6 - list-asset-instances CLI command sample output

Figure 6 – list-asset-instances CLI command sample output

The list-asset-instances CLI command also allows you to filter through numerous dimensions, such as instance type or AWS service. For example, this can be particularly useful for quickly identifying all running instances of a certain type, such as the m5d.large instance type by using the following command:

aws outposts list-asset-instances --outpost-identifier outpost-arn --instance-type-filter m5d.large

Modifying the Outposts capacity configuration

Due to the finite nature of Outposts capacity, changing operational requirements often mean that adjustments need to be made to your Outposts capacity configuration over time as new workloads are identified or applications need scaling.

From the Outposts console page, choosing Capacity Tasks from the left-hand menu gives a list of previously run capacity tasks and their status. From here, you can choose Create Capacity Task to start the process. To create a capacity task there are two options available: using an interactive capacity configuration tool through the Modify an Outpost capacity configuration option, or uploading a JSON file containing the necessary configuration through the Upload a capacity configuration option.

Figure 7 - Capacity Tasks web form

Figure 7 – Capacity Tasks web form

The interactive Modify an Outpost capacity configuration option using the simple Auto-balance feature and UI is the easiest way for those unfamiliar with Outpost capacity management to get started with making changes. Using this option, you can also choose one of two methods for the task:

  • Run once: This results in the capacity task attempting to run a single time. If any instances block the successful application of the configuration, then the task fails.
  • Run periodically over 48 hours or less: In the event of blocking instances, the capacity task is paused until the instances are stopped. The task rechecks the status every 10 minutes until it can run. If instances aren’t stopped within 48 hours, then the task is cancelled.

To build out the capacity task, capacity pools are displayed, grouped by instance families, and automatically populated with the current configuration of instance sizes and corresponding vCPU allocation. From here, you can make necessary changes to the existing capacity. Specifically, you can add new instance sizes and amend instance quantities in the corresponding fields. The total vCPU count for each instance family will update automatically to reflect your changes. The Auto-balance allows you to automatically adjust the quantities of individual instance sizes to fit within the total vCPU capacity available for the host, which is reflected for each capacity pool at the end. In the event that a capacity pool is over- or under-used, a warning is displayed. You can under provision a host if you choose, but the unprovisioned capacity is unusable, and attempting to overprovision the host results in the capacity task failing to run.

Figure 8 - Modifying existing capacity configuration

Figure 8 – Modifying existing capacity configuration

When the necessary changes have been made to the capacity pools, the second part of the capacity task configuration is choosing instances that should not be impacted by the running of the capacity task. During the run, you may not be in a position to stop certain instances, such as databases or AWS managed services such as Elastic Load Balancing (ELB) or Amazon ElastiCache, due to the impact to production workloads. Choosing these instances allows the capacity task to automatically try to find a path that avoids impacting them. However, in some situations capacity tasks may fail if the chosen instances block the successful running of the task, and there is no possible solution that avoids all the chosen instances. For example, if a capacity task was to remove all c5.xlarge instances and an instance was chosen to ‘keep as-is’ that was running on this instance size, then the task would fail. To avoid this, make sure to include these instances in your capacity configuration. For example, if you have five critical m5.4xlarge instances that must remain running, then include 5 m5.4xlarge instances in your m5 capacity pool configuration.

Figure 9 - Instances to keep as-is

Figure 9 – Instances to keep as-is

After configuring the capacity pools and choosing the necessary instances to keep as-is, an overview of the changes is presented allowing you to validate the configuration prior to running. When you have reviewed the summary, select Create Task to trigger the execution of the capacity task using the chosen method. You can observe the status of a capacity task by choosing the capacity task ID. When it’s initially submitted, the status shows as Requested. During this time, the capacity task evaluates the necessary changes to determine if the task can proceed or if instances need stopping. If the Run once option was chosen and instances do need stopping, then the task moves to a cancelled status and provides details of the blocking instances. Alternatively, if Run periodically was chosen, the task remains in the Requested status until the listed instances have been stopped. In the event that the blocking instances can’t be stopped, the capacity task is cancelled after 48 hours. While a task is running, the Outpost hosts that are impacted by the configuration changes are placed into an isolated state. This means that capacity for new instance launches may be impacted. This isolation only lasts a few minutes while the capacity task is running, but it may impact auto scaling groups if a capacity task coincides with a scaling event.

Instead of using the interactive capacity configurator UI, you can also choose to upload a JSON file to the console containing the necessary configuration. Using this method, choosing instances to keep as-is isn’t available, and the method is automatically chosen as Run once. When a capacity task JSON file is uploaded, the resulting plan is displayed in the following text box and can be amended if needed. Alternatively, rather than uploading the file, the contents can be directly pasted into the text box. Choosing Next moves to the review screen where the remainder of the process continues in line with using the interactive capacity configurator UI.

Figure 10 - Upload a Capacity Configuration using JSON

Figure 10 – Upload a Capacity Configuration using JSON

You may also prefer using the AWS CLI/Outposts API for creating capacity tasks, and a number of new CLI/API actions are now available to support this:

  • cancel-capacity-task / CancelCapacityTask
  • get-capacity-task / GetCapacityTask
  • list-blocking-instances-for-capacity-task / ListBlockingInstancesForCapacityTask
  • list-capacity-tasks / ListCapacityTasks
  • start-capacity-task / StartCapacityTask

In addition to the same options available within the console, it is possible to request a dry run of the capacity task to determine if the instance type and instance size changes are above or below the available instance capacity. Requesting a dry run doesn’t make any changes to your plan.

For example, using the CLI to submit a capacity task to homogeneously slot an Outpost with 2 x m5 and 2 x c5 hosts (192 vCPU for each capacity pool) with xlarge instance sizes, and using periodic running could be achieved by running the following:

aws outposts start-capacity-task \

--outpost-identifier outpost-arn \

--instance-pools '[{"InstanceType":"c5.xlarge","Count":48},{"InstanceType":"m5.xlarge","Count":48}]' \

--task-action-on-blocking-instances WAIT_FOR_EVACUATION

Conclusion

This post demonstrated how to run a capacity task on Outposts and view your existing capacity configuration. For more information on how to manage and monitor your capacity configuration on Outposts, see Capacity management for AWS Outposts user guide and the Capacity planning section of the AWS Outposts High Availability Design and Architecture Considerations whitepaper, and the Modify AWS Outposts instance capacity – Outposts rack/Modify AWS Outposts instance capacity – Outposts server user guide sections for your respective environment. Reach out to your AWS account team to learn more about Outposts and self-service capacity management.

Enhance your workload resilience with new Amazon EMR instance fleet features

Post Syndicated from Deepmala Agarwal original https://aws.amazon.com/blogs/big-data/enhance-your-workload-resilience-with-new-amazon-emr-instance-fleet-features/

Big data processing and analytics have emerged as fundamental components of modern data architectures. Organizations worldwide use these capabilities to extract actionable insights and facilitate data-driven decision-making processes. Amazon EMR has long been a cornerstone for big data processing in the cloud. Now, with a suite of exciting new features for EMR instance fleets that enables you to effectively manage your compute, Amazon is taking cloud-based analytics to the next level.

Amazon EMR has introduced new features for instance fleets that address critical challenges in big data operations. This post explores how these innovations improve cluster resilience, scalability, and efficiency, enabling you to build more robust data processing architectures on AWS. This comprehensive post introduces instance fleets, demonstrates using this new allocation strategy, explores how enhanced Availability Zone and subnet selection works, and examines how these features improve cluster’s resilience. This technical exploration will equip you with the knowledge to implement more resilient and efficient EMR clusters for your organization’s big data processing needs.

The current challenges

Organizations using big data operations might face several challenges:

  • When preferred instance types are unavailable, finding suitable alternatives often delays cluster launches and disrupts workflows
  • Selecting the optimal Availability Zone for cluster launch is challenging due to constantly changing available compute capacity, especially when considering future scaling needs
  • Maintaining uninterrupted operation of mission-critical long-running clusters becomes complex as data processing requirements evolve over time
  • Organizations frequently struggle to scale their operations to meet growing data processing demands, leading to performance bottlenecks and delayed insights

These challenges underscore the need for more advanced, flexible, and intelligent solutions in the realm of big data operations, driving the demand for innovative features in cloud-based data processing platforms.

Introducing improved EMR instance fleets

Amazon EMR, a cloud-based big data platform, allows you to process large datasets using various open source tools such as Apache Spark, Apache Flink, and Trino. To address the aforementioned challenges, Amazon EMR introduced instance fleets, with a robust set of features.

When setting up an EMR cluster, Amazon EMR offers two configuration options for configuring the primary, core, and task nodes: uniform instance groups or instance fleets.

Uniform instance groups offer a streamlined approach to cluster setup, allowing up to 50 instance groups per cluster. An EMR cluster has a primary instance group for primary node, a core instance group with one or more Amazon Elastic Compute Cloud (Amazon EC2) instances, and the option to add up to 48 task instance groups. Both core and task instance groups are flexible, allowing any number of EC2 instances within each group. Both core and task groups offer flexibility in instance count, and each node type (primary, core, or task) consists of instances sharing the same specifications and purchasing model (On-Demand or Spot). However, this approach limits the ability to mix different instance types or purchasing options within a single group.

Instance fleets provide a versatile approach to provisioning EC2 instances, offering unparalleled flexibility in cluster configuration. This setup assigns one instance fleet each for primary and core nodes, with the task instance fleet being optional. It allows you to specify up to five EC2 instance types (or up to 30 when using the Amazon Command Line Interface (AWS CLI) or API with an instance allocation strategy) for each node type in a cluster, providing enhanced instance diversity to optimize cost and performance while increasing the likelihood of fulfilling capacity requirements. Instance fleets automatically manage the mix of instance types to meet specified target capacities for On-Demand and Spot, reducing operational overhead and improving compute availability.

Key benefits of instance fleets include improved cluster resilience to capacity fluctuations, superior management of Spot Instances with the ability to set timeouts and specify actions if Spot capacity can’t be provisioned, and faster cluster provisioning. The feature also allows you to select multiple subnets for different Availability Zones, enabling Amazon EMR to optimally launch clusters and automatically route traffic away from impacted zones during large-scale events. Additionally, instance fleets offer capacity reservation options for On-Demand Instances and support allocation strategies that prioritize instance types based on user-defined criteria, further enhancing the flexibility and efficiency of EMR cluster management.

Achieve resiliency with instance fleets

Now that you have a good understanding of instance fleets, let’s explore how the new instance fleet capabilities help achieve resiliency for your workloads through the following methods:

  • EC2 instance allocation – Enables precise control over instance type selection and prioritization
  • Enhanced subnet selection – Optimizes cluster deployment across Availability Zones

EC2 instance allocation

EMR instance fleets now offer newer allocation strategies for both Spot and On-Demand Instances, giving you control over selection and prioritization of instance types and allowing you to optimize for greater flexibility, resilience, and cost-efficiency.

Amazon EMR supports the following allocation strategies for On-Demand Instances:

  • Prioritized (new) – Allows you to define a priority order for instance types, giving you precise control over instance selection
  • Lowest-price (existing) – Selects the lowest-priced instance type from the available options

Amazon EMR supports the following allocation strategies for Spot Instances:

  • Price-capacity optimized (new) – Selects instances with the lowest price while also considering the available capacity
  • Capacity-optimized-prioritized (new) – Similar to capacity-optimized, but respects instance type priorities that you specify, on a best-effort basis
  • Capacity-optimized (existing) – Selects instances from the pools with the most available capacity
  • Lowest-price (existing) – Selects the lowest-priced Spot Instances
  • Diversified (existing) – Distributes instances across all pools

When using the prioritized On-Demand allocation strategy, Amazon EMR applies the same priority value to both your On-Demand and Spot Instances when you set priorities.

For Spot Instances, Amazon EMR recommends the capacity-optimized allocation strategy. This approach allocates instances from the most available capacity pools, thereby reducing the chance of interruptions and enhancing cluster stability. Amazon EMR also allows you to launch a cluster without an allocation strategy. However, using an allocation strategy is recommended for faster cluster provisioning, more accurate Spot Instance allocation, and fewer Spot Instance interruptions.

Enhanced subnet selection

Amazon EMR on EC2 offers improved reliability and cluster launch experience for instance fleet clusters through the newly launched enhanced subnet selection. With this feature, EMR on EC2 reduces cluster launch failures resulting from an IP address shortage. Previously, the subnet selection for EMR clusters only considered the available IP addresses for the core instance fleet. Amazon EMR now employs subnet filtering at cluster launch and selects one of the subnets that have adequate available IP addresses to successfully launch all instance fleets. If Amazon EMR can’t find a subnet with sufficient IP addresses to launch the whole cluster, it will prioritize the subnet that can at least launch the core and primary instance fleets. In this scenario, Amazon EMR will also publish an Amazon CloudWatch alert event to notify the user. If none of the configured subnets can be used to provision the core and primary fleet, Amazon EMR will fail the cluster launch and provide a critical error event. These CloudWatch events enable you to monitor your clusters and take remedial actions as necessary. This capability is enabled by default when you configure more than one subnet for cluster launch, and you don’t need to make any configuration changes to benefit from it.

Solution overview

Now that you have a comprehensive grasp of the two new features, let’s integrate the elements of instance fleets and look at the implementation flow for each feature.

EC2 instance allocation

The following diagram illustrates the instance fleet lifecycle management architecture.

The workflow consists of the following steps:

  1. Create a cluster configuration with the prioritized allocation strategy, specifying instance types, their priority, and a list of potential subnets.
  2. When you launch an EMR cluster, it evaluates compute capacity and available IPs across the specified subnets. Amazon EMR then selects a single Availability Zone that best meets capacity and instance availability needs for the entire cluster.
  3. Amazon EMR launches the cluster using available instance types in one of the configured Availability Zones based on enhanced subnet selection.
  4. During a scale-up scenario, Amazon EMR adds new instances to the clusters while following the configured compute allocation strategy.
  5. If a specific instance type is unavailable, Amazon EMR will select the next available instance types based on the priority order. This flexibility provides capacity availability for production workloads while maintaining scalability.

The following example code provisions an EMR cluster with a primary and core instance fleet configuration with both Spot and On-Demand Instances, using the Capacity-optimized-prioritized allocation strategy for Spot Instances and the Prioritized strategy for On-Demand Instances:

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Resources": {
    "myCluster": {
      "Type": "AWS::EMR::Cluster",
      "Properties": {
        "Instances": {
          "MasterInstanceFleet": {
            "Name": "cfnPrimary",
            "InstanceTypeConfigs": [
              {
                "BidPrice": "10.50",
                "InstanceType": "m5.xlarge",
                "Priority": "1",
                "EbsConfiguration": {
                  "EbsBlockDeviceConfigs": [
                    {
                      "VolumeSpecification": {
                        "VolumeType": "gp2",
                        "SizeInGB": 32
                      }
                    }
                  ]
                }
              }
            ],
            "TargetOnDemandCapacity": 1
          },
          "CoreInstanceFleet": {
            "Name": "cfnCore",
            "InstanceTypeConfigs": [
              {
                "BidPrice": "10.50",
                "InstanceType": "m5.xlarge",
                "Priority": "1",
                "WeightedCapacity": "1",
                "EbsConfiguration": {
                  "EbsBlockDeviceConfigs": [
                    {
                      "VolumeSpecification": {
                        "VolumeType": "gp2",
                        "SizeInGB": 32
                      }
                    }
                  ]
                }
              }
            ],
            "LaunchSpecifications": {
              "SpotSpecification": {
                "TimeoutAction": "SWITCH_TO_ON_DEMAND",
                "TimeoutDurationMinutes": 20,
                "AllocationStrategy": "CAPACITY_OPTIMIZED_PRIORITIZED"
              },
              "OnDemandSpecification": {
                "AllocationStrategy": "PRIORITIZED"
              }
            },
            "TargetOnDemandCapacity": "5",
            "TargetSpotCapacity": "0"
          }
        },
        "Name": "blog-test",
        "JobFlowRole": "EMR_EC2_DefaultRole",
        "ServiceRole": "EMR_DefaultRole",
        "ReleaseLabel": "emr-7.2.0"
      }
    }
  }
}

Enhanced subnet selection

To better understand Step 3 in the preceding workflow, let’s explore how enhanced subnet selection works with instance fleet EMR clusters.

For our example, let’s configure an EMR instance fleet as follows:

  • Primary fleet (1 unit) – r8g.xlarge, r6g.xlarge, r8g.2xlarge
  • Core fleet (48 units) – r6g.xlarge, r6g.2xlarge, m7g.2xlarge
  • Task fleet (48 units) – m7g.2xlarge, r6g.xlarge, r6a.4xlarge

For this example, let’s use the lowest price allocation strategy. Next, let’s check the available IP addresses in our subnets using the AWS CLI:

aws ec2 describe-subnets 
--query "sort_by(Subnets, &SubnetId)[*].[SubnetId, AvailableIpAddressCount, AvailabilityZoneId]" 
--output table

We get the following results:

--------------------------------------------------
|                 DescribeSubnets                |
+---------------------------+-------+------------+
|subnet-XXXXXXXXXXXXXXXX1   |  27  |  us-east-1a |
|subnet-XXXXXXXXXXXXXXXX2   |  251 |  us-east-1b |
|subnet-XXXXXXXXXXXXXXXX3   |  11  |  us-east-1a |
-------------------------------------------------

When launching an EMR cluster, Amazon EMR follows a specific subnet filtering process. First, EMR on EC2 evaluates subnets based on the total IP addresses required for all node types: primary, core, and task nodes. If multiple subnets have sufficient IP capacity to accommodate all instance fleets, Amazon EMR selects one based on the cluster’s allocation strategy. However, if no subnet has enough IPs to support all node types, Amazon EMR considers subnets that can at least accommodate the primary and core nodes, again using the allocation strategy to make the final selection. In our case, Amazon EMR selected a subnet in Availability Zone us-east-1b that had 251 available IPs that can support 97 instances to launch the whole cluster, bypassing smaller subnets with only 27 or 11 available IPs because they didn’t meet the minimum IP requirements for the cluster configuration.

  • Primary fleet (1 unit) – r6g.xlarge
  • Core fleet (48 units) – m7g.2xlarge
  • Task fleet (48 units) – r6g.xlarge

The EMR and CloudWatch event for this cluster would be:

Amazon EMR cluster j-X40BEI1Oxxx (Cluster) 
is being created in subnet (subnet-XXXXXXXXXXXXXXXX2) 
in VPC (vpc-XXXXXXXXXXXXXXXX1) in Availability Zone (us-east-1b), 
which was chosen from the specified VPC options.

If Amazon EMR can’t find a subnet with sufficient IP addresses to launch the entire cluster, it will prioritize launching the core and primary instance fleets. If no configured subnet can accommodate even the core and primary fleets, Amazon EMR will fail the cluster launch and provide a critical error event. These CloudWatch events enable you to monitor your clusters and take necessary actions.

Conclusion

The latest enhancements to EMR instance fleets mark a significant advancement in cloud-based big data processing, addressing key challenges in resource allocation, scalability, and reliability. These features, including priority-based instance selection and enhanced subnet selection, provide you with greater control over resource strategies, improved cluster availability, enhanced capacity optimization across Availability Zones, and more efficient fallback mechanisms for production workloads. Instance fleets help you tackle current resource management challenges while laying the groundwork for future scalability.

Get started today by setting up an EMR cluster using the example configuration provided in this post. For additional configuration options and implementation guidance, refer here or reach out to your AWS account team.

About the Authors

Deepmala Agarwal works as an AWS Data Specialist Solutions Architect. She is passionate about helping customers build out scalable, distributed, and data-driven solutions on AWS. When not at work, Deepmala likes spending time with family, walking, listening to music, watching movies, and cooking!

Ravi Kumar Singh is a Senior Product Manager Technical-ES (PMT) at Amazon Web Services, specialized in building petabyte-scale data infrastructure and analytics platforms. With a passion for building innovative tools, he helps customers unlock valuable insights from their structured and unstructured data. Ravi’s expertise lies in creating robust data foundations using open source technologies and advanced cloud computing that power advanced artificial intelligence and machine learning use cases. A recognized thought leader in the field, he advances the data and AI ecosystem through pioneering solutions and collaborative industry initiatives. As a strong advocate for customer-centric solutions, Ravi constantly seeks ways to simplify complex data challenges and enhance user experiences. Outside of work, Ravi is an avid technology enthusiast who enjoys exploring emerging trends in data science, cloud computing, and machine learning.

Mandisa Nxumalo is a Cloud Engineer at Amazon Web Services (AWS) with over 5 years experience in topics related to cloud services (databases, automation, and others). Currently, specializing in Big data service Amazon EMR. She is passionate about engaging customers to effectively adopt and utilize data driven approaches to improve their big data workflows. Outside work, Mandisa enjoys hiking mountains, chasing waterfalls and travelling across countries.

Kashif Khan is a Sr. Analytics Specialist Solutions Architect at AWS, specializing in big data services like Amazon EMR, AWS Lake Formation, AWS Glue, Amazon Athena, and Amazon DataZone. With over a decade of experience in the big data domain, he possesses extensive expertise in architecting scalable and robust solutions. His role involves providing architectural guidance and collaborating closely with customers to design tailored solutions using AWS analytics services to unlock the full potential of their data.

Gaurav Sharma is a Specialist Solutions Architect (Analytics) at AWS, supporting US public sector customers on their cloud journey. Outside of work, Gaurav enjoys spending time with his family and reading books.

AWS Weekly Roundup: DeepSeek-R1, S3 Metadata, Elastic Beanstalk updates, and more (February 3, 2024)

Post Syndicated from Donnie Prakoso original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-deepseek-r1-s3-metadata-elastic-beanstalk-updates-and-more-february-3-2024/

Last week, I had an amazing time attending AWS Community Day Thailand in Bangkok. This event came at an exciting time, following the recent launch of the AWS Asia Pacific (Bangkok) Region. We had over 300 attendees and featured 15 speakers from the community, including an AWS Hero and 4 AWS Community Builders who shared their technical expertise and experiences.

The highlight was definitely Jeff Barr, AWS Vice President & Chief Evangelist, delivering an inspiring keynote titled “Next-Generation Software Development”, which set the perfect tone for the day. The day kicked off with welcoming remarks from Vatsun Thirapatarapong, AWS Country Manager for Thailand, and was made even more special thanks to the tremendous support from both the AWS User Group volunteers and the AWS Thailand team.

Here’s a photo capturing the excitement from the event: 

Last week’s AWS Launches
There are 30+ launches last week and here are some launches that caught my attention:

DeepSeek-R1 models now available on AWS — Channy wrote on how you can now deploy DeepSeek-R1 models in Amazon Bedrock and Amazon SageMaker AI. This helps you to build and scale generative AI applications with minimal infrastructure investment.

Amazon S3 Tables increases table limit to 10,000 per bucket — S3 Tables now supports creating up to 10,000 tables in each table bucket, allowing you to scale up to 100,000 tables across 10 buckets within an AWS Region per account.

Amazon S3 Metadata now generally available — S3 Metadata provides automated and easily queried metadata that updates in near real-time, simplifying business analytics and real-time inference applications. It supports both system-defined and custom metadata, including integration with AWS analytics services.

AWS Amplify adds TypeScript Data client support for Lambda functions — Developers can now use the Amplify Data client within AWS Lambda functions, enabling consistent type-safe data operations across frontend and backend applications.

AWS Elastic Beanstalk adds Python 3.13, .NET 9, and PHP 8.4 support on Amazon Linux 2023 — AWS Elastic Beanstalk brings the latest language features and improvements to application deployments while benefiting from Amazon Linux 2023 enhanced security and performance features.

From community.aws
Here’s my top 5 personal favorites posts from community.aws:

Upcoming AWS and community events
Check your calendars and sign up for upcoming AWS and community events:

  • AWS Korea re:Invent reCap Online, February 2-4 — A virtual event recapping key announcements and innovations from re:Invent 2023 for the Korean audience.
  • AWS Community Days – Join community-led conferences that feature technical discussions, workshops, and hands-on labs. Upcoming AWS Community Day is in Ahmedabad (February 8).
  • AWS Public Sector Day London, February 27 — Join public sector leaders and innovators to explore how AWS is enabling digital transformation in government, education, and healthcare.
  • AWS Innovate GenAI + Data Edition — A free online conference focusing on generative AI and data innovations. Available in multiple Regions: APJC and EMEA (March 6), North America (March 13), Greater China Region (March 14), and Latin America (April 8).

Browse more upcoming AWS led in-person and virtual developer-focused events.

AWS Community re:Invent re:Caps

Lastly, if you want to learn about top announcements and innovations from AWS re:Invent, the AWS Community shares a summary from a community perspective of these announcements so you can get up to speed. Download the AWS Community re:Invent re:Caps deck. 

That’s all for this week. Check back next Monday for another Weekly Roundup!

— Donnie

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

Automating Notifications for Future-Dated Amazon EC2 Capacity Reservation State Changes

Post Syndicated from aostan original https://aws.amazon.com/blogs/compute/automating-notifications-for-future-dated-amazon-ec2-capacity-reservation-state-changes/

This post is written by Ballu Singh, Principal Solutions Architect at AWS, Sandeep Rohilla, Senior Solutions Architect at AWS and Pranjal Gururani, Senior Solutions Architect at AWS.

AWS customers are able to proactively reserve future-dated Amazon EC2 On-Demand Capacity Reservations (known as future-dated CRs) to get capacity assurance for workloads and events. Because reservations can be created weeks in advance, customers are able to ensure that they can monitor their real-time status. Future-dated CRs can transition through various state of a Capacity Reservation, based on capacity availability. Over the course of several weeks, the status of a reservation might only change 1 or 2 times. Even with the low frequency of changes, most customers don’t want to manually poll the state of their reservations, instead they want to be proactively notified when something changes.

Amazon EventBridge is a serverless event bus service that facilitates the routing of events and EventBridge rules establish how events are processed, allowing you to filter events and route them to one or more targets for action. Amazon Simple Notification Service (SNS) is a fully managed messaging service that enables you to send messages or notifications to a variety of endpoints, facilitating scalable, and decoupled communication between different components of your application or with end-users.

In this blog post, we guide you through the process of automating notifications for future-dated CR state changes using AWS services such as Amazon EventBridge and Amazon Simple Notification Service (SNS).

Pre-requisites

  • Existing future-dated Capacity Reservation. To learn how to create future-dated CRs, visit our blog post here.
  • IAM access to create Amazon EventBridge rules
  • IAM access to create Amazon SNS topic and subscription

Architecture

Amazon EC2 continuously monitors the state of your Capacity Reservations and sends events to Amazon EventBridge when Capacity Reservation states change. Using Amazon EventBridge, you can create rules that trigger notifications through Amazon SNS in response to these events. Amazon SNS then pushes these notifications to a variety of supported endpoint types such as Amazon Data Firehose, Amazon Simple Queue Service (SQS), AWS Lambda, HTTP, email, mobile push notifications, and mobile text messages (SMS). In our example, we will send notification to our email address.

Send notification to email address

Walkthrough

To automate the notification process for future-dated CR state changes, we will use the following AWS services:

  1. Amazon Simple Notification Service (SNS): We will use SNS to send email notifications to the designated recipients.
  2. Amazon EventBridge: We will create an EventBridge rule to capture the state change events of future-dated CRs.

AWS Console Walkthrough

Step 1: Set Up Amazon SNS Topics and Subscriptions

  • Create a new topic for future-dated CR state change notifications.
    • Navigate to Amazon SNS console. On the Topics page, choose Create topic
    • In the Details section, for Type, choose Standard and enter the name of Topic (ex: Subscriber)

Set up amazon sns topics and subscriptions

    • Choose rest of details as is. Choose Create topic.
  • Create a subscription for the desired recipients, such as email addresses to receive the notifications.
    • Navigate to Amazon SNS console. In the left navigation pane, choose Subscriptions. Choose Create subscription.
    • On the Create subscription page, in the Details section, for Topic ARN, choose the Amazon Resource Name (ARN) of a topic enter topic ARN noted above. For Protocol, choose Email/Email-JSON
    • For Endpoint, enter your email address. Keep rest of details as is.

create subscription for the desired recipients

    • Choose Create subscription
    • Navigate to your email and choose Confirm subscription in the email from Amazon SNS.

Step 2: Create an Amazon EventBridge Rule

  • Next open the Amazon EventBridge console and navigate to the Rules
  • Click on Create rule and provide a name (ex: EC2CapacityReservationStateChange) and description for the rule. Keep all other setting as it is. Select
  • For Event source, choose AWS events or EventBridge partner events.
  • In the Creation method section, for Method, choose Use pattern form.
  • For Event source, choose AWS services. Under AWS Service, choose EC2 and then choose EC2 Capacity Reservation.
  • Optionally, you can narrow down the state and reservation ID you want to be alerted for by selecting Specific Capacity Reservation State and Specific Capacity reservation ID. Select Next.

Create an amazon eventbridge rule

  • In the Target section, under Target Type, select AWS Service.
  • For Select Target, choose SNS topic, under Topic, select SNS Topic you created in Step 1. Select
  • On Configure Tags page, select Next.
  • On Review and create page, select Create Rule.

CloudFormation Walkthrough

To simplify the setup process and make it easier for you to implement the automated notification solution, we have provided a CloudFormation template. This template automates the creation of the necessary Amazon EventBridge rule and Amazon SNS topic, along with the required configurations and permissions.

  1. Download yaml sample template.
  2. Navigate to AWS CloudFormation console and click on Create stack.
  3. Choose Upload a template file and select the downloaded template. Choose Next
  4. Provide a name for the stack under Stack Name. For CapacityReservationId, enter ID of the EC2 Capacity Reservation to monitor, (e.g., cr-1234567890abcdef0), for EmailAddress, enter email address you want to subscribe to the SNS topic, and for MonitoredStates enter comma-separated list of Capacity Reservation states to monitor (e.g. failed, expired, cancelled, pending). Choose Next

CloudFormation walkthrough

5. On Configure stack options, keep defaults. Choose

6. On Review and create page, choose

Clean up

To avoid ongoing charges, clean up your environment, by following these steps to delete the resources you created by following this blog, if they are no longer needed:

If you followed setup for AWS Console, please follow the below steps:

  1. Delete the Amazon EventBridge Rule
    1. Navigate to Amazon EventBridge console and choose to the Rules from left pane.
    2. Choose the rule you created earlier and click on Delete.
    3. Confirm the deletion by clicking Delete in the confirmation dialog.
  2. Delete the Amazon SNS Topic and Subscriptions
    1. Navigate to Amazon SNS console and navigate to the Topics from left pane.
    2. Choose the topic you created earlier and click on Delete.
    3. Confirm the deletion by clicking Delete in the confirmation dialog.

If you created any subscriptions for the topic (e.g., email or SMS), they will be automatically deleted along with the topic.

If you deployed CloudFormation template, please follow the below steps:

Conclusion

In this blog post we walked you through setting up an EventBridge rule to capture the state change events of your future-dated CRs and configure SNS to send notifications to the designated recipients. This automated approach eliminates the need for manual monitoring and ensures that you stay informed about the status of your capacity reservations.

By proactively managing your future-dated CRs through automated notifications, you can make informed decisions, adjust your reservation plans if need, and take corrective actions to ensure you have the necessary resources available for your critical events.

This solution enhances your operational efficiency, reduces the risk of capacity shortages, and allows you to focus on other important aspects of your business.

We encourage you to implement this automated notification system for your future-dated Amazon EC2 On-Demand Capacity Reservations and experience the benefits of streamlined monitoring and proactive capacity management.

Implementing backup for workloads running on AWS Outposts servers

Post Syndicated from aostan original https://aws.amazon.com/blogs/compute/implementing-backup-for-workloads-running-on-aws-outposts-servers/

This post is written by Leonardo Queirolo, Senior Cloud Support Engineer and Tareq Rajabi, Senior Solutions Architect, Hybrid Cloud

AWS Outposts servers provide fully managed AWS infrastructure, services, APIs, and tools to on-premises and edge locations with limited space or small capacity requirements, such as retail stores, branch offices, healthcare provider locations, or factory floors. Outposts servers provide local compute and networking services.

Outposts servers come with internal NVMe SSD instance storage, supporting local storage used for data access and processing on premises, and for launching Amazon Elastic Block Store (Amazon EBS)-backed Amazon Machine Images (AMIs). The data on these volumes persists after an instance reboot but does not persist after an instance termination. In order for data to persist beyond the lifetime of the instance, it is important to back up your data to a persistent AWS storage, such as an Amazon Simple Storage Service (Amazon S3) bucket or an Amazon Elastic Block Store (Amazon EBS) volume.

In this post, we explore several approaches to back up the data stored in the instance storage volumes of your EC2 instances running on an Outposts server to a persistent storage solution from AWS, and explore their benefits and use cases.

Planning for failure

When evaluating a backup strategy, it’s important to understand the failure modes you are looking to recover from. Some examples are ransomware attacks, accidental data deletion, hardware failure, or a wide scale issue impacting the whole facility where your Outposts servers and on-premises devices (such as network switches, storage appliances) reside. These failures come in many forms and are often unplanned and unexpected events. Next, understand what is considered acceptable recovery for your business. For example, what are the Recovery Time Objective (RTO) and Recovery Point Objective (RPO) for your workload running on Outposts servers? These two values, defined by your organization, profile how long a service can be down during recovery and quantify the acceptable amount of data loss, helping you define the appropriate backup strategy.

Scenario 1: Backup to AWS storage in an AWS Region

Backup to an AWS Region enables data redundancy outside of the data center or facility where your Outpost resides, taking advantage of the durability, high availability, and scalability provided natively by the storage in the Region. This approach offers flexibility for restoration to the Region or to an Outposts server in a different edge location if the original data center/facility is impacted by an irrecoverable incident. However, when restoring the data back to an Outposts server, this approach could result in relatively high RTO, depending on the throughput of the service link and the amount of data to restore. In the following sections, we will cover using the AWS Elastic Disaster Recovery (AWS DRS) and an open source solution based on operating system tools and AWS Systems Manager (AWS SSM).

Option 1: AWS Elastic Disaster Recovery (AWS DRS)

You can use AWS DRS to perform a continuous replication of workloads that reside on the Outposts C6id server powered by Intel processors (C6gd are not supported, since only 64-bit operating systems built for the x86 system architecture are supported by AWS DRS) to a staging area subnet in the Region. AWS DRS provides nearly continuous, block-level replication in the Region and creates periodic EBS Snapshots according to the Point in Time (PIT) state schedule for AWS DRS.

The following diagram shows the continuous replication of the data in the instance store volumes through AWS DRS. The PIT EBS Snapshots are used to create Amazon EBS-backed AMIs as a backup of the EC2 instances running on the Outposts server.

Figure 1 - Continuous replication of the Instance Store Volumes data from the instances running Outpost Server to a staging area in the parent region through DRS

Figure 1 – Continuous replication of the Instance Store Volumes data from the instances running Outpost Server to a staging area in the parent region through DRS

Despite AWS DRS not supporting the failback from the Region to Outposts servers, you can use the EBS snapshots taken by AWS DRS to restore the data back to the Outposts server at the desired PIT following the steps described in this post.

Prerequisites

The following prerequisites are required to complete the walkthrough:

  1. The EC2 instance to restore running on the Outposts server has been added as a source server to AWS DRS by installing the AWS Replication Agent.
  2. The initial sync has been completed and the data replication status is showing as healthy.

Restore the entire EC2 instance on the same or a different Outposts server

  1. Use the describe-recovery-snapshots command to list the PIT Snapshots taken by AWS DRS for the source server to restore.$ aws drs describe-recovery-snapshots --source-server <source-server-id>

2. Based on the time in which you want to restore your data, retrieve the corresponding EBS Snapshots in the output of the command. The following is an example of the output:

{
    "items": [
       {
            "ebsSnapshots": [
                "snap-07bf348d58151a432"
            ],
            "expectedTimestamp": "2024-06-13T16:40:00+00:00",
            "snapshotID": "pit-a4877ff6fa68561bf",
            "sourceServerID": "s-a080ceb10af7275a7",
            "timestamp": "2024-06-13T16:46:56.645979+00:00"
        },
        {
            "ebsSnapshots": [
                "snap-0496020ff7f83486d"
            ],
            "expectedTimestamp": "2024-06-13T16:30:00+00:00",
            "snapshotID": "pit-aece827519e1b0fbb",
            "sourceServerID": "s-a080ceb10af7275a7",
            "timestamp": "2024-06-13T16:37:06.600323+00:00"
        },
        {
            "ebsSnapshots": [
                "snap-0d7ebd23e56346cea"
            ],
            "expectedTimestamp": "2024-06-13T16:20:00+00:00",
            "snapshotID": "pit-a56960f89ff12579e",
            "sourceServerID": "s-a080ceb10af7275a7",
            "timestamp": "2024-06-13T16:27:01.595791+00:00"
        },
…
…

3. Open the Amazon EC2 console. In the navigation pane, choose Snapshots and filter by the Snapshot ID chosen in the previous step: snap-07bf348d58151a432.

4. Choose Actions, Create image from snapshot, and specify the Image name. You can leave the other information as default or customize as desired.

5. To perform the restore, launch a new EC2 instance on the same or a different Outposts server from the Amazon EBS-backed AMI created in the previous step.

Note that since AMIs are downloaded from the Region with every instance launch on Outposts servers, this approach could result in an RTO spanning hours, depending on the throughput of the service link and the size of the local instance storage from which the Snapshot and AMI were taken by AWS DRS. Alternatively, if you need to restore only some files and directories, you can do so by launching the EC2 instance in the Region from the AMI taken in Step 4 and then transferring the desired data from that instance to the source server running on Outposts.

Option 2: Backup to the Region using an open source solution

In addition to AWS DRS, you can use open source solutions and/or operating system (OS) functions to back up data from local instance storage to a Region. Consider this approach when you want a highly-customizable solution for workloads where lack of commercial support is acceptable. The open source solution uses AWS Systems Manager Automation and OS functions to take an Amazon EBS-backed AMI in the Region from a Linux EC2 instance running on your Outposts server. The following diagram provides a high-level overview of the solution.

Figure 2 – Wokflow of the open source solution

Figure 2 – Workflow of the open source solution

  1. The Automation creates a helper instance and a baseline EBS volume attached to it in the Region, using an AWS CloudFormation
  2. The Automation executes commands on the OS of the EC2 instance running on the Outposts server to perform preliminary checks and start syncing data from the local instance store volume to the baseline EBS volume in the Region.
  3. The sync continues until the data has been transferred successfully.
  4. When the sync completes, the Automation takes an EBS Snapshot of the baseline EBS volume and then creates an Amazon EBS-backed AMI from it.

Create the Automation document

  1. Open the github page of the open source solution backup-outposts-servers-linux-instance.
  2. Follow the Installation Instructions to create the Systems Manager Automation document.

Back up an EC2 instance running on Outposts server

  1. After creating the Automation document, follow the Usage Instructions to execute the Automation and initiate the backup.
  2. Monitor the Execution status in the System Manager Automation console.

Restore the entire EC2 instance on the same or a different Outposts server

  1. Open the Amazon EC2 console. In the navigation pane, choose AMIs and filter by the AMI names that contain the InstanceId to restore.

2. Select the desired AMI to restore and note its AMI ID.

3. To perform the restore, launch a new EC2 instance on the same or a different Outposts server from the Amazon EBS-backed AMI identified in the previous step.

Considerations for data residency and service link bandwidth

Data residency is a critical consideration for organizations that need to collect and store data in their own data centers for regulatory or compliance reasons. In this case, users cannot back up their data to the Region and need to consider backing up to another on-premises system.

Another consideration is the impact on the service link connectivity when performing backup and restore operations between the Outposts and the Region. When implementing the solutions described in the “Backup to AWS storage in an AWS Region” scenario, both your backup/restore and management/monitoring operations for your Outpost rely on the service link connectivity. Although AWS DRS provides block-level replication, the open source solution we discuss in this post only replicates data, resulting in smaller snapshot sizes for users with lower service link bandwidth requirement.

If you foresee bandwidth constraints for your service link, consider backing up to another on-premises system that is reachable through the local network interface (LNI) of your Outposts server.

Scenario 2: Backup to AWS storage in your on-premises environment

For the preceding reasons, you may need to back up your workload running on Outposts server to a persistent AWS storage system within the same geo political boundary. To do so, you can use an AWS Outposts rack that resides in the same or a different physical location and is reachable through the LNI of your Outposts server.

Outposts rack with Amazon S3 on Outposts allows you to run AWS infrastructure, services, and object storage to your on-premises to meet local data processing and data residency needs while offering the AWS durable storage that can be used to store your backup.

Thanks to this, you can use the same approaches described in the “Backup to AWS storage in an AWS Region” section at a high level to back up your data, while the storage is hosted on the Outposts rack. When evaluating this approach, keep in mind these important considerations for local snapshots.

With this approach, you can store your backup on premises to meet your data residency requirements. This also keeps the network traffic for your backup and restore within your on-premises network, without impacting the service link.

Conclusion

In this post, we showed different approaches to design backup and restore strategies for your workloads running on Outposts servers. Implementing the right approach can help protect your organization’s data against loss or corruption while meeting your performance, RTO, RPO, and data residency needs, with backup destinations ranging from AWS storage in the Region, locally on Outposts rack, or in a hybrid architecture.

Accelerate your AWS Graviton adoption with the AWS Graviton Savings Dashboard

Post Syndicated from aostan original https://aws.amazon.com/blogs/compute/accelerate-your-aws-graviton-adoption-with-the-aws-graviton-savings-dashboard/

This post is written by Rajani Guptan, Rosa Corley and Shankar Gopalan.

Are you looking to optimize your AWS infrastructure costs while maintaining high performance? AWS Graviton is a custom-built CPU developed by Amazon Web Services (AWS), and it is designed to deliver the best price performance for a broad range of cloud workloads running on Amazon Elastic Compute Cloud (Amazon EC2). Graviton-based instances provide up to 40% better price performance while using up to 60% less energy than comparable EC2 instances.

AWS users recognize that migrating existing workloads to Graviton-based instances results in better price performance. However, migrating to Graviton necessitates identifying comparable instance types, understanding the performance impacts, and estimating the savings opportunities. Furthermore, prioritizing and tracking migration efforts at scale across a diverse set of services such as Amazon EC2, Amazon Relational Database Service (Amazon RDS), Amazon ElastiCache, and Amazon OpenSearch can be challenging. Therefore, AWS has developed the AWS Graviton Savings Dashboard to help users address these complexities and accelerate their Graviton migration.

In this post, we walk you through the dashboard architecture, deployment steps, features, and capabilities. Whether you are an Executive, FinOps Practitioner, Product Owner, or in Engineering, you can use the dashboard to get the following:

  • Centralized visibility across accounts/workloads: The dashboard consolidates and tracks Graviton adoption across multiple management accounts, member accounts, and AWS Regions in a single view.
  • Graviton support across key AWS services: There are dedicated tabs allowing users to review current Graviton usage and potential savings across AWS compute and managed services.
  • Granular resource-level visibility for managed services: The dashboard provides granular resource level visibility for managed services such as Amazon RDS, ElastiCache, and OpenSearch.
  • Accurate savings and unit cost estimations: The dashboard provides accurate cost estimations for existing and comparable Graviton-based instance types by using the existing AWS Cost and Usage Report (CUR) data with the AWS public pricing API.
  • Categorization of migration effort: The dashboard categorizes Graviton migration opportunities into two main groups: Typically Easy and Requires Additional Planning, for EC2 instances. It also identifies Graviton-eligible resources for managed services, which may need version or database upgrades. This categorization helps users prioritize their engineering efforts for migration.

Architecture overview

The solution integrates AWS CUR, the AWS SDK, and AWS Public pricing API to generate comprehensive data on the usage, cost, and resource inventory. This data is stored in Amazon S3 and analyzed using Amazon Athena, providing deep insights into potential cost savings. Then, the results are visualized through Amazon QuickSight, enabling stakeholders to collaborate effectively and make informed, data-driven decisions, as shown in the following figure.

Graviton Savings Dashboard architecture diagram

Figure 1: Graviton Savings Dashboard architecture diagram

Although the solution typically costs between $50–$100 per month, the potential return on investment is substantial. The dashboard often identifies measurable cost savings that significantly outweigh its operational expenses. Moreover, it offers additional productivity benefits by streamlining the process of adopting Graviton, saving valuable time and effort for your team. For a detailed breakdown of the dashboard’s cost structure, we invite you to explore our comprehensive Graviton Savings Dashboard Cost Breakdown guide.

Deployment

The Graviton Savings Dashboard is part of the Cloud Intelligence Dashboards framework. You can deploy it using AWS CloudFormation Templates and a ‘cid-cmd’ command line tool. Prior to deploying the dashboard, make sure that you’ve met the prerequisites. These include the following:

  1. Setting up your AWS CUR: We highly recommend that you complete Steps 1 and 2 from the Cloud Intelligence Dashboard Deployment Guide. This makes sure that your CUR is set up with settings that allow for easy installation and troubleshooting if necessary.
  2. Setting up the Inventory Collector Module of the Optimization Data Collection lab: This provides automation to collect metadata and pricing for Amazon RDS, ElastiCache, and OpenSearch for all accounts in your AWS Organizations and AWS Regions.
  3. Preparing QuickSight: If you’re an existing QuickSight user, then you can skip this step. If not, then you must complete Step 3.1 to Prepare QuickSight.

When the prerequisites are in place, you can deploy the dashboard by running three simple commands (shown as follows) using a terminal application with permissions to run API requests in your AWS account.

python3 -m ensurepip --upgrade

pip3 install --upgrade cid-cmd

cid-cmd deploy --dashboard-id graviton-savings

For detailed instructions about the deployment and prerequisites, refer to the AWS Well-Architected Cost Optimization lab.

Examining the results: unlocking insights from your Graviton Savings Dashboard

Now that you’ve successfully deployed the dashboard, we can explore its powerful features and uncover valuable insights. As you read through this section, we encourage you to interact with your dashboard to familiarize yourself with the dashboard’s intuitive interface and functionality.

The Graviton Savings Dashboard is organized into service-specific tabs, each containing two key sections:

Current Graviton Usage and Savings (top section): This section highlights the tangible benefits you’ve already achieved by migrating workloads to Graviton. You can explore the following:

    • Monthly Graviton adoption trends
    • Usage distribution across different accounts, Regions, and processor types
    • Popular Graviton instance families
    • Unit cost trends
    • Realized Graviton savings

These metrics are calculated by comparing your Graviton usage to comparable non-Graviton instances, which provides a clear picture of your cost optimization efforts, as shown in the following figure.

Current Amazon EC2 Graviton Usage and Savings

Figure 2: Current Amazon EC2 Graviton Usage and Savings

Potential Graviton Savings Opportunities (bottom section): This section identifies areas where you can further optimize costs by adopting Graviton instances. It provides the following:

  • Actionable migration insights
  • Estimated implementation effort
  • Potential savings breakdowns by account, instance family/type, OS, and purchase option

These insights compare potential Graviton savings across various attributes, enabling targeted decision-making for future Graviton migrations and cost optimizations, as shown in the following figure.

Amazon EC2 Graviton Opportunity

Figure 3: Amazon EC2 Graviton Opportunity

Using dashboard insights: a FinOps team use case

In this section we explore a use case where you, as the lead of the Cloud Center of Excellence Team, use the insights from this dashboard to address concerns raised by your Chief Technology Officer (CTO).

Your CTO at your company approaches you with the following questions:

  1. Is our organization using the price-performance benefits of Graviton-based EC2 instances?
  2. How does our Graviton usage and spend and savings compare to other processor types within our overall EC2 compute spend?

Step 1: Initial analysis

You begin generating summary reports from the Current Graviton Usage (Figure 2) and Graviton Opportunity (Figure 3) sections of the dashboard. After reviewing these reports, the CTO asks you to engage with the Engineering team to evaluate potential opportunities for increasing Graviton coverage.

Step 2: Engaging with Engineering on Graviton Migration

When presenting the summary reports to the engineering manager, they expressed interest in understanding the effort level required for this project. This information can help them allocate resources and prioritize workloads, thus identifying what can be started in the short-term and what needs additional planning.

Step 3: Detailed analysis

As shown in the following figure, the Engineering team can focus on identifying candidate workloads with the most significant savings impact by segmenting the dashboard data by:

  • Implementation efforts
  • Linked accounts
  • Regions
  • Instance types
  • Operating systems

Amazon EC2 Graviton opportunity breakdown

Figure 4: Amazon EC2 Graviton opportunity breakdown

Furthermore, the team can use the dashboard to determine comparable Graviton-based instances for migration and their potential savings, as shown in the following figure.

Potential graviton Savings Details

Figure 5: Potential graviton Savings Details

Step 4: Tracking progress

Over time, the FinOps team and Engineering team can showcase the Graviton migration successes by highlighting the increasing Graviton coverage and realized savings using the dashboard’s charts (Figure 2).

Broader application:

Although this post primarily focuses on EC2 instance migration, the dashboard also provides similar insights for AWS managed services such as Amazon RDS, ElastiCache, and OpenSearch. Individual tabs with visualizations guide your Graviton adoption across these services, as shown in the following figure.

Potential graviton savings details

Figure 6: Graviton Savings Dashboard

As demonstrated by this use case, the Graviton Savings Dashboard enables various stakeholders in an organization to collaborate effectively, which leads to efficient outcomes and potential cost savings.

Conclusion

In summary, we showed how the Graviton Savings Dashboard provides clear insights into suitable workloads for Graviton migration, offers easy-to-understand visualizations for monitoring adoption, and automates resource matching and savings calculations. Streamlining the process of identifying and implementing cost-saving opportunities with Graviton-based instances means that the dashboard enables more informed decision-making about your AWS infrastructure. To learn more and get started with the Graviton Savings Dashboard, visit the Graviton Savings Dashboard page and take the first step toward more efficient and cost-effective cloud computing.

New Amazon EC2 P5en instances with NVIDIA H200 Tensor Core GPUs and EFAv3 networking

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5en-instances-with-nvidia-h200-tensor-core-gpus-and-efav3-networking/

Today, we’re announcing the general availability of Amazon Elastic Compute Cloud (Amazon EC2) P5en instances, powered by NVIDIA H200 Tensor Core GPUs and custom 4th generation Intel Xeon Scalable processors with an all-core turbo frequency of 3.2 GHz (max core turbo frequency of 3.8 GHz) available only on AWS. These processors offer 50 percent higher memory bandwidth and up to four times throughput between CPU and GPU with PCIe Gen5, which help boost performance for machine learning (ML) training and inference workloads.

P5en, with up to 3200 Gbps of third generation of Elastic Fabric Adapter (EFAv3) using Nitro v5, shows up to 35% improvement in latency compared to P5 that uses the previous generation of EFA and Nitro. This helps improve collective communications performance for distributed training workloads such as deep learning, generative AI, real-time data processing, and high-performance computing (HPC) applications.

Here are the specs for P5en instances:

Instance size vCPUs Memory (GiB) GPUs (H200) Network bandwidth (Gbps) GPU Peer to peer (GB/s) Instance storage (TB) EBS bandwidth (Gbps)
p5en.48xlarge 192 2048 8 3200 900 8 x 3.84 100

On September 9, we introduced Amazon EC2 P5e instances, powered by 8 NVIDIA H200 GPUs with 1128 GB of high bandwidth GPU memory, 3rd Gen AMD EPYC processors, 2 TiB of system memory, and 30 TB of local NVMe storage. These instances provide up to 3,200 Gbps of aggregate network bandwidth with EFAv2 and support GPUDirect RDMA, enabling lower latency and efficient scale-out performance by bypassing the CPU for internode communication.

With P5en instances, you can increase the overall efficiency in a wide range of GPU-accelerated applications by further reducing the inference and network latency. P5en instances increases local storage performance by up to two times and Amazon Elastic Block Store (Amazon EBS) bandwidth by up to 25 percent compared with P5 instances, which will further improve inference latency performance for those of you who are using local storage for caching model weights.

The transfer of data between CPUs and GPUs can be time-consuming, especially for large datasets or workloads that require frequent data exchanges. With PCIe Gen 5 providing up to four times bandwidth between CPU and GPU compared with P5eand P5e instances, you can further improve latency for model training, fine-tuning, and running inference for complex large language models (LLMs) and multimodal foundation models (FMs), and memory-intensive HPC applications such as simulations, pharmaceutical discovery, weather forecasting, and financial modeling.

Getting started with Amazon EC2 P5en instances
You can use EC2 P5en instances available in the US East (Ohio), US West (Oregon), and Asia Pacific (Tokyo) AWS Regions through EC2 Capacity Blocks for ML, On Demand, and Savings Plan purchase options.

I want to introduce how to use P5en instances with Capacity Reservation as an option. To reserve your EC2 Capacity Blocks, choose Capacity Reservations on the Amazon EC2 console in the US East (Ohio) AWS Region.

Select Purchase Capacity Blocks for ML and then choose your total capacity and specify how long you need the EC2 Capacity Block for p5en.48xlarge instances. The total number of days that you can reserve EC2 Capacity Blocks is 1–14, 21, or 28 days. EC2 Capacity Blocks can be purchased up to 8 weeks in advance.

When you select Find Capacity Blocks, AWS returns the lowest-priced offering available that meets your specifications in the date range you have specified. After reviewing EC2 Capacity Blocks details, tags, and total price information, choose Purchase.

Now, your EC2 Capacity Block will be scheduled successfully. The total price of an EC2 Capacity Block is charged up front, and the price does not change after purchase. The payment will be billed to your account within 12 hours after you purchase the EC2 Capacity Blocks. To learn more, visit Capacity Blocks for ML in the Amazon EC2 User Guide.

To run instances within your purchased Capacity Block, you can use AWS Management Console, AWS Command Line Interface (AWS CLI) or AWS SDKs.

Here is a sample AWS CLI command to run 16 P5en instances to maximize EFAv3 benefits. This configuration provides up to 3200 Gbps of EFA networking bandwidth and up to 800 Gbps of IP networking bandwidth with eight private IP address:

$ aws ec2 run-instances --image-id ami-abc12345 \
  --instance-type p5en.48xlarge \
  --count 16 \
  --key-name MyKeyPair \
  --instance-market-options MarketType='capacity-block' \
  --capacity-reservation-specification CapacityReservationTarget={CapacityReservationId=cr-a1234567}
--network-interfaces "NetworkCardIndex=0,DeviceIndex=0,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa" \
"NetworkCardIndex=1,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=2,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=3,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=4,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa" \
"NetworkCardIndex=5,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=6,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=7,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=8,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa" \
"NetworkCardIndex=9,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=10,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=11,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=12,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa" \
"NetworkCardIndex=13,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=14,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=15,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=16,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa" \
"NetworkCardIndex=17,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=18,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=19,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=20,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa" \
"NetworkCardIndex=21,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=22,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=23,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=24,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa" \
"NetworkCardIndex=25,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=26,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=27,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=28,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa" \
"NetworkCardIndex=29,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=30,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only" \
"NetworkCardIndex=31,DeviceIndex=1,Groups=security_group_id,SubnetId=subnet_id,InterfaceType=efa-only"
...

When launching P5en instances, you can use AWS Deep Learning AMIs (DLAMI) to support EC2 P5en instances. DLAMI provides ML practitioners and researchers with the infrastructure and tools to quickly build scalable, secure, distributed ML applications in preconfigured environments.

You can run containerized ML applications on P5en instances with AWS Deep Learning Containers using libraries for Amazon Elastic Container Service (Amazon ECS) or Amazon Elastic Kubernetes Service (Amazon EKS).

For fast access to large datasets, you can use up to 30 TB of local NVMe SSD storage or virtually unlimited cost-effective storage with Amazon Simple Storage Service (Amazon S3). You can also use Amazon FSx for Lustre file systems in P5en instances so you can access data at the hundreds of GB/s of throughput and millions of input/output operations per second (IOPS) required for large-scale deep learning and HPC workloads.

Now available
Amazon EC2 P5en instances are available today in the US East (Ohio), US West (Oregon), and Asia Pacific (Tokyo) AWS Regions and US East (Atlanta) Local Zone us-east-1-atl-2a through EC2 Capacity Blocks for ML, On Demand, and Savings Plan purchase options. For more information, visit the Amazon EC2 pricing page.

Give Amazon EC2 P5en instances a try in the Amazon EC2 console. To learn more, see Amazon EC2 P5 instance page and send feedback to AWS re:Post for EC2 or through your usual AWS Support contacts.

— Channy

NEW: Simplifying the use of third-party block storage with AWS Outposts

Post Syndicated from Rachel Zheng original https://aws.amazon.com/blogs/compute/new-simplifying-the-use-of-third-party-block-storage-with-aws-outposts/

This post is written by Kate Sposato, Senior Solutions Architect, EC2 Edge Compute

AWS is excited to announce deeper collaboration with industry-leading storage solutions to streamline the use of third-party storage with AWS Outposts. You can now attach and use external block data volumes from NetApp® on-premises enterprise storage arrays and Pure Storage® FlashArray™ directly from the AWS Management Console.

Outposts is a fully managed service that extends AWS infrastructure, AWS services, APIs, and tools to customer premises. By providing local access to AWS managed infrastructure, Outposts allows you to build and run applications on premises using the same application programming interfaces (APIs) as in AWS Regions. Moreover, this is done while using local compute and storage resources to meet lower latency and local data processing needs. Outposts is available in various rack and server form factors.

Many of you have block storage systems running in your on-premises environments that provide advanced data storage and management features—such as snapshots, replication, and encryption—to protect data integrity and security. There are various uses cases that would predicate you needing to access data through these external volumes backed by external storage systems from an application running in Amazon Elastic Compute Cloud (Amazon EC2) instances on Outposts. These include: regulatory auditing requirements, government and local regulation compliance, high data durability and resiliency requirements, low-latency data access, and migration of on-premises applications that are tightly coupled with existing external storage systems. To make it easier for you to use external volumes with Outposts, AWS has validated a broad range of third-party storage solutions through the AWS Outposts Ready Program. With this program, you can easily identify storage solutions that are tested to run with Outposts.

Today, we are taking our integration with storage solutions from NetApp and Pure Storage to the next level. Outposts now has a simplified and automated way to launch EC2 instances with attached block storage from external infrastructure through the AWS Management Console. The new integration includes automated user script generation and attachment of data volumes to EC2 instances running on 42U Outposts racks and 2U Outposts servers. This integration reduces the friction associated with using the advanced data management and security features of external storage infrastructure in combination with Outposts, allowing you to create a resilient, compliant, and optimized storage and compute infrastructure.

Outposts rack storage and networking overview

Outposts racks support Amazon Elastic Block Store (Amazon EBS) volumes for EC2 instances, which provide persistent local block storage.

EC2 instances running on Outposts racks can access data stored on external block storage arrays over the Outposts local gateway (LGW). An LGW enables connectivity between the Outpost subnets, where EC2 instances run, and the on-premises network. It carries storage traffic between the EC2 instances running on the Outposts rack and the local network. The LGW is created by AWS as part of the Outposts rack installation process. Each Outposts rack supports a single LGW.

The following diagram shows an EC2 instance running on an Outposts rack with an elastic network interface (ENI) and LGW configured for instance connectivity. An external storage array communicates with the EC2 instance running on the Outposts rack through the Outpost network devices (ONDs). Customer Network Devices (CNDs) that connect to EC2 instances running on Outposts racks need to support the following:

  • Link aggregation: connections to the Outposts rack network devices are added to a link aggregation group (LAG).
  • VLANs: Virtual LANs (VLANs) are configured between each Outposts rack TOR device and any customer devices, including data stores.;
  • Dynamic routing: Border Gateway Protocol (BGP) is configured between the CND and the OND for each VLAN. Two total BGP sessions are shown in the following diagram between devices.

Figure 1. Outposts rack and Amazon EC2 networking architecture

Figure 1. Outposts rack and Amazon EC2 networking architecture

Outposts server storage and networking overview

Outposts servers come with internal NVMe SSD-based high-performance instance storage. Similar to AWS Regions, instance storage is allocated directly to the EC2 instance and follows the lifecycle of the instance. For example, if an EC2 instance is terminated, then the instance storage associated with the instance is also deleted. If you want data to persist after the instance is terminated, you can use external storage solutions to complement the instance storage included with Outposts servers.

Outposts servers have a local network interface (LNI). This logical networking component connects the EC2 instances running on the Outposts servers subnet to the on-premises network and allows communication to other on-premises storage, compute, and networking appliances.

To support the Amazon EC2 on Outposts to external storage array integration, an LNI must be created then added to the EC2 instance during instance launch. An LNI can only be created through the AWS Command Line Interface (AWS CLI) or the AWS software development toolkit (SDK) using the following command. The subnet id is the Outposts server subnet and the device index should be unique to the subnet.

aws ec2 modify-subnet-attribute --subnet-id <subnet id> --enable-lni-at-device-index <device index>

In the on-premises network, you must have a Network Interface Card (NIC) at the same device index that you specified when running the preceding CLI command.

Further detailed steps for this workflow are listed in the Outposts server user guide.

When the local network interfaces are enabled on an Outpost subnet, the EC2 instances in the Outpost subnet can be configured to include this LNI in addition to the ENI. The LNI connects to the on-premises network while the ENI connects to the VPC.

The following diagram shows an EC2 instance running on an Outposts server with both an ENI and LNI configured for instance connectivity. There is an external storage array connected to the Outposts server using a CND through NVMe-over-TCP or iSCSI protocol. Figure 2. Outposts server and Amazon EC2 networking architecture

Figure 2. Outposts server and Amazon EC2 networking architecture

Supported operating systems and AWS Support

The rest of this post covers the steps for how to launch an EC2 instance running on an Outposts 2U server or Outposts rack with a connected external block storage volume for local data access from within the EC2 instance. The current release of this feature supports EC2 instances running Microsoft Windows Server 2022 and Red Hat Enterprise Linux 9 (RHEL9) based operating systems.

Support for Outposts and all Outposts integration features, including this one, needs an active AWS Enterprise Support Plan or AWS Enterprise On-Ramp Support Plan. Support for external storage arrays and configurations can be obtained from the respective storage vendor and may need an additional support plan depending on the vendor and the storage solution implemented.

This post assumes you’re familiar with the basic functionality of Outposts servers and Outposts rack. If you would like to learn more about the Outposts family in general, then the user guide, What is AWS Outposts?, is a great place to start.

Solution deployment

The following sections outline the solution deployment.

Prerequisites:

  1. An Outposts 2U server or Outposts rack is provisioned, activated, and connected to the customer network.
  2. A block storage array is connected on the same network and accessible to Outposts subnets.
  3. A block data volume is configured and running on the storage array. The unique identifier for this volume is necessary for launching the EC2 instance on the Outpost. The volume must remain provisioned after initial provisioning on the storage array.
  4. The IP address and port number (optional for iSCSI connections) of the block storage volume, which is necessary for launching the EC2 instance on the Outpost.

Deployment architecture overview

The following deployment architecture shows the workflow attaching an external storage array to an Outpost, launching an EC2 instance through the AWS Management Console, and accessing the data on the external storage array from within the EC2 instance running on the Outpost.Figure 3. Third-party block storage on Outposts architecture overview

Figure 3. Third-party block storage on Outposts architecture overview

Deployment steps for NVMe-over-TCP connections

1. (Prerequisite) If there is no block data volume already running and configured on the compatible storage array, this must be completed in the storage solution’s interface before moving to Step 2.

a. Create an NVMe device, subsystem, and namespace for the block data volume.

b. Optionally, generate a host NQN that is used for the EC2 instance connection, and add it to the allow list for the appropriate subsystems.

c. The following pieces of information are used in later steps:

i. Host NQN: Unique identifier of the EC2 instance for attachment;

ii. Target IP: Address of the connected block volume host;

iii. Target Port Number: Port number of the connected block volume host.

You can learn more about launching and configuring external storage arrays in the Outposts family documentation or in the respective storage array vendor documentation.

2. In the Console, navigate to EC2 Launch Instance Wizard by choosing EC2, Instances, Launch instances.

a. Name the instance and add any desired tags to be applied at launch.

b. Choose the desired, compatible RHEL9 based Amazon Machine Image (AMI) from the list, or choose one from the AWS Marketplace.

c. Choose the desired EC2 Instance type.

d. Expand the Network settings section and select Edit. Choose the VPC and subnet of the target Outpost.

i. Outposts servers only: You must create an LNI in the Advanced Network settings before launching the instance.

e. Expand Advanced network configuration and select Add network device. Continue to add network devices until the Device index is equal to the volume index.

Figure 4. Advanced network configurationFigure 4. Advanced network configuration

f. Expand Configure storage and select Edit next to External storage volumes settings section and choose NVMe/TCP in Storage network protocol.

Figure 5. External storage volumes configuration

g. Enter the HostNQN in the format provided for the NVMe/TCP data volume. Make sure that the HostNQN used has been added to the storage array subsystem allow list.

h. Select Add NVMe/TCP Discovery Controller and enter the IP address and port of the controller from the storage array. Enter 4420 as the Target Port, if the target port is unknown.

i. (Optional) You can add more data volumes that use a different target discovery controller at this time by choosing the Add NVMe/TCP Data Volume button under the Target IP address. Repeat Steps 2.h for each data volume to be attached to the EC2 instance.

j. Expand the Advanced details and provide any additional Amazon EC2 behavior settings as appropriate.

k. At the bottom of the Advanced details section is the automatically generated User data. If you need to manually edit this data, you can do so by selecting Edit at the bottom.

Figure 6. Automatically generated user data file

l. When the configurations are set, choose the Launch instance button in the right-side column.

3. The EC2 Launch Instance Wizard now launches an EC2 instance configured as described on the Outpost and attaches the desired external data volume(s) to the EC2 instance.

4. Applications and users can access the data on the attached external volumes from within the EC2 instance. To verify this:

a. From within the launched EC2 instance, run sudo nvme list

b. The volumes are displayed as /dev/nvme1n1 with the number increasing for each attached volume. Local instance store volumes on Outposts servers and EBS boot volumes on Outposts racks are listed first. External volumes are listed after those with sequentially increasing node numbers.

5. External storage volume and array management, configuration, and backups continue to be managed through the storage vendor-provided toolkit. You can find more information on external storage management in the respective storage array vendor documentation.

Deployment steps for iSCSI connections

1. (Prerequisite) If there is no block data volume already running and configured on the compatible storage array, this must be completed in the storage solution’s interface before moving to Step 2.

a. Create an Initiator group (igroup) and add the Initiator IQN to the igroup. Then map the logical unit number (LUN) to the igroup.

b. Optionally, generate an initiator IQN that is used for the EC2 instance connection, and add it to the allow list for the appropriate subsystems.

c. The following pieces of information are used in later steps:

i. Initiator IQN: Unique identifier of the EC2 instance for attachment;

ii. Target IQNs: Unique identifier of the storage virtual machine (SVM);

iii. Target IP: Address of the connected block volume host;

iv. (Optional) Target Port Number: Port number of the connected block volume host.

You can learn more about launching and configuring external storage arrays in the Outposts family documentation or in the respective storage array vendor documentation.

2. In the Console, navigate to EC2 Launch Instance Wizard by choosing EC2, Instances, Launch instances.

a. Name the instance and add any desired tags to be applied at launch.

b. Choose the desired, compatible RHEL9 or Windows Server 2022 based AMI from the list, or purchase one from the AWS Marketplace.

c. Choose the desired EC2 Instance type.

d. Expand the Network settings section and choose the VPC and subnet of the target Outpost.

i. Outposts servers only: You must create an LNI in the Advanced Network settings before launching the instance.

e. Expand Advanced network configuration and select Add network device. Continue to add network devices until the Device index is equal to the volume index.

Figure 7. Advanced network configurationFigure 7. Advanced network configuration

f. Expand Configure storage and select Edit next to External storage volumes settings section and choose iSCSI in Storage network protocol.

Figure 8. External storage volumes configurationFigure 8. External storage volumes configuration

g. Enter the Initiator IQN for the iSCSI data volume in the format provided. Make sure that the Initiator IQN used has been added to the allow list for the volume.

h. Select Add iSCSI Target and enter the Target IP, Target Port, and Target IQN of the storage array. Enter 4420 for the Target Port, if the target port is unknown.

i. (Optional) You can add additional data volumes with a different Target IQN at this time by selecting the Add iSCSI Target button under the Target IP address. Repeat Steps 2.h for each data volume to be attached to the EC2 instance.

j. Expand the Advanced details and provide any additional Amazon EC2 behavior settings as appropriate.

k. At the bottom of the Advanced details section is the automatically generated User data. If you need to manually edit this data, you can do so by selecting Edit at the bottom.

Figure 9. Automatically generated user data fileFigure 9. Automatically generated user data file

l. When the configurations are set, choose the Launch instance button in the right-side column.

3. The EC2 Launch Instance Wizard now launches an EC2 instance configured as described on the Outpost and attaches the desired external data volume(s) to the EC2 instance.

4. Applications and users can access the data on the attached external volumes from within the EC2 instance. To verify this:

a. From within the launched EC2 instance, run iscsiadm -m session -P3

b. The volumes are displayed as /dev/sd0 with the number increasing for each attached volume.

5. External storage volume and array management, configuration, and backups continue to be managed through the storage vendor-provided toolkit. You can find more information on external storage management in the respective storage array vendor documentation.

Conclusion

This integration offers a streamlined workflow to attach and utilize external block data volumes on Outposts directly through the AWS Management Console, eliminating manual processes. It provides the full benefits of advanced data infrastructure from trusted storage providers in conjunction with the security, reliability, and scalability of AWS managed infrastructure. This helps you accelerate cloud migration with dependencies on third-party storage and realize the full potential of your on-premises data.

To learn more about this integration, visit the NetApp on-premises enterprise storage arrays for AWS Outposts solution page and the Pure Storage FlashArray for AWS Outposts blog post. To discuss your external storage needs with an Outposts expert, submit this form. If you are attending AWS re:Invent 2024, make sure to check out the NetApp booth (booth #1748) and Pure Storage booth (booth #454) to connect with our partner specialists.

Introducing storage optimized Amazon EC2 I8g instances powered by AWS Graviton4 processors and 3rd gen AWS Nitro SSDs

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/introducing-storage-optimized-amazon-ec2-i8g-instances-powered-by-aws-graviton4-processors-and-3rd-gen-aws-nitro-ssds/

Today, we’re announcing the general availability of Amazon EC2 I8g instances, a new storage optimized instance type to provide the highest real-time storage performance among storage-optimized EC2 instances with the third generation of AWS Nitro SSDs and AWS Graviton4 processors.

AWS Graviton4 is the most powerful and energy efficient processor we have ever designed for a broad range of workloads running on EC2 instances using a 64-bit ARM instruction set architecture. AWS Nitro System SSDs are custom built by AWS and offer high I/O performance, low latency, minimal latency variability, and security with always-on encryption.

EC2 I8g instances are the first instance type to use third-generation AWS Nitro SSDs. These instances offer up to 22.5 TB local NVME SSD storage with up to 65 percent better real-time storage performance per TB and 60 percent lower latency variability compared to the previous generation I4g instances. Based on the AWS Graviton4 processors, I8g instances deliver up to 60 percent better compute performance and two times larger caches compared to I4g.

I8g instances offer up to 96 vCPUs, 768 GiB of memory, and 22.5 TB of storage to deliver more compute and storage choices compared with I4g instances.

Instance name vCPUs Memory (Gib) Storage (GB) Network bandwidth (Gbps) EBS bandwidth (Gbps)
I8g.large 2 16 468 up to 10 up to 10 Gbps
I8g.xlarge 4 32 937 up to 10 up to 10 Gbps
I8g.2xlarge 8 64 1,875 up to 12 up to 10 Gbps
I8g.4xlarge 16 128 3,750 up to 25 up to 10 Gbps
I8g.8xlarge 32 256 7,500
(2 x 3,750)		 up to 25 10 Gbps
I8g.12xlarge 48 384 11,520
(3 x 3,750)		 up to 28.125 15 Gbps
I8g.16xlarge 64 512 15,000
(4 x 3,750)		 up to 37.5 20 Gbps
I8g.24xlarge 96 768 22,500
(6 x 3,750)		 up to 56.25 20 Gbps
I8g.metal-24×1 96 768 22,500
(6 x 3,750)		 up to 56.25 30 Gbps

You can use I8g instances for I/O intensive workloads that require low latency access to data such as transactional databases (MySQL and PostgreSQL), real-time databases, NoSQL databases, (Aerospike, Apache Druid, MongoDB) and real-time analytics such as Apache Spark.

Additionally, I8g instances are built on the AWS Nitro System, which offloads CPU virtualization, storage, and networking functions to dedicated hardware and software to enhance the performance and security of your workloads. The Graviton4 processors offer you enhanced security by fully encrypting all high-speed physical hardware interfaces.

Things to know
Here are some things that you should know about EC2 I8g instances:

  • Operating system – EC2 I8g instances support Amazon Linux 2023, Amazon Linux 2, CentOS Stream 8 or newer, Ubuntu 18.04 or newer, SUSE 15 SP2 or newer, Debian 11 or newer, Red Hat Enterprise 8.2 or newer, CentOS 8.2 or newer, FreeBSD 13 or newer, Rocky Linux 8.4 or newer, Alma Linux 8.4 or newer, and Alpine Linux 3.12.7 or newer.
  • Networking – You can use I8g instances in storage intensive workloads that typically have burst network usage patterns. All I8g instance sizes have burst network bandwidth and can burst more than 60 minutes, depending on the instance sizes, to support the majority of the workloads requiring instance storage data hydration, backup, and snapshot over the network.
  • Migration – If you’re using I4g instances now, you will have straightforward experience migrating storage intensive workloads to I8g instances because these instances offer similar memory and storage ratios to existing I4g instances.

Now available
Amazon EC2 I8g instances are now available in the US East (N. Virginia) and US West (Oregon) AWS Regions through On-Demand instances, Savings Plans, Spot Instances, Dedicated Instances, or Dedicated Hosts.

Give EC2 I8g instances a try in the Amazon EC2 console. To learn more, visit the EC2 I8g instances page and send feedback to AWS re:Post for EC2 or through your usual AWS Support contacts.

— Channy

Faster scaling with Amazon EC2 Auto Scaling Target Tracking

Post Syndicated from aostan original https://aws.amazon.com/blogs/compute/faster-scaling-with-amazon-ec2-auto-scaling-target-tracking/

This post is written by Shahad Choudhury, Senior Cloud Support Engineer and Tiago Souza, Solutions Architect

Introduction

One of the key benefits of the AWS cloud is elasticity. It enables our users to provision and pay only for resources they need. To fully use the elasticity benefits, users needed a mechanism that is automated and can be widely operated with ease. Amazon EC2 Auto Scaling solves these challenges by helping our users automatically scale the number of Amazon Elastic Compute Cloud (Amazon EC2) instances to meet the changing workload demands, and it offers a wide suite of capabilities to manage the instance’s lifecycle.

To scale their Auto Scaling groups (ASG), users need to create scaling policies. Scaling policies provide ASGs with guidelines for adjusting Amazon EC2 capacity to match the workload demand. There are different types of scaling policies, with each having a different approach to manage capacity. One type of policy is Target Tracking, which offers a simpler yet effective way to scale automatically. To use it, users need to define a utilization metric and set a target value to maintain. For example, setting a 60% Average CPU Utilization policy causes the ASG to keep the metric as close to that value as possible across its fleet of EC2 instances.

In this post, we describe the recently released updates to Target Tracking. We also walk through the steps to create a Target Tracking policy that uses the new feature, and highlight the improvements and benefits users can expect from this new feature.

What’s new with Target Tracking policy

As users modernized their applications, we learned from them that a dynamic Auto Scaling solution must expand beyond our original implementation of the Target Tracking policy.

First, users found that the few minutes Target Tracking took to respond to a demand spike could lead to short-term performance degradation. We’ve seen many users mitigate this challenge by buffering their running capacity, leading to increased costs. Second, different workloads have different scaling requirements. This leads to users having to create tailored scaling policies for each workload, which is a time consuming, error prone, and operationally expensive activity for performance and cost optimizations.

To address these user challenges, we released an intelligent and highly responsive Target Tracking scaling policy. Target Tracking now automatically tunes its responsiveness to the unique usage patterns of individual applications and closely monitors application demand for faster scaling decisions. Automatic tuning allows users to enhance their application performance and maintain high usage for their Amazon EC2 resources to save costs without having to create tailored scaling policies for each workload. Users must specify a target utilization they want to maintain, and Target Tracking scales without any further input needed from users.

For faster auto scaling decisions, users can configure Target Tracking policies using high-resolution metrics in Amazon CloudWatch. This fine-grained monitoring allows Target Tracking to detect and respond to changing demand, not in minutes, but in seconds. This capability is ideal for applications that have volatile demand patterns, such as client-serving APIs, live streaming services, e-commerce websites, and on-demand data processing.

Getting started with the new Target Tracking policy

If you’re already using Target Tracking policies, then no action is necessary for you to upgrade to Target Tracking that automatically tunes itself. Target Tracking policies regularly analyze targeted metric history and determine the appropriate level of sensitivity to initiate scale-outs and scale-ins. Furthermore, it determines the amount of capacity that must be added or removed to optimize both availability and lower cost. These decisions depend on the unique characteristics of the application’s demand patterns, such as the range and frequency of demand changes, and whether spikes in usage are long or short-lived. Target Tracking continues to learn on an ongoing basis, and reevaluates itself to automatically adapt for your specific application and demand patterns.

Enabling faster scaling response from Target Tracking

Moreover, to enable the fastest response from Target Tracking policies, users can track metrics published at sub-minute granularity to CloudWatch (also known as high-resolution CloudWatch metrics). Users can update an existing Target Tracking policy or create a new one with a high-resolution metric as part of a CustomizedMetricSpecification. Users must describe the same metric namespace, metric name, and any dimension(s) and/or unit created when publishing the metric to CloudWatch. They must also define the metric period to indicate the metric granularity at which target tracking should evaluate the metric. The following steps walk you through how to get started on the AWS Management Console for ASG:

Step 1: Choose the ASG

In the console, choose the name of the ASG. This takes you to the Details page, as shown in the following figure.

List of ASGs in the Amazon EC2 console Auto Scaling section

Figure 1: In the Amazon EC2 console, choose the ASG that you want to scale

Choose the Automatic scaling tab that gives you the option to Create a dynamic scaling policy, as shown in the following figure.

Step 2: Create dynamic scaling policy

Step 2: Create dynamic scaling policy

Choose the target tracking policy as the policy type. For Metric Type, choose Custom CloudWatch metric. This shows a prefilled JSON snippet that you can edit to specify the metric name, namespace, and dimensions of the metric that you want to scale using the Target Tracking policy that you used to publish the CloudWatch metric, as shown in the following figure.

Figure 3: Updated CustomizedMetricSpecification section added to the Auto Scaling Console

Figure 3: Updated CustomizedMetricSpecification section added to the Auto Scaling Console

The minimum Period supported is ten seconds. To use the ten second metric periods, your metric should be published at a ten second or higher resolution, for example at one second. However, publishing at one second intervals can substantially increase your CloudWatch cost. We discuss the cost considerations later in this post. Auto Scaling imposes a limit of 60 seconds to make sure that Target Tracking can observe and respond to usage spikes quickly.

These two steps allow you to enable target tracking to scale on a high resolution metric.

Enabling faster scaling impact:

The preceding steps allow the ASG to detect changes in your utilization faster, thus it can add more instances when demand spikes.

In the following diagram, we see the results of running identical load tests against an environment with a default target tracking policy of a 60 second period and a target tracking policy configured with a ten second period. Each policy has a target value of 60% CPU Utilization. The load test ramps up to 20 threads over three minutes each sending http requests to simulate a spike in demand. We can see that, in the 60 second period case (the left diagram) there were three minutes where the application was above the CPU Utilization target of 60% (blue line). The capacity (green line) increased only after the system had reached a peak of 100% CPU Utilization. This may lead to application performance issues and, to avoid that, users would have to aim for lower utilization level so that more capacity can be provisioned, which would increase their cost. However, with the ten second periods (the right diagram), scaling happened rapidly to avoid application impact. The capacity increased after one minute, during which CPU Utilization remained closer to 60% and didn’t hit the peak 100% level. This allows users to reach a higher utilization level, saving the cost without impacting the application performance.

Two graphs of CPU Utilization and Auto Scaling InService Capacity comparing scaling based on 60 seconds period as opposed to 10 seconds period. In the first case, the CPU approaches 100% for multiple minutes before scaling occurs to bring CPU down. Comparatively, when scaling with a 10 second period, the CPU increases over two minutes but remained closer to the 60% target throughout.

Figure 4: Target tracking policy with 60 second periods as opposed to 10 seconds

Considerations

Before applying high resolution custom metrics, we recommend that you consider the following factors as they may impact your costs.

Metric types: Target Tracking assumes that metrics change proportionally to the number of instances in the ASG. Selecting the right metric is key for successful Target Tracking policies. Refer to the Target Tracking public documentation for more details.

Pricing: There is no further charge for EC2 Auto Scaling, including these new features. Users pay only for the AWS resources needed to run their applications and CloudWatch monitoring fees. However, you must understand the three CloudWatch billing items relevant to these features:

1) High-resolution alarms

2) API calls

3) Custom metrics

Target Tracking creates at least two alarms, one each to track high and low usage with a buffer in between their thresholds to reduce oscillation. If the metric period is less than sixty seconds, these alarms are billed as high-resolution alarms. As of this writing, the price for high-resolution alarm for the AWS US East (Ohio) Region is $0.30 per alarm metric as compared to $0.10 per alarm metric for standard resolution alarms.

If you’re using CloudWatch Agent, it sends API calls from each instance based on the metrics_collection_interval setting in the CloudWatch Agent config. Each instance sends an API call once per interval to CloudWatch. In CloudWatch, a metric is defined as a unique combination of a Namespace, MetricName, Dimension(s) (optional), and Unit (optional). Every unique combination of dimensions pushed from the CloudWatch Agent is billed as its custom metric.

The following is an example of expected monthly charges in USD using us-east-2 for an account that has passed the free tier, but is still in the first tier of paid usage (the price reduction for bulk usage). This example assumes an average of ten instances running over the month in an ASG with one target tracking policy where metrics and alarms are configured for ten second intervals.

1) High-resolution alarms:

2 alarms @ $0.30 each = $0.60/month

2) API calls:

10 instances * 30 days * 24 hours * 3600 seconds / 10 second_intervals = 2.592 million API calls

2.592 million API calls * $0.01 per 1,000 requests = $25.92/month

3) Custom metrics:

1 ASG aggregate metric @ $0.30/month = $0.30/month

Total estimate: $26.82/month for a 10 instance ASG

Multiple metrics can be pushed in a single PutMetricData API call. If you decide to configure the CloudWatch Agent to publish more than the single aggregate AutoScalingGroupName metric, then the API charges stay the same until the PutMetricData size limit is hit, and only the Custom metrics charge increases.

For example, if the ASG is running c8g.xlarge instances, then by running one fewer instance due to the higher utilization unlocked by these features, then the monthly cost saving in us-east-2 would be:

1 c8g.xlarge @ $0.15896/hour * 30 days * 24 hours = $114.45/month

Taking away the $26.82/month in estimated CloudWatch costs means a savings of $87.63/month per ASG. This is nearly 8% saving on the EC2 cost in this example.

Template to publish metrics and updating your scaling policies

To help you start publishing high resolution metrics, we have created a sample AWS CloudFormation template. The template provides the scaffolding to demonstrate the new faster scaling period for an existing ASG. It includes installing a CloudWatch agent and publishing the CPU Utilization of the ASG instances to CloudWatch at high resolution. The template also includes a Target Tracking policy, as described in this post.

Instructions on deployment and customization requirements can be found in the AWS Samples Repo for Faster Target Tracking. However, there are a few code snippets in the template that we want to highlight.

First, to install the CloudWatch agent, the template updates the UserData of the Launch Template used with the ASG.

UserData: 
          Fn::Base64: 
            !Sub |
              #!/bin/bash
              yum install amazon-cloudwatch-agent -y
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:/cw-agent-asg-aggregate-cpu -s

This command refers to an AWS Systems Manager parameter holding the Cloudwatch Agent configuration.

The following snippet of the Systems Manager parameter reports the CPU Utilization metric at a 10 second interval to a custom namespace called FasterScalingDemo. The metric is also aggregated with the name of the ASG as a dimension so that you can easily refer to it in CloudWatch.

CloudWatchMetricsSSMParameter:
    Type: AWS::SSM::Parameter
    Properties:
      Name: cw-agent-asg-aggregate-cpu
      Type: String
      Value: '{"agent":{"metrics_collection_interval":10,"run_as_user":"cwagent"},"metrics":{"force_flush_interval":10,"aggregation_dimensions":[["AutoScalingGroupName"]],"append_dimensions":{"AutoScalingGroupName":"${aws:AutoScalingGroupName}"},"namespace":"FasterScalingDemo","metrics_collected":{"cpu":{"drop_original_metrics":["cpu_usage_active"],"measurement":[{"name":"cpu_usage_active","rename":"CPUUtilization"}]}}}}'
      Tier: Intelligent-Tiering
      Description: Custom metric specification for CloudWatch Agent

Second, the template also includes an updated AWS Identity and Access Management (IAM) Role and corresponding IAM Instance Profile with permissions to PutMetricData to CloudWatch, and to retrieve Systems Manager parameters that we created previously to configure the agent.

IAMInstanceRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: FasterScalingDemo
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - cloudwatch:PutMetricData
                  - ec2:DescribeTags
                  - ssm:GetParameter
                Resource: '*'
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName}_IAMROLE

Finally, the following image depicts the architecture deployed by the CloudFormation template.

Amazon VPC with an ASG launching instances in three Availability Zones, publishing high-resolution metrics to CloudWatch

Figure 4: AWS resources created in the CloudFormation example

When the template is deployed with your chosen ASG, you should be ready to test Target Tracking set with high resolution metrics. You can perform a load test to see Target Tracking in action. The closer the load test mimics your application usage pattern, the more conclusive the test would be in determining the benefits of these features.

Conclusion

This post provides an overview of the updates we have made to the Target Tracking policy that deliver higher precision in matching your demand with Amazon EC2 capacity. Specifically, this post demonstrated the value of using high resolution CloudWatch metrics with Target Tracking to increase the Auto Scaling rate to match demand, improve availability, and open possibilities for better resource utilization. We encourage you to test the feature and apply the consideration factors outlined in this post before opting for high-resolution metric scaling. You can find more details about these new features in the Target Tracking documentation.

Hosting containers at the edge using Amazon ECS and AWS Outposts server

Post Syndicated from aostan original https://aws.amazon.com/blogs/compute/hosting-containers-at-the-edge-using-amazon-ecs-and-aws-outposts-server/

This post is written by Craig Warburton, Hybrid Cloud Senior Solutions Architect and Sedji Gaouaou, Hybrid Cloud Senior Solutions Architect

In today’s fast-paced digital landscape, businesses are increasingly looking to process data and run applications closer to the source, at the edge of the network. For those seeking to use the power of containerized workloads in edge environments, AWS Outposts servers offer a compelling solution. This fully managed service brings the AWS infrastructure, services, APIs, and tools to virtually any on-premises or edge location, allowing users to run container-based applications seamlessly across their distributed environments. In this post, we explore how Outposts servers can empower organizations to deploy and manage containerized workloads at the edge, bringing cloud-native capabilities closer to where they’re needed most.

Solution overview

Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service that can be used with Outposts servers. This combination allows users to run containerized applications at the edge with the same ease and flexibility as in the AWS cloud.

By using Outposts server with Amazon ECS, users can effectively extend their container-based workloads to the edge, enabling new use cases and improving application performance for latency-sensitive operations.

The following diagram illustrates an example architecture where a user is looking to deploy a microservices based PHP web application and instance based MySQL database. Furthermore, a container based load balancer appliance is used to receive and distribute traffic to the web application container. The example application writes its data to a MySQL database, which is hosted on an external storage array. The application is deployed on the Outpost server, and can communicate with the database across the user data center network.

In this post we will show how users can deploy an example microservice based application. Each section of this post walks through Steps 1 through 4 shown in the following diagram.

Figure 1: Solution overview

Figure 1: Solution overview

Walkthrough

Prerequisites

Before deploying the sample application, you must have ordered, received, and successfully installed an Outposts server. The server is operational and visible in the AWS Management Console.

This walkthrough assumes you have access to Amazon Elastic Container Registry (Amazon ECR) that is used for the container repository.

You need the following AWS Identity and Access Management (IAM) role provisioned with the necessary permissions included in the policy to permit the load balancer to read the required Amazon ECS attributes. Refer to the user guide Create a role to delegate permissions to an IAM user section to help you through creating an IAM role and associated policy. The Amazon ECS task IAM role needs the following policy configuration to read the necessary Amazon ECS information:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "LoadBalancerECSReadAccess",
            "Effect": "Allow",
            "Action": [
                "ecs:ListClusters",
                "ecs:DescribeClusters",
                "ecs:ListTasks",
                "ecs:DescribeTasks",
                "ecs:DescribeContainerInstances",
                "ecs:DescribeTaskDefinition",
                "ec2:DescribeInstances",
                "ssm:DescribeInstanceInformation"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

You also need the Amazon ECS task execution IAM role (ecsTaskExecutionRole) that will grant the Amazon ECS container service the necessary permissions to make AWS API calls on your behalf.

Step 1: Setting up Amazon ECS on Outposts server

Amazon ECS is used in this walkthrough to deploy our container workloads to the Outposts server. Before deploying workloads, an ECS cluster on Outposts needs to be created.

In this configuration, the Amazon ECS cluster targets the private subnets (10.0.1.0/24 and 10.0.2.0/24) and the Amazon Elastic Compute Cloud Amazon (EC2) instances configured on the Outpost server for deployments.

To assist in targeting the deployment of our Amazon ECS services to specific instances with an attached Local Network Interface (LNI), our Amazon EC2 instances are assigned a logical role using custom Amazon ECS container instance attributes. Custom attributes are used to configure task placement constraints, as shown in the following figure.

Figure 2: Amazon ECS container instances used for tasks

Figure 2: Amazon ECS container instances used for tasks

One of the container instances is assigned the role of loadbalancer, as shown in the following figure. Follow the developer guide section to Define which container instances Amazon ECS uses for tasks, and add the following custom attribute to one of your instances:

  • Name = role, Value = loadbalancer

Figure 3: Instance with the Custom Attibutes - loadbalancer

Figure 3: Instance with the Custom Attibutes – loadbalancer

The other container instance is assigned the role of webserver, as shown in the following figure. Add the following custom attribute to each of the remaining instance:

  • Name = role, Value = webserver

Figure 4: Instance with the Custom Attibutes - webserver

Figure 4: Instance with the Custom Attributes – webserver

Step 2: Deploying a load balancer with host mode to use LNI

In this section, you deploy a task for the load balancer as seen in Step 2 of the Solution overview.

First, you must enable the private subnet, where your load balancer is deployed, for LNIs:

aws ec2 modify-subnet-attribute \

    --subnet-id subnet-1a2b3c4d \

    --enable-lni-at-device-index 1

Now add an LNI to the container instance with the attibute “loadbalancer”. This instance can now access your local network.

To deploy the load balancer, create an Amazon ECS task definition named “task-definition-loadbalancer.json”, which describes the container configuration to implement the load balancer as followed:

{
    "containerDefinitions": [
        {
            "name": "loadbalancer",
            "image": "traefik:latest",
            "cpu": 0,
            "portMappings": [
                {
                    "containerPort": 80,
                    "hostPort": 80,
                    "protocol": "tcp"
                },
                {
                    "containerPort": 8080,
                    "hostPort": 8080,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "command": [
                "--api.dashboard=true",
                "--api.insecure=true",
                "--accesslog=true",
                "--providers.ecs.ecsAnywhere=false",
                "--providers.ecs.region=<AWS_REGION>",
                "--providers.ecs.autoDiscoverClusters=true",
                "--providers.ecs.clusters=<YOUR_CLUSTER_NAME>",
                "--providers.ecs.exposedByDefault=true"
            ],
            "environment": [],
            "mountPoints": [],
            "volumesFrom": [],
            "systemControls": []
        }
    ],
    "family": "loadbalancer",
    "taskRoleArn": <TASK_ROLE_ARN>,
    "executionRoleArn": <EXECUTION_ROLE_ARN>,
    "networkMode": "host",
    "volumes": [],
    "placementConstraints": [
        {
            "type": "memberOf",
            "expression": "attribute:role == loadbalancer"
        }
    ],
    "requiresCompatibilities": [
        "EC2"
    ],
    "cpu": "256",
    "memory": "128",
    "tags": []
}

Replace the string <TASK_ROLE_ARN> with the Amazon Resource Name (ARN) of the IAM role configured with the LoadBalancerECSReadAccess policy and the string <EXECUTION_ROLE_ARN> with the ARN of the IAM role configured with the ecsTaskExecutionRole policy as configured in the Prerequisites section, <AWS_REGION> with the AWS Region where you deployed your ECS cluster, <YOUR_CLUSTER_NAME> with your cluster name.

Some points to consider:

  • The Amazon ECS Network mode is set to “host”. The load balancer task uses the host’s network to access the LNI.
  • The task definition includes the placement constraint matching the loadbalancer custom attribute value.

Lastly, register the task definition with your cluster and create the loadbalancer service using the following AWS Command Line Interface (AWS CLI) command:

aws ecs register-task-definition --cli-input-json file://task-definition-loadbalancer.json

aws ecs create-service--cluster <CLUSTER_NAME> --service-name loadbalancer --task-definition loadbalancer:1 --desired-count 1

Replace the string <CLUSTER_NAME> with the target Amazon ECS cluster name.

The load balancer is now running.

Connecting to the Amazon EC2 instance with the attibute loadbalancer using Session Manager, you can get the following LNI IP address:

Figure 5: Getting the LNI IP

Figure 5: Getting the LNI IP

You can access the web user interface by browsing to the URL from your local network:

http://<HOST_IP>:8080/dashboard/

Replace the string <HOST_IP> with the Amazon EC2 instance host LNI IP address, or DNS hostname.

Step 3: Deploying sample web application in awsvpc mode

First, make sure that the AWSVPC Trunking is turned on, as shown in the following figure:

Figure 6: Enabling AWSVPC Trunking

Figure 6: Enabling AWSVPC Trunking

Create an Amazon ECS task definition for our application named “task-definition-webapp.json”, which describes the container configuration to implement the example web application as followed:

Replace the <PLACEHOLDER> values for your application.

{
    "containerDefinitions": [
        {
            "name": "whoami",
            "image": "<CONTAINER-IMAGE>:latest",
            "cpu": 0,
            "portMappings": [
                {
                    "name": "<WEBAPP>",
                    "containerPort": 80,
                    "hostPort": 80,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "environment": [],
            "mountPoints": [],
            "volumesFrom": [],
            "dockerLabels": {
"traefik.http.routers.<WEBAPP>-host.rule":     "Host(`<WEBAPP>.domain.com`)",
               "traefik.http.routers.<WEBAPP>-path.rule": "Path(`/<WEBAPP>`)",
               "traefik.http.services.<WEBAPP>.loadbalancer.server.port": "80"
            },
            "systemControls": []
        }
    ],
    "family": "<WEBAPP>",
    "networkMode": "awsvpc",
    "volumes": [],
    "placementConstraints": [
        {
            "type": "memberOf",
            "expression": "attribute:role == webserver"
        }
    ],
    "requiresCompatibilities": [
        "EC2"
    ],
    "cpu": "256",
    "memory": "128",
    "tags": []
}

In the task-definition-webapp.json, consider the following:

  • The task definition includes the placement constraint matching the webserver custom attribute value.
  • Docker label traefik.http.routers is used to configure host and path based routing rules.
  • As the example web application container exposes the single TCP port 80, Docker label traefik.http.services.<WEBAPP> is used to configure this port for private communication with the Traefik load balancer.

Register the task definition with your cluster and create the loadbalancer service using the following AWS CLI command:

aws ecs register-task-definition --cli-input-json file://task-definition-webapp.json

aws ecs create-service--cluster <CLUSTER_NAME> --service-name <WEBAPP> --task-definition <WEBAPP>:1 --desired-count 1

Replace the string <CLUSTER_NAME> with the target Amazon ECS cluster name and the string <WEBAPP> with your application.

You can access the whoami application by browsing to the URL from your local network:

http://<HOST_IP>/<WEBAPP>

Step 4: Provision DB instance and attach an external storage

The web application has been successfully deployed, so we will move on to the deployment and configuration of the database server next. First, deploy an Amazon EC2 instance to host a MySQL database. As shown in the following screenshot, use the AWS Console to choose an instance type (this is dependent on your Outposts server instance capacity configuration) and configure its network settings to target the correct VPC and the subnet deployed to the Outposts server.

Figure 7: Provisioning a database instance

Figure 7: Provisioning a database instance

When the instance is available, deploy MySQL following a standard documented approach to install on a Linux host from the vendor. After successfully installing MySQL, configure users and tables necessary for the application. The sample application configuration file can now be updated to allow the PHP web server container to connect to the MySQL database, as well as create a user and list the users, as shown in the following figures.

Figure 8: Updating the application config file to use the database instance

Figure 8: Updating the application config file to use the database instance

Figure 9: Sample application connected to database

Figure 9: Sample application connected to database

For the database instance, make sure that the data associated with the application is stored on an existing storage array in the user data center. To do this, you must complete the following:

(a) Enable connectivity to the user network through the LNI.

(b) Mount the iSCSI volume in the EC2 instance.

(c) Configure MySQL to use this iSCSI volume.

To enable connectivity, follow the same process described in step 2 of this post to add an Elastic Network Interface (ENI) with the correct device index to present the LNI to the instance. The following screenshots show a second ENI configured on the instance and associated with the LNI along with the interface and address configuration of the instance that shows two addresses (VPC and user network addresses).

Figure 10: Network interface configuration

Figure 10: Network interface configuration

Now that connectivity has been established to the user network, you can configure the storage array to present an ISCSI volume to the database instance and mount that volume. The following screenshot shows the /mnt mountpoint being used with iSCSI multi-path across four volumes.

Figure 11: iSCSI volume mount

Figure 11: iSCSI volume mount

Finally, configure MySQL to use the iSCSI volume to store data by stopping the MySQL service, updating the default configuration file /etc/my.cnf, and restarting MySQL, as shown in the following figure.

Figure 12: MySQL configuration

Figure 12: MySQL configuration

Clean up:

Please follow the below instructions to clean up after testing:

  • Delete the <WEBAPP> service
  • Delete the loadbalancer service
  • Delete your Amazon ECS cluster
  • Delete the MySQL Database EC2 instance
  • Delete all VPCs

Conclusion

This post has demonstrated how to deploy a sample container-based web application while connecting to the user network, allowing access to the application and connecting to existing storage appliances.

AWS Outposts server allows users to run containers at the edge, addressing challenges related to low latency, local data processing, and data residency. Amazon ECS allows you to deploy consistently, whether in-Region or at the edge, allowing users to develop once and deploy many times.

Get started with Outposts servers by visiting the Outposts servers webpage and learn more about Amazon ECS to begin deploying your containarized workloads at the edge!

Announcing future-dated Amazon EC2 On-Demand Capacity Reservations

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/announcing-future-dated-amazon-ec2-on-demand-capacity-reservations/

Customers use Amazon Elastic Compute Cloud (Amazon EC2) to run every type of workload imaginable, including web hosting, big data processing, high-performance computing (HPC), virtual desktops, live event streaming, and databases. Some of these workloads are so critical that customers asked for the ability to reserve capacity for them.

To help customers flexibly reserve capacity, we launched EC2 On-Demand Capacity Reservations (ODCRs) in 2018. Since then, customers have used capacity reservations (CRs) to run critical applications like hosting consumer websites, streaming lives sporting events and processing financial transactions.

Today, we’re announcing the ability to get capacity for future workloads using CRs. Many customers have future events such as product launches, large migrations, or end-of-year sales events like Cyber Monday or Diwali. These events are critical, and customers want to ensure they have the capacity when and where they need it.

While CRs helped customers reserve capacity for these events, they were only available just-in-time. So customers either needed to provision the capacity ahead of time and pay for it or plan with precision to provision CRs just-in-time at the start of the event.

Now you can plan and schedule your CRs up to 120 days in advance. To get started you specify the capacity you need, the start date, delivery preference, and the minimum duration you commit to use the capacity reservation. There are no upfront charges to schedule a capacity reservation. After Amazon EC2 evaluates and approves the request, it will activate the reservation on the start date, and customers can use it to immediately launch instances.

Getting started with future-dated capacity reservations
To reserve your future-dated capacity, choose Capacity Reservations on the Amazon EC2 console and select Create On-Demand Capacity Reservation, and choose Get started.

To create a capacity reservation, specify the instance type, platform, Availability Zone, platform, tenancy, and number of instances you are requesting.

future-dated-2a

In the Capacity Reservation details section, choose At a future date in the Capacity Reservation starts option and choose your start date and commitment duration.

future-dated-1a

You can also choose to end the capacity reservation at a specific time or manually. If you select Manually, the reservation has no end date. It will remain active in your account and continue to be billed until you manually cancel it. To reserve this capacity, choose Create.

future-dated-4

After you create your capacity request, it appears in the dashboard with an Assessing status. During this state, AWS systems will work to determine if your request is supportable which is usually done within 5 days. Once the systems determine the request is supportable, the status will be changed to Scheduled. In rare cases, your request may be unsupported.

On your scheduled date, the capacity reservation will change to an Active state, the total instance count will be increased to the amount requested, and you can immediately launch instances.

After activation, you must hold the reservation for at least the commitment duration. After the commitment duration elapses, you can continue to hold and use the reservation if you’d like or cancel it if no longer needed.

Things to know
Here are some things that you should know about the future-dated CRs:

  • Evaluation – Amazon EC2 considers multiple factors when evaluating your request. Along with forecasted supply, Amazon EC2 considers how long you plan to hold the capacity, how early you create the Capacity Reservation relative to your start date, and the size of your request. To improve the ability of Amazon EC2 to support your request, create your reservation at least 56 days (8 weeks) before the start date. You need to submit a request for at least 100 vCPUs for only C, M, R, T, I instance types. The recommended minimum commitment for most requests will be 14 days.
  • Notification – We recommend monitoring the status of your request through the console or Amazon EventBridge You can use these notifications to trigger automation or send an email or text update. To learn more, visit Send an email when events happen using Amazon EventBridge in the Amazon EventBridge User Guide.
  • Pricing – Future dated capacity reservations are billed just like regular CRs. It is charged at the equivalent On-Demand rate whether you run instances in reserved capacity or not. For example, if you create a future dated CR for 20 instances and run 15 instances, you will be charged for 15 active instances and for 5 unused instances in the reservation including the minimum duration. Savings Plans apply to both unused reservations and instances running on the reservation. To learn more, visit Capacity Reservation pricing and billing in the Amazon EC2 User Guide.

Now available
Future dated EC2 Capacity Reservations are now available today in all AWS Regions where Amazon EC2 Capacity Reservations are available.

Give Amazon EC2 Capacity Reservations a try in the Amazon EC2 console. To learn more, visit On-Demand Capacity Reservations in the Amazon EC2 User Guide and send feedback to AWS re:Post for Amazon EC2 or through your usual AWS Support contacts.

— Channy

Run high-availability long-running clusters with Amazon EMR instance fleets

Post Syndicated from Garima Arora original https://aws.amazon.com/blogs/big-data/run-high-availability-long-running-clusters-with-amazon-emr-instance-fleets/

AWS now supports high availability Amazon EMR on EC2 clusters with instance fleet configuration. With high availability instance fleet clusters, you now get the enhanced resiliency and fault tolerance of high availability architecture, along with the improved flexibility and intelligence in Amazon Elastic Compute Cloud (Amazon EC2) instance selection of instance fleets. Amazon EMR is a cloud big data platform for petabyte-scale data processing, interactive analysis, streaming, and machine learning (ML) using open source frameworks such as Apache Spark, Presto and Trino, and Apache Flink. Customers love the scalability and flexibility that Amazon EMR on EC2 offers. However, like most distributed systems running mission-critical workloads, high availability is a core requirement, especially for those with long-running workloads.

In this post, we demonstrate how to launch a high availability instance fleet cluster using the newly redesigned Amazon EMR console, as well as using an AWS CloudFormation template. We also go over the basic concepts of Hadoop high availability, EMR instance fleets, the benefits and trade-offs of high availability, and best practices for running resilient EMR clusters.

High availability in Hadoop

High availability (HA) provides continuous uptime and fault tolerance for a Hadoop cluster. The core components of Hadoop, like Hadoop Distributed File System (HDFS) NameNode and YARN ResourceManager, are single points of failure in clusters with a single primary node. In the event that any of them crash, the entire cluster goes down. High Availability removes this single point of failure by introducing redundant standby nodes that can quickly take over if the primary node fails.

In a high availability EMR cluster, one node serves as the active NameNode that handles client operations, and others act as standby NameNodes. The standby NameNodes constantly synchronize their state with the active one, enabling seamless failover to maintain service availability. To learn more, see Supported applications in an Amazon EMR Cluster with multiple primary nodes.

Key instance fleet differentiations

Amazon EMR recommends using the instance fleet configuration option for provisioning EC2 instances in EMR clusters because it offers a flexible and robust approach to cluster provisioning. Some key advantages include:

  • Flexible instance provisioning – Instance fleets provide a powerful and simple way to specify up to five EC2 instance types on the Amazon EMR console, or up to 30 when using the AWS Command Line Interface (AWS CLI) or API with an allocation strategy. This enhanced diversity helps optimize for cost and performance while increasing the likelihood of fulfilling capacity requirements.
  • Target capacity management – You can specify target capacities for On-Demand and Spot Instances for each fleet. Amazon EMR automatically manages the mix of instances to meet these targets, reducing operational overhead.
  • Improved availability – By spanning multiple instance types and purchasing options such as On-Demand and Spot, instance fleets are more resilient to capacity fluctuations in specific EC2 instance pools.
  • Enhanced Spot Instance handling – Instance fleets offer superior management of Spot Instances, including the ability to set timeouts and specify actions if Spot capacity can’t be provisioned.
  • Reliable cluster launches – You can configure your instance fleet to select multiple subnets for different Availability Zones, allowing Amazon EMR to find the best combination of instances and purchasing options across these zones to launch your cluster in. Amazon EMR will identify the best Availability Zone based on your configuration and available EC2 capacity and launch the cluster.

Prerequisites

Before you launch the high availability EMR instance fleet clusters, make sure you have the following:

  • Latest Amazon EMR release – We recommend that you use the latest Amazon EMR release to benefit from the highest level of resiliency and stability for your high availability clusters. High availability for instance fleets is supported with Amazon EMR releases 5.36.1, 6.8.1, 6.9.1, 6.10.1, 6.11.1, 6.12.0, and later.
  • Supported applications – High availability for instance fleets is supported for applications such as Apache Spark, Presto, Trino, and Apache Flink. Refer to Supported applications in an Amazon EMR Cluster with multiple primary nodes for the complete list of supported applications and their failover processes.

Launch a high availability instance fleet cluster using the Amazon EMR console

Complete the following steps on the Amazon EMR console to configure and launch a high availability EMR cluster with instance fleets:

  1. On the Amazon EMR console, create a new cluster.
  2. For Name, enter a name.
  3. For Amazon EMR release, choose the Amazon EMR release that supports high availability clusters with instance fleets. The setting will default to the latest available Amazon EMR release.

CreateHACluster-EMRRelease

  1. Under Cluster configuration, choose the desired instance types for the primary fleet. (You can select up to five when using the Amazon EMR console.)
  2. Select Use high availability to launch the cluster with three primary nodes.

CreateHACluster

  1. Choose the instance types and target On-Demand and Spot size for the core and task fleet according to your requirements.

InstanceFleet-CreateFleets

  1. Under Allocation strategy, select Apply allocation strategy.
    1. 1 We recommend that you select Price-capacity optimized for your allocation strategy for your cluster for faster cluster provisioning, more accurate Spot Instance allocation, and fewer Spot Instance interruptions.
  2. Under Networking, you can choose multiple subnets for different Availability Zones. This allows Amazon EMR to look across those subnets and launch the cluster in an Availability Zone that best suits your instance and purchasing option requirements.

allocationStrategy

  1. Review your cluster configuration and choose Create cluster.

Amazon EMR will launch your cluster in a few minutes. You can view the cluster details on the Amazon EMR console.
ClusterDetailPage

Launch a high availability cluster with AWS CloudFormation

To launch a high availability cluster using AWS CloudFormation, complete the following steps:

  1. Create a CloudFormation template with EMR resource type AWS::EMR::Cluster and JobFlowInstancesConfig property types MasterInstanceFleet, CoreInstanceFleet and (optional) TaskInstanceFleets. To launch a high availability cluster, configure TargetOnDemandCapacity=3, TargetSpotCapacity=0 for the primary instance fleet and weightedCapacity=1 for each instance type configured for the fleet. See the following code:
{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Resources": {
    "cluster": {
      "Type": "AWS::EMR::Cluster",
      "Properties": {
        "Instances": {
          "Ec2SubnetIds": [
            "subnet-003c889b8379f42d1",
            "subnet-0382aadd4de4f5da9",
            "subnet-078fbbb77c92ab099"
          ],
          "MasterInstanceFleet": {
            "Name": "HAPrimaryFleet",
            "TargetOnDemandCapacity": 3,
            "TargetSpotCapacity": 0,
            "InstanceTypeConfigs": [
              {
                "InstanceType": "m5.xlarge",
                "WeightedCapacity": 1
              },
              {
                "InstanceType": "m5.2xlarge",
                "WeightedCapacity": 1
              },
              {
                "InstanceType": "m5.4xlarge",
                "WeightedCapacity": 1
              }
            ]
          },
          "CoreInstanceFleet": {
            "Name": "cfnCore",
            "InstanceTypeConfigs": [
              {
                "InstanceType": "m5.xlarge",
                "WeightedCapacity": 1
              },
              {
                "InstanceType": "m5.2xlarge",
                "WeightedCapacity": 2
              },
              {
                "InstanceType": "m5.4xlarge",
                "WeightedCapacity": 4
              }
            ],
            "LaunchSpecifications": {
              "SpotSpecification": {
                "TimeoutAction": "SWITCH_TO_ON_DEMAND",
                "TimeoutDurationMinutes": 20,
                "AllocationStrategy": "PRICE_CAPACITY_OPTIMIZED"
              }
            },
            "TargetOnDemandCapacity": "4",
            "TargetSpotCapacity": 0
          },
          "TaskInstanceFleets": [
            {
              "Name": "cfnTask",
              "InstanceTypeConfigs": [
                {
                  "InstanceType": "m5.xlarge",
                  "WeightedCapacity": 1
                },
                {
                  "InstanceType": "m5.2xlarge",
                  "WeightedCapacity": 2
                },
                {
                  "InstanceType": "m5.4xlarge",
                  "WeightedCapacity": 4
                }
              ],
              "LaunchSpecifications": {
                "SpotSpecification": {
                  "TimeoutAction": "SWITCH_TO_ON_DEMAND",
                  "TimeoutDurationMinutes": 20,
                  "AllocationStrategy": "PRICE_CAPACITY_OPTIMIZED"
                }
              },
              "TargetOnDemandCapacity": "0",
              "TargetSpotCapacity": 4
            }
          ]
        },
        "Name": "TestHACluster",
        "ServiceRole": "EMR_DefaultRole",
        "JobFlowRole": "EMR_EC2_DefaultRole",
        "ReleaseLabel": "emr-6.15.0",
        "PlacementGroupConfigs": [
          {
            "InstanceRole": "MASTER",
            "PlacementStrategy": "SPREAD"
          }
        ]
      }
    }
  }
}

Make sure to use an Amazon EMR release that supports high availability clusters with instance fleets.

  1. Create a CloudFormation stack with the preceding template:
aws cloudformation create-stack --stack-name HAInstanceFleetCluster --template-body file://cfn-template.json --region us-east-1
  1. Retrieve the cluster ID from the list-clusters response to use in the following steps. You can further filter this list based on filters like cluster status, creation date, and time.
aws emr list-clusters --query "Clusters[?Name=='<YourClusterName>']"
  1. Run the following describe-cluster command:
aws emr describe-cluster --cluster-id j-XXXXXXXXXXX --region us-east-1

If the high availability cluster was launched successfully, the describe-cluster response will return the state of the primary fleet as RUNNING and provisionedOnDemandCapacity as 3. By this point, all three primary nodes have been started successfully.

DescribeClusterResponse

Primary node failover with High Availability clusters

To fetch information on all EC2 instances for an instance fleet, use the list-instances command:

aws emr list-instances --cluster-id j-XXXXXXXXXXX --instance-fleet-type MASTER --region us-east-1

For high availability clusters, it will return three instances in RUNNING state for the primary fleet and other attributes like public and private DNS names.

PrimaryInstance-DescribeCluster

The following screenshot shows the instance fleet status on the Amazon EMR console.

Instancefleet status

Let’s examine two cases for primary node failover.

Case 1: One of the three primary instances is accidentally stopped

When an EC2 instance is accidentally stopped by a user, Amazon EMR detects this and performs a failover for the stopped primary node. Amazon EMR also attempts to launch a new primary node with the same private IP and DNS name to recover back the quorum. During this failover, the cluster remains fully operational, providing true resiliency to single primary node failures.

The following screenshots illustrate the instance fleet details.

InstanceFleetDetail-PrimaryInstanceTerminated

instanceFleerRecovery

This automatic recovery for primary nodes is also reflected in the MultiMasterInstanceGroupNodesRunning or MultiMasterInstanceGroupNodesRunningPercentage Amazon CloudWatch metric emitted by Amazon EMR for your cluster. The following screenshot shows an example of these metrics.

CloudwatchMetrics

Case 2: One of the three primary instances becomes unhealthy

If Amazon EMR continuously receives failures when trying to connect to a primary instance, it is deemed as unhealthy and Amazon EMR will attempt to replace it. Similar to case 1, Amazon EMR will perform a failover for the stopped primary node and also attempt to launch a new primary node with the same private IP and DNS name to recover the quorum.

UnhealthyPrimaryInstance
PrimaryInstanceFailover-2

If you list the instances for the primary fleet, the response will include information for the EC2 instance that was stopped by the user and the new primary instance that replaced it with the same private IP and DNS name.
DescribeClusterResponse-instanceFailover

The following screenshot shows an example of the CloudWatch metrics.

An instance can have connection failures for multiple reasons, including but not limited to disk space unavailable on the instance, critical cluster daemons like instance controller shut down with errors, high CPU utilization, and more. Amazon EMR is continuously improving its health monitoring criteria to better identify unhealthy nodes on an EMR cluster.

Considerations and best practices

The following are some of the key considerations and best practices for using EMR instance fleets to launch a high availability cluster with multiple primary nodes:

  • Use the latest EMR release – With the latest EMR releases, you get the highest level of resiliency and stability for your high availability EMR clusters with multiple primary nodes.
  • Configure subnets for high availability – Amazon EMR can’t replace a failed primary node if the subnet is oversubscribed (there aren’t any available private IP addresses in the subnet). This results in a cluster failure as soon as the second primary node fails. Limited availability of IP addresses in a subnet can also result in cluster launch or scaling failures. To avoid such scenarios, we recommend that you dedicate an entire subnet to an EMR cluster.
  • Configure core nodes for enhanced data availability – To minimize the risk of local HDFS data loss on your production clusters, we recommend that you set the dfs.replication parameter to 3 and launch at least four core nodes. Setting dfs.replication to 1 on clusters with fewer than four core nodes can lead to data loss if a single core node goes down. For clusters with three or fewer core nodes, set dfs.replication parameter to at least 2 to achieve sufficient HDFS data replication. For more information, see HDFS configuration.
  • Use an allocation strategy – We recommend enabling an allocation strategy option for your instance fleet cluster to provide faster cluster provisioning, more accurate Spot Instance allocation, and fewer Spot Instance interruptions.
  • Set alarms for monitoring primary nodes – You should monitor the health and status of primary nodes of your long-running clusters to maintain smooth operations. Configure alarms using CloudWatch metrics such as MultiMasterInstanceGroupNodesRunning, MultiMasterInstanceGroupNodesRunningPercentage, or MultiMasterInstanceGroupNodesRequested.
  • Integrate with EC2 placement groups – You can also choose to protect primary instances against hardware failures by using a placement group strategy for your primary fleet. This will spread the three primary instances across separate underlying hardware to avoid loss of multiple primary nodes at the same time in the event of a hardware failure. See Amazon EMR integration with EC2 placement groups for more details.

When setting up a high availability instance fleet cluster with Amazon EMR on EC2, it’s important to understand that all EMR nodes, including the three primary nodes, are launched within a single Availability Zone. Although this configuration maintains high availability within that Availability Zone, it also means that the entire cluster can’t tolerate an Availability Zone outage. To mitigate the risk of cluster failures due to Spot Instance reclamation, Amazon EMR launches the primary nodes using On-Demand instances, providing an additional layer of reliability for these critical components of the cluster.

Conclusion

This post demonstrated how you can use high availability with EMR on EC2 instance fleets to enhance the resiliency and reliability of your big data workloads. By using instance fleets with multiple primary nodes, EMR clusters can withstand failures and maintain uninterrupted operations, while providing enhanced instance diversity and better Spot capacity management within a single Availability Zone. You can quickly set up these high availability clusters using the Amazon EMR console or AWS CloudFormation, and monitor their health using CloudWatch metrics.

To learn more about the supported applications and their failover process, see Supported applications in an Amazon EMR Cluster with multiple primary nodes. To get started with this feature and launch a high availability EMR on EC2 cluster, refer to Plan and configure primary nodes.

About the Authors

Garima Arora is a Software Development Engineer for Amazon EMR at Amazon Web Services. She specializes in capacity optimization and helps build services that allow customers to run big data applications and petabyte-scale data analytics faster. When not hard at work, she enjoys reading fiction novels and watching anime.

Ravi Kumar is a Senior Product Manager Technical-ES (PMT) at Amazon Web Services, specialized in building exabyte-scale data infrastructure and analytics platforms. With a passion for building innovative tools, he helps customers unlock valuable insights from their structured and unstructured data. Ravi’s expertise lies in creating robust data foundations using open-source technologies and advanced cloud computing, that powers advanced artificial intelligence and machine learning use cases. A recognized thought leader in the field, he advances the data and AI ecosystem through pioneering solutions and collaborative industry initiatives. As a strong advocate for customer-centric solutions, Ravi constantly seeks ways to simplify complex data challenges and enhance user experiences. Outside of work, Ravi is an avid technology enthusiast who enjoys exploring emerging trends in data science, cloud computing, and machine learning.

Tarun Chanana is a Software Development Manager for Amazon EMR at Amazon Web Services.

Using zonal shift with Amazon EC2 Auto Scaling

Post Syndicated from aostan original https://aws.amazon.com/blogs/compute/using-zonal-shift-with-amazon-ec2-auto-scaling/

This post is written by Michael Haken, Senior Principal Solutions Architect, AWS

Today, we’re announcing support for zonal shift in Amazon EC2 Auto Scaling. Zonal shift gives allows you to rapidly recover from application impairments in a single Availability Zone (AZ) impacting your Auto Scaling Group (ASG) resources. In this post, we describe how performing an ASG zonal shift fits in to a multi-AZ resilience strategy and considerations for how to use the feature with different architectures.

Overview

Using multiple AZs is an architectural best practice for building resilient applications on AWS. Deploying your application across multiple AZs makes your applications more available, fault tolerant, and scalable. EC2 Auto Scaling enables you to further enhance your application’s availability and fault tolerance by dynamically scaling your Amazon Elastic Compute Cloud (Amazon EC2) instances across multiple AZs and replacing them when they’re unhealthy.

AZs in AWS represent a fault isolation boundary, meaning that failures from various sources are contained to a single AZ, whether caused by a bad deployment, networking issues, power loss, or operator error. In 2023, we launched zonal shift, part of Amazon Application Recovery Controller (ARC), which allows you to Rapidly recover from application impairments in a single AZ by shifting traffic at your Elastic Load Balancing (ELB) load balancer.

Zonal shift for EC2 Auto Scaling enhances this capability for users who have already implemented recovery patterns for single AZ impairments. It also provides recovery capabilities for architectures that aren’t load balanced by allowing you to prevent new instance launches in a specified AZ. Without zonal shift, when EC2 Auto Scaling detects consistent launch failures in an AZ, the service tries to launch instances in other AZs configured for the ASG. However, certain conditions, like gray failures, can cause post-launch problems in a single AZ that EC2 Auto Scaling doesn’t detect. For example, successfully launched instances in a single AZ experience elevated error rates downloading their configuration files over a zonal Amazon S3, Amazon Virtual Private Cloud (Amazon VPC) interface endpoint. The instances can’t correctly configure their application software and respond to requests with errors. Alternatively, the single-AZ impairment could cause the instance to fail its health checks after provisioning. This causes EC2 Auto Scaling to constantly recycle instances in the impaired AZ, leading to the application running with less capacity than desired.

Although you might choose to perform a zonal shift at your load balancer to mitigate the impact caused by the event, new instances can still be launched in the impacted AZ and don’t receive incoming requests. Even if your application architecture doesn’t use load balancers, zonal shift for EC2 Auto Scaling can help you recover from single-AZ impairments by allowing you to prevent instance launches in the impaired AZ.

Using EC2 Auto Scaling zonal shift to recover

To use zonal shift on your ASG, you need to configure it with an AvailabilityZoneImpairmentPolicy parameter either when you create a new ASG or update an existing one. This parameter has two options, ZonalShiftEnabled that enables or disables the ability to perform zonal shifts, and ImpairedZoneHealthCheckBehaviour. The latter option allows you to choose between ignoring or replacing instances identified as unhealthy by EC2 Auto Scaling. First, we look at how you can use zonal shift with a standalone ASG architecture.

Standalone ASG zonal shift

This architecture uses a standalone ASG without being integrated with an ELB load balancer. Workloads with a standalone ASG commonly perform event driven work such as generating load against a target based on a schedule or processing messages from a queue. The architecture in the following figure uses an ASG that reads messages from an Amazon Simple Queue Service (Amazon SQS) queue, performs some processing on the message data, and writes the results into an Amazon Aurora database. The instances communicate with Amazon SQS using a VPC endpoint in each AZ. Each message varies in size, thus the instances use a heartbeat pattern to update the message visibility timeout until they finish processing it. EC2 Auto Scaling scales instances based on the queue depth, which helps make sure that messages are processed in a timely manner.

Figure 1: EC2 instances deployed across three AZs that process messages from an SQS queue

Figure 1: EC2 instances deployed across three AZs that process messages from an SQS queue

Say that a networking degradation causes instances in AZ 1 to experience elevated error rates when attempting to write to the Aurora database, resulting in a 2x increase in the p50 processing latency. The instances in AZ 1 continue to heartbeat until they time out, keeping the message hidden and preventing other healthy instances from taking over the work. As a result, the queue depth grows and EC2 Auto Scaling deploys a new instance, as shown in the following figure.

Figure 2: EC2 Auto Scaling launches a new instance in AZ 1 in response to the queue depth growing

Figure 2: EC2 Auto Scaling launches a new instance in AZ 1 in response to the queue depth growing

The new instance lands in AZ 1 and experiences the same problem as the other instance, thus it can’t decrease the queue depth and processing latency. Instead, it exacerbates the issue by consuming additional messages that aren’t successfully processed. The instances in AZ 1 never appeared unhealthy, thus EC2 Auto Scaling didn’t take any actions to replace them. To mitigate this problem, you can start a zonal shift for your ASG. This makes sure that any future instance launches only happen in AZ 2 or AZ 3, as shown in the following figure.

Figure 3: After the zonal shift new instances are only launched in AZ 2 and AZ 3 by EC2 Auto Scaling

Figure 3: After the zonal shift new instances are only launched in AZ 2 and AZ 3 by EC2 Auto Scaling

You have the option to mark the instances as unhealthy using the SetInstanceHealth API to force EC2 Auto Scaling to replace these instances to prevent them from continuing to contribute to additional latency and errors. Changing the instance health state is considered a mutating change and relies on the EC2 Auto Scaling control plane. Therefore, you should avoid making this a critical step in your recovery plan. When you are confident that the impairment has abated, you can cancel the zonal shift, which causes EC2 Auto Scaling to automatically rebalance capacity across your AZs.

ASG with ELB zonal shift

In this section we observe how to use zonal shift with an ASG that is serving traffic from an ELB. We also examine how the ImpairedZoneHealthCheckBehavior affects recovery in this situation. In this architecture, the instances in the ASG read data from the database when they receive HTTP requests from the ELB, as shown in the following figure.

Figure 4: A three-tier application deployed in three AZs using an ALB, ASG, and Aurora database

Figure 4: A three-tier application deployed in three AZs using an ALB, ASG, and Aurora database

In this scenario, the instances in AZ 1 start experiencing increased latency with their EBS volumes causing them to respond to requests with errors and fail their EC2 instance status checks. Initially, to mitigate the impact, you can start a zonal shift at your load balancer to prevent your users from receiving errors. Then, you can initiate a zonal shift for your ASG to prevent new capacity from being launched into the AZ that isn’t receiving traffic.

If the ASG’s ImpairedZoneHealthCheckBehavior is set to IgnoreUnhealthy, then the instances in AZ 1 that are failing their health checks aren’t terminated by EC2 Auto Scaling, as shown in the following figure. This can be helpful if you’re pre-scaled to handle the loss of an AZ’s worth of capacity by not causing EC2 Auto Scaling to attempt to launch additional instances. It can also make recovery safer by leaving capacity in the AZ, thus when you end your load balancer zonal shift after the impairment ends, the AZ can immediately start receiving traffic again.

Figure 5: Performing a zonal shift on the ALB and ASG, choosing to ignore unhealthy instances in the ASG

Figure 5: Performing a zonal shift on the ALB and ASG, choosing to ignore unhealthy instances in the ASG

Alternatively, you can set the option to ReplaceUnhealthy. Now, instances that are found to be unhealthy by EC2 Auto Scaling are replaced. This option can be helpful if you aren’t pre-scaled to handle the loss of capacity. EC2 Auto Scaling launches new instances into the remaining AZs to bring the ASG back to its desired capacity, as shown in the following figure. However, this approach also has a tradeoff: launching new instances isn’t guaranteed to be successful, thus you might experience delays in acquiring new capacity.

Figure 6: Performing a zonal shift on the ALB and ASG, this time replacing unhealthy instances in the remaining AZs

Figure 6: Performing a zonal shift on the ALB and ASG, this time replacing unhealthy instances in the remaining AZs

In both situations you must consider whether you have cross-zone load balancing enabled or disabled. When cross-zone load balancing is enabled, each instance, regardless of its AZ, receives an approximately equal share of the traffic. This means that you can end your zonal shift for both your load balancer and ASG at the same time safely. As EC2 Auto Scaling rebalances your instances across each enabled AZ, they receive the same percentage of traffic.

If cross-zone load balancing is disabled, then each AZ receives an equal percentage of the traffic, regardless of how many instances are in the AZ. If you’ve chosen to replace unhealthy instances, or if your ASG has scaled during the event, then the capacity across your AZs could have become imbalanced. When you end your load balancer zonal shift and EC2 Auto Scaling begins to rebalance your capacity, you could end up in a situation shown in the following figure, where a single or small number of instances gets an overwhelming portion of the load.

Figure 7: A three-tier architecture with an imbalance of capacity among its three AZs

Figure 7: A three-tier architecture with an imbalance of capacity among its three AZs

This imbalance can present an overload risk, thus you must specify the –skip-zonal-shift-validation parameter when you enable zonal shift to acknowledge that you understand the risk. However, you can help prevent overload from occurring due to imbalance by using the load balancer’s target_group_health.dns_failover.minimum_healthy_targets.count option and specifying the number of instances that should be present in the AZ. If you’re using three AZs and your desired capacity is 12, then you should set the value to four (which represents one third of the ASGs total capacity). This prevents traffic from being routed to the AZ until there is enough healthy capacity there to handle the load. You may need to dynamically adjust this number as the ASG scales over time. The minimum count you set in the past may not be the right minimum count today.

Zonal shift best practices

As a set of best practices, we recommend that you:

  1. Are pre-scaled to handle the loss of an AZ’s worth of capacity
  2. Configure your impairment policy to ignore unhealthy hosts
  3. Enable cross-zone load balancing

With this configuration, you can also safely use zonal autoshift. When zonal autoshift is enabled, AWS automatically starts and ends the zonal shift on your behalf whenever the AWS telemetry indicates there is an impairment affecting a single AZ. This can be used in conjunction with zonal autoshift for your ELB load balancer. If you are not using zonal autoshift, then you can still use the EventBridge observer notifications to inform your zonal shift decisions or start automated processes. Refer to the EC2 Auto Scaling zonal shift documentation for more details on the full set of best practices when using zonal shift.

Conclusion

In this post we showed you the benefits of using zonal shift with your Amazon EC2 Auto Scaling Groups as part of enhancing your resilience in multi-AZ architectures. We explored several scenarios where zonal shift can be used, and reviewed best practices for using zonal shift safely and effectively. To get started using zonal shift with your ASGs, refer to the documentation.