Post Syndicated from Macey Neff original https://aws.amazon.com/blogs/compute/hibernating-ec2-instances-in-response-to-a-cloudwatch-alarm/

This blog post is written by Jose Guay, Technical Account Manger, Enterprise Support. 

A typical option to reduce costs associated with running Amazon Elastic Compute Cloud (Amazon EC2) instances is to stop them when they are idle. However, there are scenarios where stopping an idle instance is not practical. For example, instances with development environments that take time to prepare and run which benefit from not needing to do this process every day. For these instances, hibernation is a better alternative.

This blog post explores a solution that will find idle instances using an Amazon CloudWatch alarm that monitors the instance’s CPU usage. When the CPU usage consistently drops below the alarm’s threshold, the alarm enters the ALARM state and raises an event used to identify the instance and trigger hibernation.

With this solution, the instance no longer incurs in compute costs, and only accrues storage costs for any Amazon Elastic Block Store (Amazon EBS) volumes.

Overview

To hibernate an EC2 instance, there are prerequisites and required preparation. The instance must be configured to hibernate, and this is done when first launching it. This configuration cannot be changed after launching the instance.

One way to trigger instance hibernation is to use an AWS Lambda function. The Lambda function needs specific permissions configured with AWS Identity and Access Management (IAM). To connect the function with the alarm that detects the idle instance, use an Amazon EventBridge bus.

The following architecture diagram shows a solution.

Figure 1 – Solution architecture

• An EC2 instance sends metrics to CloudWatch.
• A CloudWatch alarm detects an idle instance and sends the event to EventBridge.
• EventBridge triggers a Lambda function.
• The Lambda function evaluates the execution role permissions.
• The Lambda function identifies the instance and sends the hibernation signal.

To implement the solution, follow these steps:

1. Configure permissions with IAM
2. Create the Lambda function
3. Configure the EC2 instance to send metrics to CloudWatch
4. Configure EventBridge

a. Configure permissions with IAM

Create an IAM role with permissions to stop an EC2 instance. The Lambda function uses it as its execution role. The IAM role also needs permissions to save logs in CloudWatch. This is useful to log when an instance is entering hibernation.

1. Open the IAM console.
2. In the navigation pane, choose Policies.
3. Select Create policy.
4. For Select a service, search and select CloudWatch Logs.
5. In Actions allowed, search “createlog” and select CreateLogStream and CreateLogGroup.
6. Repeat the search, this time for “putlog”, and select PutLogEvents.
7. In Resources, choose All.
8. Select + Add more permissions.
9. For Select a service, select EC2.
10. In Actions allowed, search “stop” and select StopInstances from the results.
11. In Resources, choose Specific, and select the Add Arn
12. From the pop-up window select Resource in this account, type the region where the instance is, and the instance ID. This forms the ARN of the instances to monitor.
13. Select Add ARNs.
14. Select Next.
15. Name the policy AllowHibernateEC2InstancePolicy.

Figure 2 – IAM policy to access EC2 instances and CloudWatch logs

Figure 3 – Viewing the IAM policy in JSON format

16. In the navigation page, select Roles.
17. Select Create role.
18. For Trusted entity type, select AWS Service.
19. For Use case, select Lambda.
20. Select Next.
21. In the Permissions policies list, search and select Allow HibernateEC2InstancePolicy.
22. Select Next.
23. Name the role AllowHibernateEC2InstanceFromLambdaRole.
24. Select Create role.

Figure 4 – IAM role implementing the IAM policy

b. Create the Lambda function

Create a Lambda function that will find the ID of the idle instance using the event data from the CloudWatch alarm to hibernate it. The event data will be in a function parameter.

The event data is in the JSON format. The following is an example of what this data looks like.

{
	"version": "0",
	"id": "77b0f9cf-ebe3-3893-f60e-1950d2b8ef26",
	"detail-type": "CloudWatch Alarm State Change",
	"source": "aws.cloudwatch",
	"account": "<account>",
	"time": "2023-08-10T21:27:58Z",
	"region": "us-east-1",
	"resources": [
		"arn:aws:cloudwatch:<region>:<account>:alarm:alarm-name"
	],
	"detail": {
		"alarmName": "alarm-name",
		"state": {
			"value": "ALARM",
			"reason": "TEST",
			"timestamp": "2023-07-05T21:27:58.659+0000"
		},
		"previousState": {
			"value": "OK",
			"reason": "Unchecked: Initial alarm creation",
			"timestamp": "2023-07-05T21:13:51.658+0000"
		},
		"configuration": {
			"metrics": [
				{
					"id": "26c493f3-c295-4454-ff19-70ce482dca64",
					"metricStat": {
						"metric": {
							"namespace": "AWS/EC2",
							"name": "CPUUtilization",
							"dimensions": {
								"InstanceId": "<instance id>"
							}
						},
						"period": 300,
						"stat": "Average"
					},
					"returnData": true
				}
			],
			"description": "Created from EC2 Console"
		}
	}
}

Follow these steps to create the Lambda function.

1. Open the Functions page of the Lambda console.
2. Choose Create function.
3. Select Author from scratch.
4. Name the function HibernateEC2InstanceFunction.
5. For the Runtime, select Python 3.10 (or the latest Python version).
6. For Architecture, choose arm64.
7. Expand Change default execution role and select Use an existing role.
8. Select AllowHibernateEC2InstanceFromLambdaRole from the list of existing roles.
9. Select Create function at the bottom of the page.

In the Lambda function page, scroll down to view the Code tab at the bottom. Copy the following code onto the editor for the lambda_function.py file.

import boto3

def lambda_handler(event, context):
    instancesToHibernate = []
    region = getRegion(event)
    ec2Client = boto3.client('ec2', region_name=region)
    id = getInstanceId(event)

    if id is not None:
        instancesToHibernate.append(id)
        ec2Client.stop_instances(InstanceIds=instancesToHibernate, Hibernate=True)
        print('stopped instances: ' + str(instancesToHibernate) + ' in region ' + region)
    else:
        print('No instance id found')

def getRegion(payload):
    if 'region' in payload:
        region = payload['region']
        return region 
    
    #default to N. Virginia
    return 'us-east-1'

def getInstanceId(payload):
    if 'detail' in payload:
        detail = payload['detail']
        if 'configuration' in detail:
            configuration = detail['configuration']
            if 'metrics' in configuration:
                if len(configuration['metrics']) > 0:
                    firstMetric = configuration['metrics'][0] 
                    if 'metricStat' in firstMetric:
                        metricStat = firstMetric['metricStat']
                        if 'metric' in metricStat:
                            metric = metricStat['metric']
                            if 'dimensions' in metric:
                                dimensions = metric['dimensions']
                                if 'InstanceId' in dimensions:
                                    id = dimensions['InstanceId']
                                    return id
    
    return None

Figure 5 – Lambda function code editor

The code has the following contents:

1. Imports section. In this section, import the libraries that the function uses. In our case, the boto3
2. The main method, called lambda_handler, is the execution entry point. This is the method called whenever the Lambda function runs.
   1. It defines an array to store the IDs of the instances that enter hibernation. This is necessary because the method stop_instances expects an array as opposed to a single value.
   2. Using the event data, it finds the AWS Region and instance ID of the instance to hibernate.
   3. It initializes the Amazon EC2 client by calling the client method.
   4. If it finds an instance ID, then it adds it to the instances array.
   5. Calls stop_instances passing as parameters the instances array and True to indicate the hibernation operation.

c. Configure the EC2 instance to send metrics to CloudWatch

In the scenario, an idle EC2 instance has its CPU utilization under 10% during a 15-minute period. Adjust the utilization percentage and/or period to meet your needs. To enable alarm tracking, the EC2 instance must send the CPU Usage metric to CloudWatch.

1. Open the Amazon EC2 console.
2. In the navigation pane, choose Instances.
3. Select an instance to monitor with the checkbox on the left.
4. Find the Alarm status column, and select the plus sign to add a new alarm.

Figure 6 – Creating a new CloudWatch alarm from the EC2 console

5. In the Manage CloudWatch alarms page, select Create an alarm. Then, turn off Alarm action. Use Alarm notification to notify when hibernating an instance, otherwise, turn off.

Figure 7 – CloudWatch alarm notification and action settings

6. In the Alarm thresholds section, select:
   1. Group samples by Average.
   2. Type of data to sample CPU utilization.
   3. Alarm when less than (<).
   4. Percent 10.
   5. Consecutive periods 1.
   6. Period 15 Minutes.
   7. Alarm name Idle-EC2-Instance-LessThan10Pct-CPUUtilization-15Min.

Figure 8 – CloudWatch alarm thresholds

7. Select Create at the bottom of the page.
8. A successful creation shows a green banner at the top of the page.
9. Select the Alarm status column for the instance, then select the link that shows in the pop-up window to go to the new CloudWatch alarm details.

Figure 9 – Accessing the CloudWatch alarm from the EC2 console

10. Scroll down to view the alarm details and copy its ARN, which shows in the lower right corner. The EventBridge rule needs this.

Figure 10 – Finding the CloudWatch alarm ARN

d. Configure EventBridge to consume events from CloudWatch

When the alarm enters the ALARM state, it means it has detected an idle EC2 instance. It will then generate an event that EventBridge can consume and act upon. For this, EventBridge uses rules. EventBridge rules rely on patterns to identify the events and trigger the appropriate actions.

1. Open the Amazon EventBridge console.
2. In the navigation pane, choose Rules.
3. Choose Create rule.
4. Enter a name and description for the rule. A rule cannot have the same name as another rule in the same Region and on the same event bus.
5. For Event bus, choose an event bus to associate with this rule. To match events that come from the same account, select AWS default event bus. When an AWS service in the account emits an event, it always goes to the account’s default event bus.
6. For Rule type, choose Rule with an event pattern.
7. Select Next.
8. For Event source, choose AWS services.
9. Scroll down to Creation method and select Custom pattern (JSON editor).
10. Enter the following pattern on the Event Pattern

{
  "source": ["aws.cloudwatch"],
  "detail-type": ["CloudWatch Alarm State Change"],
  "detail": {
    "state": {
      "value": ["ALARM"]
    },
    "resources":[
    "<ARN of CW alarms to respond to>"
    ]
  }
}

11. In the resources element of the pattern, add the ARN of the CloudWatch alarm created for the EC2 instance. The resources element is an array. Add the ARN of every alarm to which this rule monitors and responds. Doing this allows using a single rule to handle the same action for multiple alarms.
12. Select Next.
13. Select a target. This is the action that EventBridge executes once it has identified an event. Choose AWS service and select Lambda function.
14. Select HibernateEC2InstanceFunction.
15. Select Next.
16. Add tags to the rule as needed.
17. Select Next.
18. Review the rule configuration, and select Create rule.

Figure 11 – EventBridge rule event pattern

Figure 12 – EventBridge rule targets

Testing the implementation

To test the solution, wait for the instance’s CPU utilization to fall below the 10% threshold for 15 minutes. Alternatively, force the alarm to enter the ALARM state with the following AWS CLI command.

aws cloudwatch set-alarm-state --alarm-name
"Idle-EC2-Instance-LessThan10Pct-CPUUtilization-15Min"
--state-value ALARM --state-reason "testing"

Conclusion

Hibernating EC2 instances brings savings during periods of low utilization. Another benefit is that when they start again, they continue their work from where they left off. To hibernate the instance, set the hibernation configuration when launching it. Detect the idle instance with a CloudWatch alarm, and use EventBridge to capture the alarms and trigger a Lambda function to call the Amazon EC2 stop API with the hibernate parameter.

To learn more

• Amazon EC2 User Guide for Linux Instances
• Amazon EC2 User Guide for Windows Instances
• Hibernate your On-Demand Linux instance
• Hibernate your On-Demand Windows instance
• AWS Lambda Developer Guide
• Amazon CloudWatch User Guide
• Amazon EventBridge User Guide
• AWS Identity and Access Management (IAM) User Guide

Post Syndicated from Григор original http://www.gatchev.info/blog/?p=2615

Мисля си – хич не се разбираме напоследък за лявото и дясното. Кое диктаторско, кое не. Кое свободно, кое не. Кое умно, кое не. Кое свястно, кое не… Откъде дойде тая бъркотия?

Моите 5 ст. – от технологично-икономическия прогрес.

Политическите измамници с мераци да станат диктатори винаги са били популисти. До степен всеки друг да изглежда на техен фон не-популист, така че това си е само тяхна категория.

Но и така имат избор – леви или десни? Е, поне е лесен. Където има повече наивни и тъпи хора, които по-лесно може да бъдат излъгани.

От зората на изборната демокрация – да кажем, 1800-те – та докъм средата, че и края на втората трета на 20 век, това място беше лявото. Там бяха бедните. Които не бяха имали как да се образоват, бяха видиотени от мизерия, притиснати дотам да вярват на евтини лъжи – накратко, електоратът на измамника кандидат-диктатор. Десните през тези времена бяха интелигентните, образованите, културните, мислещите, свободните да избират разумно.

Затова и през тези времена политическите измамници носеха „леви“ маски. И диктатури разцъфваха където идваше на власт тяхното „ляво“. И овладените от тях държави – Русия/СССР, Китай, Северна Корея… – също носеха „леви“ маски. Е, под маските установяваха феодално-робовладелски общества…

С напредъка на технологично-икономическия прогрес обаче нещата се промениха. Бедните получиха достатъчно възможност да се образоват, да не са видиотени от мизерия и готови да вярват на идиотски лъжи. И не всички, но много от тях престанаха. Започнаха да се опитват да мислят и избират разумно – не винаги успешно, но принципно с желание и донякъде с потенциал.

Десните пък не можаха да понесат натиска на ускоряващия се прогрес – те по начало по не носят на промени. Започнаха да си слагат дебели капаци на очите, за да се отградят от него колкото могат. Но това ги отгради и от възможностите да се образоват и да са актуални. А и масово не знаеха, че в комплект с капаците вървят и седло, юзда и камшик.

И политическите измамници с диктаторските мераци се преместиха при тях. Нахлупиха „десни“ маски, както преди нахлупваха „леви“. (Много от тях са съвсем буквално бивши „леви“ – леви тогава колкото сега са десни, естествено. Примерно Тръмп е бивш регистриран поддръжник на Демократическата партия.) Подпомогнати от славата на разумно и мъдро, която си беше създало дясното на фона на олайненото от тях ляво…

Обективно тези хора не са нито леви, нито десни – те са измамници с амбиции. Ще ви лъжат с на каквото вярвате, няма да ги е гнус или да им пречи. Да, обективно изгражданата от тях система е по-близка до дясна, отколкото до лява – феодализмът неизбежно е по-десен и консервативен и от най-екстремния съвременен капитализъм. С твърде много, за да си струва да се мисли дали е по-десен или по-ляв. Той просто е добър само за феодалите.

(И само ако те са психопати. Нормалният човек не изпитва радост, че той има какво да яде, но хиляди нямат. Нито че е успял да ликвидира съседа си и да му вземе къщата или феода. Нито пък му се нрави да живее живот, в който няма да оцелее и месец без дузина гвардейци, дето да го пазят и в тоалетната, и без човек, който да опитва храната му преди него, да не е отровена. Защото феодализмът е така. Каквато музиката, такова хорото.)

Та, пазете се от тия измамници. Включително и особено ако ви се пробутват за „свои“. Съседът ви на другия край на идеологическия спектър обикновено няма да ви наклевети, за да ви прати в затвора и да ви вземе дома. Тези „свои“ ще ви вземат не само дома – всичко. Докато ви гледат в очите и усмихнато ви обясняват колко мъдри сте, щом ги подкрепяте, и колко много печелите така…

Това е от мен.