Tag Archives: Uncategorized

AWS Security Profiles: Ben Potter, Security Lead, Well-Architected

Post Syndicated from Becca Crockett original https://aws.amazon.com/blogs/security/aws-security-profiles-ben-potter-security-lead-well-architected/

Amazon Spheres with author info

In the weeks leading up to re:Invent, we’ll share conversations we’ve had with people at AWS who will be presenting at the event so you can learn more about them and some of the interesting work that they’re doing.


How long have you been at AWS, and what do you do in your current role?

I’ve been with AWS for four and a half years. I started as a one of the first mid-market territory Solution Architects in Sydney, then I moved to professional services doing security, risk, and compliance. For the last year, I’ve been the security lead for Well-Architected, which is a global role.

What is Well-Architected?

It’s a framework that contains best practices, allowing you to measure your architecture and implement continuous improvements against those measurements. It’s designed to help your architecture evolve in alignment with five pillars: Operational Excellence, Security, Reliability, Performance Efficiency, and Cost Optimization. The framework is based on customer data that we’ve gathered, and learnings that our customers have shared. We want to share these learnings with everyone else.

How do you explain your job to non-tech friends?

Basically, I listen to customers a lot. I work with specialists and service teams around the world to help create security best practices for AWS that drive the Well-Architected framework. My work helps customers make better cloud security decisions.

What are you currently working on that you’re excited about?

I’ve been developing some in-depth, hands-on training material for Well-Architected, which you can find on GitHub. It’s all open-source, and the community is welcome to contribute. We’re just getting started with this sort of hands-on content, but we’ve run AWS-led sessions around the globe using this particular content, including at our AWS Security Lofts throughout the USA — plus Sydney, London, and Singapore — and we’ve gotten very positive feedback.

What’s the most challenging part of your job?

Everyone has different priorities and opinions on security. What a Singapore financial startup thinks is a priority is completely different from what an established bank in London thinks — which is completely different from the entertainment industry. The priorities of startups often center around short time-to-market and low cost, with less focus on security.

I’m trying to make it easy for everyone to be what we call Well-Architected in security from the start, so that the only way to do something is via automated, repeatable, secure mechanisms. AWS is great at providing building blocks, but if we can combine those building blocks into different solution sets and guidance, then we can help every customer be Well-Architected from the beginning. Most of the time, it doesn’t cost anything additional. People like me just need to spend the time developing examples, solutions, and labs, and getting them out there.

What does cloud security mean to you, personally?

Cloud security is an opportunity to rethink cybersecurity — to rethink the boundaries of what’s possible. It’s not just a security guard in front of a data center, with a big, old-fashioned firewall protecting the network. It’s a lot deeper than that. The cloud lets you influence security at every layer, from developers all the way to end users. Everyone needs to be thinking about it. I had a big presentation earlier this year, and I asked the audience, “Put your hand up if you’re responsible for your organization’s security.” Only about a quarter of the audience put their hands up. But that’s not true — it’s everyone’s responsibility. The cloud provides opportunities for businesses to innovate, improve their agility and ability to drive business value, but security needs to go hand-in-hand with all of that.

What’s the biggest issue that you see customers struggling with when it comes to cloud security?

A lot of customers don’t think about the need for incident response. They think: I don’t want to think about it. It’s never gonna happen to me. No, my access keys will never be lost. It’s fine. We’ve got processes in place, and our developers know what they’re doing. We’re never gonna lose any access keys or credentials. But it happens, people make mistakes. And it’s very important for anyone, regardless of whether or not they’re in the cloud, to be prepared for an incident, by investing in the tools that they need, by actually practicing responding to an incident, and by having run books. If X does happen, then where do I start? What do I need to do? Who do I need to communicate with? AWS can help with that, but it’s all very reactive. Incident response needs to be proactive because your organization’s reputation and business could be on the line.

In your opinion, what’s the biggest challenge facing the cloud security industry right now?

I think the biggest challenge is just staying up to date with what’s happening in the industry. Any company that develops software or tools or services is going to have a predefined plan of work. But often, security is forgotten about in that development process. Say you’re developing a mobile game: you’d probably have daily agile-style stand-ups, and you’d develop the game until you’ve got a minimum viable product. Then you’d put it out there for testing. But what if the underlying software libraries that you used to develop the game had vulnerabilities in them, and you didn’t realize this because you didn’t build in a process for hourly or daily checking of vulnerabilities in the external libraries you pulled in?

Keeping up-to-date is always a challenge, and this is where the cloud actually has a lot of power, because the cloud can drive the automated infrastructure combined with the actual code. It’s part of the whole dev ops thing — combining infrastructure code with the actual application code. You can take it all and run automated tools across it to verify your security posture and provide more granular control. In the old days, nearly everyone had keys to the data center to go in and reboot stuff. Now, you can isolate different application teams to different portions of their cloud environment. If something bad does happen, it’s much easier to contain the issue through the segmentation and micro-segmentation of services.

Five years from now, what changes do you think we’ll see across the security landscape?

I think we’re going to see a lot of change for the better. If you look at ransomware statistics that McAfee has published, new infection rates have actually gone down. More people are becoming aware of security, including end users and the general public. Cyber criminals go where the money is. This means organizations are under increasing pressure to do the right thing in terms of public safety and security.

For ransomware specifically, there’s also nomoreransom.org, a global project for which I was the “Chief Architect” — I worked with Europol, McAfee, and Kaspersky to create this website. It’s been around for a couple years now, and I think it’s already helping drive awareness of security and best practices for the public, like, don’t click on this phishing email. I co-presented a re:Invent presentation on this project few years ago, if you want more info about it.

Tell us about the chalk talk you’re giving at re:Invent this year.

The Well-Architected for Security chalk talk is meant to help customers get started by helping them identify which best practices they should follow. It’s an open QA. I’ll start by giving an overview of the Well-Architected framework, some best practices, and some design principles, and then I’ll do a live Q&A with whiteboarding. It’ll be really interactive. I like to question the audience about what they think their challenges are. Last year, I ran a session on advanced web application security that was really awesome because I actually got a lot of feedback, and I had some service team members in the room who were also able to use a lot of feedback from that session. So it’s not just about sharing, it’s also listening to customers’ challenges, which helps drive our content road map on what we need to do for customer enablement in the coming months.

Your second re:Invent session, the Security Framework Shakedown, says it will walk you through a complete security journey. What does that mean?

This session that Steve Laino and I are delivering is about where you should start in terms of design: How to know you’re designing a secure architecture, and how the Cloud Adoption and Well-Architected frameworks can help. As your company evolves, you’re going to have priorities, and you can’t do everything right the first time. So you’ll need to think about what your priorities are and create your own roadmap for an evolving architecture that becomes continually more secure. We’ve got National Australia Bank co-presenting with us. They’ll share their journey, including how they used the Cloud Adoption Framework to get started, and how they use Well-Architected daily to drive improvement across their platform.

Broadly, what are you hoping that your audience will take away from your sessions? What do you want them to do differently?

I want people to start prioritizing security in their day-to-day job roles. That prioritization means asking questions like, “What are some principles that I should include in my day to day work life? Are we using tools and automation to make security effective?” And if you’re not using automation and tools, then what’s out there that you can start using?

Any tips for first-time conference attendees?

Get out there and socialize. Talk to your peers, and try to find some mentors in the community. You’ll find that many people in the industry, both in AWS and among our customers and partners, are very willing to help you on a personal basis to develop your career.

Any tips for returning attendees?

Think about your goals, and go after that. You should be willing to give your honest feedback, too, and seek out service team members and individuals that have influenced you in the past.

You’re from Adelaide. If somebody is visiting your hometown, what would you advise them to do?

The “Mad March” festivities should not be missed. If you like red wine, you should visit the wine regions of Barossa Valley or McLaren Vale — or both. My favorite is definitely Barossa Valley.

The AWS Security team is hiring! Want to find out more? Check out our career page.

Want more AWS Security news? Follow us on Twitter.

Author

Ben Potter

Ben is the global security leader for the AWS Well-Architected Framework and is responsible for sharing best practices in security with customers and partners. Ben is also an ambassador for the No More Ransom initiative helping fight cyber crime with Europol, McAfee and law enforcement across the globe.

Trick or (the ultimate) treat!

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/trick-or-treat/

I’ll keep today’s blog post short and sweet, because Liz, Helen, and I are all still under the weather.

Raspberry Pi 4!

Don’t tell Eben, Liz, or the rest of the team I showed you this, but here’s your Halloween ‘trick or treat’ gift: an exclusive sneak peek at the Raspberry Pi 4.

We’ll be back to our regularly scheduled programming from tomorrow.

The post Trick or (the ultimate) treat! appeared first on Raspberry Pi.

We have the plague

Post Syndicated from Liz Upton original https://www.raspberrypi.org/blog/we-have-the-plague/

Apologies to our daily visitors (we love you guys); we don’t have a proper blog post for you today because we’re all really ill. (I have food poisoning, Helen is coughing up goo and can barely speak or breathe, and Alex is being sick.)

You’ve got a day until Halloween; if you’re looking for inspiration, we’ve got several years of archived spooky project posts for you to check out. And now, if you’ll excuse me, I’m going to go and have a little lie down.

The post We have the plague appeared first on Raspberry Pi.

Първи годишен доклад на британския регулатор за БиБиСи

Post Syndicated from nellyo original https://nellyo.wordpress.com/2018/10/27/ofcom_bbc-2/

Публикуван  е първият годишен доклад на  британския регулатор за обществената медия БиБиСи за периода от април 2017 г. до март 2018 г. Всяка година Ofcom ще публикува доклад, който  оценява  съответствието на Би Би Си с изискванията на закона, лицензията и свързаните с нея документи. В този доклад регулаторът предоставя оценка за това как БиБиСи се представя на фона на бързо променящия се медиен пейзаж,  за изпълнението на  мисията и обществените цели на БиБиСи.  Според първия доклад Би Би Си като цяло изпълнява своята мисия, заедно с това регулаторът посочва  четири области, в които трябват още усилия. 

Този доклад показва една обществена медия от друг свят.

Тук с обществената медия сме  така: ето така – за което СЕМ наложил санкция от 3000 лева – взети откъде? – от бюджета, много ясно.

Среден пръст, за който не ни обезщетяват  – а плащаме, когато го видим.

kushi

 

*

Оценката на БиБиСи:

 Letter to the BBC 

Ofcom’s Annual Report on the BBC (PDF, 2.3 MB)

Annex 1: Compliance with regulatory requirements (PDF, 231.7 KB)

Annex 2: BBC Performance Report (PDF, 1.1 MB)

Annex 3: Methodology overview (PDF, 267.4 KB)

View an interactive report providing visualisations of key data

Title Summary
BBC Performance Report: data download (CSV, 39.7 KB) Download of the data used in the charts from the BBC Performance Report.
BBC Performance Tracker data tables (PDF, 13.6 MB) and questionnaire (PDF, 405.9 KB) The BBC performance tracker provides Ofcom with an evidence base to assess audience opinions on the BBC’s performance against its delivery of four public purposes. The data tables contain the results from fieldwork period October 2017 to April 2018.
PSB Tracker (PDF, 10.1 MB) The PSB Tracker monitors the satisfaction and attitudes towards public service broadcasting channels.
News Consumption in the UK This report provides the findings of Ofcom’s 2018 research into news consumption across television, radio, print and online. The aim is to inform an understanding of news consumption across the UK and within each UK nation.
Children’s Media Literacy The Children’s Media Literacy Tracker is a face-to-face survey run once a year between April and June. The objective of the survey is to provide detailed evidence on media use and understanding among children and young people aged 5-15, as well as in-depth information about media access and use among children aged 3-4.

 

Deploying a Burstable and Event-driven HPC Cluster on AWS Using SLURM, Part 2

Post Syndicated from Geoff Murase original https://aws.amazon.com/blogs/compute/deploy-a-burstable-and-event-driven-hpc-cluster-on-aws-using-slurm-part-2/

Contributed by Amr Ragab, HPC Application Consultant, AWS Professional Services

In part 1 of this series, you deployed the base components to create the HPC cluster. This unique deployment stands up the SLURM headnode. For every job submitted to the queue, the headnode provisions the needed compute resources to run the job, based on job submission parameters.

By provisioning the compute nodes dynamically, you can immediately see the benefit of elasticity, scale, and optimized operational compute costs. As new technologies are released, you can take advantage of heterogeneous deployments, such as scaling high, tightly coupled, CPU-bound workloads independently from high memory or distributed GPU-based workloads.

To further extend a cloud-native approach to designing HPC architectures, you can integrate with existing AWS services and provide additional benefits by abstracting the underlying compute resources. It is possible for the HPC cluster to be event-driven in response to requests from a web application or from direct API calls.

Additional frontend components can be added to take advantage of an API-instantiated execution of an HPC workload. The following reference architecture describes the pattern.

 

The difference from the previous reference architecture in Part 1 is that the user submits the job described as JSON through an HTTP call to Amazon API Gateway, which is then processed by an AWS Lambda function to submit the job.

Deployment

I recommend that you start this section after completing the deployment in Part I . Write down the private IP address of the SLURM controller.

In the Amazon EC2 console, select the SLURM headnode and retrieve the private IPv4 address. In the Lambda console, create a new function based on Python 2.7 authored from scratch.

Under the environment variables, add a new entry for “HEADNODE”, “SLURM_BUCKET_S3”, “SLURM_KEY_S3” and set the value to the private IPv4 address of the SLURM controller noted earlier, plus the bucket and key pair. This allows the Lambda function to connect to the instance using SSH.

In the AWS GitHub repo that you cloned in part 1, find the lambda/hpc_worker.zip file and upload the contents to the Function Code section of the Lambda function. A derivative of this function was referenced by Puneet Agarwal, in the Scheduling SSH jobs using AWS Lambda post.

The Lambda function needs to launch in the VPC as the SLURM node and have the same security groups as the SLURM headnode. This is because the Lambda function connects to the SLURM controller using SSH. Ignore the error about creating the Lambda function across two Availability Zones for high availability (HA).

The default memory settings, with a timeout of 20 seconds, are sufficient. The Lambda execution role needs access to Amazon EC2, Amazon CloudWatch, and Amazon S3.

In the API Gateway console, create a new API from scratch and name it “hpc.” Under Resources, create a new resource as “hpc.” Then, create a new method under the “hpc” resource for POST.

Under the POST method, set the integration method to the Lambda function created earlier.

Under the resource “hpc”, choose to deploy the API for staging, calling the endpoint “dev.” You get an endpoint to execute:

curl -H "Content-Type: application/json" -X POST https://<endpoint>.execute-api.us-west-2.amazonaws.com/dev/hpc -d @test.json

Then, create a JSON file with the following code.

{
    "username": "awsuser", 
    "jobname": "hpc_test", 
    "nodes": 2, 
    "tasks-per-node": 1, 
    "cpus-per-task": 4, 
    "feature": "us-west-2a|us-west-2b|us-west-2c", 
    "io": 
        [{"workdir": "/home/centos/job123"},
         {"input": "s3://ar-job-input/test.input"},
         {"output": "s3://ar-job-output"}],
    "launch": "env && sleep 60"
}

Next, in the API Gateway console, watch the following four events happen:

  1. The API gateway passes the input JSON to the Lambda function.
  2. The Lambda function writes out a SLURM sbatch job submission file.
  3. The job is executed and held until the instance is provisioned
  4. After the instance is running, the job script executes, copies data from S3, and completes the job.

In the response body of the API call, you return the job ID.

{
"body": "{\"error\": \"\", \"name\": \"awsuser\", \"jobid\": \"Submitted batch job 5\\n\"}",
"statusCode": 200
}

When the job completes, the instance is held for 60 seconds in case another job is submitted. If no jobs are submitted, the instance is terminated by the SLURM cluster.

Conclusion

End-to-end scalable job submission and instance provisioning is one way to execute your HPC workloads in a scalable and elastic fashion. Now, go power your HPC workloads on AWS!

Automating rollback of failed Amazon ECS deployments

Post Syndicated from Anuneet Kumar original https://aws.amazon.com/blogs/compute/automating-rollback-of-failed-amazon-ecs-deployments/

Contributed by Vinay Nadig, Associate Solutions Architect, AWS.

With more and more organizations moving toward Agile development, it’s not uncommon to deploy code to production multiple times a day. With the increased speed of deployments, it’s imperative to have a mechanism in place where you can detect errors and roll back problematic deployments early. In this blog post, we look at a solution that automates the process of monitoring Amazon Elastic Container Service (Amazon ECS) deployments and rolling back the deployment if the container health checks fail repeatedly.

The normal flow for a service deployment on Amazon ECS is to create a new task definition revision and update an Amazon ECS service with the new task definition. Based on the values of minimumHealthyPercent and maximumHealthyPercent, Amazon ECS replaces existing containers in batches to complete the deployment. After the deployment is complete, you typically monitor the service health for errors and make a call on rolling back the deployment.

In March 2018, AWS announced support for native Docker health checks on Amazon ECS. Amazon ECS also supports Application Load Balancer health checks for services that are integrated with a load balancer. Leveraging these two features, we can build a solution that automatically rolls back Amazon ECS deployments if health checks fail.

Solution overview

The solution consists of the following components:

·      An Amazon CloudWatch event to listen for the UpdateService event of an Amazon ECS cluster

·      An AWS Lambda function that listens for the Amazon ECS events generated from the cluster after the service update

·      A Lambda function that calculates the failure percentage based on the events in the Amazon ECS event stream

·      A Lambda function that triggers rollback of the deployment if there are high error rates in the event

·      An AWS Step Functions state machine to orchestrate the entire flow

 

The following diagram shows the solution’s components and workflow.

Assumptions

The following assumptions are important to understand before you implement the solution:

·      The solution assumes that with every revision of the task definition, you use a new Docker tag instead of using the default “latest” tag. As a best practice, we advise that you do every release with a different Docker image tag and a revision of the task definition.

·      If there are continuous healthcheck failures even after the deployment is automatically rolled back using this setup, another rollback is triggered due to the health check failures. This might introduce a runaway deployment rollback loop. Make sure that you use the solution where you know that a one-step rollback will bring the Amazon ECS service into a stable state.

·      This blog post assumes deployment to the US West (Oregon) us-west-2 Region. If you want to deploy the solution to other Regions, you need to make minor modifications to the Lambda code.

·      The Amazon ECS cluster launches in a new VPC. Make sure that your VPC service limit allows for a new VPC.

Prerequisites

You need the following permissions in AWS Identity and Access Management (IAM) to implement the solution:

·      Create IAM Roles

·      Create ECS Cluster

·      Create CloudWatch Rule

·      Create Lambda Functions

·      Create Step Functions

Creating the Amazon ECS cluster

First, we create an Amazon ECS cluster using the AWS Management Console.

1. Sign in to the AWS Management Console and open the Amazon ECS console.

2. For Step 1: Select cluster template, choose EC2 Linux + Networking and then choose Next step.

3. For Step 2: Configure cluster, under Configure cluster, enter the Amazon ECS cluster name as AutoRollbackTestCluster.

 

4. Under Instance configuration, for EC2 instance type, choose t2.micro.

5. Keep the default values for the rest of the settings and choose Create.

 

This provisions an Amazon ECS cluster with a single Amazon ECS container instance.

Creating the task definition

Next, we create a new task definition using the Nginx Alpine image.

1. On the Amazon ECS console, choose Task Definitions in the navigation pane and then choose Create new Task Definition.

2. For Step 1: Select launch type compatibility, choose EC2 and then choose Next step.

3. For Task Definition Name, enter Web-Service-Definition.

4. Under Task size, under Container Definitions, choose Add Container.

5.  On the Add container pane, under Standard, enter Web-Service-Container for Container name.

6.  For Image, enter nginx:alpine. This pulls the nginx:alpine Docker image from Docker Hub.

7.  For Memory Limits (MiB), choose Hard limit and enter 128.

8.  Under Advanced container configuration, enter the following information for Healthcheck:

 

·      Command:

CMD-SHELL, wget http://localhost/ && rm index.html || exit 1

·      Interval: 10

·      Timeout: 30

·      Start period: 10

·      Retries: 2

9.  Keep the default values for the rest of the settings on this pane and choose Add.

10. Choose Create.

Creating the Amazon ECS service

Next, we create an Amazon ECS service that uses this task definition.

1.  On the Amazon ECS console, choose Clusters in the navigation pane and then choose AutoRollbackTestCluster.

2.  On the Services view, choose Create.

3.  For Step 1: Configure service, use the following settings:

·      Launch type: EC2.

·      Task Definition Family: Web-Service-Definition. This automatically selects the latest revision of the task definition.

·      Cluster: AutoRollbackTestCluster.

·      Service name: Nginx-Web-Service.

·      Number of tasks: 3.

4.  Keep the default values for the rest of the settings and choose Next Step.

5.  For Step 2: Configure network, keep the default value for Load balancer type and choose Next Step.

6. For Step 3: Set Auto Scaling (optional), keep the default value for Service Auto Scaling and choose Next Step.

7. For Step 4: Review, review the settings and choose Create Service.

After creating the service, you should have three tasks running in the cluster. You can verify this on the Tasks view in the service, as shown in the following image.

Implementing the solution

With the Amazon ECS cluster set up, we can move on to implementing the solution.

Creating the IAM role

First, we create an IAM role for reading the event stream of the Amazon ECS service and rolling back any faulty deployments.

 

1.  Open the IAM console and choose Policies in the navigation pane.

2.  Choose Create policy.

3.  On the Visual editor view, for Service, choose EC2 Container Service.

4.  For Actions, under Access Level, select DescribeServices for Read and UpdateServices for Write.

5.  Choose Review policy.

6.  For Name, enter ECSRollbackPolicy.

7.  For Description, enter an appropriate description.

8.  Choose Create policy.

Creating the Lambda service role

Next, we create a Lambda service role that uses the previously created IAM policy. The Lambda function to roll back faulty deployments uses this role.

 

1.  On the IAM console, choose Roles in the navigation pane and then choose Create role.

2.  For the type of trusted entity, choose AWS service.

3.  For the service that will use this role, choose Lambda.

4.  Choose Next: Permissions.

5.  Under Attach permissions policies, select the ECSRollbackPolicy policy that you created.

6. Choose Next: Review.

7.  For Role name, enter ECSRollbackLambdaRole and choose Create role.

Creating the Lambda function for the Step Functions workflow and Amazon ECS event stream

The next step is to create the Lambda function that will collect Amazon ECS events from the Amazon ECS event stream. This Lambda function will be part of the Step Functions state machine.

 

1.  Open the Lambda console and choose Create function.

2.  For Name, enter ECSEventCollector.

3.  For Runtime, choose Python 3.6.

4.  For Existing role, choose the ECSRollbackLambdaRole IAM role that you created.

5. Choose Create function.

6.  On the Configuration view, under Function code, enter the following code.

import time
import boto3
from datetime import datetime

ecs = boto3.client('ecs', region_name='us-west-2')


def lambda_handler(event, context):
    service_name = event['detail']['requestParameters']['service']
    cluster_name = event['detail']['requestParameters']['cluster']
    _update_time = event['detail']['eventTime']
    _update_time = datetime.strptime(_update_time, "%Y-%m-%dT%H:%M:%SZ")
    start_time = _update_time.strftime("%s")
    seconds_from_start = time.time() - int(start_time)
    event.update({'seconds_from_start': seconds_from_start})

    _services = ecs.describe_services(
        cluster=cluster_name, services=[service_name])
    service = _services['services'][0]
    service_events = service['events']
    events_since_update = [event for event in service_events if int(
        (event['createdAt']).strftime("%s")) > int(start_time)]
    [event.pop('createdAt') for event in events_since_update]
    event.update({"events": events_since_update})
    return event

 

7. Under Basic Settings, set Timeout to 30 seconds.

 

8.  Choose Save.

Creating the Lambda function to calculate failure percentage

Next, we create a Lambda function that calculates the failure percentage based on the number of failed container health checks derived from the event stream.

 

1.     On the Lambda console, choose Create function.

2.     For Name, enter ECSFailureCalculator.

3.     For Runtime, choose Python 3.6.

4.     For Existing role, choose the ECSRollbackLambdaRole IAM role that you created.

5.     Choose Create function.

6.     On the Configuration view, under Function code, enter the following code.

 

import re

lb_hc_regex = re.compile("\(service (.*)?\) \(instance (i-[a-z0-9]{7,17})\) \(port ([0-9]{4,5})\) is unhealthy in \(target-group (.*)?\) due to \((.*)?: \[(.*)\]\)")
docker_hc_regex = re.compile("\(service (.*)?\) \(task ([a-z0-9]{8}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{12})\) failed container health checks\.")
task_registration_formats = ["\(service (.*)?\) has started ([0-9]{1,9}) tasks: (.*)\."]


def lambda_handler(event, context):
    cluster_name = event['detail']['requestParameters']['cluster']
    service_name = event['detail']['requestParameters']['service']

    messages = [m['message'] for m in event['events']]
    failures = get_failure_messages(messages)
    registrations = get_registration_messages(messages)
    failure_percentage = get_failure_percentage(failures, registrations)
    print("Failure Percentage = {}".format(failure_percentage))
    return {"failure_percentage": failure_percentage, "service_name": service_name, "cluster_name": cluster_name}


def get_failure_percentage(failures, registrations):
    no_of_failures = len(failures)
    no_of_registrations = sum([float(x[0][1]) for x in registrations])
    return no_of_failures / no_of_registrations * 100 if no_of_registrations > 0 else 0


def get_failure_messages(messages):
    failures = []
    for message in messages:
        failures.append(lb_hc_regex.findall(message)) if lb_hc_regex.findall(message) else None
        failures.append(docker_hc_regex.findall(message)) if docker_hc_regex.findall(message) else None
    return failures


def get_registration_messages(messages):
    registrations = []
    for message in messages:
        for registration_format in task_registration_formats:
            if re.findall(registration_format, message):
                registrations.append(re.findall(registration_format, message))
    return registrations

7.     Under Basic Settings, set Timeout to 30 seconds.

8.     Choose Save.

Creating the Lambda function to roll back a deployment

Next, we create a Lambda function to roll back an Amazon ECS deployment.

 

1.     On the Lambda console, choose Create function.

2.     For Name, enter ECSRollbackfunction.

3.     For Runtime, choose Python 3.6.

4.     For Existing role, choose the ECSRollbackLambdaRole IAM role that you created.

5.     Choose Create function.

6.     On the Configuration view, under Function code, enter the following code.

 

import boto3

ecs = boto3.client('ecs', region_name='us-west-2')

def lambda_handler(event, context):
    service_name = event['service_name']
    cluster_name = event['cluster_name']

    _services = ecs.describe_services(cluster=cluster_name, services=[service_name])
    task_definition = _services['services'][0][u'taskDefinition']
    previous_task_definition = get_previous_task_definition(task_definition)

    ecs.update_service(cluster=cluster_name, service=service_name, taskDefinition=previous_task_definition)
    print("Rollback Complete")
    return {"Rollback": True}

def get_previous_task_definition(task_definition):
    previous_version_number = str(int(task_definition.split(':')[-1])-1)
    previous_task_definition = ':'.join(task_definition.split(':')[:-1]) + ':' + previous_version_number
    return previous_task_definition


7.     Under Basic Settings, set Timeout to 30 seconds.

8.     Choose Save.

Creating the Step Functions state machine

Next, we create a Step Functions state machine that performs the following steps:

 

1.     Collect events of a specified service for a specified duration from the event stream of the Amazon ECS cluster.

2.     Calculate the percentage of failures after the deployment.

3.     If the failure percentage is greater than a specified threshold, roll back the service to the previous task definition.

 

To create the state machine:

1.     Open the Step Functions console and choose Create state machine.

2.     For Name, enter ECSAutoRollback.

For IAM role, keep the default selection of Create a role for me and select the check box. This will create a new IAM role with necessary permissions for the execution of the state machine.

Note
If you have already created a Step Functions state machine, IAM Role is populated.

3.     For State machine definition, enter the following code, replacing the Amazon Resource Name (ARN) placeholders with the ARNs of the three Lambda functions that you created.

{
    "StartAt": "VerifyClusterAndService",
    "States":
    {
        "VerifyClusterAndService":
        {
            "Type": "Choice",
            "Choices": [
            {
                "And": [
                {
                    "Variable": "$.detail.requestParameters.cluster",
                    "StringEquals": "AutoRollbackTestCluster"
                },
                {
                    "Variable": "$.detail.requestParameters.service",
                    "StringEquals": "Nginx-Web-Service"
                }],
                "Next": "GetTasksStatus"
            },
            {
                "Not":
                {
                    "And": [
                    {
                        "Variable": "$.detail.requestParameters.cluster",
                        "StringEquals": "AutoRollbackTestCluster"
                    },
                    {
                        "Variable": "$.detail.requestParameters.service",
                        "StringEquals": "Nginx-Web-Service"
                    }]
                },
                "Next": "EndState"
            }]
        },
        "GetTasksStatus":
        {
            "Type": "Task",
            "Resource": "<ARN-of-ECSEventCollector-Lambda-Function>",
            "Next": "WaitForInterval"
        },
        "WaitForInterval":
        {
            "Type": "Wait",
            "Seconds": 5,
            "Next": "IntervalCheck"
        },
        "IntervalCheck":
        {
            "Type": "Choice",
            "Choices": [
            {
                "Variable": "$.seconds_from_start",
                "NumericGreaterThan": 300,
                "Next": "FailureCalculator"
            },
            {
                "Variable": "$.seconds_from_start",
                "NumericLessThan": 300,
                "Next": "GetTasksStatus"
            }]
        },
        "FailureCalculator":
        {
            "Type": "Task",
            "Resource": "<ARN-of-ECSFailureCalculator-Lambda-Function-here>",
            "Next": "RollbackDecider"
        },
        "RollbackDecider":
        {
            "Type": "Choice",
            "Choices": [
            {
                "Variable": "$.failure_percentage",
                "NumericGreaterThan": 10,
                "Next": "RollBackDeployment"
            },
            {
                "Variable": "$.failure_percentage",
                "NumericLessThan": 10,
                "Next": "EndState"
            }]
        },
        "RollBackDeployment":
        {
            "Type": "Task",
            "Resource": "<ARN-of-ECSRollbackFunction-Lambda-Function-here>",
            "Next": "EndState"
        },
        "EndState":
        {
            "Type": "Succeed"
        }
    }
}

4.     Choose Create state machine.

 

Now we have a mechanism to roll back a deployment if there are more than a configurable percentage of errors after a deployment to a specific Amazon ECS service.

(Optional) Monitoring and rolling back all services in the Amazon ECS cluster

Step Functions hard-codes the Amazon ECS service name in the state machine so that you monitor only a specific service in the cluster. The following image shows these lines in the state machine’s definition.

If you want to monitor all services and automatically roll back any Amazon ECS deployment in the cluster  based on failures, modify the state machine definition to verify only the cluster name and to not verify the service name. To do this, remove the service name check in the definition, as shown in the following image.

The following code verifies only the cluster name. It monitors any Amazon ECS service and performs a rollback if there are errors.

{
    "StartAt": "VerifyClusterAndService",
    "States":
    {
        "VerifyClusterAndService":
        {
            "Type": "Choice",
            "Choices": [
            {
                "Variable": "$.detail.requestParameters.cluster",
                "StringEquals": "AutoRollbackTestCluster",
                "Next": "GetTasksStatus"
            },
            {
                "Not":
                {
                    "Variable": "$.detail.requestParameters.cluster",
                    "StringEquals": "AutoRollbackTestCluster"
                },
                "Next": "EndState"
            }]
        },
        "GetTasksStatus":
        {
            "Type": "Task",
            "Resource": "<ARN-of-ECSEventCollector-Lambda-Function>",
            "Next": "WaitForInterval"
        },
        "WaitForInterval":
        {
            "Type": "Wait",
            "Seconds": 5,
            "Next": "IntervalCheck"
        },
        "IntervalCheck":
        {
            "Type": "Choice",
            "Choices": [
            {
                "Variable": "$.seconds_from_start",
                "NumericGreaterThan": 300,
                "Next": "FailureCalculator"
            },
            {
                "Variable": "$.seconds_from_start",
                "NumericLessThan": 300,
                "Next": "GetTasksStatus"
            }]
        },
        "FailureCalculator":
        {
            "Type": "Task",
            "Resource": "<ARN-of-ECSFailureCalculator-Lambda-Function-here>",
            "Next": "RollbackDecider"
        },
        "RollbackDecider":
        {
            "Type": "Choice",
            "Choices": [
            {
                "Variable": "$.failure_percentage",
                "NumericGreaterThan": 10,
                "Next": "RollBackDeployment"
            },
            {
                "Variable": "$.failure_percentage",
                "NumericLessThan": 10,
                "Next": "EndState"
            }]
        },
        "RollBackDeployment":
        {
            "Type": "Task",
            "Resource": "<ARN-of-ECSRollbackFunction-Lambda-Function-here>",
            "Next": "EndState"
        },
        "EndState":
        {
            "Type": "Succeed"
        }
    }
}

 

Configuring the state machine to execute automatically upon Amazon ECS deployment

Next, we configure a trigger for the state machine so that its execution automatically starts when there is an Amazon ECS deployment. We use Amazon CloudWatch to configure the trigger.

 

1.     Open the CloudWatch console and choose Rules in the navigation pane.

2.     Choose Create rule and use the following settings:

·      Event Source

o   Service Name: EC2 Container Service (ECS)

o   Event Type: AWS API Call via CloudTrail

o   Operations: choose ‘Specific Operations and enter UpdateService

·      Targets

o   Step Functions state machine

o   State machine: ECSAutoRollback

3.     Choose Configure details.

4.     For Name, enter ECSServiceUpdateRule.

5.     For Description, enter an appropriate description.

6.     For State, make sure that Enabled is selected.

7.     Click Create rule.

 

Setting up the CloudWatch trigger is the last step in linking the Amazon ECS UpdateService events to the Step Functions state machine that we set up. With this step complete, we can move on to testing the solution.

Testing the solution

Let’s update the task definition and force a failure of the container health checks so that we can confirm that the deployment rollback occurs as expected.

 

To test the solution:

 

1.     Open the Amazon ECS console and choose Task Definitions in the navigation pane.

2.     Select the check box next to Web-Service-Definition and choose Create new revision.

3.     Under Container Definitions, choose Web-Service-Container.

4.     On the Edit container pane, under Healthcheck, update Command to

CMD-SHELL, wget http://localhost/does-not-exist.html && rm index.html || exit 1 

and choose Update.

5.     Choose Create. This creates the task definition revision.

6.     Open the Nginx-Web-Service page of the Amazon ECS console and choose Update.

7.     For Task Definition, select the latest revision.

8.    Keep the default values for the rest of the settings by choosing Next Step until you reach Review.

9.     Choose Update Service. This creates a new Amazon ECS deployment.

This service update triggers the CloudWatch rule, which in turn triggers the state machine. The state machine collects the Amazon ECS events for 300 seconds. If the percentage of errors due to health check failures is more than 10%, the deployment is automatically rolled back. You can verify this on the Step Functions console. On the Executions view, you should see a new execution that the deployment is triggering, as shown in the following image.

Choose the execution to see the workflow in progress. After the workflow is complete, you can check the outcome of the workflow by choosing EndState in Visual Workflow. The output should show {“Rollback”: true}.

You can also verify in the service details that the service has been updated with the previous version of the task definition.

Conclusion

With this solution, you can detect issues with Amazon ECS deployments early on and automate failure responses. You can also integrate the solution into your existing systems by triggering an Amazon SNS notification to send email or SMS instead of rolling back the deployment automatically. Though this blog uses Amazon ECS, you can follow similar steps to have automatic rollback for AWS Fargate.

If you want to customize the duration for monitoring your deployments before deciding to rollback and the error percentage threshold beyond which a rollback should be triggered, modify the values highlighted in the following image of the state machine definition.

New – How to better monitor your custom application metrics using Amazon CloudWatch Agent

Post Syndicated from Helen Lin original https://aws.amazon.com/blogs/devops/new-how-to-better-monitor-your-custom-application-metrics-using-amazon-cloudwatch-agent/

This blog was contributed by Zhou Fang, Sr. Software Development Engineer for Amazon CloudWatch and Helen Lin, Sr. Product Manager for Amazon CloudWatch

Amazon CloudWatch collects monitoring and operational data from both your AWS resources and on-premises servers, providing you with a unified view of your infrastructure and application health. By default, CloudWatch automatically collects and stores many of your AWS services’ metrics and enables you to monitor and alert on metrics such as high CPU utilization of your Amazon EC2 instances. With the CloudWatch Agent that launched last year, you can also deploy the agent to collect system metrics and application logs from both your Windows and Linux environments. Using this data collected by CloudWatch, you can build operational dashboards to monitor your service and application health, set high-resolution alarms to alert and take automated actions, and troubleshoot issues using Amazon CloudWatch Logs.

We recently introduced CloudWatch Agent support for collecting custom metrics using StatsD and collectd. It’s important to collect system metrics like available memory, and you might also want to monitor custom application metrics. You can use these custom application metrics, such as request count to understand the traffic going through your application or understand latency so you can be alerted when requests take too long to process. StatsD and collectd are popular, open-source solutions that gather system statistics for a wide variety of applications. By combining the system metrics the agent already collects, with the StatsD protocol for instrumenting your own metrics and collectd’s numerous plugins, you can better monitor, analyze, alert, and troubleshoot the performance of your systems and applications.

Let’s dive into an example that demonstrates how to monitor your applications using the CloudWatch Agent.  I am operating a RESTful service that performs simple text encoding. I want to use CloudWatch to help monitor a few key metrics:

  • How many requests are coming into my service?
  • How many of these requests are unique?
  • What is the typical size of a request?
  • How long does it take to process a job?

These metrics help me understand my application performance and throughput, in addition to setting alarms on critical metrics that could indicate service degradation, such as request latency.

Step 1. Collecting StatsD metrics

My service is running on an EC2 instance, using Amazon Linux AMI 2018.03.0. Make sure to attach the CloudWatchAgentServerPolicy AWS managed policy so that the CloudWatch agent can collect and publish metrics from this instance:

Here is the service structure:

 

The “/encode” handler simply returns the base64 encoded string of an input text.  To monitor key metrics, such as total and unique request count as well as request size and method response time, I used StatsD to define these custom metrics.

@RestController

public class EncodeController {

    @RequestMapping("/encode")
    public String encode(@RequestParam(value = "text") String text) {
        long startTime = System.currentTimeMillis();
        statsd.incrementCounter("totalRequest.count", new String[]{"path:/encode"});
        statsd.recordSetValue("uniqueRequest.count", text, new String[]{"path:/encode"});
        statsd.recordHistogramValue("request.size", text.length(), new String[]{"path:/encode"});
        String encodedString = Base64.getEncoder().encodeToString(text.getBytes());
        statsd.recordExecutionTime("latency", System.currentTimeMillis() - startTime, new String[]{"path:/encode"});
        return encodedString;
    }
}

Note that I need to first choose a StatsD client from here.

The “/status” handler responds with a health check ping.  Here I am monitoring my available JVM memory:

@RestController
public class StatusController {

    @RequestMapping("/status")
    public int status() {
        statsd.recordGaugeValue("memory.free", Runtime.getRuntime().freeMemory(), new String[]{"path:/status"});
        return 0;
    }
}

 

Step 2. Emit custom metrics using collectd (optional)

collectd is another popular, open-source daemon for collecting application metrics. If I want to use the hundreds of available collectd plugins to gather application metrics, I can also use the CloudWatch Agent to publish collectd metrics to CloudWatch for 15-months retention. In practice, I might choose to use either StatsD or collectd to collect custom metrics, or I have the option to use both. All of these use cases  are supported by the CloudWatch agent.

Using the same demo RESTful service, I’ll show you how to monitor my service health using the collectd cURL plugin, which passes the collectd metrics to CloudWatch Agent via the network plugin.

For my RESTful service, the “/status” handler returns HTTP code 200 to signify that it’s up and running. This is important to monitor the health of my service and trigger an alert when the application does not respond with a HTTP 200 success code. Additionally, I want to monitor the lapsed time for each health check request.

To collect these metrics using collectd, I have a collectd daemon installed on the EC2 instance, running version 5.8.0. Here is my collectd config:

LoadPlugin logfile
LoadPlugin curl
LoadPlugin network

<Plugin logfile>
  LogLevel "debug"
  File "/var/log/collectd.log"
  Timestamp true
</Plugin>

<Plugin curl>
    <Page "status">
        URL "http://localhost:8080/status";
        MeasureResponseTime true
        MeasureResponseCode true
    </Page>
</Plugin>

<Plugin network>
    <Server "127.0.0.1" "25826">
        SecurityLevel Encrypt
        Username "user"
        Password "secret"
    </Server>
</Plugin>

 

For the cURL plugin, I configured it to measure response time (latency) and response code (HTTP status code) from the RESTful service.

Note that for the network plugin, I used Encrypt mode which requires an authentication file for the CloudWatch Agent to authenticate incoming collectd requests.  Click here for full details on the collectd installation script.

 

Step 3. Configure the CloudWatch agent

So far, I have shown you how to:

A.  Use StatsD to emit custom metrics to monitor my service health
B.  Optionally use collectd to collect metrics using plugins

Next, I will install and configure the CloudWatch agent to accept metrics from both the StatsD client and collectd plugins.

I installed the CloudWatch Agent following the instructions in the user guide, but here are the detailed steps:

Install CloudWatch Agent:

wget https://s3.amazonaws.com/amazoncloudwatch-agent/linux/amd64/latest/AmazonCloudWatchAgent.zip -O AmazonCloudWatchAgent.zip && unzip -o AmazonCloudWatchAgent.zip && sudo ./install.sh

Configure CloudWatch Agent to receive metrics from StatsD and collectd:

{
  "metrics": {
    "append_dimensions": {
      "AutoScalingGroupName": "${aws:AutoScalingGroupName}",
      "InstanceId": "${aws:InstanceId}"
    },
    "metrics_collected": {
      "collectd": {},
      "statsd": {}
    }
  }
}

Pass the above config (config.json) to the CloudWatch Agent:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:config.json -s

In case you want to skip these steps and just execute my sample agent install script, you can find it here.

 

Step 4. Generate and monitor application traffic in CloudWatch

Now that I have the CloudWatch agent installed and configured to receive StatsD and collect metrics, I’m going to generate traffic through the service:

echo "send 100 requests"
for i in {1..100}
do
   curl "localhost:8080/encode?text=TextToEncode_${i}[email protected]#%"
   echo ""
   sleep 1
done

 

Next, I log in to the CloudWatch console and check that the service is up and running. Here’s a graph of the StatsD metrics:

 

Here is a graph of the collectd metrics:

 

Conclusion

With StatsD and collectd support, you can now use the CloudWatch Agent to collect and monitor your custom applications in addition to the system metrics and application logs it already collects. Furthermore, you can create operational dashboards with these metrics, set alarms to take automated actions when free memory is low, and troubleshoot issues by diving into the application logs.  Note that StatsD supports both Windows and Linux operating systems while collectd is Linux only.  For Windows, you can also continue to use Windows Performance Counters to collect custom metrics instead.

The CloudWatch Agent with custom metrics support (version 1.203420.0 or later) is available in all public AWS Regions, AWS GovCloud (US), with AWS China (Beijing) and AWS China (Ningxia) coming soon.

The agent is free to use; you pay the usual CloudWatch prices for logs and custom metrics.

For more details, head over to the CloudWatch user guide for StatsD and collectd.

Podcast: AI tech named automated reasoning provides next-gen cloud security

Post Syndicated from Supriya Anand original https://aws.amazon.com/blogs/security/podcast-automated-reasoning-aws-next-gen-security-ai/

AWS just released a new podcast on how next generation security technology, backed by automated reasoning, is providing you higher levels of assurance for key components of your AWS architecture. Byron Cook, Director of the AWS Automated Reasoning Group, discusses how automated reasoning is embedded within AWS services and code and the tools customers can take advantage of today to achieve provable security.

Here’s a direct link to listen to it.

As the AWS cloud continues to grow, offering more services and features for you to architect your environment, AWS is working to ensure that the security of the cloud meets the pace of growth and your needs. To address the evolving threat landscape, AWS has made it easier to operate workloads securely in the cloud with a host of services and features that strengthen your security posture. Using automated reasoning, a branch of artificial intelligence, AWS is implementing next-generation security technology to help secure its platform. Automated reasoning at AWS helps customers verify and ensure continuous security of key components in the cloud, providing provable security—the highest assurance of cloud security.

Automated reasoning is powered by mathematical logic and consists of designing and implementing mechanized mathematical proofs that key components in the cloud are operating in alignment with customers’ intended security measures. With automated reasoning, AWS enables customers to detect entire classes of misconfigurations that could potentially expose vulnerable data. This relieves the customers’ burden of having to manually verify increasingly granular configurations for a complex organization, providing new levels of assurance that security verification scales with enterprise growth.

We hope you enjoy the podcast! If you have feedback about this blog post, submit comments in the Comments section below.

Want more AWS Security news? Follow us on Twitter.

Author

Supriya Anand

Supriya is a Content Strategist at AWS working with the Automated Reasoning Group.

How to rotate a WordPress MySQL database secret using AWS Secrets Manager in Amazon EKS

Post Syndicated from Paavan Mistry original https://aws.amazon.com/blogs/security/how-to-rotate-a-wordpress-mysql-database-secret-using-aws-secrets-manager-in-amazon-eks/

AWS Secrets Manager recently announced a feature update to rotate credentials for all Amazon RDS database types. This allows you to automatically rotate credentials for all types of databases hosted on Amazon RDS. In this post, I show you how to rotate database secrets for a non-RDS database using AWS Secrets Manager. I use a containerized WordPress application with a MySQL database to demonstrate the secret rotation.

Enabling regular rotation of database secrets helps secure application databases, protects customer data, and helps meet compliance requirements. You’ll use Amazon Elastic Container Service for Kubernetes (Amazon EKS) to help deploy, manage, and scale the WordPress application, and Secrets Manager to perform secret rotation on a containerized MySQL database.

Prerequisites

You’ll need an Amazon EKS or a Kubernetes cluster running and accessible for this post. Getting Started with Amazon EKS provides instructions on setting up the cluster and node environment. I recommend that you have a basic understanding of Kubernetes concepts, but Kubernetes reference document links are provided throughout this walk-through. For this post, I use the placeholder EKSClusterName to denote the existing EKS cluster. Remember to replace this with the name of your EKS cluster.

You’ll also need AWS Command Line Interface (AWS CLI) installed and configured on your machine. For this blog, I assume that the default AWS CLI region is set to Oregon (us-west-2) and that you have access to the AWS services described in this post. If you use other regions, you should check the availability of AWS services in those regions.

Architecture overview

 

Figure 1: Architecture and data flow diagram within Amazon EKS nodes

Figure 1: Architecture and data flow diagram within Amazon EKS nodes

The architecture diagram shows the overall deployment architecture with two data flows, a user data flow and a Secrets Manager data flow within the EKS three-node cluster VPC (virtual private cloud).

To access the WordPress site, user data flows through an internet gateway and an external load balancer as part of the WordPress frontend Kubernetes deployment. The AWS Secrets Manager data flow uses the recently announced Secrets Manager VPC Endpoint and an AWS Lambda function within the VPC that rotates the MySQL database secret through an internal load balancer. MySQL database and the internal load balancer are provisioned as part of the WordPress MySQL Kubernetes deployment. The internal load balancer allows the database service to be exposed internally for Secrets Manager to perform the rotation within a VPC. It removes the need for the database to be exposed to the Internet for secret rotation, which is not a good security practice.

Solution overview

The blog post consists of the following steps:

  1. Host WordPress and MySQL services on Amazon EKS.
  2. Store the database secret in AWS Secrets Manager.
  3. Set up the rotation of the database secret using Secrets Manager VPC Endpoint.
  4. Rotate the database secret and update the WordPress frontend deployment.

Note: This post helps you implement MySQL secret rotation for non-RDS databases. It should be used as a reference guide on database secret rotation. For guidance on using WordPress on AWS, please refer to the WordPress: Best Practices on AWS whitepaper.

Step 1: Host WordPress and MySQL services on Amazon EKS

To deploy WordPress on an EKS cluster, you’ll use three YAML templates.

  1. First, create a StorageClass in Amazon EKS that uses Amazon Elastic Block Store (Amazon EBS) for persistent volumes, using the command below if the volumes don’t exist. Then, create a Kubernetes namespace. The Kubernetes namespace creates a separate environment within your Kubernetes cluster to create objects specific to this walk-through. This helps with object management and logical separation.
    
    kubectl create -f https://raw.githubusercontent.com/paavan98pm/eks-secret-rotation/master/templates/gp2-storage-class.yaml
    
    kubectl create namespace wp
    

  2. Next, use Kubernetes secrets to store your MySQL database password with an Amazon EKS cluster. The password is generated by the AWS Secret Manager get-random-password API using the command below. This allows a random password to be created and stored in the Kubernetes secret object through the Secrets Manager API feature without the need to manually create it. The get-random-password API allows various password length and type restrictions to be enforced based on your organizational security policy.
    
    kubectl create secret generic mysql-pass --from-literal=password=$(aws secretsmanager get-random-password --password-length 20 --no-include-space | jq -r .RandomPassword) --namespace=wp
    

You’ll use the Kubernetes secret mysql-pass to create your MySQL and WordPress deployments using the YAML manifests in the next step.

Deploying MySQL and WordPress in Amazon EKS

Now, you’ll run the MySQL and WordPress templates provided by the Kubernetes community to deploy your backend MySQL services, deployments, and persistent volume claims.

  1. To deploy an internal load balancer that AWS Secrets Manager will use to perform its secret rotation, you’ll use the Kubernetes service annotation service.beta.kubernetes.io/aws-load-balancer-internal within the MySQL service YAML. For WordPress deployment, you’ll add three replicas for availability across the three availability zones.
    
    kubectl create -f https://raw.githubusercontent.com/paavan98pm/eks-secret-rotation/master/templates/mysql-deployment.yaml --namespace=wp
    
    kubectl create -f https://raw.githubusercontent.com/paavan98pm/eks-secret-rotation/master/templates/wordpress-deployment.yaml --namespace=wp
    

    Run the commands above with the Kubernetes YAML manifests to create MySQL and WordPress services, deployments, and persistent volume claims. As shown in the architecture diagram, these services also provision an external and an internal load balancer. The external load balancer is accessible over the internet for your WordPress frontend, while you’ll use the internal load balancer to rotate database secrets. It takes a few minutes for the load balancers to be provisioned.

  2. Once the load balancers are provisioned, you should be able to access your WordPress site in your browser by running the following command. The browser will open to show the WordPress setup prompt in Figure 2.
    
    open http://$(kubectl get svc -l app=wordpress --namespace=wp -o=jsonpath='{.items[0].status.loadBalancer.ingress[0].hostname}')
    

     

    Figure 2: WordPress setup page for the Amazon EKS deployment

    Figure 2: WordPress setup page for the Amazon EKS deployment

  3. From here, follow the WordPress instructions to complete installation and set up credentials to access your WordPress administration page. This completes the hosting and setup of your WordPress blog on Amazon EKS.

Step 2: Store the database secret in AWS Secrets Manager

Next, you’re going to store your database secret in AWS Secrets Manager. AWS Secrets Manager is a fully managed AWS service that enables you to rotate, manage, and retrieve secrets such as database credentials and application programming interface keys (API keys) throughout their lifecycle. Follow the steps below to use Secrets Manager to store the MySQL database secret and the rotation configuration details.
 

Figure 3: Store a new secret in AWS Secrets Manager

Figure 3: Store a new secret in AWS Secrets Manager

From the Secrets Manager console, select Store a new secret to open the page shown in Figure 3 above. Follow the instructions below to store the MySQL database secret parameters.

  1. Select Credentials for other database and provide the username and password you created in the previous step. The username value will be root, and you should paste the password value from the output of the command below. This command copies the MySQL password stored in the Kubernetes secret object, allowing you to store it in Secrets Manager.

    Password:

    
    kubectl get secret mysql-pass -o=jsonpath='{.data.password}' --namespace=wp | base64 --decode | pbcopy
    

  2. Secrets Manager encrypts secrets by default using the service-specific encryption key for your AWS account. Since you’re storing the secret for MySQL database, choose the MySQL tile and provide the server details. For Server address, you can paste the output of the command below. The database name and port values are mysql and 3306, respectively.

    Server address:

    
    kubectl get svc -l app=wordpress-mysql --namespace=wp -o=jsonpath='{.items[0].status.loadBalancer.ingress[0].hostname}' | pbcopy
    

  3. Select Next and give the secret a name and a description that enables you to reference and manage the secret easily. I use the placeholder yourSecretName to reference your secret throughout the rest of my post; be sure to swap this placeholder with your own secret name.
  4. On the next screen, accept the default setting: Disable automatic rotation.
  5. Finally, select Store to store your MySQL secret in AWS Secrets Manager.

Step 3: Set MySQL secret rotation using Secrets Manager VPC Endpoint

Next, you’ll create a Lambda Function within the Amazon EKS node cluster VPC to rotate this secret. Within the node cluster VPC, you’re also going to configure the AWS Secrets Manager recently announced support for Amazon Virtual Private Cloud (VPC) endpoints powered by AWS PrivateLink.

Note: If you have a database hosted on Amazon RDS, Secrets Manager natively supports rotating secrets. However, if you’re using a container-based MySQL database, you must create and configure the Lambda rotation function for AWS Secrets Manager, and then provide the Amazon Resource Name (ARN) of the completed function to the secret.

Creating a Lambda rotation function

The following commands apply a WordPress MySQL rotation template to a new Lambda function. CloudFormation uses an AWS Serverless Application Repository template to automate most of the steps for you. The values you need to replace for your environment are denoted in red.

  1. The command below will return a ChangeSetId. It uses the Secrets Manager MySQL single user template from the AWS Serverless Application Repository and creates a CloudFormation Change Set by providing the Secrets Manager endpoint and Lambda function name as parameters. You need to change the Secrets Manager region within the endpoint parameter and provide a name for the Lambda function that you’re creating.
    
    aws serverlessrepo create-cloud-formation-change-set --application-id arn:aws:serverlessrepo:us-east-1:297356227824:applications/SecretsManagerRDSMySQLRotationSingleUser --stack-name MyLambdaCreationStack --parameter-overrides '[{"Name":"endpoint","Value":"https://secretsmanager.<REGION>.amazonaws.com"},{"Name":"functionName","Value":"<EKSMySQLRotationFunction>"}]' | jq -r '.ChangeSetId'
    

  2. The next command runs the change set that you just created to create a Lambda function. The change-set-name parameter comes from the ChangeSetId output of the previous command. Copy the ChangeSetId value into the instruction below.
    
    aws cloudformation execute-change-set --change-set-name <ChangeSetId>
    

  3. The following command grants Secrets Manager permission to call the Lambda function on your behalf. You should provide the Lambda function name that you set earlier.
    
    aws lambda add-permission --function-name <EKSMySQLRotationFunction> --principal secretsmanager.amazonaws.com --action lambda:InvokeFunction --statement-id SecretsManagerAccess
    

Configuring the Lambda rotation function to use Secrets Manager VPC Endpoint

Instead of connecting your VPC to the internet, you can connect directly to Secrets Manager through a private endpoint that you configure within your VPC. When you use a VPC service endpoint, communication between your VPC and Secrets Manager occurs entirely within the AWS network and requires no public internet access.

  1. To enable AWS Secrets Manager VPC endpoint, first store the Node Instance security group in a NODE_INSTANCE_SG environment variable to use it as an argument for creating the Secrets Manager VPC Endpoint. Using the VPC ID from the EKS cluster, this command filters the Security Groups attached to the nodes and stores them in the NODE_INSTANCE_SG environment variable. This variable will be used as an argument in the next step to create a Secrets Manager VPC endpoint.
    
    export NODE_INSTANCE_SG=$(aws ec2 describe-security-groups --filters Name=vpc-id,Values=$(aws eks describe-cluster --name <EKSClusterName> | jq -r '.cluster.resourcesVpcConfig.vpcId') Name=tag:aws:cloudformation:logical-id,Values=NodeSecurityGroup | jq -r '.SecurityGroups[].GroupId')
    

  2. Next, create a Secrets Manager VPC endpoint attached to the EKS node cluster subnets and the related security group. This command retrieves the VPC ID and Subnet IDs from the EKS cluster to create a Secrets Manager VPC endpoint.
    
    aws ec2 create-vpc-endpoint --vpc-id $(aws eks describe-cluster --name <EKSClusterName> | jq -r '.cluster.resourcesVpcConfig.vpcId') --vpc-endpoint-type Interface --service-name com.amazonaws.<region>.secretsmanager --subnet-ids $(aws eks describe-cluster --name <EKSClusterName> | jq -r '.cluster.resourcesVpcConfig.subnetIds | join(" ")') --security-group-id $NODE_INSTANCE_SG --private-dns-enabled
    

  3. Next, update the Lambda function to attach it to the EKS node cluster subnets and the related security group. This command updates the Lambda function’s configuration by retrieving its ARN and EKS cluster Subnet IDs and Security Group ID.
    
    aws lambda update-function-configuration --function-name $(aws lambda list-functions | jq -r '.Functions[] | select(.FunctionName == "<EKSMySQLRotationFunction>") | .FunctionArn') --vpc-config SubnetIds=$(aws eks describe-cluster --name <EKSClusterName> | jq -r '.cluster.resourcesVpcConfig.subnetIds | join(",")'),SecurityGroupIds=$NODE_INSTANCE_SG
    

Step 4: Rotate the database secret and update your WordPress frontend deployment

Now that you’ve configured the Lambda rotation function, use the rotate-secret command to schedule a rotation of this secret. The command below uses the previously stored secret name with the Lambda function ARN to rotate your secret automatically every 30 days. You can adjust the rotation frequency value, if you want. The minimum rotation frequency is 1 day.


aws secretsmanager rotate-secret --secret-id <yourSecretName> --rotation-lambda-arn $(aws lambda list-functions | jq -r '.Functions[] | select(.FunctionName == "<EKSMySQLRotationFunction>") | .FunctionArn') --rotation-rules AutomaticallyAfterDays=30

The figure below shows the Secrets Manager data flow for secret rotation using the VPC endpoint.
 

Figure 4: MySQL database secret rotation steps

Figure 4: MySQL database secret rotation steps

Based on the frequency of the rotation, Secrets Manager will follow the steps below to perform automatic rotations:

  1. Secrets Manager will invoke <EKSMySQLRotationFunction> within the EKS node VPC using the Secrets Manager VPC endpoint.
  2. Using the credentials provided for MySQL database, <EKSMySQLRotationFunction> will rotate the database secret and update the secret value in Secrets Manager.

Once the Lambda function has executed, you can test the updated database secret value that’s stored in AWS Secrets Manager by using the command below.


aws secretsmanager get-secret-value --secret-id <yourSecretName>

Next, update the Kubernetes secret (mysql-pass) with the rotated secret stored in AWS Secrets Manager, then update the WordPress frontend deployment with the rotated MySQL password environment variable using the command below. These instructions should be automated using Kubernetes and AWS API once you’ve completed your secret rotation.

The command below retrieves the rotated secret from Secrets Manager using the GetSecretValue API call and passes the updated value to the Kubernetes secret. The next command patches the WordPress deployment for the frontend WordPress containers to use the updated Kubernetes secret. It also adds a rotation date annotation for the deployment.


kubectl create secret generic mysql-pass --namespace=wp --from-literal=password=$(aws secretsmanager get-secret-value --secret-id <yourSecretName> | jq --raw-output '.SecretString' | jq -r .password) -o yaml --dry-run | k replace -f -

kubectl patch deploy wordpress -p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"rotation_date\":\"`date +'%s'`\"}}}}}" --namespace=wp

You should now be able to access the updated WordPress site with database secret rotation enabled.


open http://$(kubectl get svc -l app=wordpress --namespace=wp -o=jsonpath='{.items[0].status.loadBalancer.ingress[0].hostname}')

Clean-up

To delete the environment used in this walk-through in EKS, the secret in Secrets Manager, and the accompanying Lambda function, run the following commands. You only need to follow these steps if you intend to remove the secret, Lambda function, VPC endpoint, and EKS objects created in this walk-through.


kubectl delete namespace wp

aws secretsmanager delete-secret --secret-id <yourSecretName> --force-delete-without-recovery

aws lambda delete-function --function-name <EKSMySQLRotationFunction>

aws ec2 delete-vpc-endpoints --vpc-endpoint-ids $(aws ec2 describe-vpc-endpoints | jq -r '.VpcEndpoints[] | select(.ServiceName == "com.amazonaws.<REGION>.secretsmanager") | .VpcEndpointId')

Pricing

Next, I review the pricing and estimated cost of this example. AWS Secrets Manager offers a 30-day trial period that starts when you store your first secret. Storage of each secret costs $0.40 per secret per month. For secrets that are stored for less than a month, the price is prorated based on the number of hours. There is an additional cost of $0.05 per 10,000 API calls. You can learn more by visiting the AWS Secrets Manager pricing service details page.

You pay $0.20 per hour for each Amazon EKS cluster that you create. You can use a single Amazon EKS cluster to run multiple applications by taking advantage of Kubernetes namespaces and IAM security policies. You pay for AWS resources (for example, EC2 instances or EBS volumes) that you create to run your Kubernetes worker nodes. You only pay for what you use, as you use it; there are no minimum fees and no upfront commitments. See detailed pricing information on the Amazon EC2 pricing page.

Assuming you allocate two hours to follow the blog instructions, the cost will be approximately $1.

  1. Amazon EKS – 2 hours x ($0.20 per EKS cluster + 3 nodes x $0.096 m5.large instance)
  2. AWS Secrets Manager – 2 hours x ($0.40 per secret per month / 30 days / 24 hours + $0.05 per 10,000 API calls)

Summary

In this post, I showed you how to rotate WordPress database credentials in Amazon EKS using AWS Secrets Manager.

For more details on secrets management within Amazon EKS, check out the Github workshop for Kubernetes, particularly the section on ConfigMaps and Secrets, to understand different secret management alternatives available. To get started using Kubernetes on AWS, open the Amazon EKS console. To learn more, read the EKS documentation. To get started managing secrets, open the Secrets Manager console. To learn more, read the Secrets Manager documentation.

If you have comments about this post, submit them in the Comments section below. If you have questions about anything in this post, start a new thread on the EKS forum or Secrets Manager forum.

Want more AWS Security news? Follow us on Twitter.

Author

Paavan Mistry

Paavan is a Security Specialist Solutions Architect at AWS where he enjoys solving customers’ cloud security, risk, and compliance challenges. Outside of work, he enjoys reading about leadership, politics, law, and human rights.

How Annalect built an event log data analytics solution using Amazon Redshift

Post Syndicated from Eric Kamm original https://aws.amazon.com/blogs/big-data/how-annalect-built-an-event-log-data-analytics-solution-using-amazon-redshift/

Ingesting and analyzing event log data into a data warehouse in near real-time is challenging. Data ingest must be fast and efficient. The data warehouse must be able to scale quickly to handle incoming data volumes. Storing massive amounts of historical data in the data warehouse for infrequent access is prohibitively expensive. If you store infrequently accessed data outside the data warehouse, loading time is unacceptable.

At Annalect, we found a way to overcome these challenges. In this article, we discuss how we built a solution on AWS to manage, enhance, and analyze event log data from our ad technology partners. We use Amazon S3 for storage, Amazon EC2 and AWS Batch for compute, AWS Glue Data Catalog, and Amazon Redshift and Amazon Redshift Spectrum for analytics. This scalable, on-demand architecture has proven to be a high-performance solution for our analytics users and is very cost-effective.

Architecture overview

In mid-2016, we were using Amazon Redshift with local tables to provide SQL access to event-level ad tech datasets for analysts and modelers to improve ad buying strategies. We retained up to six months of historical data for select data sources; this represented about 230 billion events. By early 2017, the same data footprint grew to over 385 billion events, and we increased our Amazon Redshift capacity to keep pace. As we looked forward, we wanted to support more data, both in terms of history and data source coverage. At the same time, we wanted to maintain or improve query performance, ideally without increasing costs.

Because we were already using Amazon Redshift as our data warehouse, we looked at Redshift Spectrum. Redshift Spectrum is a feature of Amazon Redshift that queries data stored in Amazon S3. It uses the same SQL capabilities as Amazon Redshift, without the need to load the data into Amazon Redshift tables stored on disks within an Amazon Redshift cluster.

With Redshift Spectrum, a table’s storage is separate from a cluster’s compute resources, which enables each to scale independently of the other. We can keep large data tables in S3 instead of local Amazon Redshift storage, which reduces the size of our Amazon Redshift clusters and lowers costs. Queries that use external Redshift Spectrum tables use Redshift Spectrum compute resources (separate from the Amazon Redshift cluster) to scan the data in S3. Queries are billed by AWS at a metered rate based on the amount of data scanned. For query efficiency, we convert the data stored in S3 into Parquet files using AWS Batch.

The result is an efficient, mostly serverless analytics environment where the data is always available to query with Redshift Spectrum. Although we manage some always-on Amazon Redshift clusters, we think of this as “mostly serverless.” This is because the cluster footprint is very small relative to the serverless compute power we use through Redshift Spectrum for analysis and through AWS Batch for data preparation.

The Amazon Redshift clusters are dedicated to performing analytics for our analysts, rather than having a portion of their time dedicated to load jobs. This is because no time is spent copying data into local tables in the Amazon Redshift clusters and because we do the data preparation work outside of Amazon Redshift.

With the evolution to Redshift Spectrum and storing table data in S3, by early 2018 we increased the amount of data that we deliver to analysts by an order of magnitude. At the same time, we kept our costs at the same level as 2017. Using Redshift Spectrum is a great cost saving to us. We now have up to 36 months of historical data available for many more data sources, representing trillions of events.

Our data management infrastructure, the Enhanced Event-Level Data Environment (EELDE), is structured as shown following.

The EELDE provides four primary functions:

  • Data ingest
  • Data transformation
  • Compute-intensive enrichment
  • Analyst ad hoc query environment

These four functions are described following in more detail.

Data ingest

We architected our ingest system to ensure smooth and rapid downstream data availability. We took into account the inherent inconsistencies of vendor data feeds, both in the contents of the feeds themselves and in different vendors’ data push conventions. We also took into account the large volume of data files that data vendors might deliver at a given time. The diagram following shows our data ingest process.

We set up two storage areas in Amazon S3 to initially hold data from our vendors. The Landing Zone serves as a repository for long-term storage of historical, unaltered raw log files. Log files that are pulled by us directly from vendors are stored immediately in the Landing Zone. The Datafeeds storage serves as a transient place for log files that are pushed out by the vendors. We curate and arrange the data files delivered into the Datafeeds storage by vendors as we pull them into the Landing Zone.

The pull processes are executed in the EELDE’s Job Execution Layer, which is autoscaling and can handle many data files in parallel. This ensures that data processing and availability are not delayed by sudden spikes in data volume. Some spikes might occur from the need to backfill a large dataset from a new vendor. Others might occur when a vendor delivers a large volume of logs accumulated after a delay on their side.

Whether the vendor logs are pushed or pulled, we monitor the Landing Zone. We send alerts when we detect changes in file formats or gaps in data that might have an impact on downstream analysis. We found that this architecture and process gives us the control, flexibility, and agility that we need to respond rapidly to any volume of data and any changes in the data.

Data transformation

The EELDE can process, enhance, and analyze large volumes of event data in a compute-efficient and cost-efficient manner. An important part of this ability is the initial transformation of files in the Landing Zone. The EELDE’s Job Execution Layer uses AWS Batch to parallelize the compute-intensive data transformation from the original source data. This data is typically in comma-separated value (CSV) or JSON format. AWS Batch parallelizes it into a columnar, compressed storage format, Apache Parquet. In the diagrams, you can see the Parquet storage location labeled as the “Level 2 (columnar)” bucket.

For our Parquet transformation jobs, we have had a good experience running many containers with local-mode Apache Spark using AWS Batch. The goal for us is supporting a high throughput of file transformations where each file’s size is in the range of a few megabytes to a few gigabytes. A given Parquet transformation might take anywhere from a few seconds to 30 minutes.

The resource isolation and job management provided by AWS Batch makes running hundreds of independent data transformation jobs straightforward. At the same time, it removes some of the challenges of running many tasks simultaneously within Amazon EMR. In contrast, to transform larger datasets, the parallel compute per job that EMR can provide is attractive.

The diagram following shows data transformation and cataloging.

We partition the large Parquet-backed event data tables by date. Date partitioning is strategically implemented to optimize both query cost and query performance by providing an efficient path to the required data for a given analytics process. After the files are converted to Parquet, they are exposed as table partitions in the AWS Glue Data Catalog. Although the data storage format has been transformed, the data still retains all of its original fields, which is important for supporting ad hoc analyses.

Once the Parquet conversion and AWS Glue Data Catalog management processes have run, the data is accessible through SQL queries using Redshift Spectrum. Redshift Spectrum uses the AWS Glue Data Catalog to reference external tables stored in S3 so that you can query them through the Amazon Redshift cluster. Using a columnar format like Parquet increases the speed and cost efficiency of performing queries with Redshift Spectrum.

Compute-intensive enrichment

Before analytics work, the Job Execution Layer might also run other compute intensive data transformations to enhance the logs so that they can support more advanced types of analysis. Event-level log combination and event attribute augmentation are two examples.

Although these enrichment tasks might have different compute requirements, they can all use Redshift Spectrum to gather relevant datasets, which are then processed in the Job Execution Layer. The results are made available again through Redshift Spectrum. This enables complex enrichment of large datasets in acceptable timeframes and at a low cost.

Analyst ad hoc query environment

Our environment provides analysts with a data landscape and SQL query environment for mission-critical analytics. The data is organized within the EELDE into separate external schemas by ad tech vendor and system seat or account. Our agency-based analysts and data scientists are granted access to specific schemas according to project needs.

By having access to various datasets within a single platform, analysts can gain unique insights that are not possible by analyzing data in separate silos. They can create new datasets from a combination of source vendors for use during specific analytic projects while respecting data separation requirements. Doing this helps support use cases such as predictive modeling, channel planning, and multisource attribution.

Best practices to keep costs low

Along the way, we have developed some best practices for Redshift Spectrum:

  • Use compressed columnar format. We use Apache Parquet for our large event-level tables. Doing this reduces query run time, data scan costs, and storage costs. Our large tables typically have 40–200 columns, yet most of our queries use only 4–8 columns at a time. Using Parquet commonly lowers the scan cost by over 90 percent compared to noncolumnar alternatives.
  • Partition tables. We partition our large event level tables by date, and we train users to use the partition column in WHERE clauses. Doing this provides cost savings proportional to the portion of dates used by the queries. For example, suppose that we expose six months of data and a query looks at only one month. In this case, the cost for that query is reduced by five-sixths by using the appropriate WHERE clause.
  • Actively manage table partitions. Although we have up to 36 months of historical data available as Parquet files, by default we expose only the most recent 6 months of data as external table partitions. This practice reduces the cost of queries when users fail to filter on the partition column. On request, we temporarily add older date partitions to the necessary tables to support queries that need more than 6 months of data. In this way, we support longer timeframes when needed. At the same time, we usually maintain a lower cost limit by not exposing more data than is commonly necessary.
  • Move external tables into Amazon Redshift when necessary. When a large number of queries repeatedly hit the same external table, we often create a temporary local table in Amazon Redshift to query against. Depending on the number of queries and their scope, it can be helpful to load the data into a local Amazon Redshift table with the appropriate distribution key and sort keys. This practice helps eliminate the repeated Redshift Spectrum scan costs.
  • Use spectrum_scan_size_mb WLM rules. Most of our users and jobs operate with an effective Redshift Spectrum data scan limit of 1 terabyte by setting a spectrum_scan_size_mb WLM rule on the default WLM queue. For a single analysis or modeling query, this limit rarely needs to be exceeded.
  • Add clusters to increase concurrency when needed. Multiple Amazon Redshift clusters have access to the same external tables in S3. If we have a lot of users running queries at the same time against external tables, we might choose to add more clusters. Redshift Spectrum and the management tools we have created for our environment make this an easy task.
  • Use short-lived Amazon Redshift clusters. For some scheduled production jobs, we use short-lived Amazon Redshift clusters with the necessary data made available through external schemas. Redshift Spectrum allows us to avoid lengthy data load times, so it is practical to launch a Amazon Redshift cluster for a couple of hours of work and then terminate it every day. Doing this can save 80–90 percent compared to having an always-on cluster for a project. It also avoids contention with other query activity, because the short-lived cluster is dedicated to the project.

Summary

By establishing a data warehouse strategy using Amazon S3 for storage and Redshift Spectrum for analytics, we increased the size of the datasets we support by over an order of magnitude. In addition, we improved our ability to ingest large volumes of data quickly, and maintained fast performance without increasing our costs. Our analysts and modelers can now perform deeper analytics to improve ad buying strategies and results.

 


Additional Reading

If you found this post useful, be sure to check out Using Amazon Redshift Spectrum, Amazon Athena, and AWS Glue with Node.js in Production and How I built a data warehouse using Amazon Redshift and AWS services in record time.

 


About the Author

Eric Kamm is a senior engineer and architect at Annalect. He has been creating and managing large-scale ETL workflows and providing data and compute environments for teams of analysts and modelers for over twenty years. He enjoys working with cloud technologies because they can accelerate project planning and implementation while increasing reliability and operational efficiency.

Running high-scale web applications on Amazon EC2 Spot Instances

Post Syndicated from Roshni Pary original https://aws.amazon.com/blogs/compute/running-high-scale-web-on-spot-instances/

Contributed by Ran Sheinberg, Spot Specialist Solutions Architect, and the Appnext team

At re:Invent in 2017, AWS made some significant changes to the pricing model of Amazon EC2 Spot Instances. Instead of bidding on spare capacity, you could request to run Spot Instances, and get the capacity if it’s available.

One of the results of that change is fewer interruptions to Spot Instances. In fact, more than 95% of Spot Instances are terminated by the customer and not by EC2 interruptions. For more information about the average interruption rate for each instance type in an AWS Region, see the Spot Instance Advisor.

Customers use Spot Instances differently, depending on the type of workload:

  • Time-insensitive—These are workloads that are not SLA-bound, or are possibly internal and not public-facing. Examples: scientific computations, dev/test, batch jobs, model training in machine learning or deep learning, etc.
  • Time sensitive workloads—These are workloads that serve end users or which must be highly available and fault tolerant. Examples: website hosting and web services, APIs, containers running behind load balancers, etc.

When you run a time-insensitive workload such as a batch job, Spot interruptions could still affect some of your EC2 capacity. Generally, your job continues and tasks are rescheduled on new instances (for example, if you’re using AWS Batch).

In this post, I focus on time-sensitive workloads where Spot instance interruptions can potentially impact end users. In these scenarios, it is required to architect the workload properly to handle the two-minute warning to avoid outages or disconnections by following Cloud best practices.

My recommendations include avoiding state or end-user stickiness to specific instances, architecting for fault tolerance, decoupling the components of the application with queueing mechanisms, and keeping compute and storage decoupled as well.

Spot Instances

Spot Instances are spare compute capacity in the AWS Cloud. They are available to customers at up to 90% discount compared to On-Demand prices.

Spot Instances can be interrupted by Amazon EC2 with a two-minute warning, in case EC2 must reclaim capacity. Spot Instances are available in different capacity pools (comprised of an instance type and an Availability Zone in each AWS Region). We recommend that you deploy your workload across several (or many) Spot capacity pools to maximize the cost savings and obtain desired scale.

The Taking Advantage of Amazon EC2 Spot Instance Interruption Notices post walks you through preparing a workload for the interruption, by draining connections from the instance and detaching it from the Application Load Balancer (ALB).

Appnext

AWS and Spot Instances are popular in the ad tech industry, which is a great example of customers running business-critical web applications. Their workloads require a high request rate, low latency, and support for spiky traffic rates throughout the day in real-time bidding and similar applications.

Ad tech startup Appnext is a great example of a tech-savvy AWS customer, and they chose to run the EC2 instances for their business critical applications on Spot Instances, for cost saving purposes. Appnext’s technology allows advertisers to tap into users’ daily timelines, reaching them at crucial moments throughout their day, so the application’s availability is key for Appnext.

Instance diversification and capacity pools

To achieve their desired scale, Appnext uses a Spot Fleet to diversify their instances easily across many Spot Instance pools. Diversification of capacity is key when using Spot Instances for highly available applications. Customers like Appnext configure their Spot Fleets to use as many EC2 instance types as possible in all the Availability Zones in each region where the application is running. They choose the instance types that are supported by their application use case and required hardware specifications. Diversification also enables you to get Spot capacity at the lowest prices, using the suitable allocation strategy.

Appnext is prepared for rare situations where Spot capacity is unavailable in all their chosen Spot capacity pools, by automatically adding On-Demand Instances behind the Classic Load Balancer. According to the Appnext team, this is something they hardly ever encounter.

Each instance family and instance size in each Availability Zone in every Region is a separate Spot capacity pool. Each orange square in the following graphic is a capacity pool. This is true for each instance family (C4, C5, M5, R4…) in each AWS Region (the prices are just examples).

High-scale web applications on Spot Instances

Appnext’s goal is to run EC2 capacity behind a Classic Load Balancer that is sufficient to provide both high performance and highly available service. It should use as much Spot capacity as possible during automatic scaling. At the same time, it’s important to decrease the risk of application impact due to Spot interruptions. If Spot capacity is insufficient, additional On-Demand capacity behind the ALB is ready to scale out fast.

To achieve this goal, Appnext set up the following components. The actual CloudWatch metric thresholds are examples, as these vary across the different applications and regions.

  • Spot Fleet—This is the main supplier of Spot Instances that are automatically joined as targets behind the ALB when launched. The Spot Fleet is configured with an automatic scaling policy. It only adds Spot Instances if the average CPU of the On-Demand Instances passes the 75% threshold for 10 minutes. It terminates instances if the CPU utilization goes below a 55% CPU threshold.
  • Auto Scaling group for On-Demand Instances—This is the secondary supplier of EC2 instances for the ALB. The Auto Scaling group is configured with 1–2 static On-Demand Instances. It scales out when the average CPU utilization of the instances in the Auto Scaling group passes the 90% threshold for 10 minutes. It terminates instances when the threshold goes below 75%.

If Spot Fleet is unable to fulfill the target capacity and scale out due to lack of capacity in the selected Spot capacity pools, On-Demand Instances are started instead. The CPU utilization of the existing On-Demand Instances in the EC2 Auto Scaling group increases. This setup has proven to be a great fit for Appnext’s application, because it is CPU-bound, and increased application usage is always reflected in increased CPU utilization.

Because the Spot Fleet is well-diversified across a large number of Spot capacity pools, there is hardly ever any need to launch On-Demand Instances. That keeps the EC2 compute costs of the application low while it is still being architected for availability.

Lastly, the Auto Scaling group for the On-Demand Instance is configured to use an instance type that is not configured in the Spot Fleet. This is a best practice to avoid trying to launch On-Demand Instances to replace Spot capacity in a pool that is already depleted.

By launching Spot Instances using Spot Fleet and On-Demand Instances using Auto Scaling groups, Appnext is able to decouple the metrics and scaling policies used by each of the instance groups. This is despite the fact that you can launch On-Demand Instances in Spot Fleet.

The following diagram shows how the components work together.

Coming soon

In the EC2 Fleet Manages Thousands of On-Demand and Spot Instances with One Request post that introduced EC2 Fleet, the author mentions that there will be an integration to EC2 Auto Scaling groups. If you were already using Auto Scaling groups, you would be able to diversify the usage of Spot and On-Demand Instances with different instance types. When this feature is released, you will be able to do this all from within Auto Scaling groups without using Spot Fleet or EC2 Fleet directly.

Conclusion

I hope you found this post useful. If you haven’t tried Spot Instances yet, I suggest testing them for your web applications to optimize compute costs.

Learn about AWS – October AWS Online Tech Talks

Post Syndicated from Robin Park original https://aws.amazon.com/blogs/aws/learn-about-aws-october-aws-online-tech-talks/

AWS Tech Talks

AWS Online Tech Talks are live, online presentations that cover a broad range of topics at varying technical levels. Join us this month to learn about AWS services and solutions. We’ll have experts online to help answer any questions you may have.

Featured this month, check out the webinars under AR/VR, End-User Computing, Industry Solutions. Also, register for our second fireside chat discussion on Amazon Redshift.

Register today!

Note – All sessions are free and in Pacific Time.

Tech talks featured this month:

AR/VR

October 16, 2018 | 01:00 PM – 02:00 PM PTCreating and Publishing AR, VR and 3D Applications with Amazon Sumerian – Learn about Amazon Sumerian, the fastest and easiest way to create and publish immersive applications.

Compute

October 25, 2018 | 09:00 AM – 10:00 AM PTRunning Cost Effective Batch Workloads with AWS Batch and Amazon EC2 Spot Instances – Learn how to run complex workloads, such as analytics, image processing, and machine learning applications efficiently and cost-effectively.

Data Lakes & Analytics

October 18, 2018 | 01:00 PM – 01:45 PM PTFireside Chat: The Evolution of Amazon Redshift – Join Vidhya Srinivasan, General Manager of Redshift, in a candid conversation as she discusses the product’s evolution recently shipped features and improvements.

October 15, 2018 | 01:00 PM – 01:45 PM PTCustomer Showcase: The Secret Sauce Behind GroupM’s Marketing Analytics Platform – Learn how GroupM – the world’s largest media investment group with more than $113.8bn in billings – created a modern data analytics platform using Amazon Redshift and Matillion.

Databases

October 15, 2018 | 11:00 AM – 12:00 PM PTSupercharge Query Caching with AWS Database Services – Learn how AWS database services, including Amazon Relational Database Service (RDS) and Amazon ElastiCache, work together to make it simpler to add a caching layer to your database, delivering high availability and performance for query-intensive apps.

October 17, 2018 | 09:00 AM – 09:45 AM PTHow to Migrate from Cassandra to DynamoDB Using the New Cassandra Connector in the AWS Database Migration Service – Learn how to migrate from Cassandra to DynamoDB using the new Cassandra Connector in the AWS Database Migration Service.

End-User Computing

October 23, 2018 | 01:00 PM – 02:00 PM PTHow to use Amazon Linux WorkSpaces for Agile Development – Learn how to integrate your Amazon Linux WorkSpaces development environment with other AWS Developer Tools.

Enterprise & Hybrid

October 23, 2018 | 09:00 AM – 10:00 AM PTMigrating Microsoft SQL Server 2008 Databases to AWS – Learn how you can provision, monitor, and manage Microsoft SQL Server on AWS.

Industry Solutions

October 24, 2018 | 11:00 AM – 12:00 PM PTTape-to-Cloud Media Migration Walkthrough – Learn from media-specialist SAs as they walk through a content migration solution featuring machine learning and media services to automate processing, packaging, and metadata extraction.

IoT

October 22, 2018 | 01:00 PM – 01:45 PM PTUsing Asset Monitoring in Industrial IoT Applications – Learn how AWS IoT is used in industrial applications to understand asset health and performance.

Machine Learning

October 15, 2018 | 09:00 AM – 09:45 AM PTBuild Intelligent Applications with Machine Learning on AWS – Learn how to accelerate development of AI applications using machine learning on AWS.

Management Tools

October 24, 2018 | 01:00 PM – 02:00 PM PTImplementing Governance and Compliance in a Multi-Account, Multi-Region Scenario – Learn AWS Config best practices on how to implement governance and compliance in a multi-account, multi-Region scenario.

Networking

October 23, 2018 | 11:00 AM – 11:45 AM PTHow to Build Intelligent Web Applications @ Edge – Explore how [email protected] can help you deliver low latency web applications.

October 25, 2018 | 01:00 PM – 02:00 PM PTDeep Dive on Bring Your Own IP – Learn how to easily migrate legacy applications that use IP addresses with Bring Your Own IP.

re:Invent

October 10, 2018 | 08:00 AM – 08:30 AM PTEpisode 6: Mobile App & Reserved Seating – Discover new innovations coming to the re:Invent 2018 mobile experience this year. Plus, learn all about reserved seating for your priority sessions.

Security, Identity & Compliance

October 22, 2018 | 11:00 AM – 11:45 AM PTGetting to Know AWS Secrets Manager – Learn how to protect your secrets used to access your applications, services, and IT resources.

Serverless

October 17, 2018 | 11:00 AM – 12:00 PM PTBuild Enterprise-Grade Serverless Apps – Learn how developers can design, develop, deliver, and monitor cloud applications as they take advantage of the AWS serverless platform and developer toolset.

Storage

October 24, 2018 | 09:00 AM – 09:45 AM PTDeep Dive: New AWS Storage Gateway Hardware Appliance – Learn how you can use the AWS Storage Gateway hardware appliance to connect on-premises applications to AWS storage.

How to clone an AWS CloudHSM cluster across regions

Post Syndicated from Tracy Pierce original https://aws.amazon.com/blogs/security/how-to-clone-an-aws-cloudhsm-cluster-across-regions/

You can use AWS CloudHSM to generate, store, import, export, and manage your cryptographic keys. It also permits hash functions to compute message digests and hash-based message authentication codes (HMACs), as well as cryptographically sign data and verify signatures. To help ensure redundancy of data and simplification of the disaster recovery process, you’ll typically clone your AWS CloudHSM cluster into a different AWS region. This then allows you to synchronize keys, including non-exportable keys, across regions. Non-exportable keys are keys that can never leave the CloudHSM device in plaintext. They reside on the CloudHSM device and are encrypted for security purposes.

You clone a cluster to another region in a two-step process. First, you copy a backup to the destination region. Second, you create a new cluster from this backup. In this post, I’ll show you how to set up one cluster in region 1, and how to use the new CopyBackupToRegion feature to clone the cluster and hardware security modules (HSMs) to a virtual private cloud (VPC) in region 2.

Note: This post doesn’t include instructions on how to set up a cross-region VPC to synchronize HSMs across the two cloned clusters. If you need to do that, read this article.

Solution overview

To complete this solution, you can use either the AWS Command Line Interface (AWS CLI)
or the AWS CloudHSM API. For this post, I’ll use the AWS CLI to copy the cluster backup from region 1 to region 2, and then I’ll launch a new cluster from that copied backup.

The following diagram illustrates the process covered in the post.
 

Figure 1: Architecture diagram

Figure 1: Architecture diagram

Here’s how the process works:

  1. AWS CloudHSM creates a backup of the cluster and stores it in an S3 bucket owned by AWS CloudHSM.
  2. You run the CLI/API command to copy the backup to another AWS region.
  3. When the backup is completed, you use that backup to then create a cluster and HSMs.
  4. Note: Backups can’t be copied into or out of AWS GovCloud (US) because it’s a restricted region.

As with all cluster backups, when you copy the backup to a new AWS region, it’s stored in an Amazon S3 bucket owned by an AWS CloudHSM account. AWS CloudHSM manages the security and storage of cluster backups for you. This means the backup in both regions will also have the durability of Amazon S3, which is 99.999999999%. The backup in region 2 will also be encrypted and secured in the same way as your backup in region 1. You can read more about the encryption process of your AWS CloudHSM backups here.

Any HSMs created in this cloned cluster will have the same users and keys as the original cluster at the time the backup was taken. From this point on, you must manually keep the cloned clusters in sync. Specifically:

  • If you create users after creating your new cluster from the backup, you must create them on both clusters manually.
  • If you change the password for a user in one cluster, you must change the password on the cloned clusters to match.
  • If you create more keys in one cluster, you must sync them to at least one HSM in the cloned cluster. Note that after you sync the key from cluster 1 to cluster 2, the CloudHSM automated cluster synchronization will take care of syncing the keys within the 2nd cluster.

Prerequisites

Some items that will need to be in place for this to work are:

Important note: Syncing keys across clusters in more than one region will only work if all clusters are created from the same backup. This is because synchronization requires the same secret key, called a masking key, to be present on the source and destination HSM. The masking key is specific to each cluster. It can’t be exported, and can’t be used for any purpose other than synchronizing keys across HSMs in a cluster.

Step 1: Create your first cluster in region 1

Follow the links in each sub-step below to the documentation page for more information and setup requirements:

  1. Create the cluster. To do this, you will run the command below via CLI. You will want to replace the placeholder <SUBNET ID 1> with one of your private subnets.
    $ aws cloudhsmv2 create-cluster –hsm-type hsm1.medium –subnet-ids <SUBNET ID 1>
  2. Launch your Amazon Elastic Compute Cloud (Amazon EC2) client (in the public subnet). You can follow the steps here to launch an EC2 Instance.
  3. Create the first HSM (in the private subnet). To do this, you will run the command below via CLI. You will want to replace the placeholder <CLUSTER ID> with the ID given from the ‘Create the cluster’ command above. You’ll replace <AVAILABILITY ZONE> with the AZ matching your private subnet. For example, us-east-1a.
    $ aws cloudhsmv2 create-hsm –cluster-id <CLUSTER ID> –availability-zone <AVAILABILITY ZONE>
  4. Initialize the cluster. Initializing your cluster requires creating a self-signed certificate and using that to sign the cluster’s Certificate Signing Request (CSR). You can view an example here of how to create and use a self-signed certificate. Once you have your certificate, you will run the command below to initialize the cluster with it. You will want to replace the placeholder <CLUSTER ID> with your cluster id from step 1.
    $ aws cloudhsmv2 initialize-cluster –cluster-id <CLUSTER ID> –signed-cert file://<CLUSTER ID>_CustomerHsmCertificate.crt –-trust-anchor file://customerCA.crt

    Note: Don’t forget to place a copy of the certificate used to sign your cluster’s CSR into the /opt/cloudhsm/etc directory to ensure a continued secure connection.

  5. Install the cloudhsm-client software. Once the Amazon EC2 client is launched, you’ll need to download and install the cloudhsm-client software. You can do this by running the command below from the CLI:
    wget https://s3.amazonaws.com/cloudhsmv2-software/CloudHsmClient/EL6/cloudhsm-client-latest.el6.x86_64.rpm

    Once downloaded, you’ll install by running this command:
    sudo yum install -y ./cloudhsm-client-latest.el6.x86_64.rpm

  6. The last step in initializing the cluster requires you to configure the cloudhsm-client to point to the ENI IP of your first HSM. You do this on your EC2 client by running this command:
    $ sudo /opt/cloudhsm/bin/configure -a <IP ADDRESS>
    Replace the <IP ADDRESS> placeholder with your HSM’s ENI IP. The cloudhsm-client comes pre-installed with a Python script called “configure” located in the /opt/cloudhsm/bin/ directory. This will update your /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg and
    /opt/cloudhsm/etc/cloudhsm_client.cfg files with your HSM’s IP address. This ensures your client can connect to your cluster.
  7. Activate the cluster. To activate, you must launch the cloudhsm-client by running this command, which logs you into the cluster:

    $ /opt/cloudhsm/bin/cloudhsm_mgmt_util /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg

    Then, you need to enable the secure communication by running this command:

    aws-cloudhsm>enable_e2e

    If you’ve placed the certificate in the correct directory, you should see a response like this on the command line:

    E2E enabled on server 0(server1)

    If you run the command listUsers you’ll see a PRECO user:

    
    aws-cloudhsm>listUsers
    Users on server 0(server1):
    Number of users found:2
    	User ID		User Type	User Name
    	     1		PRECO		admin
    	     2		AU		app_user
    
    

    Change the password for this user to complete the activation process. You do this by first logging in using the command below:

    aws-cloudhsm>loginHSM PRECO admin password

    Once logged in, change the password using this command:

    
    aws-cloudhsm>changePswd PRECO admin <NEW PASSWORD>
    ***************************CAUTION******************************
    This is a CRITICAL operation, should be done on all nodes in the
    cluster. Cav server does NOT synchronize these changes with the 
    nodes on which this operation is not executed or failed, please
    ensure this operation is executed on all nodes in the cluster.
    ****************************************************************
    
    Do you want to continue(y/n)?Y
    Changing password for admin(PRECO) on 1 nodes
    

    Once completed, log out using the command logout, then log back in with the new password, using the command loginHSM PRECO admin <NEW PASSWORD>.

    Doing this allows you to create the first crypto user (CU). You create the user by running the command:
    aws-cloudhsm>createUser <USERTYPE (ex: CO, CU)> <USERNAME> <PASSWORD>
    Replace the red values in this command. The <USERTYPE> can be a CO (crypto officer) or a CU (crypto user). You can find more information about usertypes here. You’ll replace the placeholders <USERNAME> <PASSWORD> with a real user and password combo. Crypto Users are permitted to create and share keys on the CloudHSM.

    Run the command quit to exit this tool.

Step 2: Trigger a backup of your cluster

To trigger a backup that will be copied to region 2 to create your new cluster, add an HSM to your cluster in region 1. You can do this via the console or CLI. The backup that is created will contain all users (COs, CUs, and appliance users), all key material on the HSMs, and the configurations and policies associated with them. The user portion is extremely important because keys can only be synced across clusters to the same user. Make a note of the backup ID because you will need it later. You can find this by logging into the AWS console and navigating to the CloudHSM console, then selecting Backups. There will be a list of backup IDs, cluster IDs, and creation times. Make sure to select the backup ID specifically created for the cross-region copy.

Step 3: Create a key on your cluster in Region 1

There are many ways to create a key. I’m using key_mgmt_util because it’s an easy and straightforward method using CLI commands instead of SDK libraries. Start by connecting to the EC2 client instance that you launched above and ensuring the cloudhsm-client is running. If you aren’t sure, run this command:

$ sudo start cloudhsm-client

Now, launch the key_mgmt_util by running this command:

$ /opt/cloudhsm/bin/key_mgmt_util

When you see the prompt, log in as a CU to create your key, replacing <USERNAME> and <PASSWORD> with an actual CU user’s username and password:

Command: loginHSM -u CU -s <USERNAME> -p <PASSWORD>

To create the key for this example, we’re going to use the key_mgmt_util to generate a symmetric key. Note the -nex parameter is what makes this key non-exportable. An example command is below:

Command: genSymKey -t 31 -s 32 -l aes256 -nex

In the above command:

  1. genSymKey creates the Symmetric key
  2. -t chooses the key type, which in this case is AES
  3. -s states the key size, which in this case is 32 bytes
  4. -l creates a label to easily recognize the key by
  5. -nex makes the key non-exportable

The HSM will return a key handle. This is used as an identifier to reference the key in future commands. Make a note of the key handle because you will need it later. Here’s an example of the full output in which you can see the key handle provided is 37:


Command:genSymKey -t 31 -s 32 -l aes256 -nex
	Cfm3GenerateSymmetricKey returned: 0x00 : HSM Return: SUCCESS
	Symmetric Key Created.   Key Handle: 37
	Cluster Error Status
	Node id 0 and err state 0x00000000 : HSM Return: SUCCESS

Step 4: Copy your backup from region 1 to region 2 and create a cluster from the backup

To copy your backup from region 1 to region 2, from your EC2 client you’ll need to run the command that appears after these important notes:

  • Make sure the proper permissions are applied for the IAM role or user configured for the CLI. You’ll want to be a CloudHSM administrator for these actions. The instructions here show you how to create an admin user for this process, and here is an example of the permissions policy:
    
    {
       "Version":"2018-06-12",
       "Statement":{
          "Effect":"Allow",
          "Action":[
             "cloudhsm:*",
             "ec2:CreateNetworkInterface",
             "ec2:DescribeNetworkInterfaces",
             "ec2:DescribeNetworkInterfaceAttribute",
             "ec2:DetachNetworkInterface",
             "ec2:DeleteNetworkInterface",
             "ec2:CreateSecurityGroup",
             "ec2:AuthorizeSecurityGroupIngress",
             "ec2:AuthorizeSecurityGroupEgress",
             "ec2:RevokeSecurityGroupEgress",
             "ec2:DescribeSecurityGroups",
             "ec2:DeleteSecurityGroup",
             "ec2:CreateTags",
             "ec2:DescribeVpcs",
             "ec2:DescribeSubnets",
             "iam:CreateServiceLinkedRole"
          ],
          "Resource":"*"
       }
    }
    
    

  • To copy the backup over, you need to know the destination region, the source cluster ID, and/or the source backup ID. You can find the source cluster ID and/or the source backup ID in the CloudHSM console.
  • If you use only the cluster ID, the most recent backup of the associated cluster will be chosen for copy. If you specify the backup ID, that associated backup will be copied. If you don’t know these IDs, run the describe-clusters or describe-backups commands.

Here’s the example command:


$ aws cloudhsmv2 copy-backup-to-region --destination-region <DESTINATION REGION> --cluster-id <CLUSTER ID> --backup-id <BACKUP ID>
{
    “DestinationBackup”: {
	“BackupId”: “backup-gdnekhcxf4n”,
	“CreateTimestamp”: 1531742400,
	“BackupState”: “CREATE_IN_PROGRESS”,
	“SourceCluster”: “cluster-kzlczlspnho”,
	“SourceBackup”: “backup-4kuraxsqetz”,
	“SourceRegion”: “us-east-1”
     }
}

Once the backup has been copied to region 2, you’ll see a new backup ID in your console. This is what you’ll use to create your new cluster. You can follow the steps here to create your new cluster from this backup. This cluster will launch already initialized for you, but it will still need HSMs added into it to complete the activation process. Make sure you copy over the cluster certificate from the original cluster to the new region. You can do this by opening two terminal sessions, one for each HSM. Open the certificate file on the HSM in cluster 1 and copy it. On the HSM in cluster 2, create a file and paste the certificate over. You can use any text editor you like to do this. This certificate is required to establish the encrypted connection between your client and HSM instances.

You should also make sure you’ve added the cloned cluster’s Security Group to your EC2 client instance to allow connectivity. You do this by selecting the Security Group for your EC2 client in the EC2 console, and selecting Add rules. You’ll add a rule allowing traffic, with the source being the Security Group ID of your cluster.

Finally, take note of the ENI IP for the HSM because you’ll need it later. You can find this in your CloudHSM Console by clicking on the cluster for more information.

Step 5: Create a new configuration file with one ENI IP from both clusters

To sync a key from a cluster in region 1 to a cluster in region 2, you must create a configuration file that contains at least one ENI IP of an HSM in both clusters. This is required to allow the cloudhsm-client to communicate with both clusters at the same time. This is where the masking key we mentioned earlier comes into play as the syncKey command uses that to copy keys between clusters. This is why the cluster in region 2 must be created from a backup of the cluster in region 1. For the new configuration file, I’m going to copy over the original file /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg to a new file. Name this SyncClusters.cfg. You’re going to edit this new configuration file to have the ENI IP of the HSM in the cluster of region 1 and the ENI IP of the HSM in the cluster of region 2. It should look something like this:


{
    "scard": {
        "certificate": "cert-sc",
        "enable": "no",
        "pkey": "pkey-sc",
        "port": 2225
    },
    "servers": [
        {
            "CAfile": "",
            "CApath": "/opt/cloudhsm/etc/certs",
            "certificate": "/opt/cloudhsm/etc/client.crt",
            "e2e_encryption": {
                "enable": "yes",
                "owner_cert_path": "/opt/cloudhsm/etc/customerCA.crt"
            },
            "enable": "yes",
            "hostname": "",
            "name": "",
            "pkey": "/opt/cloudhsm/etc/client.key",
            "port": 2225,
            "server_ssl": "yes",
            "ssl_ciphers": ""
        },
        {
            "CAfile": "",
            "CApath": "/opt/cloudhsm/etc/certs",
            "certificate": "/opt/cloudhsm/etc/client.crt",
            "e2e_encryption": {
                "enable": "yes",
                "owner_cert_path": "/opt/cloudhsm/etc/customerCA.crt"
            },
            "enable": "yes",
            "hostname": "",
            "name": "",
            "pkey": "/opt/cloudhsm/etc/client.key",
            "port": 2225,
            "server_ssl": "yes",
            "ssl_ciphers": ""
        }
    ]
}

To verify connectivity to both clusters, start the cloudhsm-client using the modified configuration file. The command will look similar to this:

$ /opt/cloudhsm/bin/cloudhsm_mgmt_util.cfg /opt/cloudhsm/etc/SyncClusters.cfg

After connection, you should see something similar to this, with one IP from cluster 1 and one IP from cluster 2:


Connecting to the server(s), it may take time
depending on the server(s) load, please wait...
Connecting to server '<CLUSTER-1-IP>': hostname '<CLUSTER-1-IP>', port 2225...
Connected to server '<CLUSTER-1-IP>': hostname '<CLUSTER-1-IP>', port 2225.
Connecting to server '<CLUSTER-2-IP>': hostname '<CLUSTER-2-IP>', port 2225...
Connected to server '<CLUSTER-2-IP>': hostname '<CLUSTER-2-IP>', port 2225.

If you run the command info server from the prompt, you’ll see a list of servers your client is connected to. Make note of these because they’ll be important when syncing your keys. Typically, you’ll see server 0 as your first HSM in cluster 1 and server 1 as your first HSM in cluster 2.

Step 6: Sync your key from the cluster in region 1 to the cluster in region 2

You’re ready to sync your keys. Make sure you’ve logged in as the Crypto Officer (CO) user. Only the CO user can perform management functions on the cluster (for example, syncing keys).

Note: These steps are all performed at the server prompt, not the aws-cloudhsm prompt.

First, run the command listUsers to get the user IDs of the user that created the keys. Here’s an example:


server0>listUsers
Users on server 0(<CLUSTER-A-IP>):
Number of users found:3

User Id    User Type	User Name     MofnPubKey     LoginFailureCnt	 2FA
1		CO   	admin         NO               0	 	 NO
2		AU   	app_user      NO               0	 	 NO
3		CU   	<USERNAME>    NO               0	 	 NO

Make note of the user ID because you’ll need it later; in this case, it’s 3. Now, you need to see the key handles that you want to sync. You either noted this from earlier, or you can find this by running the findAllKeys command with the parameter for user 3. Here’s an example:


server0>findAllKeys 3 0
Keys on server 0(<CLUSTER-1-IP<):
Number of keys found 1
number of keys matched from start index 0::1
37
findAllKeys success

In this case, the key handle I want to sync is 37. When running the command syncKey, you’ll input the key handle and the server you want to sync it to (the destination server). Here’s an example:

server0>syncKey 37 1

In this example, 37 is the key handle, and 1 is the destination HSM. You’ll run the exit command to back out to the cluster prompt, and from here you can run findAllKeys again, which should show the same key handle on both clusters.


aws-cloudhsm>findAllKeys 3 0
Keys on server 0(<CLUSTER-1-IP>):
Number of keys found 1
number of keys matched from start index 0::1
37
findAllKeys success on server 0(<CLUSTER-1-IP>)
Keys on server 0(<CLUSTER-2-IP>):
Number of keys found 1
number of keys matched from start index 0::1
37
findAllKeys success on server 0(<CLUSTER-2-IP>)

Repeat this process with all keys you want to sync between clusters.

Summary

I walked you through how to create a cluster, trigger a backup, copy that backup to a new region, launch a new cluster from that backup, and then sync keys across clusters. This will help reduce disaster recovery time, while helping to ensure that your keys are secure in multiple regions should a failure occur.

Remember to always manually update users across clusters after the initial backup copy and cluster creation because these aren’t automatic. You must also run the syncKey command on any keys created after this, as well.

You’re now set up for fault tolerance in your AWS CloudHSM environment.

If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread on the AWS CloudHSM forum.

Want more AWS Security news? Follow us on Twitter.

Author

Tracy Pierce

Tracy Pierce is a Senior Cloud Support Engineer at AWS. She enjoys the peculiar culture of Amazon and uses that to ensure every day is exciting for her fellow engineers and customers alike. Customer Obsession is her highest priority and she shows this by improving processes, documentation, and building tutorials. She has her AS in Computer Security & Forensics from SCTD, SSCP certification, AWS Developer Associate certification, and AWS Security Specialist certification. Outside of work, she enjoys time with friends, her Great Dane, and three cats. She keeps work interesting by drawing cartoon characters on the walls at request.

Changes to our sending review process

Post Syndicated from Dustin Taylor original https://aws.amazon.com/blogs/messaging-and-targeting/changes-to-our-sending-review-process/

We’re changing some of the language we use in our sending review process to make our communications clearer and more helpful.

If you’re not familiar with the sending review process, it refers to the actions that we take when there are issues with the email sent from an Amazon SES or Amazon Pinpoint account. Usually, these issues are a result of senders making honest mistakes. However, when email providers receive problematic email from a sender, they can’t tell if the sender made a mistake, or if they’re doing something malicious. If an email provider detects a problem that’s severe enough, they might block all incoming email from the sender’s IP address. If that happens, email sent from other senders who use the same IP address is blocked as well.

For this reason, we look for certain patterns and behaviors that could cause deliverability problems, and then work with our customers to help resolve the issues with the email sent from their accounts. We used to call this our enforcement process, but we now refer to it as our sending review process. This name is a much better description of the process (not to mention a bit friendlier).

You might notice some other changes as well. When the reputation metrics for an account (such as the account’s bounce or complaint rate) exceed certain levels, or another issue occurs that could impact the reputation of that account, we’ll monitor the email sending behaviors of that account for a certain period of time. During this time, we make a note of whether the problem gets better or worse. Previously, this period was called a probation period; we now call it a review period.

If an account is under review, but the sender isn’t able to correct the issue before the end of the review period, we’ll temporarily disable the account’s ability to send any more email. We take this action to protect the reputation of the sender, and to ensure that other customers can send email without experiencing deliverability issues. We used to call this a suspension, but that name seemed very permanent and punitive. We now refer to these events as sending pauses, because in the majority of cases, they’re temporary and reversible.

Finally, if a sender disagrees with our decision to place a review period or sending pause on their account, they can contact us to let us know why we made this decision in error. This used to be known as an appeal, but we now call it a review.

If we ever change the status of your account, such as by implementing a review period or sending pause, we’ll contact you by email at the address associated with your AWS account. We recommend that you make sure that we have the right email address. For information about changing the email address associated with your AWS account, see Managing an AWS Account in the AWS Billing and Cost Management User Guide.

In addition to sending you a notification by email, we’ll also update the reputation dashboard in the Amazon SES console to show the current status of your account. To learn more about the reputation dashboard, see Using the Reputation Dashboard in the Amazon SES Developer Guide.

Learn about AWS Services and Solutions – September AWS Online Tech Talks

Post Syndicated from Robin Park original https://aws.amazon.com/blogs/aws/learn-about-aws-services-and-solutions-september-aws-online-tech-talks/

AWS Online Tech Talks are live, online presentations that cover a broad range of topics at varying technical levels. Join us this month to learn about AWS services and solutions. We’ll have experts online to help answer any questions you may have.

Featured this month is our first ever fireside chat discussion. Join Debanjan Saha, General Manager of Amazon Aurora and Amazon RDS, to learn how customers are using our relational database services and leveraging database innovations.

Register today!

Note – All sessions are free and in Pacific Time.

Tech talks featured this month:

Compute

September 24, 2018 | 09:00 AM – 09:45 AM PT – Accelerating Product Development with HPC on AWS – Learn how you can accelerate product development by harnessing the power of high performance computing on AWS.

September 26, 2018 | 09:00 AM – 10:00 AM PT – Introducing New Amazon EC2 T3 Instances – General Purpose Burstable Instances – Learn about new Amazon EC2 T3 instance types and how they can be used for various use cases to lower infrastructure costs.

September 27, 2018 | 09:00 AM – 09:45 AM PT – Hybrid Cloud Customer Use Cases on AWS: Part 2 – Learn about popular hybrid cloud customer use cases on AWS.

Containers

September 19, 2018 | 11:00 AM – 11:45 AM PT – How Talroo Used AWS Fargate to Improve their Application Scaling – Learn how Talroo, a data-driven solution for talent and jobs, migrated their applications to AWS Fargate so they can run their application without worrying about managing infrastructure.

Data Lakes & Analytics

September 17, 2018 | 11:00 AM – 11:45 AM PT – Secure Your Amazon Elasticsearch Service Domain – Learn about the multi-level security controls provided by Amazon Elasticsearch Service (Amazon ES) and how to set the security for your Amazon ES domain to prevent unauthorized data access.

September 20, 2018 | 11:00 AM – 12:00 PM PT – New Innovations from Amazon Kinesis for Real-Time Analytics – Learn about the new innovations from Amazon Kinesis for real-time analytics.

Databases

September 17, 2018 | 01:00 PM – 02:00 PM PT – Applied Live Migration to DynamoDB from Cassandra – Learn how to migrate a live Cassandra-based application to DynamoDB.

September 18, 2018 | 11:00 AM – 11:45 AM PT – Scaling Your Redis Workloads with Redis Cluster – Learn how Redis cluster with Amazon ElastiCache provides scalability and availability for enterprise workloads.

**Featured: September 20, 2018 | 09:00 AM – 09:45 AM PT – Fireside Chat: Relational Database Innovation at AWS – Join our fireside chat with Debanjan Saha, GM, Amazon Aurora and Amazon RDS to learn how customers are using our relational database services and leveraging database innovations.

DevOps

September 19, 2018 | 09:00 AM – 10:00 AM PT – Serverless Application Debugging and Delivery – Learn how to bring traditional best practices to serverless application debugging and delivery.

Enterprise & Hybrid

September 26, 2018 | 11:00 AM – 12:00 PM PT – Transforming Product Development with the Cloud – Learn how to transform your development practices with the cloud.

September 27, 2018 | 11:00 AM – 12:00 PM PT – Fueling High Performance Computing (HPC) on AWS with GPUs – Learn how you can accelerate time-to-results for your HPC applications by harnessing the power of GPU-based compute instances on AWS.

IoT

September 24, 2018 | 01:00 PM – 01:45 PM PT – Manage Security of Your IoT Devices with AWS IoT Device Defender – Learn how AWS IoT Device Defender can help you manage the security of IoT devices.

September 26, 2018 | 01:00 PM – 02:00 PM PT – Over-the-Air Updates with Amazon FreeRTOS – Learn how to execute over-the-air updates on connected microcontroller-based devices with Amazon FreeRTOS.

Machine Learning

September 17, 2018 | 09:00 AM – 09:45 AM PT – Build Intelligent Applications with Machine Learning on AWS – Learn how to accelerate development of AI applications using machine learning on AWS.

September 18, 2018 | 09:00 AM – 09:45 AM PT – How to Integrate Natural Language Processing and Elasticsearch for Better Analytics – Learn how to process, analyze and visualize data by pairing Amazon Comprehend with Amazon Elasticsearch.

September 20, 2018 | 01:00 PM – 01:45 PM PT – Build, Train and Deploy Machine Learning Models on AWS with Amazon SageMaker – Dive deep into building, training, & deploying machine learning models quickly and easily using Amazon SageMaker.

Management Tools

September 19, 2018 | 01:00 PM – 02:00 PM PT – Automated Windows and Linux Patching – Learn how AWS Systems Manager can help reduce data breach risks across your environment through automated patching.

re:Invent

September 12, 2018 | 08:00 AM – 08:30 AM PT – Episode 5: Deep Dive with Our Community Heroes and Jeff Barr – Get the insider secrets with top recommendations and tips for re:Invent 2018 from AWS community experts.

Security, Identity, & Compliance

September 24, 2018 | 11:00 AM – 12:00 PM PT – Enhanced Security Analytics Using AWS WAF Full Logging – Learn how to use AWS WAF security incidence logs to detect threats.

September 27, 2018 | 01:00 PM – 02:00 PM PT – Threat Response Scenarios Using Amazon GuardDuty – Discover methods for operationalizing your threat detection using Amazon GuardDuty.

Serverless

September 18, 2018 | 01:00 PM – 02:00 PM PT – Best Practices for Building Enterprise Grade APIs with Amazon API Gateway – Learn best practices for building and operating enterprise-grade APIs with Amazon API Gateway.

Storage

September 25, 2018 | 09:00 AM – 10:00 AM PT – Ditch Your NAS! Move to Amazon EFS – Learn how to move your on-premises file storage to Amazon EFS.

September 25, 2018 | 11:00 AM – 12:00 PM PT – Deep Dive on Amazon Elastic File System (EFS): Scalable, Reliable, and Elastic File Storage for the AWS Cloud – Get live demos and learn tips & tricks for optimizing your file storage on EFS.

September 25, 2018 | 01:00 PM – 01:45 PM PT – Integrating File Services to Power Your Media & Entertainment Workloads – Learn how AWS file services deliver high performance shared file storage for media & entertainment workflows.

Autonomous drones (only slightly flammable)

Post Syndicated from Liz Upton original https://www.raspberrypi.org/blog/autonomous-drones-only-slightly-flammable/

I had an email a little while ago, which opened: “I don’t know if you remember me, but…”

As it happens, I remembered Andy Baker very well, in large part because an indoor autonomous drone demo he ran at a Raspberry Pi birthday party a couple of years ago ACTUALLY CAUGHT FIRE. Here’s a refresher.

Raspberry Pi Party Autonomous drone demo + fire

At the Raspberry Pi IV party and there is a great demo of an Autonomous drone which is very impressive with only using a Pi. However it caught on fire. But i believe it does actually work.

We’ve been very careful since then to make sure that speakers are always accompanied by a fire extinguisher.

I love stories like Andy’s. He started working with the Raspberry Pi shortly after our first release in 2012, and had absolutely no experience with drones or programming them; there’s nothing more interesting than watching someone go from a standing start to something really impressive. It’s been a couple of years since we were last in touch, but Andy mailed me last week to let me know he’s just completed his piDrone project, after years of development. I thought you’d like to hear about it too. Over to Andy!

Building an autonomous drone from scratch

I suffer from “terminal boredom syndrome”; I always need a challenging hobby to keep me sane. In 2012, the Raspberry Pi was launched just as my previous hobby had come to an end. After six months of playing (including a Raspberry Pi version of a BBC Micro Turtle robot I did at school 30+ years ago), I was looking for something really challenging. DIY drones were emerging, so I set out making one with a Raspberry Pi and Python, from absolute ignorance but loads of motivation.  Six years later, with only one fire (at the Raspberry Pi 4th Birthday Party, no less!), the job is done.

Here’s smaller Zoë, larger Hermione and their remote-controller, Ivy:

Zoë (as in “Ball”), the smallest drone, is based on a Pi ZeroW, supporting preset- and manual-flight controls. Hermione (as in “Granger”) is a Pi3 drone, supporting the above along with GPS and obstacle-avoidance.

Penelope (as in “Pitstop”), not shown above, is a B3+ with mix of the two above.

Development history

It probably took four years(!) to get the drone to simply hover stably for more than a few seconds. For example, the accelerometer (IMU) tells gravity and acceleration in 3D; and from sum math(s), angles, speed and distance. But IMU output is very noisy. It drifts with temperature, and because gravity is huge compared to the propeller changes, it doesn’t take long before the calculated speed and distance values drift significantly. It took a lot of time, experimentation and guesswork to get accelerometer, gyrometer, ground-facing LiDAR and a Raspberry Pi camera to work together to get a stable hover for minutes rather than seconds. And during that experimentation, there were plenty of crashes: replacement parts were needed many many times! However, with a sixty-second stable hover finally working, adding cool features like GPS tracking, object avoidance and human control were trivial in comparison.

GNSS waypoint tracked successfully!

See http://blog.pistuffing.co.uk/whoohoo/

Obstruction avoidance test 2 – PASSED!!!!!

Details at http://pidrone.io/posts/obstruction-avoidance-test-2-passed/

Human control (iPhone)

See http://pidrone.io/posts/human-i-am-human/

In passing, I’m a co-founder and assistant at the Cotswold Raspberry Jam (cotswoldjam.org). I’m hoping to take Zoë to the next event on September 15th – tickets are free – and there’s so much more learn, interact and play with beyond the piDrone.

Finally, a few years ago, my goal became getting the piDrone exploring a maze: all but minor tweaks are now in places. Sadly, piDrone battery power for exploring a large maze currently doesn’t exist. Perhaps my next project will be designing a nuclear-fusion battery pack?  Deuterium oxide (heavy water) is surprisingly cheap, it seems…

More resources

If you want to learn more, there’s years of development on Andy’s blog at http://pidrone.io, and he’s made considerable documentation available at GitHub if you want to explore things further after this blog post. Thanks Andy!

The post Autonomous drones (only slightly flammable) appeared first on Raspberry Pi.

Learn to write games for the BBC Micro with Eben

Post Syndicated from Eben Upton original https://www.raspberrypi.org/blog/learn-to-write-games-for-the-bbc-micro-with-eben/

Long-time fans of the Raspberry Pi will know that we were inspired to make a programmable computer for kids by our own experiences with a machine called the BBC Micro, which many of us learned with in the 1980s.

This post is the first of what’s going to be an irregular series where I’ll walk you through building the sort of game we used to play when we were kids. You’ll need a copy of BeebEm (scroll down for a Linux port if you’re using a Pi – but this tutorial can be carried out on a PC or Mac as well as on an original BBC Micro if you have access to one).

I’m going to be presenting the next game in this series, tentatively titled Eben Goes Skiing, at the Centre for Computing History in Cambridge at 2pm this afternoon – head on down if you’d like to learn how to make scrolling ascii moguls.

Helicopter tutorial

We’re going to build a simple helicopter game in BBC BASIC. This will demonstrate a number of neat features, including user-defined characters, non-blocking keyboard input using INKEY, and positioning text and graphics using PRINT TAB.

Let’s start with user-defined characters. These provide us with an easy way to create a monochrome 8×8-pixel image by typing in 8 small numbers. As an example, let’s look at our helicopter sprite:

Each column pixel position in a row is “worth” a different power of 2, from 1 for the rightmost pixel up to 128 for the leftmost. To generate our 8 numbers, we process one row at a time, adding up the value for each occupied pixel position. We can now create custom character number 226 using the VDU 23 command. To display the character, we change to a graphics mode using the MODE command and display it using the PRINT command.

Type the following:

10MODE 2

70VDU 23,226,0,248,32,116,126,116,112,0

RUN

PRINT CHR$(226)

You should see the little helicopter on the screen just above your prompt. Let’s define some more characters for our game, with character numbers 224 through 229. These represent leftward and rightward flying birds, a rightward flying helicopter, the surface of the sea, and a landing pad.

Type the following:

50VDU 23,224,0,14,12,104,16,28,8,0

60VDU 23,225,0,112,48,22,8,56,16,0

80VDU 23,227,0,31,4,46,126,46,14,0

90VDU 23,228,0,102,255,255,255,255,255,255

100VDU 23,229,255,255,0,0,0,0,0,0

Trying running your program and using print to view the new characters!

Now we’re ready to use our sea and platform characters to build the game world. Mode 2 on the BBC Micro has 20 character positions across, and 32 down. We’ll draw 20 copies of the sea character in row 30 (remember, rows and columns are numbered from zero) using a FOR loop and the PRINT TAB command, and pick a random position for the platform using the RND() function.

Type the following:

110FOR I%=0 TO 19

120PRINT TAB(I%,30) CHR$(228);

130NEXT

140P%=RND(20)-1

150PRINT TAB(P%,30) CHR$(229);

RUN

You should see something like this:

Don’t worry about that cursor and prompt: they won’t show up in the finished game.

It’s time to add the helicopter. We’ll create variables X% and Y% to hold the position of the helicopter, and Z% to tell us if it last moved left or right. We’ll initialise X% to a random position, Y% to the top of the screen, and Z% to zero, meaning “left”. We can use PRINT TAB again to draw the helicopter (either character 226 or 227 depending on Z%) at its current position. The whole thing is wrapped up in a REPEAT loop, which keeps executing until the helicopter reaches the ground (in row 29).

Type the following:

160X%=RND(20)-1:Y%=0:Z%=0

180REPEAT

260PRINT TAB(X%,Y%) CHR$(226+Z%);

290UNTIL Y%=29

RUN

You’ll see the helicopter sitting at the top of the screen.

We’re almost there: let’s give our helicopter the ability to move left, right and down. On each trip round the loop, we move down one row, and use the INKEY() function to read the Z and X keys on the keyboard. If Z is pressed, and we’re not already at the left of the
screen, we move one column left. If X is pressed, and we’re not already at the right of the screen, we move one column right.

Type the following:

210IF INKEY(-98) AND X%>0 THEN X%=X%-1:Z%=0

220IF INKEY(-67) AND X%<19 THEN X%=X%+1:Z%=1

230Y%=Y%+1

RUN

You should see something like this:

The game is much, much too fast to control, and the helicopter leaves trails: not surprising, as we didn’t do anything to erase the previous frame. Let’s use PRINT TAB to place a “space” character over the previous position of the helicopter, and add an empty FOR loop to slow things down a bit.

Type the following:

190PRINT TAB (%,Y%)"";

280FOR I%=1 TO 200:NEXT

RUN

Much better! This is starting to feel like a real game. Let’s finish it off by:

  • Adding a bird that flies back and forth
  • Detecting whether you hit the pad or not
  • Getting rid of the annoying cursor using a “magic” VDU 23 command
  • Putting an outer loop in to let you play again

Type the following:

20REPEAT

30CLS

40VDU 23,1,0;0;0;0;

170A%=RND(18):B%=10:C%=RND(2)-1

200PRINT TAB(A%,B%) "";

240A%=A%+2*C%-1

250IF A%=0 OR A%=19 THEN C%=1-C%

270PRINT TAB(A%,B%) CHR$(224+C%);

300IF X%=P% PRINT TAB(6,15) "YOU WIN" ELSE PRINT TAB(6,15) "YOU
LOSE"

310PRINT TAB(4,16) "PRESS SPACE"

320REPEAT UNTIL INKEY(-99)

330UNTIL FALSE

RUN

And here it is in all its glory.

You might want to try adding some features to the game: collision with the bird, things to collect, vertical scrolling. The sky’s the limit!

I created a full version of the game, using graphics from our very own Sam Alder, for the Hackaday 1K challenge; you can find it here.

Appendix

Here’s the full source for the game in one block. If you get errors when you run your code, type:

MODE 0
LIST

And compare the output very carefully with what you see here.

10MODE 2
20REPEAT
30CLS
40VDU 23,1,0;0;0;0;
50VDU 23,224,0,14,12,104,16,28,8,0   
60VDU 23,225,0,112,48,22,8,56,16,0
70VDU 23,226,0,248,32,116,126,116,112,0
80VDU 23,227,0,31,4,46,126,46,14,0
90VDU 23,228,0,102,255,255,255,255,255,255
100VDU 23,229,255,255,0,0,0,0,0,0
110FOR I%=0 TO 19
120PRINT TAB(I%,30) CHR$(228);
130NEXT
140P%=RND(20)-1
150PRINT TAB(P%,30) CHR$(229);
160X%=RND(20)-1:Y%=0:Z%=0
170A%=RND(18):B%=10:C%=RND(2)-1
180REPEAT
190PRINT TAB(X%,Y%) " ";
200PRINT TAB(A%,B%) " ";  
210IF INKEY(-98) AND X%>0 THEN X%=X%-1:Z%=0  
220IF INKEY(-67) AND X%<19 THEN X%=X%+1:Z%=1
230Y%=Y%+1
240A%=A%+2*C%-1
250IF A%=0 OR A%=19 THEN C%=1-C%
260PRINT TAB(X%,Y%) CHR$(226+Z%);
270PRINT TAB(A%,B%) CHR$(224+C%);
280FOR I%=1 TO 200:NEXT
290UNTIL Y%=29
300IF X%=P% PRINT TAB(6,15) "YOU WIN" ELSE PRINT TAB(6,15) "YOU LOSE"
310PRINT TAB(4,16) "PRESS SPACE"
320REPEAT UNTIL INKEY(-99)
330UNTIL FALSE


The post Learn to write games for the BBC Micro with Eben appeared first on Raspberry Pi.

How to use AWS Secrets Manager to rotate credentials for all Amazon RDS database types, including Oracle

Post Syndicated from Apurv Awasthi original https://aws.amazon.com/blogs/security/how-to-use-aws-secrets-manager-rotate-credentials-amazon-rds-database-types-oracle/

You can now use AWS Secrets Manager to rotate credentials for Oracle, Microsoft SQL Server, or MariaDB databases hosted on Amazon Relational Database Service (Amazon RDS) automatically. Previously, I showed how to rotate credentials for a MySQL database hosted on Amazon RDS automatically with AWS Secrets Manager. With today’s launch, you can use Secrets Manager to automatically rotate credentials for all types of databases hosted on Amazon RDS.

In this post, I review the key features of Secrets Manager. You’ll then learn:

  1. How to store the database credential for the superuser of an Oracle database hosted on Amazon RDS
  2. How to store the Oracle database credential used by an application
  3. How to configure Secrets Manager to rotate both Oracle credentials automatically on a schedule that you define

Key features of Secrets Manager

AWS Secrets Manager makes it easier to rotate, manage, and retrieve database credentials, API keys, and other secrets throughout their lifecycle. The key features of this service include the ability to:

  1. Secure and manage secrets centrally. You can store, view, and manage all your secrets centrally. By default, Secrets Manager encrypts these secrets with encryption keys that you own and control. You can use fine-grained IAM policies or resource-based policies to control access to your secrets. You can also tag secrets to help you discover, organize, and control access to secrets used throughout your organization.
  2. Rotate secrets safely. You can configure Secrets Manager to rotate secrets automatically without disrupting your applications. Secrets Manager offers built-in integrations for rotating credentials for all Amazon RDS databases (MySQL, PostgreSQL, Oracle, Microsoft SQL Server, MariaDB, and Amazon Aurora.) You can also extend Secrets Manager to meet your custom rotation requirements by creating an AWS Lambda function to rotate other types of secrets.
  3. Transmit securely. Secrets are transmitted securely over Transport Layer Security (TLS) protocol 1.2. You can also use Secrets Manager with Amazon Virtual Private Cloud (Amazon VPC) endpoints powered by AWS Privatelink to keep this communication within the AWS network and help meet your compliance and regulatory requirements to limit public internet connectivity.
  4. Pay as you go. Pay for the secrets you store in Secrets Manager and for the use of these secrets; there are no long-term contracts, licensing fees, or infrastructure and personnel costs. For example, a typical production-scale web application will generate an estimated monthly bill of $6. If you follow along the instructions in this blog post, your estimated monthly bill for Secrets Manager will be $1. Note: you may incur additional charges for using Amazon RDS and Amazon Lambda, if you’ve already consumed the free tier for these services.

Now that you’re familiar with Secrets Manager features, I’ll show you how to store and automatically rotate credentials for an Oracle database hosted on Amazon RDS. I divided these instructions into three phases:

  1. Phase 1: Store and configure rotation for the superuser credential
  2. Phase 2: Store and configure rotation for the application credential
  3. Phase 3: Retrieve the credential from Secrets Manager programmatically

Prerequisites

To follow along, your AWS Identity and Access Management (IAM) principal (user or role) requires the SecretsManagerReadWrite AWS managed policy to store the secrets. Your principal also requires the IAMFullAccess AWS managed policy to create and configure permissions for the IAM role used by Lambda for executing rotations. You can use IAM permissions boundaries to grant an employee the ability to configure rotation without also granting them full administrative access to your account.

Phase 1: Store and configure rotation for the superuser credential

From the Secrets Manager console, on the right side, select Store a new secret.

Since I’m storing credentials for database hosted on Amazon RDS, I select Credentials for RDS database. Next, I input the user name and password for the superuser. I start by securing the superuser because it’s the most powerful database credential and has full access to the database.
 

Figure 1: For "Select secret type," choose "Credentials for RDS database"

Figure 1: For “Select secret type,” choose “Credentials for RDS database”

For this example, I choose to use the default encryption settings. Secrets Manager will encrypt this secret using the Secrets Manager DefaultEncryptionKey in this account. Alternatively, I can choose to encrypt using a customer master key (CMK) that I have stored in AWS Key Management Service (AWS KMS). To learn more, read the Using Your AWS KMS CMK documentation.
 

Figure 2: Choose either DefaultEncryptionKey or use a CMK

Figure 2: Choose either DefaultEncryptionKey or use a CMK

Next, I view the list of Amazon RDS instances in my account and select the database this credential accesses. For this example, I select the DB instance oracle-rds-database from the list, and then I select Next.

I then specify values for Secret name and Description. For this example, I use Database/Development/Oracle-Superuser as the name and enter a description of this secret, and then select Next.
 

Figure 3: Provide values for "Secret name" and "Description"

Figure 3: Provide values for “Secret name” and “Description”

Since this database is not yet being used, I choose to enable rotation. To do so, I select Enable automatic rotation, and then set the rotation interval to 60 days. Remember, if this database credential is currently being used, first update the application (see phase 3) to use Secrets Manager APIs to retrieve secrets before enabling rotation.
 

Figure 4: Select "Enable automatic rotation"

Figure 4: Select “Enable automatic rotation”

Next, Secrets Manager requires permissions to rotate this secret on my behalf. Because I’m storing the credentials for the superuser, Secrets Manager can use this credential to perform rotations. Therefore, on the same screen, I select Use a secret that I have previously stored in AWS Secrets Manager, and then select Next.

Finally, I review the information on the next screen. Everything looks correct, so I select Store. I have now successfully stored a secret in Secrets Manager.

Note: Secrets Manager will now create a Lambda function in the same VPC as my Oracle database and trigger this function periodically to change the password for the superuser. I can view the name of the Lambda function on the Rotation configuration section of the Secret Details page.

The banner on the next screen confirms that I’ve successfully configured rotation and the first rotation is in progress, which enables me to verify that rotation is functioning as expected. Secrets Manager will rotate this credential automatically every 60 days.
 

Figure 5: The confirmation notification

Figure 5: The confirmation notification

Phase 2: Store and configure rotation for the application credential

The superuser is a powerful credential that should be used only for administrative tasks. To enable your applications to access a database, create a unique database credential per application and grant these credentials limited permissions. You can use these database credentials to read or write to database tables required by the application. As a security best practice, deny the ability to perform management actions, such as creating new credentials.

In this phase, I will store the credential that my application will use to connect to the Oracle database. To get started, from the Secrets Manager console, on the right side, select Store a new secret.

Next, I select Credentials for RDS database, and input the user name and password for the application credential.

I continue to use the default encryption key. I select the DB instance oracle-rds-database, and then select Next.

I specify values for Secret Name and Description. For this example, I use Database/Development/Oracle-Application-User as the name and enter a description of this secret, and then select Next.

I now configure rotation. Once again, since my application is not using this database credential yet, I’ll configure rotation as part of storing this secret. I select Enable automatic rotation, and set the rotation interval to 60 days.

Next, Secrets Manager requires permissions to rotate this secret on behalf of my application. Earlier in the post, I mentioned that applications credentials have limited permissions and are unable to change their password. Therefore, I will use the superuser credential, Database/Development/Oracle-Superuser, that I stored in Phase 1 to rotate the application credential. With this configuration, Secrets Manager creates a clone application user.
 

Figure 6: Select the superuser credential

Figure 6: Select the superuser credential

Note: Creating a clone application user is the preferred mechanism of rotation because the old version of the secret continues to operate and handle service requests while the new version is prepared and tested. There’s no application downtime while changing between versions.

I review the information on the next screen. Everything looks correct, so I select Store. I have now successfully stored the application credential in Secrets Manager.

As mentioned in Phase 1, AWS Secrets Manager creates a Lambda function in the same VPC as the database and then triggers this function periodically to rotate the secret. Since I chose to use the existing superuser secret to rotate the application secret, I will grant the rotation Lambda function permissions to retrieve the superuser secret. To grant this permission, I first select role from the confirmation banner.
 

Figure 7: Select the "role" link that's in the confirmation notification

Figure 7: Select the “role” link that’s in the confirmation notification

Next, in the Permissions tab, I select SecretsManagerRDSMySQLRotationMultiUserRolePolicy0. Then I select Edit policy.
 

Figure 8: Edit the policy on the "Permissions" tab

Figure 8: Edit the policy on the “Permissions” tab

In this step, I update the policy (see below) and select Review policy. When following along, remember to replace the placeholder ARN-OF-SUPERUSER-SECRET with the ARN of the secret you stored in Phase 1.


{
  "Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "ec2:CreateNetworkInterface",
			"ec2:DeleteNetworkInterface",
			"ec2:DescribeNetworkInterfaces",
			"ec2:DetachNetworkInterface"
		],
		"Resource": "*"
	},
	{
	    "Sid": "GrantPermissionToUse",
		"Effect": "Allow",
		"Action": [
            "secretsmanager:GetSecretValue"
        ],
		"Resource": "ARN-OF-SUPERUSER-SECRET"
	}
  ]
}

Here’s what it will look like:
 

Figure 9: Edit the policy

Figure 9: Edit the policy

Next, I select Save changes. I have now completed all the steps required to configure rotation for the application credential, Database/Development/Oracle-Application-User.

Phase 3: Retrieve the credential from Secrets Manager programmatically

Now that I have stored the secret in Secrets Manager, I add code to my application to retrieve the database credential from Secrets Manager. I use the sample code from Phase 2 above. This code sets up the client and retrieves and decrypts the secret Database/Development/Oracle-Application-User.

Remember, applications require permissions to retrieve the secret, Database/Development/Oracle-Application-User, from Secrets Manager. My application runs on Amazon EC2 and uses an IAM role to obtain access to AWS services. I attach the following policy to my IAM role. This policy uses the GetSecretValue action to grant my application permissions to read secret from Secrets Manager. This policy also uses the resource element to limit my application to read only the Database/Development/Oracle-Application-User secret from Secrets Manager. You can refer to the Secrets Manager Documentation to understand the minimum IAM permissions required to retrieve a secret.


{
 "Version": "2012-10-17",
 "Statement": {
    "Sid": "RetrieveDbCredentialFromSecretsManager",
    "Effect": "Allow",
    "Action": "secretsmanager:GetSecretValue",
    "Resource": "arn:aws:secretsmanager:<AWS-REGION>:<ACCOUNT-NUMBER>:secret: Database/Development/Oracle-Application-User     
 }
}

In the above policy, remember to replace the placeholder <AWS-REGION> with the AWS region that you’re using and the placeholder <ACCOUNT-NUMBER> with the number of your AWS account.

Summary

I explained the key benefits of Secrets Manager as they relate to RDS and showed you how to help meet your compliance requirements by configuring Secrets Manager to rotate database credentials automatically on your behalf. Secrets Manager helps you protect access to your applications, services, and IT resources without the upfront investment and on-going maintenance costs of operating your own secrets management infrastructure. To get started, visit the Secrets Manager console. To learn more, visit Secrets Manager documentation.

If you have comments about this post, submit them in the Comments section below. If you have questions about anything in this post, start a new thread on the Secrets Manager forum.

Want more AWS Security news? Follow us on Twitter.

Apurv Awasthi

Apurv is the product manager for credentials management services at AWS, including AWS Secrets Manager and IAM Roles. He enjoys the “Day 1” culture at Amazon because it aligns with his experience building startups in the sports and recruiting industries. Outside of work, Apurv enjoys hiking. He holds an MBA from UCLA and an MS in computer science from University of Kentucky.

Helen’s hoglet: an adorable adventure

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/helens-hoglet-an-adorable-adventure/

Today is a bank holiday here in England, as well as for lucky people in Wales and Northern Ireland. Pi Towers UK is running on a skeleton crew of Babbage Bear, several automated Raspberry Pis, and Noel Fielding, who lives behind the red door we never open.

So, as a gift for you all while we’re busy doing bank holiday things, here’s a video that Helen Lynn just recorded of one of the baby hedgehogs who live in her garden.

Helen’s hoglet

Uploaded by Raspberry Pi on 2018-08-24.

You’re welcome. See you tomorrow!

The post Helen’s hoglet: an adorable adventure appeared first on Raspberry Pi.