Tag Archives: AWS STS

How to Eliminate the Need for Hardcoded AWS Credentials in Devices by Using the AWS IoT Credentials Provider

2018-04-30 Ramkishore Bhattacharyya

Post Syndicated from Ramkishore Bhattacharyya original https://aws.amazon.com/blogs/security/how-to-eliminate-the-need-for-hardcoded-aws-credentials-in-devices-by-using-the-aws-iot-credentials-provider/

The Internet of Things (IoT) has precipitated to an influx of connected devices and data that can be mined to gain useful business insights. If you own an IoT device, you might want the data to be uploaded seamlessly from your connected devices to the cloud so that you can make use of cloud storage and the processing power to perform sophisticated analysis of data. To upload the data to the AWS Cloud, devices must pass authentication and authorization checks performed by the respective AWS services. The standard way of authenticating AWS requests is the Signature Version 4 algorithm that requires the caller to have an access key ID and secret access key. Consequently, you need to hardcode the access key ID and the secret access key on your devices. Alternatively, you can use the built-in X.509 certificate as the unique device identity to authenticate AWS requests.

AWS IoT has introduced the credentials provider feature that allows a caller to authenticate AWS requests by having an X.509 certificate. The credentials provider authenticates a caller using an X.509 certificate, and vends a temporary, limited-privilege security token. The token can be used to sign and authenticate any AWS request. Thus, the credentials provider relieves you from having to manage and periodically refresh the access key ID and secret access key remotely on your devices.

In the process of retrieving a security token, you use AWS IoT to create a thing (a representation of a specific device or logical entity), register a certificate, and create AWS IoT policies. You also configure an AWS Identity and Access Management (IAM) role and attach appropriate IAM policies to the role so that the credentials provider can assume the role on your behalf. You also make an HTTP-over-Transport Layer Security (TLS) mutual authentication request to the credentials provider that uses your preconfigured thing, certificate, policies, and IAM role to authenticate and authorize the request, and obtain a security token on your behalf. You can then use the token to sign any AWS request using Signature Version 4.

In this blog post, I explain the AWS IoT credentials provider design and then demonstrate the end-to-end process of retrieving a security token from AWS IoT and using the token to write a temperature and humidity record to a specific Amazon DynamoDB table.

Note: This post assumes you are familiar with AWS IoT and IAM to perform steps using the AWS CLI and OpenSSL. Make sure you are running the latest version of the AWS CLI.

Overview of the credentials provider workflow

The following numbered diagram illustrates the credentials provider workflow. The diagram is followed by explanations of the steps.

To explain the steps of the workflow as illustrated in the preceding diagram:

The AWS IoT device uses the AWS SDK or custom client to make an HTTPS request to the credentials provider for a security token. The request includes the device X.509 certificate for authentication.
The credentials provider forwards the request to the AWS IoT authentication and authorization module to verify the certificate and the permission to request the security token.
If the certificate is valid and has permission to request a security token, the AWS IoT authentication and authorization module returns success. Otherwise, it returns failure, which goes back to the device with the appropriate exception.
On successful validation, the credentials provider invokes the AWS Security Token Service (AWS STS) to assume the preconfigured IAM role.
If assuming the role succeeds, AWS STS returns a temporary, limited-privilege security token to the credentials provider.
The credentials provider returns the security token to the device.
The AWS SDK on the device uses the security token to sign an AWS request with AWS Signature Version 4.
The requested service invokes IAM to validate the signature and authorize the request against access policies attached to the preconfigured IAM role.
If IAM validates the signature successfully and authorizes the request, the request goes through.

In another solution, you could configure an AWS Lambda rule that ingests your device data and sends it to another AWS service. However, in applications that require the uploading of large files such as videos or aggregated telemetry to the AWS Cloud, you may want your devices to be able to authenticate and send data directly to the AWS service of your choice. The credentials provider enables you to do that.

Outline of the steps to retrieve and use security token

Perform the following steps as part of this solution:

Create an AWS IoT thing: Start by creating a thing that corresponds to your home thermostat in the AWS IoT thing registry database. This allows you to authenticate the request as a thing and use thing attributes as policy variables in AWS IoT and IAM policies.
Register a certificate: Create and register a certificate with AWS IoT, and attach it to the thing for successful device authentication.
Create and configure an IAM role: Create an IAM role to be assumed by the service on behalf of your device. I illustrate how to configure a trust policy and an access policy so that AWS IoT has permission to assume the role, and the token has necessary permission to make requests to DynamoDB.
Create a role alias: Create a role alias in AWS IoT. A role alias is an alternate data model pointing to an IAM role. The credentials provider request must include a role alias name to indicate which IAM role to assume for obtaining a security token from AWS STS. You may update the role alias on the server to point to a different IAM role and thus make your device obtain a security token with different permissions.
Attach a policy: Create an authorization policy with AWS IoT and attach it to the certificate to control which device can assume which role aliases.
Request a security token: Make an HTTPS request to the credentials provider and retrieve a security token and use it to sign a DynamoDB request with Signature Version 4.
Use the security token to sign a request: Use the retrieved token to sign a request to DynamoDB and successfully write a temperature and humidity record from your home thermostat in a specific table. Thus, starting with an X.509 certificate on your home thermostat, you can successfully upload your thermostat record to DynamoDB and use it for further analysis. Before the availability of the credentials provider, you could not do this.

Deploy the solution

1. Create an AWS IoT thing

Register your home thermostat in the AWS IoT thing registry database by creating a thing type and a thing. You can use the AWS CLI with the following command to create a thing type. The thing type allows you to store description and configuration information that is common to a set of things.

aws iot create-thing-type --thing-type-name Thermostat

The following is sample output of the create-thing-type command. It contains the thingTypeName, thingTypeId, and thingTypeArn.

{
    "thingTypeName": "Thermostat", 
    "thingTypeId": "11f6d708-919e-479d-8a37-790ce48204d8",
    "thingTypeArn": "arn:aws:iot:us-east-1:<your_aws_account_id>:thingtype/Thermostat"
}

Run the following command in the AWS CLI to create a thing.

aws iot create-thing --thing-name MyHomeThermostat --thing-type-name Thermostat --attribute-payload "{\"attributes\": {\"Owner\":\"Alice\"}}"

The following is sample output of the create-thing command. It contains the thingArn, thingName, and thingId.

{
    "thingArn": "arn:aws:iot:us-east-1:<your_aws_account_id>:thing/MyHomeThermostat", 
    "thingName": "MyHomeThermostat",
    "thingId": "a9b098ac-2ee9-4ba6-aa40-163113eb18d0"
}

2. Register a certificate

Now, you need to have a Certificate Authority (CA) certificate, sign a device certificate using the CA certificate, and register both certificates with AWS IoT before your device can authenticate to AWS IoT. If you do not already have a CA certificate, you can use OpenSSL to create a CA certificate, as described in Use Your Own Certificate. To register your CA certificate with AWS IoT, follow the steps on Registering Your CA Certificate.

You then have to create a device certificate signed by the CA certificate and register it with AWS IoT, which you can do by following the steps on Creating a Device Certificate Using Your CA Certificate. Save the certificate and the corresponding key pair; you will use them when you request a security token later. Also, remember the password you provide when you create the certificate.

Run the following command in the AWS CLI to attach the device certificate to your thing so that you can use thing attributes in policy variables.

aws iot attach-thing-principal --thing-name MyHomeThermostat --principal <certificate-arn>

If the attach-thing-principal command succeeds, the output is empty.

3. Configure an IAM role

Next, configure an IAM role in your AWS account that will be assumed by the credentials provider on behalf of your device. You are required to associate two policies with the role: a trust policy that controls who can assume the role, and an access policy that controls which actions can be performed on which resources by assuming the role.

The following trust policy grants the credentials provider permission to assume the role. Put it in a text document and save the document with the name, trustpolicyforiot.json.

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Principal": {"Service": "credentials.iot.amazonaws.com"},
    "Action": "sts:AssumeRole"
  }
}

Run the following command in the AWS CLI to create an IAM role with the preceding trust policy.

aws iam create-role --role-name dynamodb-access-role --assume-role-policy-document file://trustpolicyforiot.json

The following is sample output of the create-role command.

{
    "Role": {
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17", 
            "Statement": {
                "Action": "sts:AssumeRole", 
                "Effect": "Allow", 
                "Principal": {
                    "Service": "credentials.iot.amazonaws.com"
                }
            }
        }, 
        "RoleId": "AROAJWE6XX2DBF3PMINT6", 
        "CreateDate": "2018-01-18T08:40:03.788Z", 
        "RoleName": "dynamodb-access-role", 
        "Path": "/", 
        "Arn": "arn:aws:iam::<your_aws_account_id>:role/dynamodb-access-role"
    }
}

The following access policy allows DynamoDB operations on the table that has the same name as the thing name that you created in Step 1, MyHomeThermostat, by using credentials-iot:ThingName as a policy variable. I explain after Step 5 about using thing attributes as policy variables. Put the following policy in a text document and save the document with the name, accesspolicyfordynamodb.json.

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Action": [
      "dynamodb:GetItem",
      "dynamodb:BatchGetItem",
      "dynamodb:PutItem",
      "dynamodb:UpdateItem",
      "dynamodb:DeleteItem"
    ],
    "Resource": "arn:aws:dynamodb:us-east-1:<your_aws_account_id>:table/${credentials-iot:ThingName}"
  }
}

Run the following command in the AWS CLI to create the access policy.

aws iam create-policy --policy-name accesspolicyfordynamodb --policy-document file://accesspolicyfordynamodb.json

The following is sample output of the create-policy command.

{
    "Policy": {
        "PolicyName": "accesspolicyfordynamodb", 
        "CreateDate": "2018-01-18T08:47:38.368Z", 
        "AttachmentCount": 0, 
        "IsAttachable": true, 
        "PolicyId": "ANPAJMRTTZH25SEHZOBYG", 
        "DefaultVersionId": "v1", 
        "Path": "/", 
        "Arn": "arn:aws:iam::<your_aws_account_id>:policy/accesspolicyfordynamodb", 
        "UpdateDate": "2018-01-18T08:47:38.368Z"
    }
}

Finally, run the following command in the AWS CLI to attach the access policy to your role.

aws iam attach-role-policy --role-name dynamodb-access-role --policy-arn arn:aws:iam::<your_aws_account_id>:policy/accesspolicyfordynamodb

If the attach-role-policy command succeeds, the output is empty.

Configure the PassRole permissions

The IAM role that you have created must be passed to AWS IoT to create a role alias, as described in Step 4. The user who performs the operation requires iam:PassRole permission to authorize this action. You also should add permission for the iam:GetRole action to allow the user to retrieve information about the specified role. Create the following policy to grant iam:PassRole and iam:GetRole permissions. Name this policy, passrolepermission.json.

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Action": [
        "iam:GetRole",
        "iam:PassRole"
    ],
    "Resource": "arn:aws:iam::<your_aws_account_id>:role/dynamodb-access-role"
  }
}

Run the following command in the AWS CLI to create the policy in your AWS account.

aws iam create-policy --policy-name passrolepermission --policy-document file://passrolepermission.json

The following is sample output of the create-policy command.

{
    "Policy": {
        "PolicyName": "passrolepermission", 
        "CreateDate": "2018-01-18T08:53:35.016Z", 
        "AttachmentCount": 0, 
        "IsAttachable": true, 
        "PolicyId": "ANPAJ6HSQYSBLAUCR5XRC", 
        "DefaultVersionId": "v1", 
        "Path": "/", 
        "Arn": "arn:aws:iam::<your_aws_account_id>:policy/passrolepermission", 
        "UpdateDate": "2018-01-18T08:53:35.016Z"
    }
}

Now, run the following command to attach the policy to the user.

aws iam attach-user-policy --policy-arn arn:aws:iam::<your_aws_account_id>:policy/passrolepermission --user-name <user_name>

If the attach-user-policy command succeeds, the output is empty.

4. Create a role alias

Now that you have configured the IAM role, you will create a role alias with AWS IoT. You must provide the following pieces of information when creating a role alias:

RoleAlias: This is the primary key of the role alias data model and hence a mandatory attribute. It is a string; the minimum length is 1 character, and the maximum length is 128 characters.
RoleArn: This is the Amazon Resource Name (ARN) of the IAM role you have created. This is also a mandatory attribute.
CredentialDurationSeconds: This is an optional attribute specifying the validity (in seconds) of the security token. The minimum value is 900 seconds (15 minutes), and the maximum value is 3,600 seconds (60 minutes); the default value is 3,600 seconds, if not specified.

Run the following command in the AWS CLI to create a role alias. Use the credentials of the user to whom you have given the iam:PassRole permission.

aws iot create-role-alias --role-alias Thermostat-dynamodb-access-role-alias --role-arn arn:aws:iam::<your_aws_account_id>:role/dynamodb-access-role --credential-duration-seconds 3600

The following is sample output of the create-role-alias command. It contains the roleAlias as specified in the request and the roleArn.

{
    "roleAlias": "Thermostat-dynamodb-access-role-alias",
    "roleAliasArn": "arn:aws:iot:us-east-1:<your_aws_account_id>:rolealias/Thermostat-dynamodb-access-role-alias"
}

5. Attach a policy

You created and registered a certificate with AWS IoT earlier for successful authentication of your device. Now, you need to create and attach a policy to the certificate to authorize the request for the security token.

Let’s say you want to allow a thing to get credentials for the role alias, Thermostat-dynamodb-access-role-alias, with thing owner Alice, thing type thermostat, and the thing attached to a principal. The following policy, with thing attributes as policy variables, achieves these requirements. After this step, I explain more about using thing attributes as policy variables. Put the policy in a text document, and save it with the name, alicethermostatpolicy.json.

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Action": "iot:AssumeRoleWithCertificate",
    "Resource": "arn:aws:iot:us-east-1:<your_aws_account_id>:rolealias/${iot:Connection.Thing.ThingTypeName}-dynamodb-access-role-alias",
    "Condition": {
      "StringEquals": {
        "iot:Connection.Thing.Attributes[Owner]": "Alice",
        "iot:Connection.Thing.ThingTypeName": "Thermostat"
      },
      "Bool": {
        "iot:Connection.Thing.IsAttached": "true"
      }
    }
  }
}

Run the following command in the AWS CLI to create the policy in your AWS IoT database.

aws iot create-policy --policy-name AliceThermostatPolicy --policy-document file://alicethermostatpolicy.json

The following is sample output of the create-policy command. It contains the policyName, policyArn, policyDocument, and policyVersionId.

{
    "policyName": "AliceThermostatPolicy", 
    "policyArn": "arn:aws:iot:us-east-1:<your_aws_account_id>:policy/AliceThermostatPolicy", 
    "policyDocument": "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": {\n    \"Effect\": \"Allow\",\n    \"Action\": \"iot:AssumeRoleWithCertificate\",\n    \"Resource\": \"arn:aws:iot:us-east-1:<your_aws_account_id>:rolealias/${iot:Connection.Thing.ThingTypeName}-dynamodb-access-role-alias\",\n    \"Condition\": {\n      \"StringEquals\": {\n        \"iot:Connection.Thing.Attributes[Owner]\": \"Alice\",\n        \"iot:Connection.Thing.ThingTypeName\": \"Thermostat\"\n      },\n      \"Bool\": {\n        \"iot:Connection.Thing.IsAttached\": \"true\"\n      }\n    }\n  }\n}\n", 
    "policyVersionId": "1"
}

Use the following command to attach the policy with the certificate you registered earlier.

aws iot attach-policy --policy-name AliceThermostatPolicy --target <certificate-arn>

If the attach-policy command succeeds, the output is empty.

You have completed all the necessary steps to request an AWS security token from the credentials provider!

Using thing attributes as policy variables

Before I show how to request a security token, I want to explain more about how to use thing attributes as policy variables and the advantage of using them. As a prerequisite, a device must provide a thing name in the credentials provider request.

Thing substitution variables in AWS IoT policies

AWS IoT Simplified Permission Management allows you to associate a connection with a specific thing, and allow the thing name, thing type, and other thing attributes to be available as substitution variables in AWS IoT policies. You can write a generic AWS IoT policy as in alicethermostatpolicy.json in Step 5, attach it to multiple certificates, and authorize the connection as a thing. For example, you could attach alicethermostatpolicy.json to certificates corresponding to each of the thermostats you have that you want to assume the role alias, Thermostat-dynamodb-access-role-alias, and allow operations only on the table with the name that matches the thing name. For more information, see the full list of thing policy variables.

Thing substitution variables in IAM policies

You also can use the following three substitution variables in the IAM role’s access policy (I used credentials-iot:ThingName in accesspolicyfordynamodb.json in Step 3):

credentials-iot:ThingName
credentials-iot:ThingTypeName
credentials-iot:AwsCertificateId

When the device provides the thing name in the request, the credentials provider fetches these three variables from the database and adds them as context variables to the security token. When the device uses the token to access DynamoDB, the variables in the role’s access policy are replaced with the corresponding values in the security token. Note that you also can use credentials-iot:AwsCertificateId as a policy variable; AWS IoT returns certificateId during registration.

6. Request a security token

Make an HTTPS request to the credentials provider to fetch a security token. You have to supply the following information:

Certificate and key pair: Because this is an HTTP request over TLS mutual authentication, you have to provide the certificate and the corresponding key pair to your client while making the request. Use the same certificate and key pair that you used during certificate registration with AWS IoT.
RoleAlias: Provide the role alias (in this example, Thermostat-dynamodb-access-role-alias) to be assumed in the request.
ThingName: Provide the thing name that you created earlier in the AWS IoT thing registry database. This is passed as a header with the name, x-amzn-iot-thingname. Note that the thing name is mandatory only if you have thing attributes as policy variables in AWS IoT or IAM policies.

Run the following command in the AWS CLI to obtain your AWS account-specific endpoint for the credentials provider. See the DescribeEndpoint API documentation for further details.

aws iot describe-endpoint --endpoint-type iot:CredentialProvider

The following is sample output of the describe-endpoint command. It contains the endpointAddress.

{
    "endpointAddress": "<your_aws_account_specific_prefix>.credentials.iot.us-east-1.amazonaws.com"
}

Note that if you are on Mac OS X, you need to export your certificate to a .pfx or .p12 file before you can pass it in the https request. Use OpenSSL with the following command to convert the device certificate from .pem to .pfx format. Remember the password because you will need it subsequently in a curl command.

openssl pkcs12 -export -out deviceCert.pfx -inkey <device-certificate-key-pair> -in <device-certificate-pem>

Now, make an HTTPS request to the credentials provider to fetch a security token. You may use your preferred HTTP client for the request. I use curl in the following examples.

curl --cert deviceCert.pfx --pass <certificate-password> --key <device-certificate-key-pair> -H "x-amzn-iot-thingname: MyHomeThermostat" https://<your_credentials_provider_endpoint>/role-aliases/Thermostat-dynamodb-access-role-alias/credentials

This command returns a security token object that has an accessKeyId, a secretAccessKey, a sessionToken, and an expiration. The following is sample output of the curl command.

{"credentials":{"accessKeyId":"ASIAJAKDHYFVFZCETC4A","secretAccessKey":"6RcojWDX0uWj3hRekTu/kOhXyyfqMU+T5U81LZzf","sessionToken":"AgoGb3JpZ2luEHIaCXVzLWVhc3QtMSKAAlEYX58Gy0MT+PLwB3SJohrUPVQP0Ai65XgwWy+k7j2knjGgvTc5XL/s5AEZsCHtCGdhXZrjk3SKnE2WG5+HQUz+H8qems7yuphjDsbRSVdBRLRDwu5FSlks2fMzSrjY8yrjjkmjixzClcWEE680+0vu5/Zmfb81uGarXvWQnibl+HNMPMePblcat/rgKvxwQujI5oQX+BEyKPmcKPi0u5WlWlOCEe2KsHA//Em6nAYIZjbLK9dWQ5MZrEgJ8wibbMsKfGu6gfqm1Qp/cqD1KFLHUNNlKHSQGD7GDsExV268FVltX+6mskAwRYS4wQFhIDPGglPc4+KbDFwUp2meJAwqiwMI+P//////////ARABGgw4ODA2ODQ1MzQxNDEiDIIRUr2U84+UW3syVyrfAult54skNMSlm2cEhNZnU6FtdFbcNvxP602OfnzF2ovtTanipMeP9fhO5Mcbnb/fuAK2HZ4W3MLVY+nzWGloDWrcrQ0tH1sxdeDp1fPJlwOg/vUIPZcLICGubsMxbPCLz5g6oX4C2fgJcxDiV3pHfCQR2wpvdfltct0eJVT+b9VFWtESsC+DCmoZGOf1QygT16UjLEDQiMwoH0a2KwItl338VbjJdTNTqx11Rzmm6PIHGu5J5efCmaN+ZUoDwbedTCdT1BBZpOihaEqi2tcsUx9BGx1jjkfyfjBO9XhJXgEpYHv7yBSJRkPJOXbcBlWcy02Pp8rQvdSu8c5nQvt+2pzZC8xAB5oFqb6OVJa1wa8/u4k82/jjjrxeka5kSkZQ751eykeWn/ZPo092E07ZDSCxqF1fEGBZ+ieTIfH3MqRcvhTnZADfdRj4v2unVPTXp7lH5cMQuSgqKC0hyR3pNDC6yKXSBQ==","expiration":"2018-01-18T09:18:06Z"}}

7. Use the security token to sign a request

Create a DynamoDB table called MyHomeThermostat in your AWS account. You will have to choose the hash (partition key) and the range (sort key) while creating the table to uniquely identify a record. Make the hash the serial_number of the thermostat and the range the timestamp of the record. Create a text file with the following JSON to put a temperature and humidity record in the table. Name the file, item.json.

{
  "serial_number": {"S": "123456789"},
  "timestamp": {"S": "2017-11-20T06:00:00.000Z"},
  "current_temp": {"N": "65"},
  "target_temp": {"N": "70"},
  "humidity": {"N": "45"}
}

You can use the accessKeyId, secretAccessKey, and sessionToken retrieved from the output of the curl command to sign a request that writes the temperature and humidity record to the DynamoDB table. Use the following commands to accomplish this.

export AWS_ACCESS_KEY_ID=<access-key-id>
export AWS_SECRET_ACCESS_KEY=<secret-access-key>
export AWS_SESSION_TOKEN=<session-token>
aws dynamodb put-item --table-name MyHomeThermostat --item file://item.json --return-consumed-capacity TOTAL

The following is sample output of the put-item command.

{
    "ConsumedCapacity": {
        "CapacityUnits": 1.0, 
        "TableName": "MyHomeThermostat"
    }
}

Conclusion

In this blog post, I demonstrated how to retrieve a security token by using an X.509 certificate and then writing an item to a DynamoDB table by using the security token. Similarly, you could run applications on surveillance cameras or sensor devices that exchange the X.509 certificate for an AWS security token and use the token to upload video streams to Amazon Kinesis or telemetry data to Amazon CloudWatch.

If you have comments about this blog post, submit them in the “Comments” section below. If you have questions about or issues implementing this solution, start a new thread on the AWS IoT forum.

– Ramkishore

Managing Cross-Account Serverless Microservices

2018-03-29 Chris Munns

Post Syndicated from Chris Munns original https://aws.amazon.com/blogs/compute/managing-cross-account-serverless-microservices/

This post courtesy of Michael Edge, Sr. Cloud Architect – AWS Professional Services

Applications built using a microservices architecture typically result in a number of independent, loosely coupled microservices communicating with each other, synchronously via their APIs and asynchronously via events. These microservices are often owned by different product teams, and these teams may segregate their resources into different AWS accounts for reasons that include security, billing, and resource isolation. This can sometimes result in the following challenges:

Cross-account deployment: A single pipeline must deploy a microservice into multiple accounts; for example, a microservice must be deployed to environments such as DEV, QA, and PROD, all in separate accounts.
Cross-account lookup: During deployment, a resource deployed into one AWS account may need to refer to a resource deployed in another AWS account.
Cross-account communication: Microservices executing in one AWS account may need to communicate with microservices executing in another AWS account.

In this post, I look at ways to address these challenges using a sample application composed of a web application supported by two serverless microservices. The microservices are owned by different product teams and deployed into different accounts using AWS CodePipeline, AWS CloudFormation, and the Serverless Application Model (SAM). At runtime, the microservices communicate using an event-driven architecture that requires asynchronous, cross-account communication via an Amazon Simple Notification Service (Amazon SNS) topic.

Sample application

First, look at the sample application I use to demonstrate these concepts. In the following overview diagram, you can see the following:

The entire application consists of three main services:
- A Booking microservice, owned by the Booking account.
- An Airmiles microservice, owned by the Airmiles account.
- A web application that uses the services exposed by both microservices, owned by the Web Channel account.
The Booking microservice creates flight bookings and publishes booking events to an SNS topic.
The Airmiles microservice consumes booking events from the SNS topic and uses the booking event to calculate the airmiles associated with the flight booking. It also supports querying airmiles for a specific flight booking.
The web application allows an end user to make flight bookings, view flights bookings, and view the airmiles associated with a flight booking.
In the sample application, the Booking and Airmiles microservices are implemented using AWS Lambda. Together with Amazon API Gateway, Amazon DynamoDB, and SNS, the sample application is completely serverless.

Sample Serverless microservices application

The typical booking flow would be triggered by an end user making a flight booking using the web application, which invokes the Booking microservice via its REST API. The Booking microservice persists the flight booking and publishes the booking event to an SNS topic to enable sharing of the booking with other interested consumers. In this sample application, the Airmiles microservice subscribes to the SNS topic and consumes the booking event, using the booking information to calculate the airmiles. In line with microservices best practices, both the Booking and Airmiles microservices store their information in their own DynamoDB tables, and expose an API (via API Gateway) that is used by the web application.

Setup

Before you delve into the details of the sample application, get the source code and deploy it.

Cross-account deployment of Lambda functions using CodePipeline has been previously discussed by my colleague Anuj Sharma in his post, Building a Secure Cross-Account Continuous Delivery Pipeline. This sample application builds upon the solution proposed by Anuj, using some of the same scripts and a similar account structure. To make it feasible for you to deploy the sample application, I’ve reduced the number of accounts needed down to three accounts by consolidating some of the services. In the following diagram, you can see the services used by the sample application require three accounts:

Tools: A central location for the continuous delivery/deployment services such as CodePipeline and AWS CodeBuild. To reduce the number of accounts required by the sample application, also deploy the AWS CodeCommit repositories here, though typically they may belong in a separate Dev account.
Booking: Account for the Booking microservice.
Airmiles: Account for the Airmiles microservice.

Without consolidation, the sample application may require up to 10 accounts: one for Tools, and three accounts each for Booking, Airmiles and Web Application (to support the DEV, QA, and PROD environments).

Account structure for sample application

To follow the rest of this post, clone the repository in step 1 below. To deploy the application on AWS, follow steps 2 and 3:

Clone this repository. It contains the AWS CloudFormation templates to use in this walkthrough.
```
git clone https://github.com/aws-samples/aws-cross-account-serverless-microservices.git
```
Install the AWS CLI. To prepare your access keys or a role to make calls to AWS, configure the AWS CLI settings.
Follow the instructions in the repository README to build the CodePipeline pipelines and deploy the microservices and web application.

Challenge 1: Cross-account deployment using CodePipeline

Though the Booking pipeline executes in the Tools account, it deploys the Booking Lambda functions into the Booking account.

In the sample application code repository, open the ToolsAcct/code-pipeline.yaml CloudFormation template.
Scroll down to the Pipeline resource and look for the DeployToTest pipeline stage (shown below). There are two AWS Identity and Access Management (IAM) service roles used in this stage that allow cross-account activity. Both of these roles exist in the Booking account:
- Under Actions.RoleArn, find the service role assumed by CodePipeline to execute this pipeline stage in the Booking account. The role referred to by the parameter NonProdCodePipelineActionServiceRole allows access to the CodePipeline artifacts in the S3 bucket in the Tools account, and also access to the AWS KMS key needed to encrypt/decrypt the artifacts.
- Under Actions.Configuration.RoleArn, find the service role assumed by CloudFormation when it carries out the CHANGE_SET_REPLACE action in the Booking account.

These roles are created in the CloudFormation template NonProdAccount/toolsacct-codepipeline-cloudformation-deployer.yaml.

- Name: DeployToTest
  Actions:
    - Name: CreateChangeSetTest
      ActionTypeId:
        Category: Deploy
        Owner: AWS
        Version: 1
        Provider: CloudFormation
      Configuration:
        ChangeSetName: !Join ['-',[!Ref ProjectName, 'lambda', 'CS']]
        ActionMode: CHANGE_SET_REPLACE
        StackName: !Join ['-',[!Ref ProjectName, 'lambda']]
        Capabilities: CAPABILITY_NAMED_IAM
        TemplatePath: BuildOutput::output-sam-template.yml
        TemplateConfiguration: BuildOutput::sam-config.json
        RoleArn: !Ref NonProdCloudFormationServiceRole
      InputArtifacts:
        - Name: BuildOutput
      RunOrder: 1
      RoleArn: !Ref NonProdCodePipelineActionServiceRole
    - Name: DeployChangeSetTest
      ActionTypeId:
        Category: Deploy
        Owner: AWS
        Version: 1
        Provider: CloudFormation
      Configuration:
        ChangeSetName: !Join ['-',[!Ref ProjectName, 'lambda', 'CS']]
        ActionMode: CHANGE_SET_EXECUTE
        StackName: !Join ['-',[!Ref ProjectName, 'lambda']]
      InputArtifacts:
        - Name: BuildOutput
      RunOrder: 2
      RoleArn: !Ref NonProdCodePipelineActionServiceRole

Challenge 2: Cross-account stack lookup using custom resources

Asynchronous communication is a fairly common pattern in microservices architectures. A publishing microservice publishes an event that consumers may be interested in, without any concern for who those consumers may be.

In the case of the sample application, the publisher is the Booking microservice, publishing a flight booking event onto an SNS topic that exists in the Booking account. The consumer is the Airmiles microservice in the Airmiles account. To enable the two microservices to communicate, the Airmiles microservice must look up the ARN of the Booking SNS topic at deployment time in order to set up a subscription to it.

To enable CloudFormation templates to be reused, you are not hardcoding resource names in the templates. Because you allow CloudFormation to generate a resource name for the Booking SNS topic, the Airmiles microservice CloudFormation template must look up the SNS topic name at stack creation time. It’s not possible to use cross-stack references, as these can’t be used across different accounts.

However, you can use a CloudFormation custom resource to achieve the same outcome. This is discussed in the next section. Using a custom resource to look up stack exports in another stack does not create a dependency between the two stacks, unlike cross-stack references. A stack dependency would prevent one stack being deleted if another stack depended upon it. For more information, see Fn::ImportValue.

Using a Lambda function as a custom resource to look up the booking SNS topic

The sample application uses a Lambda function as a custom resource. This approach is discussed in AWS Lambda-backed Custom Resources.
Walk through the sample application and see how the custom resource is used to list the stack export variables in another account and return these values to the calling AWS CloudFormation stack. Examine each of the following aspects of the custom resource:

Deploying the custom resource.
Custom resource IAM role.
A calling CloudFormation stack uses the custom resource.
The custom resource assumes a role in another account.
The custom resource obtains the stack exports from all stacks in the other account.
The custom resource returns the stack exports to the calling CloudFormation stack.
The custom resource handles all the event types sent by the calling CloudFormation stack.

Step1: Deploying the custom resource

The custom Lambda function is deployed using SAM, as are the Lambda functions for the Booking and Airmiles microservices. See the CustomLookupExports resource in Custom/custom-lookup-exports.yml.

Step2: Custom resource IAM role

The CustomLookupExports resource in Custom/custom-lookup-exports.yml executes using the CustomLookupLambdaRole IAM role. This role allows the custom Lambda function to assume a cross account role that is created along with the other cross account roles in the NonProdAccount/toolsacct-codepipeline-cloudformation-deployer.yml. See the resource CustomCrossAccountServiceRole.

Step3: The CloudFormation stack uses the custom resource

The Airmiles microservice is created by the Airmiles CloudFormation template Airmiles/sam-airmile.yml, a SAM template that uses the custom Lambda resource to look up the ARN of the Booking SNS topic. The custom resource is specified by the CUSTOMLOOKUP resource, and the PostAirmileFunction resource uses the custom resource to look up the ARN of the SNS topic and create a subscription to it.

Because the Airmiles Lambda function is going to subscribe to an SNS topic in another account, it must grant the SNS topic in the Booking account permissions to invoke the Airmiles Lambda function in the Airmiles account whenever a new event is published. Permissions are granted by the LambdaResourcePolicy resource.

Step4: The custom resource assumes a role in another account

When the Lambda custom resource is invoked by the Airmiles CloudFormation template, it must assume a role in the Booking account (see Step 2) in order to query the stack exports in that account.

This can be seen in Custom/custom-lookup-exports.py, where the AWS Simple Token Service (AWS STS) is used to obtain a temporary access key to allow access to resources in the account referred to by the environment variable: ‘CUSTOM_CROSS_ACCOUNT_ROLE_ARN’. This environment variable is defined in the Custom/custom-lookup-exports.yml CloudFormation template, and refers to the role created in Step 2.

sts = boto3.client('sts')
assumedRole = sts.assume_role(
    RoleArn=os.environ['CUSTOM_CROSS_ACCOUNT_ROLE_ARN'],
    RoleSessionName='LambdaCloudFormationSession'
)
credentials = assumedRole['Credentials']
accessKey = credentials['AccessKeyId']
secretAccessKey = credentials['SecretAccessKey']
sessionToken = credentials['SessionToken']
cfn = boto3.client('cloudformation',  
	region_name=os.environ['AWS_DEFAULT_REGION'], 
aws_access_key_id=accessKey,
aws_secret_access_key=secretAccessKey, 
	aws_session_token=sessionToken)

Step5: The custom resource obtains the stack exports from all stacks in the other account

The function get_exports() in Custom/custom-lookup-exports.py uses the AWS SDK to list all the stack exports in the Booking account.

Step 6: The custom resource returns the stack exports to the calling CloudFormation stack

The calling CloudFormation template, Airmiles/sam-airmile.yml, uses the custom resource to look up a stack export with the name of booking-lambda-BookingTopicArn. This is exported by the CloudFormation template Booking/sam-booking.yml.

PostAirmileFunction:
  Type: AWS::Serverless::Function
  Properties:
    Handler: post-airmiles-lambda.handler
    Runtime: python2.7
    Policies: AmazonDynamoDBFullAccess
    Events:
      SubmitBooking:
        Type: SNS
        Properties:
          Topic:
            !GetAtt CUSTOMLOOKUP.booking-lambda-BookingTopicArn

Step7: The custom resource handles all the event types sent by the calling CloudFormation stack

Custom resources used in a stack are called whenever the stack is created, updated, or deleted. They must therefore handle CREATE, UPDATE, and DELETE stack events. If your custom resource fails to send a SUCCESS or FAILED notification for each of these stack events, your stack may become stuck in a state such as CREATE_IN_PROGRESS or UPDATE_ROLLBACK_IN_PROGRESS. To handle this cleanly, use the crhelper custom resource helper. This strongly encourages you to handle the CREATE, UPDATE, and DELETE CloudFormation stack events.

Challenge 3: Cross-account SNS subscription

The SNS topic is specified as a resource in the Booking/sam-booking.yml CloudFormation template. To allow an event published to this topic to trigger the Airmiles Lambda function, permissions must be granted on both the Booking SNS topic and the Airmiles Lambda function. This is discussed in the next section.

Permissions on booking SNS topic

SNS topics support resource-based policies, which allow a policy to be attached directly to a resource specifying who can access the resource. This policy can specify which accounts can access a resource, and what actions those accounts are allowed to perform. This is the same approach as used by a small number of AWS services, such as Amazon S3, KMS, Amazon SQS, and Lambda. In Booking/sam-booking.yml, the SNS topic policy allows resources in the Airmiles account (referenced by the parameter NonProdAccount in the following snippet) to subscribe to the Booking SNS topic:

BookingTopicPolicy:
  Type: AWS::SNS::TopicPolicy
  Properties:
    PolicyDocument:
      Id: BookingTopicPolicy
      Version: '2012-10-17'
      Statement:
      - Sid: BookingTopicPolicy-stmt1
        Effect: Allow
        Principal:
            AWS:
              - !Sub arn:aws:iam::${AWS::AccountId}:root
        Action:
          - sns:Publish
          - sns:RemovePermission
          - sns:SetTopicAttributes
          - sns:DeleteTopic
          - sns:ListSubscriptionsByTopic
          - sns:GetTopicAttributes
          - sns:Receive
          - sns:AddPermission
          - sns:Subscribe
        Resource: !Ref BookingTopic
      - Sid: BookingTopicPolicy-stmt2
        Effect: Allow
        Principal:
            AWS:
              - !Sub arn:aws:iam::${NonProdAccount}:root
        Action:
          - sns:Subscribe
          - sns:ListSubscriptionsByTopic
        Resource: !Ref BookingTopic
    Topics:
    - !Ref BookingTopic

Permissions on the Airmiles Lambda function

The Airmiles microservice is created by the Airmiles/sam-airmile.yml CloudFormation template. The template uses SAM to specify the Lambda functions together with their associated API Gateway configurations. SAM hides much of the complexity of deploying Lambda functions from you.

For instance, by adding an ‘Events’ resource to the PostAirmileFunction in the CloudFormation template, the Airmiles Lambda function is triggered by an event published to the Booking SNS topic. SAM creates the SNS subscription, as well as the permissions necessary for the SNS subscription to trigger the Lambda function.

PostAirmileFunction:
  Type: AWS::Serverless::Function
  Properties:
    Handler: post-airmiles-lambda.handler
    Runtime: python2.7
    Policies: AmazonDynamoDBFullAccess
    Events:
      SubmitBooking:
        Type: SNS
        Properties:
          Topic:
            !GetAtt CUSTOMLOOKUP.booking-lambda-BookingTopicArn
    Environment:
      Variables: 
        TABLE_NAME: !Ref AirmileDBTable

However, the Lambda permissions generated automatically by SAM are not sufficient for cross-account SNS subscription, which means you must specify an additional Lambda permissions resource in the CloudFormation template. LambdaResourcePolicy, in the following snippet, specifies that the Booking SNS topic is allowed to invoke the Lambda function PostAirmileFunction in the Airmiles account.

LambdaResourcePolicy:
  Type: AWS::Lambda::Permission
  Properties:
    FunctionName: !GetAtt PostAirmileFunction.Arn
    Principal: sns.amazonaws.com
    Action: lambda:InvokeFunction
    SourceArn : !GetAtt CUSTOMLOOKUP.booking-lambda-BookingTopicArn

Summary

In this post, I showed you how product teams can use CodePipeline to deploy microservices into different AWS accounts and different environments such as DEV, QA, and PROD. I also showed how, at runtime, microservices in different accounts can communicate securely with each other using an asynchronous, event-driven architecture. This allows product teams to maintain their own AWS accounts for billing and security purposes, while still supporting communication with microservices in other accounts owned by other product teams.

Acknowledgments

Thanks are due to the following people for their help in preparing this post:

Kevin Yung for the great web application used in the sample application
Jay McConnell for crhelper, the custom resource helper used with the CloudFormation custom resources

Enable Federated API Access to your AWS Resources for up to 12 hours Using IAM Roles

2018-03-29 Ujjwal Pugalia

Post Syndicated from Ujjwal Pugalia original https://aws.amazon.com/blogs/security/enable-federated-api-access-to-your-aws-resources-for-up-to-12-hours-using-iam-roles/

Now, your applications and federated users can complete longer running workloads in a single session by increasing the maximum session duration up to 12 hours for an IAM role. Users and applications still retrieve temporary credentials by assuming roles using AWS Security Token Service (AWS STS), but these credentials can now be valid for up to 12 hours when using the AWS SDK or CLI. This change allows your users and applications to perform longer running workloads, such as a batch upload to S3 or a CloudFormation template, using a single session. You can extend the maximum session duration using the IAM console or CLI. Once you increase the maximum session duration, users and applications assuming the IAM role can request temporary credentials that expire when the IAM role session expires.

In this post, I show you how to configure the maximum session duration for an existing IAM role to 4 hours (maximum allowed duration is 12 hours) using the IAM console. I’ll use 4 hours because AWS recommends configuring the session duration for a role to the shortest duration that your federated users would require to access your AWS resources. I’ll then show how existing federated users can use the AWS SDK or CLI to request temporary security credentials that are valid until the role session expires.

Prerequisites

In this post, I’ll use a federation example. If you have an existing identity provider, you might have enabled federation by using SAML (Security Assertion Markup Language) to allow your users to access your AWS resources. This post assumes you have created an IAM role for a third-party identity provider that defines the permissions for your federated users. You could also have configured SAML-based federation for API access to AWS. For my example, I’ve chosen to configure SAML-based federation using Microsoft Active Directory Federation Service (ADFS) as the identity provider (IdP).

Configure the maximum session duration for an existing IAM role to 4 hours

Let’s assume you have an existing IAM role called ADFS-Production that allows your federated users to upload objects to an S3 bucket in your AWS account. You want to extend the maximum session duration for this role to 4 hours. By default, IAM roles in your AWS accounts have a maximum session duration of one hour. To extend a role’s maximum session duration to 4 hours, follow the steps below:

Sign in to the IAM console.
In the left navigation pane, select Roles and then select the role for which you want to increase the maximum session duration. For this example, I select ADFS-Production and verify the maximum session duration for this role. This value is set to 1 hour (3,600 seconds) by default.
Select Edit, and then define the maximum session duration.

Select one of the predefined durations or provide a custom duration. For this example, I set the maximum session duration to be 4 hours.
Select Save changes.

Alternatively, you can use the latest AWS CLI and call Update-Role to set the maximum session duration for the role ADFS-Production. Here’s an example to set the maximum session duration to 14,400 seconds (4 hours).

$ aws iam update-role -–role-name ADFS-Production -–MaxSessionDuration 14400

Now that you’ve successfully extended the maximum session for your IAM role, ADFS-Production, your federated users can use AWS STS to retrieve temporary credentials that are valid for 4 hours to access your S3 buckets.

Access AWS resources with temporary security credentials using AWS CLI/SDK

To enable federated SDK and CLI access for your users who use temporary security credentials, you might have implemented the solution described in the blog post on How to Implement Federated API and CLI Access Using SAML 2.0 and AD FS. That blog post demonstrates how to use the AWS Python SDK and some additional client-side integration code provided in the post to implement federated SDK and CLI access for your users. To enable your users to request longer temporary security credentials, you can make the following changes suggested in this blog to the solution provided in that post.

When calling AssumeRoleWithSAML API to request AWS temporary security credentials, you need to include the DurationSeconds parameter. The value of this parameter is the duration the user requests and, therefore, the duration their temporary security credentials are valid. In this example, I am using boto to request the maximum length of 14,400 seconds (4 hours) using code from the How to Implement Federated API and CLI Access Using SAML 2.0 and AD FS post that I have updated:

# Use the assertion to get an AWS STS token using Assume Role with SAML conn = boto.sts.connect_to_region(region) token = conn.assume_role_with_saml(role_arn, principal_arn, assertion, 14400)

By adding a value for the DurationSeconds parameter in the AssumeRoleWithSAML call, your federated user can retrieve temporary security credentials that are valid for up to 14,400 seconds (4 hours). If you don’t provide this value, the default session duration is 1 hour. If you provide a value of 5 hours for your temporary security credentials, AWS STS will throw an error since this is longer than the role session duration of 4 hours.

Conclusion

I demonstrated how you can configure the maximum session duration for a role from 1 hour (default) up to 12 hours. Then, I showed you how your federated users can retrieve temporary security credentials that are valid for longer durations to access AWS resources using AWS CLI/SDK for up to 12 hours.

Similarly, you can also increase the maximum role session duration for your applications and users who use Web Identity or OpenID Connect Federation or Cross-Account Access with Assume Role. If you have comments about this blog, submit them in the Comments section below. If you have questions or suggestions, please start a new thread on the IAM forum.

AWS Federated Authentication with Active Directory Federation Services (AD FS)

2018-03-03 Kevin Higgins

Post Syndicated from Kevin Higgins original https://aws.amazon.com/blogs/security/aws-federated-authentication-with-active-directory-federation-services-ad-fs/

Today we’d like to walk you through AWS Identity and Access Management (IAM), federated sign-in through Active Directory (AD) and Active Directory Federation Services (ADFS). With IAM, you can centrally manage users, security credentials such as access keys, and permissions that control which resources users can access. Customers have the option of creating users and group objects within IAM or they can utilize a third-party federation service to assign external directory users access to AWS resources. To streamline the administration of user access in AWS, organizations can utilize a federated solution with an external directory, allowing them to minimize administrative overhead. Benefits of this approach include leveraging existing passwords and password policies, roles and groups. This guide provides a walk-through on how to automate the federation setup across multiple accounts/roles with an Active Directory backing identity store. This will establish the minimum baseline for the authentication architecture, including the initial IdP deployment and elements for federation.

ADFS Federated Authentication Process

The following describes the process a user will follow to authenticate to AWS using Active Directory and ADFS as the identity provider and identity brokers:

Corporate user accesses the corporate Active Directory Federation Services portal sign-in page and provides Active Directory authentication credentials.
AD FS authenticates the user against Active Directory.
Active Directory returns the user’s information, including AD group membership information.
AD FS dynamically builds ARNs by using Active Directory group memberships for the IAM roles and user attributes for the AWS account IDs, and sends a signed assertion to the users browser with a redirect to post the assertion to AWS STS.
Temporary credentials are returned using STS AssumeRoleWithSAML.
The user is authenticated and provided access to the AWS management console.

Configuration Steps

Configuration requires setup in the Identity Provider store (e.g. Active Directory), the identity broker (e.g. Active Directory Federation Services), and AWS. It is possible to configure AWS to federate authentication using a variety of third-party SAML 2.0 compliant identity providers, more information can be found here.

AWS Configuration

The configuration steps outlined in this document can be completed to enable federated access to multiple AWS accounts, facilitating a single sign on process across a multi-account AWS environment. Access can also be provided to multiple roles in each AWS account. The roles available to a user are based on their group memberships in the identity provider (IdP). In a multi-role and/or multi-account scenario, role assumption requires the user to select the account and role they wish to assume during the authentication process.

Identity Provider

A SAML 2.0 identity provider is an IAM resource that describes an identity provider (IdP) service that supports the SAML 2.0 (Security Assertion Markup Language 2.0) standard. AWS SAML identity provider configurations can be used to establish trust between AWS and SAML-compatible identity providers, such as Shibboleth or Microsoft Active Directory Federation Services. These enable users in an organization to access AWS resources using existing credentials from the identity provider.

A SAML identify provider can be configured using the AWS console by completing the following steps.

1. Access the “Identity Providers” section of the AWS IAM console at the following URL: https://console.aws.amazon.com/iam/home?region=us-east-1#/providers. Click on the “Create Provider” button.

2. Select SAML for the provider type. Select a provider name of your choosing (this will become the logical name used in the identity provider ARN). Lastly, download the FederationMetadata.xml file from your ADFS server to your client system file (https://yourADFSserverFQDN/FederationMetadata/2007-06/FederationMetadata.xml). Click “Choose File” and upload it to AWS.

3. Click “Next Step” and then verify the information you have entered. Click “Create” to complete the AWS identity provider configuration process.

IAM Role Naming Convention for User Access
Once the AWS identity provider configuration is complete, it is necessary to create the roles in AWS that federated users can assume via SAML 2.0. An IAM role is an AWS identity with permission policies that determine what the identity can and cannot do in AWS. In a federated authentication scenario, users (as defined in the IdP) assume an AWS role during the sign-in process. A role should be defined for each access delineation that you wish to define. For example, create a role for each line of business (LOB), or each function within a LOB. Each role will then be assigned a set of policies that define what privileges the users who will be assuming that role will have.

The following steps detail how to create a single role. These steps should be completed multiple times to enable assumption of different roles within AWS, as required.

1. Access the “Roles” section of the AWS IAM console at the following URL: https://console.aws.amazon.com/iam/home?region=us-east-1#/roles. Click on the “Create Role” button.

2. Select “SAML” as the trusted entity type. Click Next Step.

3. Select your previously created identity provider. Click Next: Permissions.

4. The next step requires selection of policies that represent the desired permissions the user should obtain in AWS, once they have authenticated and successfully assumed the role. This can be either a custom policy or preferably an AWS managed policy. AWS recommends leveraging existing AWS access policies for job functions for common levels of access. For example, the “Billing” AWS Managed policy should be utilized to provide financial analyst access to AWS billing and cost information.

5. Provide a name for your role. All roles should be created with the prefix ADFS-<rolename> to simplify the identification of roles in AWS that are accessed through the federated authentication process. Next click, “Create Role”.

Active Directory Configuration

Determining how you will create and delineate your AD groups and IAM roles in AWS is crucial to how you secure access to your account and manage resources. SAML assertions to the AWS environment and the respective IAM role access will be managed through regular expression (regex) matching between your on-premises AD group name to an AWS IAM role.

One approach for creating the AD groups that uniquely identify the AWS IAM role mapping is by selecting a common group naming convention. For example, your AD groups would start with an identifier, for example AWS-, as this will distinguish your AWS groups from others within the organization. Next, include the 12-digit AWS account number. Finally, add the matching role name within the AWS account. Here is an example:

You should do this for each role and corresponding AWS account you wish to support with federated access. Users in Active Directory can subsequently be added to the groups, providing the ability to assume access to the corresponding roles in AWS. If a user is associated with multiple Active Directory groups and AWS accounts, they will see a list of roles by AWS account and will have the option to choose which role to assume. A user will not be able to assume more than one role at a time, but has the ability to switch between them as needed.

Note: Microsoft imposes a limit on the number of groups a user can be a member of (approximately 1,015 groups) due to the size limit for the access token that is created for each security principal. This limitation, however, is not affected by how the groups may or may not be nested.

Active Directory Federation Services Configuration

ADFS federation occurs with the participation of two parties; the identity or claims provider (in this case the owner of the identity repository – Active Directory) and the relying party, which is another application that wishes to outsource authentication to the identity provider; in this case Amazon Secure Token Service (STS). The relying party is a federation partner that is represented by a claims provider trust in the federation service.

Relying Party

You can configure a new relying party in Active Directory Federation Services by doing the following.

1. From the ADFS Management Console, right-click ADFS and select Add Relying Party Trust.

2. In the Add Relying Party Trust Wizard, click Start.

3. Check Import data about the relying party published online or on a local network, enter
https://signin.aws.amazon.com/static/saml-metadata.xml, and then click Next. The metadata XML file is a standard SAML metadata document that describes AWS as a relying party.

Note: SAML federations use metadata documents to maintain information about the public keys and certificates that each party utilizes. At run time, each member of the federation can then use this information to validate that the cryptographic elements of the distributed transactions come from the expected actors and haven’t been tampered with. Since these metadata documents do not contain any sensitive cryptographic material, AWS publishes federation metadata at https://signin.aws.amazon.com/static/saml-metadata.xml

4. Set the display name for the relying party and then click Next.

5. We will not choose to enable/configure the MFA settings at this time.

6. Select “Permit all users to access this relying party” and click Next.

7. Review your settings and then click Next.

8. Choose Close on the Finish page to complete the Add Relying Party Trust Wizard. AWS is now configured as a relying party.

Custom Claim Rules

Microsoft Active Directory Federation Services (AD FS) uses Claims Rule Language to issue and transform claims between claims providers and relying parties. A claim is information about a user from a trusted source. The trusted source is asserting that the information is true, and that source has authenticated the user in some manner. The claims provider is the source of the claim. This can be information pulled from an attribute store such as Active Directory (AD). The relying party is the destination for the claims, in this case AWS.

AD FS provides administrators with the option to define custom rules that they can use to determine the behavior of identity claims with the claim rule language. The Active Directory Federation Services (AD FS) claim rule language acts as the administrative building block to help manage the behavior of incoming and outgoing claims. There are four claim rules that need to be created to effectively enable Active Directory users to assume roles in AWS based on group membership in Active Directory.

Right-click on the relying party (in this case Amazon Web Services) and then click Edit Claim Rules

Here are the steps used to create the claim rules for NameId, RoleSessionName, Get AD Groups and Roles.

1. NameId

a) In the Edit Claim Rules for <relying party> dialog box, click Add Rule.
b) Select Transform an Incoming Claim and then click Next.
c) Use the following settings:

i) Claim rule name: NameId
ii) Incoming claim type: Windows Account Name
iii) Outgoing claim type: Name ID
iv) Outgoing name ID format: Persistent Identifier
v) Pass through all claim values: checked

d) Click OK.

Rule language:
c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname"] => issue(Type = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier", Issuer = c.Issuer, OriginalIssuer = c.OriginalIssuer, Value = c.Value, ValueType = c.ValueType, Properties["http://schemas.xmlsoap.org/ws/2005/05/identity/claimproperties/format"] = "urn:oasis:names:tc:SAML:2.0:nameid-format:persistent");

2. RoleSessionName

a) Click Add Rule
b) In the Claim rule template list, select Send LDAP Attributes as Claims.
c) Use the following settings:

i) Claim rule name: RoleSessionName
ii) Attribute store: Active Directory
iii) LDAP Attribute: E-Mail-Addresses
iv) Outgoing Claim Type: https://aws.amazon.com/SAML/Attributes/RoleSessionName

d) Click OK

Rule language:
c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname", Issuer == "AD AUTHORITY"] => issue(store = "Active Directory", types = ("https://aws.amazon.com/SAML/Attributes/RoleSessionName"), query = ";mail;{0}", param = c.Value);

3. Get AD Groups

a) Click Add Rule.
b) In the Claim rule template list, select Send Claims Using a Custom Rule and then click Next.
c) For Claim Rule Name, select Get AD Groups, and then in Custom rule, enter the following:

Rule language:
c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname", Issuer == "AD AUTHORITY"] => add(store = "Active Directory", types = ("http://temp/variable"), query = ";tokenGroups;{0}", param = c.Value);

This custom rule uses a script in the claim rule language that retrieves all the groups the authenticated user is a member of and places them into a temporary claim named http://temp/variable. Think of this as a variable you can access later.

Note: Ensure there’s no trailing whitespace to avoid unexpected results.

4. Role Attributes

a) Unlike the two previous claims, here we used custom rules to send role attributes. This is done by retrieving all the authenticated user’s AD groups and then matching the groups that start with to IAM roles of a similar name. I used the names of these groups to create Amazon Resource Names (ARNs) of IAM roles in my AWS account (i.e., those that start with AWS-). Sending role attributes requires two custom rules. The first rule retrieves all the authenticated user’s AD group memberships and the second rule performs the transformation to the roles claim.

i) Click Add Rule.
ii) In the Claim rule template list, select Send Claims Using a Custom Rule and then click Next.
iii) For Claim Rule Name, enter Roles, and then in Custom rule, enter the following:

Rule language:
c:[Type == "http://temp/variable", Value =~ "(?i)^AWS-([\d]{12})"] => issue(Type = "https://aws.amazon.com/SAML/Attributes/Role", Value = RegExReplace(c.Value, "AWS-([\d]{12})-", "arn:aws:iam::$1:saml-provider/idp1,arn:aws:iam::$1:role/"));

This custom rule uses regular expressions to transform each of the group memberships of the form AWS-<Account Number>-<Role Name> into in the IAM role ARN, IAM federation provider ARN form AWS expects.

Note: In the example rule language above idp1 represents the logical name given to the SAML identity provider in the AWS identity provider setup. Please change this based on the logical name you chose in the IAM console for your identity provider.

Adjusting Session Duration

By default, the temporary credentials that are issued by AWS IAM for SAML federation are valid for an hour. Depending on your organizations security stance, you may wish to adjust. You can allow your federated users to work in the AWS Management Console for up to 12 hours. This can be accomplished by adding another claim rule in your ADFS configuration. To add the rule, do the following:

1. Access ADFS Management Tool on your ADFS Server.
2. Choose Relying Party Trusts, then select your AWS Relying Party configuration.
3. Choose Edit Claim Rules.
4. Choose Add Rule to configure a new rule, and then choose Send claims using a custom rule. Finally, choose Next.
5. Name your Rule “Session Duration” and add the following rule syntax.
6. Adjust the value of 28800 seconds (8 hours) as appropriate.

Rule language:
=> issue(Type = "https://aws.amazon.com/SAML/Attributes/SessionDuration", Value = "28800");

Note: AD FS 2012 R2 and AD FS 2016 tokens have a sixty-minute validity period by default. This value is configurable on a per-relying party trust basis. In addition to adding the “Session Duration” claim rule, you will also need to update the security token created by AD FS. To update this value, run the following command:

Set-ADFSRelyingPartyTrust -TargetName “[Display Name]” -TokenLifetime 480

The Parameter “-TokenLifetime” determines the Lifetime in Minutes. In this example, we set the Lifetime to 480 minutes, eight hours.

These are the main settings related to session lifetimes and user authentication. Once updated, any new console session your federated users initiate will be valid for the duration specified in the SessionDuration claim.

API/CLI Access
Access to the AWS API and command-line tools using federated access can be accomplished using techniques in the following blog article:

https://blogs.aws.amazon.com/security/post/Tx1LDN0UBGJJ26Q/How-to-Implement-Federated-API-and-CLI-Access-Using-SAML-2-0-and-AD-FS

This will enable your users to access your AWS environment using their domain credentials through the AWS CLI or one of the AWS SDKs.

Conclusion
In this post, I’ve shown you how to provide identity federation, and thus SSO, to the AWS Management Console for multiple accounts using SAML assertions. With this approach, the AWS Security Token service (STS) will provide temporary credentials (via SAML) for the user to ‘assume’ a role (that they have access to use, as denoted by AD Group membership) that has specific permissions associated; as opposed to providing long-term access credentials to the AWS resources. By adopting this model, you will have a secure and robust IAM approach for accessing AWS resources that align with AWS security best practices.

AWS Single Sign-On Now Enables Command Line Interface Access for AWS Accounts Using Corporate Credentials

2018-02-23 Anand Murugesan

Post Syndicated from Anand Murugesan original https://aws.amazon.com/blogs/security/aws-single-sign-on-now-enables-command-line-interface-access-for-aws-accounts-using-corporate-credentials/

Today, AWS made it easier to use the AWS Command Line Interface (CLI) to manage services in your AWS accounts. Now you can sign into the AWS Single Sign-On (AWS SSO) user portal using your existing corporate credentials, choose an AWS account and a specific permission set, and get temporary credentials to manage your AWS services through the AWS CLI.

AWS SSO is a service that enables you to centrally manage single sign-on access to multiple AWS accounts and business applications. AWS temporary security credentials are an easy way to get short-term credentials to manage your AWS services through the AWS CLI or a programmatic client.

Previously, when you issued commands from the CLI to access resources in each of several AWS accounts, you had to remember the password for each account, sign in to each AWS account individually, and fetch the credentials for each account one at a time. Now, AWS SSO eliminates the need to sign in to each AWS account individually to get temporary credentials. Instead, you can sign in to the AWS SSO user portal once using your existing corporate credentials and then fetch temporary credentials for any of your authorized AWS accounts to use with the AWS CLI to access the resources in that account, limited by the permissions granted to you.

In this blog post, I’ll show how to fetch temporary credentials from the AWS SSO user portal to use with the AWS CLI to access resources in your AWS accounts. First, I’ll show you how to obtain short-term credentials for any account for a permission set for which you are authorized. Next, I’ll show you three ways to use these credentials.

For this scenario, let’s say I am an administrator at “AnyCompany” and I want to list instances in two AWS accounts by using the AWS CLI command, aws ec2 describe-instances. “AnyCompany” has enabled access to AWS accounts through AWS SSO.

Prerequisites

You need to install the AWS CLI to use this feature. You also need to configure AWS SSO, connect a corporate directory, and grant access to users or groups to access AWS accounts with permission sets. To learn more, see, “Introducing AWS Single Sign-On“.

How to access resources in your AWS accounts by using AWS SSO and the AWS CLI

1. Sign in to the AWS SSO user portal using your corporate credentials. If you don’t know the URL of your AWS SSO user portal, ask your IT administrator. This URL can be found in AWS SSO Console in the Dashboard menu, under “User portal URL” section. In the user portal, you will see the AWS accounts to which you have been granted access.

2. Choose “AWS Account” to expand the list of AWS accounts.

3. Choose the AWS account that you want to access using the AWS CLI. This expands the list of permission sets in the account that you can use to access the account. For this example, I choose “Administrator” permission set which has the necessary permissions to create security groups in accounts. I then choose “Command Line” or “Programmatic Access” associated with the “Administrator” permissions set.

4. AWS SSO shows the credentials you requested in the appropriate format for your operating system. If you need credentials for an operating system that is different from the one shown, you can switch between the MacOS and Linux and Windows tabs. AWS SSO offers three options to use the temporary security credentials (these credentials are valid for up to 60 minutes; see the following screenshot for examples of each option):

a. To run commands from the AWS CLI against the selected AWS account, copy the commands in the “Setup AWS CLI environment variables” section and paste the commands in the terminal window to set the necessary environment variables. These environment variables will be effective in the current terminal window.

b. To run commands from multiple terminal windows against the same AWS account, copy the profile in the “Setup AWS CLI profile” section to setup a new named profile in your AWS credentials file. To learn more, see: “Configuration and Credential Files“. You then will be able to use the –profile option with your AWS CLI command to use this credential. This will be effective in all terminal windows that use the same credential file.

c. To access AWS resources from an AWS service client, use the credentials under the “Copy individual values” section to initialize your client. For more information, see the “Use the temporary credentials to access AWS resources” section on “Getting Temporary Credentials with AWS STS“.

5. Move your mouse over the option you want to copy credentials. I chose option 1.

6. I have copied, pasted, and run the AWS CLI environment variables commands in my terminal window:

$ export AWS_ACCESS_KEY_ID=“ASIAJWOHLDZASDEXAMPLE" $ export AWS_SECRET_ACCESS_KEY="feTxcGI2aus2m4RZh+eDASvqw3vOq/jS+EXAMPLE" $ export AWS_SESSION_TOKEN="FQoDYXdzEFQaDIiq9STHISISEXAMPLE”

7. Optionally, you can verify that the credentials are set up correctly by running the “aws configure list” command. Verify that the access_key and secret_key have values assigned.

8. Now you can run any applicable AWS CLI commands (based on the permission set granted to you by your administrator). In the following example, I list instances in my AWS account.

9. To run the same (or different) AWS CLI command against a different AWS account, repeat this process, starting with Step 3. By keeping the AWS SSO user portal open in a browser window, you can easily switch to another AWS account without needing to sign in again. Every time you want to switch between accounts/permission sets or do additional work in an account after the temporary credentials expire, just copy fresh credentials for that account/permission set from the user portal.

Conclusion

In this post, in order to manage services using the AWS CLI, I’ve showed you how to use your existing corporate username and password to get temporary credentials from AWS SSO. If you have questions, please start a new thread in the AWS SSO Forum.

– Anand Murugesan

The Top 20 Most Viewed AWS IAM Documentation Pages in 2017

2018-01-05 Stephenie Swope

Post Syndicated from Stephenie Swope original https://aws.amazon.com/blogs/security/the-top-20-most-viewed-aws-iam-documentation-pages-in-2017/

The following 20 pages were the most viewed AWS Identity and Access Management (IAM) documentation pages in 2017. I have included a brief description with each link to explain what each page covers. Use this list to see what other AWS customers have been viewing and perhaps to pique your own interest in a topic you’ve been meaning to learn about.

What Is IAM?
Learn more about IAM, a web service that helps you securely control access to AWS resources for your users. You use IAM to control who can use your AWS resources (authentication) and how they can use resources (authorization).
Creating an IAM User in Your AWS Account
You can create one or more IAM users in your AWS account. You might create an IAM user when someone joins your organization, or when you have a new application that needs to make API calls to AWS.
Managing Access Keys for IAM Users
Users need their own access keys to make programmatic calls to AWS from the AWS Command Line Interface (AWS CLI), Tools for Windows PowerShell, the AWS SDKs, or direct HTTP calls using the APIs for individual AWS services. To fill this need, you can create, modify, view, or rotate access keys (access key IDs and secret access keys) for IAM users.
IAM JSON Policy Elements Reference
Learn more about the elements that you can use when you create a JSON policy. View additional JSON policy examples and learn about conditions, supported data types, and how they are used in various services.
IAM Best Practices
To help secure your AWS resources, follow these best practices for IAM.
Tutorial: Delegate Access to the Billing Console
Learn how to delegate access to specific IAM users who need to view or manage AWS Billing and Cost Management data for an AWS account.
Using Multi-Factor Authentication (MFA) in AWS
For an additional layer of security when signing in to your AWS account, AWS recommends that you configure MFA to help protect your AWS resources. MFA adds extra security because it requires users to enter a unique authentication code from an approved authentication device when they access AWS websites or services.
The IAM Console and the Sign-in Page
Learn about the IAM-enabled AWS Management Console sign-in page and how to sign in as an AWS account root user or as an IAM user. To help your users sign in easily, create a unique sign-in URL for your account.
Enabling a Virtual MFA Device
Learn how to enable and manage virtual MFA devices from the AWS Management Console.
How Users Sign In to Your Account
After you create IAM users and passwords for each, your users can sign in to the AWS Management Console using your account ID or alias, or from a special URL that includes your account ID.
Working with Server Certificates
Some AWS services can use server certificates that you manage with IAM or AWS Certificate Manager (ACM). ACM is the preferred tool to provision, manage, and deploy your server certificates. Use IAM as a certificate manager only when you must support HTTPS connections in a region that is not supported by ACM.
Your AWS Account ID and Its Alias
Learn how to find your AWS account ID and its alias.
IAM Roles
A role is an AWS identity with permission policies that determine what the identity can and cannot do in AWS using temporary security credentials that are created dynamically and provided to the user. A role is intended to be assumable by anyone who needs it using these temporary security credentials.
IAM Policies
Read an overview of policies, which are entities in AWS that, when attached to an identity or resource, define their permissions. Policies are stored in AWS as JSON documents attached to principals as identity-based policies or to resources as resource-based policies.
Example Policies
This collection of policies can help you define permissions for your IAM identities, such as granting access to a specific Amazon DynamoDB table or launching Amazon EC2 instances in a specific subnet.
Tutorial: Delegate Access Across AWS Accounts Using IAM Roles
Learn how to use an IAM role to delegate access to resources that are in different AWS accounts that you own.
Using an IAM Role to Grant Permissions to Applications Running on Amazon EC2 Instances
Use an IAM role to manage temporary credentials for applications that run on an EC2 instance. When you use a role, you do not have to distribute long-term credentials to an EC2 instance. Instead, the role supplies temporary permissions that applications can use when they make calls to other AWS resources.
Creating Your First IAM Admin User and Group
As a best practice, do not use the AWS account root user for any task where it’s not required. Instead, learn how to create an IAM administrator user and group for yourself.
Temporary Security Credentials
You can use the AWS Security Token Service (AWS STS) to create and provide trusted users with temporary security credentials that can control access to your AWS resources. Temporary security credentials work almost identically to the long-term access key credentials that your IAM users can use.
The AWS Account Root User
When you first create an AWS account, you begin with a single sign-in identity that has complete access to all AWS services and resources in the account. This identity is called the AWS account root user and is accessed by signing in with the email address and password that you used to create the account. To manage your root user, follow the steps on this page.

In the “Comments” section below, let us know if you would like to see anything on these or other IAM documentation pages expanded or updated to make them more useful to you.

– Stephenie

Create an Amazon Redshift Data Warehouse That Can Be Securely Accessed Across Accounts

2017-11-18 Ryan Hood

Post Syndicated from Ryan Hood original https://aws.amazon.com/blogs/big-data/create-an-amazon-redshift-data-warehouse-that-can-be-securely-accessed-across-accounts/

Data security is paramount in many industries. Organizations that shift their IT infrastructure to the cloud must ensure that their data is protected and that the attack surface is minimized. This post focuses on a method of securely loading a subset of data from one Amazon Redshift cluster to another Amazon Redshift cluster that is located in a different AWS account. You can accomplish this by dynamically controlling the security group ingress rules that are attached to the clusters.

Disclaimer: This solution is just one component of creating a secure infrastructure for loading and migrating data. It should not be thought of as an all-encompassing security solution. This post doesn’t touch on other important security topics, such as connecting to Amazon Redshift using Secure Sockets Layer (SSL) and encrypting data at rest.

The case for creating a segregated data loading account

From a security perspective, it is easier to restrict access to sensitive infrastructure if the respective stages (dev, QA, staging, and prod) are each located in their own isolated AWS account. Another common method for isolating resources is to set up separate virtual private clouds (VPCs) for each stage, all within a single AWS account. Because many services live outside the VPC (for example, Amazon S3, Amazon DynamoDB, and Amazon Kinesis), it requires careful thought to isolate the resources that should be associated with dev, QA, staging, and prod.

The segregated account model setup does create more overhead. But it gives administrators more control without them having to create tags and use cumbersome naming conventions to define a logical stage. In the segregated account model, all the data and infrastructure that are located in an account belong to that particular stage of the release pipeline (dev, QA, staging, or prod).

But where should you put infrastructure that does not belong to one particular stage?

Infrastructure to support deployments or to load data across accounts is best located in another segregated account. By deploying infrastructure or loading data from a separate account, you can’t depend on any existing roles, VPCs, subnets, etc. Any information that is necessary to deploy your infrastructure or load the data must be captured up front. This allows you to perform repeatable processes in a predictable and secure manner. With the recent addition of the StackSets feature in AWS CloudFormation, you can provision and manage infrastructure in multiple AWS accounts and Regions from a single template. This four-part blog series discusses different ways of automating the creation of cross-account roles and capturing account-specific information.

Loading OpenFDA data into Amazon Redshift

Before you get started with loading data from one Amazon Redshift cluster to another, you first need to create an Amazon Redshift cluster and load some data into it. You can use the following AWS CloudFormation template to create an Amazon Redshift cluster. You need to create Amazon Redshift clusters in both the source and target accounts.

AWSTemplateFormatVersion: '2010-09-09'
Description: This template creates a Redshift cluster given with the supplied username and password.
Parameters:
  username:
    Type: String
    Description: The master username for the Redshift cluster.
  password:
    Type: String
    Description: The master password for the Redshift cluster.
    MinLength: 8
Resources:
  DataWarehouse:
    Type: AWS::Redshift::Cluster
    Properties:
      ClusterType: single-node
      DBName: openfda
      MasterUsername: 
        Ref: username
      MasterUserPassword: 
        Ref: password
      NodeType: dc1.large
Outputs:
    RedshiftEndpoint:
        Description: The endpoint address of the Redshift cluster.
        Value: 
            Fn::GetAtt:
              - DataWarehouse
              - Endpoint.Address

After you create your Amazon Redshift clusters, you can go ahead and load some data into the cluster that is located in your source account. One of the great benefits of AWS is the ability to host and share public datasets on Amazon S3. When you test different architectures, these datasets serve as useful resources to get up and running without a lot of effort. For this post, we use the OpenFDA food enforcement dataset because it is a relatively small file and is easy to work with.

In the source account, you need to spin up an Amazon EMR cluster with Apache Spark so that you can unzip the file and format it properly before loading it into Amazon Redshift. The following AWS CloudFormation template provides the EMR cluster that you need.

AWSTemplateFormatVersion: '2010-09-09'
Description: This template creates an EMR cluster to load OpenFDA data into the source Redshift cluster.
Parameters:
  keyPairName:
    Type: String
    Description: The name of the KeyPair to SSH into the EMR instances.
Resources:
  Cluster:
    Type: AWS::EMR::Cluster
    Properties: 
      Applications:
        - Name: Hadoop
        - Name: Spark
        - Name: Zeppelin
        - Name: Livy
      Instances: 
          MasterInstanceGroup: 
            InstanceCount: 1
            InstanceType: m3.xlarge
            Market: ON_DEMAND
            Name: Master
          CoreInstanceGroup: 
            InstanceCount: 2
            InstanceType: m3.xlarge
            Market: ON_DEMAND
            Name: Core
          TerminationProtected: false
          Ec2KeyName:
            Ref: keyPairName
      JobFlowRole: EMR_EC2_DefaultRole 
      Name: OpenFDALoader
      ServiceRole: EMR_DefaultRole 
      ReleaseLabel: emr-5.9.0
      VisibleToAllUsers: true

Note: As an alternative, you can load the data using AWS Glue, which now supports Scala.

Now that your EMR cluster is up and running, you can submit this Scala code over a REST API call to Apache Livy. You also have the option of running this code inside of an Apache Zeppelin notebook.

To be able to run this Scala code, you must install the spark-redshift driver on your EMR cluster.

val df =  spark.read.json(sc.wholeTextFiles("s3://<Your S3 Bucket Name>/food/enforcement/").values)

var results = df.withColumn("results", explode($"results"))

results = results.select("results.address_1",
                            "results.address_2", 
                            "results.center_classification_date",
                            "results.city",
                            "results.classification",
                            "results.code_info",
                            "results.country",
                            "results.distribution_pattern",
                            "results.event_id",
                            "results.initial_firm_notification",
                            "results.more_code_info",
                            "results.postal_code",
                            "results.product_description",
                            "results.product_quantity",
                            "results.product_type",
                            "results.reason_for_recall",
                            "results.recall_initiation_date",
                            "results.recall_number",
                            "results.recalling_firm",
                            "results.report_date",
                            "results.state",
                            "results.status",
                            "results.termination_date",
                            "results.voluntary_mandated")

//Extend the varchar length for strings larger than 250 characters
val columnLengthMap = Map("code_info" -> 32600, "distribution_pattern" -> 1294, "more_code_info" -> 9161,"product_description"-> 4001,"reason_for_recall" -> 1138)

// Apply each column metadata customization
columnLengthMap.foreach { case (colName, length) =>
  val metadata = new MetadataBuilder().putLong("maxlength", length).build()
  results = results.withColumn(colName, results(colName).as(colName, metadata))
}

results.write
  .format("com.databricks.spark.redshift")
  .option("url", "jdbc:redshift://<Your Redshift Host>:5439/dev?user=<Your Redshift Username>&password=<Your Redshift Password>")
  .option("dbtable", "food_enforcement")
  .option("tempdir", "s3n:// <Your S3 Bucket Name>/tmp/")
  .option("aws_iam_role", "<Your Redshift Role Arn>")
  .mode("overwrite")
  .save()

Connect to your source Amazon Redshift cluster in your source account, and verify that the data is present by running a quick query:

select count(*) from public.food_enforcement;

Opening up the security groups

Now that the data has been loaded in the source Amazon Redshift cluster, it can be moved over to the target Amazon Redshift cluster. Because the security groups that are associated with the two clusters are very restrictive, there is no way to load the data from the centralized data loading AWS account without modifying the ingress rules on both security groups. Here are a few possible options:

Add an ingress rule to allow all traffic to port 5439 (the default Amazon Redshift port).

This option is not recommended because you are widening your attack surface significantly and exposing yourself to a potential attack.

Peer the VPC in the data loader account to the source and target Amazon Redshift VPCs, and modify the ingress rule to allow all traffic from the private IP range of the data loader VPC.

This solution is reasonably secure but does require some manual setup. Because the ingress rules in the source and target Amazon Redshift clusters allow access from the VPC private IP range, any resources in the data loader account can access both clusters, which is suboptimal.

Leave long-running Amazon EC2 instances or EMR clusters in the data loader AWS account and manually create specific ingress rules in the source and target Amazon Redshift security groups to allow for those specific IPs.

This option creates a lot of wasted cost because it requires leaving EC2 instances or an EMR cluster running indefinitely whether or not they are actually being used.

None of these three options is ideal, so let’s explore another option. One of the more powerful features of running EC2 instances in the cloud is the ability to dynamically manage and configure your environment using instance metadata. The AWS Cloud is dynamic by nature and incentivizes you to reduce costs by terminating instances when they are not being used. Therefore, instance metadata can serve as the glue to performing repeatable processes to these dynamic instances.

To load the data from the source Amazon Redshift cluster to the target Amazon Redshift cluster, perform the following steps:

Spin up an EC2 instance in the data loader account.

Use instance metadata to look up the IP of the EC2 instance.

Assume roles in the source and target accounts using the AWS Security Token Service (AWS STS), and create ingress rules for the IP that was retrieved from instance metadata.

client = boto3.client('sts')
sts_response = client.assume_role(RoleArn='<your role arn>', RoleSessionName='CrossAccountSecurityGroupIngress')
ec2 = boto3.client('ec2', region_name=region, aws_session_token=sts_response['Credentials']['SessionToken'], aws_secret_access_key=sts_response['Credentials']['SecretAccessKey'],  aws_access_key_id=sts_response['Credentials']['AccessKeyId'])
ec2.authorize_security_group_ingress(CidrIp='<your ip>/32',FromPort=5439,ToPort=5439,GroupId='<your security group id>', IpProtocol='tcp')

Run a simple Python or Java program to perform a simple transformation and unload the data from the source Amazon Redshift cluster. Then load the results into the target Amazon Redshift cluster.

UNLOAD('select case when product_description ilike ''%milk%'' then 1 else 0 end as milk_flag
                        , *
                        from dev.public.food_enforcement 
                        where left(recall_initiation_date, 4) >= 2016')
TO 's3://<Your S3 Bucket>/milk-food-enforcement.csv'
IAM_ROLE '<Your Redshift Role>'
ADDQUOTES


COPY public.milk_food_enforcement
FROM 's3://<Your S3 Bucket>/milk-food-enforcement.csv'
IAM_ROLE '<Your Redshift Role>'
REMOVEQUOTES
DELIMITER '|'

Assume roles in the source and target accounts using AWS STS, and remove the ingress rules that were created in step 3.

client = boto3.client('sts')
sts_response = client.assume_role(RoleArn='<your role arn>', RoleSessionName='CrossAccountSecurityGroupIngress')
ec2 = boto3.client('ec2', region_name=region, aws_session_token=sts_response['Credentials']['SessionToken'], aws_secret_access_key=sts_response['Credentials']['SecretAccessKey'],  aws_access_key_id=sts_response['Credentials']['AccessKeyId'])
ec2.revoke_security_group_ingress(CidrIp='<your ip>/32',FromPort=5439,ToPort=5439,GroupId='<your security group id>', IpProtocol='tcp')

Once step 5 is completed, you should see that the security groups for both Amazon Redshift clusters don’t allow traffic from any IP. Manually add your IP as an ingress rule to the target Amazon Redshift cluster’s security group on port 5439. When you run the following query, you should see that the data has been populated within the target Amazon Redshift cluster.

select * 
from milk_food_enforcement

Recap

This post highlighted the importance of loading data in a secure manner across accounts. It mentioned reasons why you might want to provision infrastructure and load data from a centralized account. Several candidate solutions were discussed. Ultimately, the solution that we chose involved opening up security groups for a single IP and then closing them back up after the data was loaded. This solution minimizes the attack surface to a single IP and can be completely automated.

Additional Reading

Be sure to check out the 10 Best Practices for Amazon Redshift Spectrum, and the Top 10 Tuning Techniques for Amazon Redshift.

About the Author

Ryan Hood is a Data Engineer for AWS. He works on big data projects leveraging the newest AWS offerings. In his spare time, he attempts to Sous-vide anything he can find in his refrigerator.

The Full List of the Security, Compliance, and Identity Sessions, Workshops, and Chalk Talks Being Offered at AWS re:Invent 2017

2017-10-24 Craig Liebendorfer

Post Syndicated from Craig Liebendorfer original https://aws.amazon.com/blogs/security/a-full-list-of-the-security-compliance-and-identity-sessions-workshops-and-chalk-talks-available-at-aws-reinvent-2017/

Now that you can reserve seating in AWS re:Invent 2017 breakout sessions, workshops, chalk talks, and other events, the time is right to review the list of introductory, advanced, and expert content being offered this year. To learn more about breakout content types and levels, see Breakout Content.

Jump to: Advanced level | Expert level

Introductory level

SID201 – IAM for Enterprises: How Vanguard Strikes the Balance Between Agility, Governance, and Security
For Vanguard, managing the creation of AWS Identity and Access Management (IAM) objects is key to balancing developer velocity and compliance. In this session, you learn how Vanguard designs IAM roles to control the blast radius of AWS resources and maintain simplicity for developers. Vanguard will also share best practices to help you manage governance and improve your visibility across your AWS resources.

SID202 – Deep dive about how Capital One automates the delivery of directory services across AWS accounts
Traditional solutions for using Microsoft Active Directory across on-premises and AWS Cloud Windows workloads can require complex networking or syncing identities across multiple systems. AWS Directory Service for Microsoft Active Directory, also known as AWS Managed AD, offers you actual Microsoft Active Directory in the AWS Cloud as a managed service. In this session, you will learn how Capital One uses AWS Managed AD to provide highly available authentication and authorization services for its Windows workloads, such as Amazon RDS for SQL Server.

SID205 – Building the Largest Repo for Serverless Compliance-as-Code
When you use the cloud to enable speed and agility, how do you know if you’ve done it correctly? We are on a mission to help builders follow industry best practices within security guardrails by creating the largest compliance-as-code repository, available to all. Compliance-as-code is the idea to translate best practices, guardrails, policies, and standards into codified unit testing. Apply this to your AWS environment to provide insights about what can or must be improved. Learn why compliance-as-code matters to gain speed (by getting developers, architects, and security pros on the same page), how it is currently used (demo), and how to start using it or being part of building it.

SID206 – Best Practices for Managing Security Operation on AWS
To help prevent unexpected access to your AWS resources, it is critical to maintain strong identity and access policies and track, detect, and react to changes. In this session, you will learn how to use AWS Identity and Access Management (IAM) to control access to AWS resources and integrate your existing authentication system with IAM. We will cover how to deploy and control AWS infrastructure using code templates, including change management policies with AWS CloudFormation.

SID207 – Feedback Security in the Cloud
Like many security teams, Riot has been challenged by new paradigms that came with the move to the cloud. We discuss how our security team has developed a security culture based on feedback and self-service to best thrive in the cloud. We detail how the team assessed the security gaps and challenges in our move to AWS, and then describe how the team works within Riot’s unique feedback culture.

SID208 – Less (Privilege) Is More: Getting Least Privilege Right in AWS
AWS services are designed to enable control through AWS Identity and Access Management (IAM) and Amazon Virtual Private Cloud (VPC). Join us in this chalk talk to learn how to apply these toward the security principal of least privilege for applications and data and how to practically integrate them in your security operations.

SID209 – Designing and Deploying an AWS Account Factory
AWS customers start off with one AWS account, but quickly realize the benefits of having multiple AWS accounts. A common learning curve for customers is how to securely baseline and set up new accounts at scale. This talk helps you understand how to use AWS Organizations, AWS Identity and Access Management (IAM), AWS CloudFormation, and other tools to baseline new accounts, set them up for federation, and make a secure and repeatable account factory to create new AWS accounts. Walk away with demos and tools to use in your own environment.

SID210 – A CISO’s Journey at Vonage: Achieving Unified Security at Scale
Making sense of the risks of IT deployments that sit in hybrid environments and span multiple countries is a major challenge. When you add in multiple toolsets and global compliance requirements, including GDPR, it can get overwhelming. Listen to Vonage’s Chief Information Security Officer, Johan Hybinette, share his experiences tackling these challenges.

SID212 – Maximizing Your Move to AWS – Five Key Lessons from Vanguard and Cloud Technology Partners
CTP’s Robert Christiansen and Mike Kavis describe how to maximize the value of your AWS initiative. From building a Minimum Viable Cloud to establishing a cloud robust security and compliance posture, we walk through key client success stories and lessons learned. We also explore how CTP has helped Vanguard, the leading provider of investor communications and technology, take advantage of AWS to delight customers, drive new revenue streams, and transform their business.

SID213 – Managing Regulator Expectations – Lessons Learned on Positioning AWS Services from an Audit Perspective
Cloud migration in highly regulated industries can stall without a solid understanding of how (and when) to address regulatory expectations. This session provides a guide to explaining the aspects of AWS services that are most frequently the subject of an internal or regulatory audit. Because regulatory agencies and internal auditors might not share a common understanding of the cloud, this session is designed to help you to help them, regardless of their level of technical fluency.

SID214 – Best Security Practices in the Intelligence Community
Executives from the Intelligence community discuss cloud security best practices in a field where security is imperative to operations. CIA security cloud chief John Nicely and NGA security cloud chief Scot Kaplan share success stories of migrating mass data to the cloud from a security perspective. Hear how they migrated their IT portfolios while managing their organizations’ unique blend of constraints, budget issues, politics, culture, and security pressures. Learn how these institutions overcame barriers to migration, and ask these panelists what actions you can take to better prepare yourself for the journey of mass migration to the cloud.

SID216 – Defending Diverse Applications Against Common Threats
In this session, you learn how to adapt application defenses and operational responses based on your unique requirements. You also hear directly from customers about how they architected their applications on AWS to protect their applications. There are many ways to build secure, high-availability applications in the cloud. Services such as Amazon API Gateway, Amazon VPC, ALB, ELB, and Amazon EC2 are the basic building blocks that enable you to address a wide range of use cases. Best practices for defending your applications against Distributed Denial of Service (DDoS) attacks, exploitation attempts, and bad bots can vary with your choices in architecture.

Advanced level

SID301 – Using AWS Lambda as a Security Team
Operating a security practice on AWS brings many new challenges that haven’t been faced in data center environments. The dynamic nature of infrastructure, the relationship between development team members and their applications, and the architecture paradigms have all changed as a result of building software on top of AWS. In this session, learn how your security team can leverage AWS Lambda as a tool to monitor, audit, and enforce your security policies within an AWS environment.

SID302 – Force Multiply Your Security Team with Automation and Alexa
Adversaries automate. Who says the good guys can’t as well? By combining AWS offerings like AWS CloudTrail, Amazon Cloudwatch, AWS Config, and AWS Lambda with the power of Amazon Alexa, you can do more security tasks faster, with fewer resources. Force multiplying your security team is all about automation! Last year, we showed off penetration testing at the push of an (AWS IoT) button, and surprise-previewed how to ask Alexa to run Inspector as-needed. Want to see other ways to ask Alexa to be your cloud security sidekick? We have crazy new demos at the ready to show security geeks how to sling security automation solutions for their AWS environments (and impress and help your boss, too).

SID303 – How You Can Use AWS’s Identity Services to be Successful on Your AWS Cloud Journey
Every journey to the AWS Cloud is unique. Some customers are migrating existing applications, while others are building new applications using cloud-native services. Along each of these journeys, identity and access management helps customers protect their applications and resources. In this session, you will learn how AWS’s identity services provide you a secure, flexible, and easy solution for managing identities and access on the AWS Cloud. With AWS’’s Identity Services, you do not have to adapt to AWS. Instead, you have a choice of services designed to meet you anywhere along your journey to the AWS Cloud. Every journey to the AWS Cloud is unique. Some customers are migrating existing applications, while others are building new applications using cloud-native services.

SID304 – SecOps 2021 Today: Using AWS Services to Deliver SecOps
This talk dives deep on how to build end-to-end security capabilities using AWS. Our goal is orchestrating AWS Security services with other AWS building blocks to deliver enhanced security. We cover working with AWS CloudWatch Events as a queueing mechanism for processing security events, using Amazon DynamoDB to provide a stateful layer to provide tailored response to events and other ancillary functions, using DynamoDB as an attack signature engine, and the use of analytics to derive tailored signatures for detection with AWS Lambda.

SID305 – How CrowdStrike Built a Real-time Security Monitoring Service on AWS
The CrowdStrike motto is “We Stop Breaches.” To do that, it needed to build a real-time security monitoring service to detect threats. Join this session to learn how Crowdstrike uses Amazon EC2 and Amazon EBS to help its customers identify vulnerabilities before they become large-scale problems.

SID306 – How Chick-fil-A Embraces DevSecOps on AWS
As Chick-fil-A became a cloud-first organization, their security team didn’t want to become the bottleneck for agility. But the security team also wanted to raise the bar for their security posture on AWS. Robert Davis, security architect at Chick-fil-A, provides an overview about how he and his team recognized that writing code was the best way for their security policies to scale across the many AWS accounts that Chick-fil-A operates.

SID307 – Serverless for Security Officers: Paradigm Walkthrough and Comprehensive Security Best Practices
For security practitioners, serverless represents a context switch from the familiar servers and networks to a decentralized set of code snippets and AWS platform constructs. This new ecosystem also represents new operational teams, data flows, security tooling, and faster-then-ever change velocity. In this talk, we perform live demos and provide code samples for a wide array of security best practices aligned to industry standards such as NIST 800-53 and ISO 27001.

SID308 – Multi-Account Strategies
We will explore a multi-account architecture and how to approach the design/thought process around it. This chalk talk will allow attendees to dive deep into the topic and discuss the nuances of the architecture as well as provide feedback around the approach.

SID309 – Credentials, Credentials, Credentials, Oh My!
For new and experienced customers alike, understanding the various credential forms and exchange mechanisms within AWS can be a daunting exercise. In this chalk talk, we clear up the confusion by performing a cartography exercise. We visually depict the right source credentials (for example, enterprise user name and password, IAM keys, AWS STS tokens, and so on) and transformation mechanisms (for example, AssumeRole and so on) to use depending on what you’re trying to do and where you’re coming from.

SID310 – Moving from the Shadows to the Throne
What do you do when leadership embraces what was called “shadow IT” as the new path forward? How do you onboard new accounts while simultaneously pushing policy to secure all existing accounts? This session walks through Cisco’s journey consolidating over 700 existing accounts in the Cisco organization, while building and applying Cisco’s new cloud policies.

SID311 – Designing Security and Governance Across a Multi-Account Strategy
When organizations plan their journey to cloud adoption at scale, they quickly encounter questions such as: How many accounts do we need? How do we share resources? How do we integrate with existing identity solutions? In this workshop, we present best practices and give you the hands-on opportunity to test and develop best practices. You will work in teams to set up and create an AWS environment that is enterprise-ready for application deployment and integration into existing operations, security, and procurement processes. You will get hands-on experience with cross-account roles, consolidated logging, account governance and other challenges to solve.

SID312 – DevSecOps Capture the Flag
In this Capture the Flag workshop, we divide groups into teams and work on AWS CloudFormation DevSecOps. The AWS Red Team supplies an AWS DevSecOps Policy that needs to be enforced via CloudFormation static analysis. Participant Blue Teams are provided with an AWS Lambda-based reference architecture to be used to inspect CloudFormation templates against that policy. Interesting items need to be logged, and made visible via ChatOps. Dangerous items need to be logged, and recorded accurately as a template fail. The secondary challenge is building a CloudFormation template to thwart the controls being created by the other Blue teams.

SID313 – Continuous Compliance on AWS at Scale
In cloud migrations, the cloud’s elastic nature is often touted as a critical capability in delivering on key business initiatives. However, you must account for it in your security and compliance plans or face some real challenges. Always counting on a virtual host to be running, for example, causes issues when that host is rebooted or retired. Managing security and compliance in the cloud is continuous, requiring forethought and automation. Learn how a leading, next generation managed cloud provider uses automation and cloud expertise to manage security and compliance at scale in an ever-changing environment.

SID314 – IAM Policy Ninja
Are you interested in learning how to control access to your AWS resources? Have you wondered how to best scope permissions to achieve least-privilege permissions access control? If your answer is “yes,” this session is for you. We look at the AWS Identity and Access Management (IAM) policy language, starting with the basics of the policy language and how to create and attach policies to IAM users, groups, and roles. We explore policy variables, conditions, and tools to help you author least privilege policies. We cover common use cases, such as granting a user secure access to an Amazon S3 bucket or to launch an Amazon EC2 instance of a specific type.

SID315 – Security and DevOps: Agility and Teamwork
In this session, you learn pragmatic steps to integrate security controls into DevOps processes in your AWS environment at scale. Cybersecurity expert and founder of Alert Logic Misha Govshteyn shares insights from high performing teams who are embracing the reality that an agile security program can enable faster and more secure workload deployments. Joining Misha is Joey Peloquin, Director of Cloud Security Operations at Citrix, who discusses Citrix’s DevOps experiences and how they manage their cybersecurity posture within the AWS Cloud. Session sponsored by Alert Logic.

SID316 – Using Access Advisor to Strike the Balance Between Security and Usability
AWS provides a killer feature for security operations teams: Access Advisor. In this session, we discuss how Access Advisor shows the services to which an IAM policy grants access and provides a timestamp for the last time that the role authenticated against that service. At Netflix, we use this valuable data to automatically remove permissions that are no longer used. By continually removing excess permissions, we can achieve a balance of empowering developers and maintaining a best-practice, secure environment.

SID317 – Automating Security and Compliance Testing of Infrastructure-as-Code for DevSecOps
Infrastructure-as-Code (IaC) has emerged as an essential element of organizational DevOps practices. Tools such as AWS CloudFormation and Terraform allow software-defined infrastructure to be deployed quickly and repeatably to AWS. But the agility of CI/CD pipelines also creates new challenges in infrastructure security hardening. This session provides a foundation for how to bring proven software hardening practices into the world of infrastructure deployment. We discuss how to build security and compliance tests for infrastructure analogous to unit tests for application code, and showcase how security, compliance and governance testing fit in a modern CI/CD pipeline.

SID318 – From Obstacle to Advantage: The Changing Role of Security & Compliance in Your Organization
A surprising trend is starting to emerge among organizations who are progressing through the cloud maturity lifecycle: major improvements in revenue growth, customer satisfaction, and mission success are being directly attributed to improvements in security and compliance. At one time thought of as speed bumps in the path to deployment, security and compliance are now seen as critical ingredients that help organizations differentiate their offerings in the market, win more deals, and achieve mission-critical goals faster. This session explores how organizations like Jive Software and the National Geospatial Agency use the Evident Security Platform, AWS, and AWS Quick Starts to automate security and compliance processes in their organization to accomplish more, do it faster, and deliver better results.

SID319 – Incident Response in the Cloud
In this session, we walk you through a hypothetical incident response managed on AWS. Learn how to apply existing best practices as well as how to leverage the unique security visibility, control, and automation that AWS provides. We cover how to set up your AWS environment to prevent a security event and how to build a cloud-specific incident response plan so that your organization is prepared before a security event occurs. This session also covers specific environment recovery steps available on AWS.

SID320 – Fraud Prevention, Detection, Lessons Learned, and Best Practices
Fighting fraud means countering human actors that quickly adapt to whatever you do to stop them. In this presentation, we discuss the key components of a fraud prevention program in the cloud. Additionally, we provide techniques for detecting known and unknown fraud activity and explore different strategies for effectively preventing detected patterns. Finally, we discuss lessons learned from our own prevention activities as well as the best practices that you can apply to manage risk.

SID321 – How Capital One Applies AWS Organizations Best Practices to Manage Multiple AWS Accounts
In this session, we review best practices for managing multiple AWS accounts using AWS Organizations. We cover how to think about the master account and your account strategy, as well as how to roll out changes. You learn how Capital One applies these best practices to manage its AWS accounts, which number over 160, and PCI workloads.

SID322 – The AWS Philosophy of Security
AWS distinguished engineer Eric Brandwine speaks with hundreds of customers each year, and noticed one question coming up more than any other, “How does AWS operationalize its own security?” In this session, Eric details both strategic and tactical considerations, along with an insider’s look at AWS tooling and processes.

SID324 – Automating DDoS Response in the Cloud
If left unmitigated, Distributed Denial of Service (DDoS) attacks have the potential to harm application availability or impair application performance. DDoS attacks can also act as a smoke screen for intrusion attempts or as a harbinger for attacks against non-cloud infrastructure. Accordingly, it’s crucial that developers architect for DDoS resiliency and maintain robust operational capabilities that allow for rapid detection and engagement during high-severity events. In this session, you learn how to build a DDoS-resilient application and how to use services like AWS Shield and Amazon CloudWatch to defend against DDoS attacks and automate response to attacks in progress.

SID325 – Amazon Macie: Data Visibility Powered by Machine Learning for Security and Compliance Workloads
In this session, Edmunds discusses how they create workflows to manage their regulated workloads with Amazon Macie, a newly-released security and compliance management service that leverages machine learning to classify your sensitive data and business-critical information. Amazon Macie uses recurrent neural networks (RNN) to identify and alert potential misuse of intellectual property. They do a deep dive into machine learning within the security ecosystem.

SID326 – AWS Security State of the Union
Steve Schmidt, chief information security officer at AWS, addresses the current state of security in the cloud, with a particular focus on feature updates, the AWS internal “secret sauce,” and what’s on horizon in terms of security, identity, and compliance tooling.

SID327 – How Zocdoc Achieved Security and Compliance at Scale With Infrastructure as Code
In less than 12 months, Zocdoc became a cloud-first organization, diversifying their tech stack and liberating data to help drive rapid product innovation. Brian Lozada, CISO at Zocdoc, and Zhen Wang, Director of Engineering, provide an overview on how their teams recognized that infrastructure as code was the most effective approach for their security policies to scale across their AWS infrastructure. They leveraged tools such as AWS CloudFormation, hardened AMIs, and hardened containers. The use of DevSecOps within Zocdoc has enhanced data protection with the use of AWS services such as AWS KMS and AWS CloudHSM and auditing capabilities, and event-based policy enforcement with Amazon Elasticsearch Service and Amazon CloudWatch, all built on top of AWS.

SID328 – Cloud Adoption in Regulated Financial Services
Macquarie, a global provider of financial services, identified early on that it would require strong partnership between its business, technology and risk teams to enable the rapid adoption of AWS cloud technologies. As a result, Macquarie built a Cloud Governance Platform to enable its risk functions to move as quickly as its development teams. This platform has been the backbone of Macquarie’s adoption of AWS over the past two years and has enabled Macquarie to accelerate its use of cloud technologies for the benefit of clients across multiple global markets. This talk will outline the strategy that Macquarie embarked on, describe the platform they built, and provide examples of other organizations who are on a similar journey.

SID329 – A Deep Dive into AWS Encryption Services
AWS Encryption Services provide an easy and cost-effective way to protect your data in AWS. In this session, you learn about leveraging the latest encryption management features to minimize risk for your data.

SID330 – Best Practices for Implementing Your Encryption Strategy Using AWS Key Management Service
AWS Key Management Service (KMS) is a managed service that makes it easy for you to create and manage the encryption keys used to encrypt your data. In this session, we will dive deep into best practices learned by implementing AWS KMS at AWS’s largest enterprise clients. We will review the different capabilities described in the AWS Cloud Adoption Framework (CAF) Security Perspective and how to implement these recommendations using AWS KMS. In addition to sharing recommendations, we will also provide examples that will help you protect sensitive information on the AWS Cloud.

SID331 – Architecting Security and Governance Across a Multi-Account Strategy
Whether it is per business unit or per application, many AWS customers use multiple accounts to meet their infrastructure isolation, separation of duties, and billing requirements. In this session, we discuss considerations, limitations, and security patterns when building out a multi-account strategy. We explore topics such as identity federation, cross-account roles, consolidated logging, and account governance. Thomson Reuters shared their journey and their approach to a multi-account strategy. At the end of the session, we present an enterprise-ready, multi-account architecture that you can start leveraging today.

SID332 – Identity Management for Your Users and Apps: A Deep Dive on Amazon Cognito
Learn how to set up an end-user directory, secure sign-up and sign-in, manage user profiles, authenticate and authorize your APIs, federate from enterprise and social identity providers, and use OAuth to integrate with your app—all without any server setup or code. With clear blueprints, we show you how to leverage Amazon Cognito to administer and secure your end users and enable identity for the applied patterns of mobile, web, and enterprise apps.

SID333 – Security at Scale: How Autodesk Leverages Native AWS Technologies to Provide Uniformly Scalable Security Capabilities
Learn how Autodesk implemented security at scale, moved to native AWS security products and features, as well as attained SOC certification.

SID334 – How Amazon Business Uses Amazon Cloud Directory as the Data Store for Its Account Management Platform
Join the Amazon Business team to learn how it uses Amazon Cloud Directory as the data store for its account management platform. You will learn how Amazon Business uses Amazon DynamoDB with Cloud Directory to manage user authorization and business process workflows. You also will learn how Cloud Directory helps to manage hierarchical datasets and how to get started modeling these datasets in Cloud Directory.

SID335 – Implementing Security and Governance across a Multi-Account Strategy
As existing or new organizations expand their AWS footprint, managing multiple accounts while maintaining security quickly becomes a challenge. In this chalk talk, we will demonstrate how AWS Organizations, IAM roles, identity federation, and cross-account manager can be combined to build a scalable multi-account management platform. By the end of this session, attendees will have the understanding and deployment patterns to bring a secure, flexible and automated multi-account management platform to their organizations.

SID336 – Use AWS to Effectively Manage GDPR Compliance
The General Data Protection Regulation (GDPR) is considered to be the most stringent privacy regulation ever enacted. Complying with GDPR could be a challenge for organizations, and AWS services can help get you ahead of the May 2018 enforcement deadline. In this chalk talk, the Legal and Compliance GDPR leadership at AWS discusses what enforcement of GDPR might mean for you and your customer’s compliance programs.

SID337 – Best Practices for Managing Access to AWS Resources Using IAM Roles
In this chalk talk, we discuss why using temporary security credentials to manage access to your AWS resources is an AWS Identity and Access Management (IAM) best practice. IAM roles help you follow this best practice by delivering and rotating temporary credentials automatically. We discuss the different types of IAM roles, the assume role functionality, and how to author fine-grained trust and access policies that limit the scope of IAM roles. We then show you how to attach IAM roles to your AWS resources, such as Amazon EC2 instances and AWS Lambda functions. We also discuss migrating applications that use long-term AWS access keys to temporary credentials managed by IAM roles.

SID338 – [email protected]
Once a customer achieves success with using AWS in a few pilot projects, most look to rapidly adopt an “all-in” enterprise migration strategy. Along this journey, several new challenges emerge that quickly become blockers and slow down migrations if they are not addressed properly. At this scale, customers will deal with the governance of hundreds of accounts, as well as thousands of IT resources residing within those accounts. Humans and traditional IT management processes cannot scale at the same pace and inevitably challenging questions emerge. In this session, we discuss those questions about governance at scale.

SID339 – Deep Dive on AWS CloudHSM
Organizations building applications that handle confidential or sensitive data are subject to many types of regulatory requirements and often rely on hardware security modules (HSMs) to provide validated control of encryption keys and cryptographic operations. AWS CloudHSM is a cloud-based hardware security module (HSM) that enables you to easily generate and use your own encryption keys on the AWS Cloud using FIPS 140-2 Level 3 validated HSMs. This chalk talk will provide you a deep-dive on CloudHSM, and demonstrate how you can quickly and easily use CloudHSM to help secure your data and meet your compliance requirements.

SID340 – Using Infrastructure as Code to Inject Security Best Practices as Part of the Software Deployment Lifecycle
A proactive approach to security is key to securing your applications as part of software deployment. In this chalk talk, T. Rowe Price, a financial asset management institution, outlines how they built their security automation process in enabling their numerous developer teams to rapidly and securely build and deploy applications at scale on AWS. Learn how they use services like AWS Identity and Access Management (IAM), HashiCorp tools, Terraform for automation, and Vault for secrets management, and incorporate certificate management and monitoring as part of the deployment process. T. Rowe Price discusses lessons learned and best practices to move from a tightly controlled legacy environment to an agile, automated software development process on AWS.

SID341 – Using AWS CloudTrail Logs for Scalable, Automated Anomaly Detection
This workshop gives you an opportunity to develop a solution that can continuously monitor for and detect a realistic threat by analyzing AWS CloudTrail log data. Participants are provided with a CloudTrail data source and some clues to get started. Then you have to design a system that can process the logs, detect the threat, and trigger an alarm. You can make use of any AWS services that can assist in this endeavor, such as AWS Lambda for serverless detection logic, Amazon CloudWatch or Amazon SNS for alarming and notification, Amazon S3 for data and configuration storage, and more.

SID342 – Protect Your Web Applications from Common Attack Vectors Using AWS WAF
As attacks and attempts to exploit vulnerabilities in web applications become more sophisticated, having an effective web request filtering solution becomes key to keeping your users’ data safe. In this workshop, discover how the OWASP Top 10 list of application security risks can help you secure your web applications. Learn how to use AWS services, such as AWS WAF, to mitigate vulnerabilities. This session includes hands-on labs to help you build a solution. Key learning goals include understanding the breadth and complexity of vulnerabilities customers need to protect from, understanding the AWS tools and capabilities that can help mitigate vulnerabilities, and learning how to configure effective HTTP request filtering rules using AWS WAF.

SID343 – User Management and App Authentication with Amazon Cognito
Are you curious about how to authenticate and authorize your applications on AWS? Have you thought about how to integrate AWS Identity and Access Management (IAM) with your app authentication? Have you tried to integrate third-party SAML providers with your app authentication? Look no further. This workshop walks you through step by step to configure and create Amazon Cognito user pools and identity pools. This workshop presents you with the framework to build an application using Java, .NET, and serverless. You choose the stack and build the app with local users. See the service being used not only with mobile applications but with other stacks that normally don’t include Amazon Cognito.

SID344 – Soup to Nuts: Identity Federation for AWS
AWS offers customers multiple solutions for federating identities on the AWS Cloud. In this session, we will embark on a tour of these solutions and the use cases they support. Along the way, we will dive deep with demonstrations and best practices to help you be successful managing identities on the AWS Cloud. We will cover how and when to use Security Assertion Markup Language 2.0 (SAML), OpenID Connect (OIDC), and other AWS native federation mechanisms. You will learn how these solutions enable federated access to the AWS Management Console, APIs, and CLI, AWS Infrastructure and Managed Services, your web and mobile applications running on the AWS Cloud, and much more.

SID345 – AWS Encryption SDK: The Busy Engineer’s Guide to Client-Side Encryption
You know you want client-side encryption for your service but you don’t know exactly where to start. Join us for a hands-on workshop where we review some of your client-side encryption options and explore implementing client-side encryption using the AWS Encryption SDK. In this session, we cover the basics of client-side encryption, perform encrypt and decrypt operations using AWS KMS and the AWS Encryption SDK, and discuss security and performance considerations when implementing client-side encryption in your service.

Expert level

SID401 – Let’s Dive Deep Together: Advancing Web Application Security
Beginning with a recap of best practices in CloudFront, AWS WAF, Route 53, and Amazon VPC security, we break into small teams to work together on improving the security of a typical web application. How can we creatively use the services? What additional features would help us? This technically advanced chalk talk requires certification at the solutions architect associate level or greater.

SID402 – An AWS Security Odyssey: Implementing Security Controls in the World of Internet, Big Data, IoT and E-Commerce Platforms
This workshop will give participants the opportunity to take a security-focused journey across various AWS services and implement automated controls along the way. You will learn how to apply AWS security controls to services such as Amazon EC2, Amazon S3, AWS Lambda, and Amazon VPC. In short, you will learn how to use the cloud to protect the cloud. We will talk about how to: Adopt a workload-centric approach to your security strategy, Address security issues in a cost-effective manner Automate your security responses to promote maturity and auditability. In order to complete this workshop, attendees will need a laptop with wireless access, an AWS account and an IAM user that has full administrative privileges within their account. AWS credits will be provided as attendees depart the session to cover the cost of running the workshop in their own account.

SID404 – Amazon Inspector – Automating the “Sec” in DevSecOps
Adopting DevSecOps can be challenging using traditional security tools that are designed for on-premises infrastructure. Amazon Inspector is an automated security assessment service that helps you adopt DevSecOps by integrating security assessments directly into the development process of applications running on Amazon EC2. We dive deep on how to use Inspector to automate host security assessments. We show you how to integrate Inspector with other AWS Cloud services to provide automated security assessments throughout your development process. We demo installing the AWS agent, setting up assessment targets and templates, and running assessments. We review the findings and discuss how you can automate the management and remediation of those findings with your available AWS services.

SID405 – Five New Security Automation Improvements You Can Make by Using Amazon CloudWatch Events and AWS Config Rules
This presentation will include a deep dive into the code behind multiple security automation and remediation functions. This session will consider potential use cases, as well as feature a demonstration of a proposed script, and then walk through the code set to explain the various challenges and solutions of the intended script. All examples of code will be previously unreleased and will feature integration with services such as Trusted Advisor and Macie. All code will be released as OSS after re:Invent.

– Craig

The Top 20 AWS IAM Documentation Pages so Far in 2017

2017-10-02 Stephenie Swope

Post Syndicated from Stephenie Swope original https://aws.amazon.com/blogs/security/the-top-20-aws-iam-documentation-pages-so-far-in-2017/

The following 20 pages have been the most viewed AWS Identity and Access Management (IAM) documentation pages so far this year. I have included a brief description with each link to explain what each page covers. Use this list to see what other AWS customers have been viewing and perhaps to pique your own interest about a topic you’ve been meaning to learn about.

What Is IAM?
Learn more about IAM, a web service that helps you securely control access to AWS resources for your users. You use IAM to control who can use your AWS resources (authentication) and how they can use resources (authorization).
Creating an IAM User in Your AWS Account
You can create one or more IAM users in your AWS account. You might create an IAM user when someone joins your organization, or when you have a new application that needs to make API calls to AWS.
IAM Policy Elements Reference
Learn more about the elements that you can use when you create a policy. View additional policy examples and learn about conditions, supported data types, and how they are used in various services.
Managing Access Keys for IAM Users
Users need their own access keys to make programmatic calls to AWS from the AWS Command Line Interface (AWS CLI), Tools for Windows PowerShell, the AWS SDKs, or direct HTTP calls using the APIs for individual AWS services. To fill this need, you can create, modify, view, or rotate access keys (access key IDs and secret access keys) for IAM users.
IAM Best Practices
To help secure your AWS resources, follow these best practices for IAM.
Tutorial: Delegate Access to the Billing Console
Learn how to delegate access to specific IAM users who need to view or manage AWS Billing and Cost Management data for an AWS account.
The IAM Console and the Sign-in Page
Learn about the IAM-enabled AWS Management Console sign-in page and how to sign in as an AWS account root user or as an IAM user. To help your users sign in easily, create a unique sign-in URL for your account.
How Users Sign In to Your Account
After you create IAM users and passwords for each, your users can sign in to the AWS Management Console for your AWS account using your account ID or alias, or from a special URL that includes your account ID.
Using Multi-Factor Authentication (MFA) in AWS
For increased security, AWS recommends that you configure MFA to help protect your AWS resources. MFA adds extra security because it requires users to enter a unique authentication code from an approved authentication device or SMS text message when they access AWS websites or services.
Working with Server Certificates
Some AWS services can use server certificates that you manage with IAM or AWS Certificate Manager (ACM). ACM is the preferred tool to provision, manage, and deploy your server certificates. Use IAM as a certificate manager only when you must support HTTPS connections in a region that is not supported by ACM.
Enabling a Virtual MFA Device
Learn how to enable and manage virtual MFA devices from the AWS Management Console.
Overview of IAM Policies
Read an overview of IAM policies, which define permissions.
Your AWS Account ID and Its Alias
Learn how to find your AWS account ID and its alias.
IAM Roles
You can delegate access to AWS resources using an IAM role. A role is similar to a user because it is an AWS identity with permission policies that determine what the identity can and cannot do in AWS. However, instead of being uniquely associated with one person, a role is intended to be assumable by anyone who needs it.
Example Policies
This collection of policies can help you define permissions for your IAM identities.
Using an IAM Role to Grant Permissions to Applications Running on Amazon EC2 Instances
Use an IAM role to manage temporary credentials for applications that run on an EC2 instance. When you use a role, you do not have to distribute long-term credentials to an EC2 instance. Instead, the role supplies temporary permissions that applications can use when they make calls to other AWS resources.
Tutorial: Delegate Access Across AWS Accounts Using IAM Roles
Learn how to use an IAM role to delegate access to resources that are in different AWS accounts that you own.
Creating Your First IAM Admin User and Group
Learn how to create an IAM group, grant the group full permissions for all AWS services, and then create an administrative IAM user for yourself by adding the user to the IAM group.
Using Instance Profiles
An instance profile is a container for an IAM role that you can use to pass role information to an EC2 instance when the instance starts. Use the commands on this page to work with instance profiles in an AWS account
Temporary Security Credentials
You can use the AWS Security Token Service (AWS STS) to create and provide trusted users with temporary security credentials that can control access to your AWS resources. Temporary security credentials work almost identically to the long-term access key credentials that your IAM users can use.

In the “Comments” section below, let us know if you would like to see anything on these or other IAM documentation pages expanded or updated to make it more useful to you.

– Stephenie

How to Use AWS Organizations to Automate End-to-End Account Creation

2017-07-24 David Schonbrun

Post Syndicated from David Schonbrun original https://aws.amazon.com/blogs/security/how-to-use-aws-organizations-to-automate-end-to-end-account-creation/

AWS Organizations offers new capabilities for managing AWS accounts, including automated account creation via the Organizations API. For example, you can bring new development teams onboard by using the Organizations API to create an account, AWS CloudFormation templates to configure the account (such as for AWS Identity and Access Management [IAM] and networking), and service control policies (SCPs) to help enforce corporate policies.

In this blog post, I demonstrate the step-by-step process for end-to-end account creation in Organizations as well as how to automate the entire process. I also show how to move a new account into an organizational unit (OU).

Process overview

The following process flow diagram illustrates the steps required to create an account, configure the account, and then move it into an OU so that the account can take advantage of the centralized SCP functionality in Organizations. The tasks in the blue nodes occur in the master account in the organization in question, and the task in the orange node occurs in the new member account I create. In this post, I provide a script (in both Bash/CLI and Python) that you can use to automate this account creation process, and I walk through each step shown in the diagram to explain the process in detail. For the purposes of this post, I use the AWS CLI in combination with CloudFormation to create and configure an account.

The account creation process

Follow the steps in this section to create an account, configure it, and move it into an OU. I am also providing a script and CloudFormation templates that you can use to automate the entire process.

1. Sign in to AWS Organizations

In order to create an account, you must sign in to your organization’s master account with a minimum of the following permissions:

organizations:DescribeOrganization
organizations:CreateAccount

2. Create a new member account

After signing in to your organization’s master account, create a new member account. Before you can create the member account, you need three pieces of information:

An account name – The friendly name of the member account, which you can find on the Accounts tab in the master account.
An email address – The email address of the owner of the new member account. This email address is used by AWS when we need to contact the account owner.
An IAM role name – The name of an IAM role that Organizations automatically preconfigures in the new member account. This role trusts the master account, allowing users in the master account to assume the role, as permitted by the master account administrator. The role also has administrator permissions in the new member account. If you do not change the role’s name, the name defaults to OrganizationAccountAccessRole.

The following AWS CLI command creates a new member account.

aws organizations create-account --email <newAccEmail> --account-name "<newAccName>" --role-name <roleName>

To explain the placeholder values in the preceding command that you must update with your own values:

newAccEmail – The email address of the owner of the new member account. This email address must not already be associated with another AWS account.
newAccName – The friendly name of the new member account.
roleName – The name of an IAM role that Organizations automatically preconfigures in the new member account. The default name is OrganizationAccountAccessRole.

This CLI command returns a request_id that uniquely identifies the request, a value that is required for in Step 3.

Important: When you create an account using Organizations, you currently cannot remove this account from your organization. This, in turn, can prevent you from later deleting the organization.

3. Verify account creation

Account creation may take a few seconds to complete, so before doing anything with the newly created account, you must first verify the account creation status. To check the status, you must have at least the following permission:

organizations:DescribeCreateAccountStatus

The following CLI command, with the request_id returned in the previous step as an input parameter, verifies that the account was created:

aws organizations describe-create-account-status --create-account-request-id <request_id>

The command returns the state of your account creation request and can have three different values: IN_PROGRESS, SUCCEEDED, and FAILED.

4. Assume a role

After you have verified that the new account has been created, configure the account. In order to configure the newly created account, you must sign in with a user who has permission to assume the role submitted in the createAccount API call. In the example in Step 1, I named the role OrganizationAccountAccessRole; however, if you revised the name of the role, you must use that revised name when assuming the role. Note that when an account is created from within an organization, cross-account trust between the master and programmatically created accounts is automatically established.

The following CLI command assumes a role.

aws sts assume-role --role-arn <role-arn> --role-session-name <"role-session-name">

To explain the placeholder values in the preceding command that you must update with your own values:

role-arn – The Amazon Resource Name (ARN) of the role to assume.
role-session-name – An identifier for the assumed role session.

5. Configure the new account

After you assume the role, build the new account’s networking, IAM, and governance resources as explained in this section. Again, to learn more about and download the account creation script and the templates that can automate this process, see “Automating the entire end-to-end process” later in this post.

Networking – Amazon VPC, web access control lists (ACLs), and Internet gateway:
1. Create a new Amazon VPC to enable you to launch AWS resources in a virtual network that you define.
2. Run the script at the end of this post to create a VPC with two subnets (one public subnet and one private subnet) in each of two Availability Zones.
3. Set up web ACLs to control traffic in and out of the subnets. You can set up network ACLs with rules similar to your security groups in order to add an additional layer of security to your VPC.
4. Connect your VPC to remote networks by using a VPN connection.
5. If the resources in the VPC require access to the Internet, create an Internet gateway to allow communication between instances in your VPC and the Internet.
IAM – Identity provider (IdP), IAM policies, and IAM roles:
1. Many customers use enterprise federated authentication (such as Active Directory) to manage users and permissions centrally outside AWS. If you use federated authentication, set up an IdP.
2. After you set up the IdP, author the customer managed IAM policies you will use.
3. Use AWS managed policies or your customer managed policies to manage access to your AWS resources.
Governance – AWS Config Rules:
1. Create AWS Config rules to help manage and enforce standards for resources deployed on AWS.
2. Develop a tagging strategy that specifies a minimum set of tags required on every taggable resource. A tagging rule checks that all resources created or edited fulfill this requirement. A noncompliance report is created to document resources that do not meet the AWS Config rule. AWS Lambda scripts can also be launched as a result of AWS Config rules.

6. Move the new account into an OU

Before allowing your development teams to access the new member account that you configured in the previous steps, apply an SCP to the account to limit the API calls that can be made by all users. To do this, you must move the member account into an OU that has an SCP attached to it.

An OU is a container for accounts. It can contain other OUs, allowing you to create a hierarchy that resembles an upside-down tree with a “root” at the top and OU “branches” that reach down, ending with accounts that are the “leaves” of the tree. When you attach a policy to one of the nodes in the hierarchy, it affects all the branches (OUs) and leaves (accounts) under it. An OU can have exactly one parent, and currently, each account can be a member of exactly one OU.

The following CLI command moves an account into an OU.

aws organizations move-account --account-id <account_id> --source-parent-id <source_parent_id> --destination-parent-id <destination_parent_id>

To explain the placeholder values in the preceding command that you must update with your own values:

account_id – The unique identifier (ID) of the account you want to move.
source_parent_id – The unique ID of the root or OU from which you want to move the account.
destination_parent_id – The unique ID of the root or OU to which you want to move the account.

7. Reduce the IAM role permissions

The OrganizationAccountAccessRole is created with full administrative permissions to enable the creation and development of the new member account. After you complete the development process and you have moved the member account into an OU, reduce the permissions of OrganizationAccountAccessRole to match your anticipated use of this role going forward.

Automating the entire end-to-end process

To help you fully automate the process of creating new member accounts, setting up those accounts, and moving new member accounts into an OU, I am providing a script in both Bash/CLI and Python. You can modify or call additional CloudFormation templates as needed.

Download the script and CloudFormation templates

Download the script and CloudFormation templates to help you automate this end-to-end process. The global variables in the script are set in the opening lines of code. Update these variables’ values, and they will flow as input parameters to the API commands when the script is executed. I have prepopulated the roleName by using AWS best practices nomenclature, but you can use a custom name.

I am including the following descriptions of the elements of the script to give you a better idea of how the script works.

Bash/CLI:

Organization-new-acc.sh – An example shell script that includes parameters, account creation, and a call to the JSON sample templates for each of three subtasks in Step 5 earlier in this post.
CF-VPC.json – An example Cloud Formation template that creates and configures a VPC in the new member account. Each AWS account must have at least one VPC as a networking construct where you can deploy customer resources. Though AWS does create a default VPC when an account is created, you must configure that VPC to meet your needs. This includes creating subnets with specific IP Classless Inter-Domain Routing (CIDR) blocks, creating gateways (including an Internet gateway, a customer gateway, a VPN tunnel, AWS Storage Gateway, Amazon API Gateway, and a NAT gateway), and VPC peering connections. Web ACLs are also part of this process to limit the source IP addresses and ports that can access the VPC. The VPC created by this script includes four subnets across two Availability Zones. Two of the subnets are public and two are private.
CF-IAM.json – An example Cloud Formation template that creates IAM roles in the new member account. As part of a security baseline, you should develop a standard set of IAM roles and related policies. Update this template with the IAM role definitions and policies you want to create in the member account to controls privilege and access.
CF-ConfigRules.json – An example Cloud Formation template that creates an AWS Config rule to enforce tagging standards on resources created in the new account.
Organization_Output.docx – Example output of the results from running Organization-new-acc.sh.

Python:

Create_account_with_iam.py – An example Python template that creates an account, moves it into an OU, applies an SCP, and then calls additional templates to deploy resources. CF-VPC.JSON can be called by this template if you first customize the .json file.

Baseline.yml – An example CloudFormation template for creating a new IAM administrative user, IAM user group, IAM role, and IAM policy in the account.

Summary

In this post, I have demonstrated the step-by-step process for end-to-end account creation in Organizations as well as how to automate the entire process. I also showed how to move a new account into an OU. This solution should save you some time and help you avoid common issues that tend to crop up in the manual account-creation process. To learn more about the features of Organizations, see the AWS Organizations User Guide. For more information about the APIs used in this post, see the Organizations API Reference.

If you have comments about this blog post, submit them in the “Comments” section below. If you have implementation or troubleshooting questions, start a new thread on the Organizations forum.

– David

Secure API Access with Amazon Cognito Federated Identities, Amazon Cognito User Pools, and Amazon API Gateway

2017-06-08 Ed Lima

Post Syndicated from Ed Lima original https://aws.amazon.com/blogs/compute/secure-api-access-with-amazon-cognito-federated-identities-amazon-cognito-user-pools-and-amazon-api-gateway/

Ed Lima, Solutions Architect

Our identities are what define us as human beings. Philosophical discussions aside, it also applies to our day-to-day lives. For instance, I need my work badge to get access to my office building or my passport to travel overseas. My identity in this case is attached to my work badge or passport. As part of the system that checks my access, these documents or objects help define whether I have access to get into the office building or travel internationally.

This exact same concept can also be applied to cloud applications and APIs. To provide secure access to your application users, you define who can access the application resources and what kind of access can be granted. Access is based on identity controls that can confirm authentication (AuthN) and authorization (AuthZ), which are different concepts. According to Wikipedia:

“The process of authorization is distinct from that of authentication. Whereas authentication is the process of verifying that “you are who you say you are,” authorization is the process of verifying that “you are permitted to do what you are trying to do.” This does not mean authorization presupposes authentication; an anonymous agent could be authorized to a limited action set.”

Amazon Cognito allows building, securing, and scaling a solution to handle user management and authentication, and to sync across platforms and devices. In this post, I discuss the different ways that you can use Amazon Cognito to authenticate API calls to Amazon API Gateway and secure access to your own API resources.

Amazon Cognito Concepts

It’s important to understand that Amazon Cognito provides three different services:

Today, I discuss the use of the first two. One service doesn’t need the other to work; however, they can be configured to work together.

Amazon Cognito Federated Identities

To use Amazon Cognito Federated Identities in your application, create an identity pool. An identity pool is a store of user data specific to your account. It can be configured to require an identity provider (IdP) for user authentication, after you enter details such as app IDs or keys related to that specific provider.

After the user is validated, the provider sends an identity token to Amazon Cognito Federated Identities. In turn, Amazon Cognito Federated Identities contacts the AWS Security Token Service (AWS STS) to retrieve temporary AWS credentials based on a configured, authenticated IAM role linked to the identity pool. The role has appropriate IAM policies attached to it and uses these policies to provide access to other AWS services.

Amazon Cognito Federated Identities currently supports the IdPs listed in the following graphic.

Continue reading Secure API Access with Amazon Cognito Federated Identities, Amazon Cognito User Pools, and Amazon API Gateway →

Securely Analyze Data from Another AWS Account with EMRFS

2017-04-07 Jigar Mistry

Post Syndicated from Jigar Mistry original https://aws.amazon.com/blogs/big-data/securely-analyze-data-from-another-aws-account-with-emrfs/

Sometimes, data to be analyzed is spread across buckets owned by different accounts. In order to ensure data security, appropriate credentials management needs to be in place. This is especially true for large enterprises storing data in different Amazon S3 buckets for different departments. For example, a customer service department may need access to data owned by the research department, but the research department needs to provide that access in a secure manner.

This aspect of securing the data can become quite complicated. Amazon EMR uses an integrated mechanism to supply user credentials for access to data stored in S3. When you use an application (Hive, Spark, etc.) on EMR to read or write a file to or from an S3 bucket, the S3 API call needs to be signed by proper credentials to be authenticated.

Usually, these credentials are provided by the EC2 instance profile that you specify during cluster launch. What if the EC2 instance profile credentials are not enough to access an S3 object, because that object requires a different set of credentials?

This post shows how you can use a custom credentials provider to access S3 objects that cannot be accessed by the default credentials provider of EMRFS.

EMRFS and EC2 instance profiles

When an EMR cluster is launched, it needs an IAM role to be specified as the Amazon EC2 instance profile. An instance profile is a container that is used to pass the permissions contained in an IAM role when the EC2 instance is starting up. The IAM role essentially defines the permissions for anyone who assumes the role.

In the case of EMR, the IAM role contained in the instance profile has permissions to access other AWS services such as Amazon S3, Amazon CloudWatch, Amazon Kinesis, etc. This role obtains temporary credentials via the EC2 instance metadata service and provides them to the application that needs to access other AWS services.

For example, when a Hive application on EMR needs to read input data from an S3 bucket (where the S3 bucket path is specified by the s3:// URI), it invokes a default credentials provider function of EMRFS. The provider in turn obtains the temporary credentials from the EC2 instance profile and uses those credentials to sign the S3 GET request.

Custom credentials providers

In certain cases, the credentials obtained by the default credentials provider might not be enough to sign requests to an S3 bucket to which your IAM user does not have permissions to access. Maybe the bucket has a different owner, or restrictive bucket policies that allow access only to a specific IAM user or role.

In situations like this, you have other options that allow your IAM user to access the data. You could modify the S3 bucket policy to allow access to your IAM user but this might be a security risk. A better option is to implement a custom credentials provider for EMRFS to ensure that your S3 requests are signed by the correct credentials. A custom credentials provider ensures that only a configured EMR cluster has access to the data in S3. It provides much better control over who can access the data.

Configuring a custom credentials provider for EMRFS

Create a credentials provider by implementing both the AWSCredentialsProvider (from the AWS Java SDK) and the Hadoop Configurable classes for use with EMRFS when it makes calls to Amazon S3.

Each implementation of AWSCredentialsProvider can choose its own strategy for loading credentials depending on the use case. You can either load credentials using the AWS STS AssumeRole API action or from a Java properties file if you would like to make API calls using the credentials of a specific IAM user. Then, package your custom credentials provider in a JAR file, upload the JAR file to your EMR cluster, and specify the class name by setting fs.s3.customAWSCredentialsProvider in the emrfs-site configuration classification.

Walkthrough

Suppose you would like to analyze data stored in an S3 bucket owned by the research department of your company which has its own AWS account. You can launch an EMR cluster in your account and leverage EMRFS to access data stored in the bucket owned by the research department.

For this example, the two accounts of your company are:

Research: [email protected] (Account ID: 123456789012)
Your department: [email protected] (Account ID: 111222333444)

Also, you have an IAM user called “data_analyst” in [email protected] This user should be granted cross-account access to read data from the bucket called “research-data” in [email protected] In order to enable cross-account access, follow these procedures:

Configure IAM inside account [email protected]
Configure IAM inside account [email protected]

Configure IAM inside account [email protected]

Sign in to the IAM console.
Choose Roles, Create New Role
Enter a name for the role, such as “demo-role”
Expand the Role for Cross-Account Access section and select role type Provide access between AWS accounts you own.
Add [email protected] as the account from which IAM users can access [email protected] This can be done by specifying the AWS account ID for [email protected] The account ID can be obtained from the My Account page in the AWS Management Console.
On the Attach Policy page, choose Next Step
Review the details and choose Create Role
In the left navigation, choose Policies, Create Policy

Choose Create Your Own Policy and name the policy as demo-role-policy, where the policy document is:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::research-data",
                "arn:aws:s3:::research-data/*"
            ]
        }
    ]
}

In the left navigation pane, choose Roles page, open the demo-role role, and choose Attach Policy. From the list of displayed policies, open demo-role-policy (which you just created in Step 9) and attach the policy
To configure the trust relationship for the role, choose Trust Relationships, Edit Trust Relationship

On the Edit Trust Relationship page, the policy document should specify the IAM user data_analyst who can assume this role

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam:: 111222333444:data_analyst"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Configure IAM inside account [email protected]

Attach an IAM policy to the user “data_analyst” which allows the user to assume the IAM role “demo-role” created in the account [email protected]

Sign in to the IAM console
Choose Policies, Create Policy

Choose Create Your Own Policy and name the policy as “data-analyst-policy”, where the policy document is:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam:: 123456789012:role/demo-role"
        }
    ]
}

Choose Create Policy
Select the policy just created, choose Policy actions, Attach
For Principal entity, select the user “data_analyst” and choose Attach policy

Implement the custom credential provider

After the IAM configurations are finished, you implement a custom credentials provider to enable the EMR cluster to access objects stored in the S3 bucket “research-data”.

The following is sample code for the custom credentials provider that reads the IAM user data_analyst credentials from a Java properties file. If the bucket URI points to a bucket in the [email protected] account, the provider then assumes the role to obtain temporary credentials. On the other hand, if the bucket URI points to a bucket in the user’s own account, the credentials are read from the EC2 instance profile for the EMR cluster.

import java.io.IOException;
import java.net.URI;
import org.apache.hadoop.conf.Configurable;
import org.apache.hadoop.conf.Configuration;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.BasicSessionCredentials;
import com.amazonaws.auth.InstanceProfileCredentialsProvider;
import com.amazonaws.auth.PropertiesCredentials;
import com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient;
import com.amazonaws.services.securitytoken.model.AssumeRoleRequest;
import com.amazonaws.services.securitytoken.model.AssumeRoleResult;

final class MyAWSCredentialsProviderWithUri implements AWSCredentialsProvider, Configurable {

	private Configuration configuration;
	private static AWSCredentials credentials, iamUserCredentials;
	private static final String role_arn =
			"arn:aws:iam::123456789012:role/demo-role";
	
	//Specifying the S3 bucket URI for the other account
	private static final String bucket_URI = "s3://research-data/";

	public MyAWSCredentialsProviderWithUri(URI uri, Configuration conf) {
		this.configuration = conf;
		if (uri.toString().startsWith(bucket_URI)) {
			try {
				//Reading the credentials of the IAM users from Java properties file
				iamUserCredentials = new PropertiesCredentials
						(MyAWSCredentialsProviderWithUri.class.getResourceAsStream("Credentials.properties"));
			} catch (IOException e) {
				e.printStackTrace();
			}
			AWSSecurityTokenServiceClient stsClient = new
					AWSSecurityTokenServiceClient(iamUserCredentials);
			//Assuming the role in the other account to obtain temporary credentials
			AssumeRoleRequest assumeRequest = new AssumeRoleRequest()
			.withRoleArn(role_arn)
			.withDurationSeconds(3600)
			.withRoleSessionName("demo");
			AssumeRoleResult assumeResult =
					stsClient.assumeRole(assumeRequest);
			BasicSessionCredentials temporaryCredentials =
					new BasicSessionCredentials(
							assumeResult.getCredentials().getAccessKeyId(),
							assumeResult.getCredentials().getSecretAccessKey(),
							assumeResult.getCredentials().getSessionToken());
			credentials = temporaryCredentials;
		} else {
			//Extracting the credentials from EC2 metadata service
			Boolean refreshCredentialsAsync = true;
			InstanceProfileCredentialsProvider creds = new InstanceProfileCredentialsProvider
					(refreshCredentialsAsync);
			credentials = creds.getCredentials();
		}
	}

	@Override
	public AWSCredentials getCredentials() {
		//Returning the credentials to EMRFS to make S3 API calls
		return credentials;
	}

	@Override
	public void refresh() {}

	@Override
	public void setConf(Configuration conf) {
	}

	@Override
	public Configuration getConf() {
		return configuration;
	}
}

As you might have noticed in the above code you are using the file “Credentials.properties” to read the IAM user data_analyst credentials, where the contents of that file are:

accessKey=AKIAxxxxxxxxxxxxxxxx
secretKey=iXbXxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Now, compile and package the code in to a JAR file to be used by your EMR cluster for cross-account S3 access. Use the following steps:

SSH in to the master node of a running EMR cluster, navigate to the home directory of the Hadoop user, create a folder called MyAWSCredentialsProvider, and add the following files in the folder:

MyAWSCredentialsProviderWithUri.java
Credentials.properties

In the folder, execute the following commands to compile and package the Java code in to a JAR file:

$ javac -cp $(hadoop classpath) MyAWSCredentialsProviderWithURI.java
$ jar -cvf MyAWSCredentialsProviderWithUri.jar MyAWSCredentialsProviderWithUri.class Credentials.properties

Upload the generated JAR file to your S3 bucket:

$ aws s3 cp MyAWSCredentialsProviderWithUri.jar s3://<YOUR-BUCKET>/

After you have executed the steps above, create a bootstrap action script (configure_emrfs_lib.sh) whose contents are as follows:

sudo aws s3 cp s3://<YOUR-BUCKET>/MyAWSCredentialsProviderWithUri.jar /usr/share/aws/emr/emrfs/auxlib/

Now, launch a new EMR cluster. Specify the full class name of the credentials provider by setting fs.s3.customAWSCredentialsProvider in the emrfs-site configuration classification. Add a custom bootstrap action (configure_emrfs_lib.sh), which copies the JAR file to the auxiliary library folder of EMRFS on all nodes of the cluster.

Using AWS CLI, run the following command to launch an EMR cluster configured with a custom credentials provider. The configuration also includes a bootstrap action to place your custom JAR file in the relevant directory.

aws emr create-cluster --applications Name=Hive --bootstrap-actions '[{"Path":"s3://<YOUR-BUCKET>/configure_emrfs_lib.sh","Name":"Custom action"}]' --ec2-attributes '{"KeyName":"<YOUR-KEY-PAIR>","InstanceProfile":"EMR_EC2_DefaultRole","SubnetId":"subnet-xxxxxxxx","EmrManagedSlaveSecurityGroup":"sg-xxxxxxxx","EmrManagedMasterSecurityGroup":"sg-xxxxxxxx"}' --service-role EMR_DefaultRole --enable-debugging --release-label emr-5.4.0 --log-uri 's3n://<YOUR-BUCKET/' --name 'test-awscredentialsprovider-emrfs' --instance-type=m3.xlarge --instance-count 3  --configurations '[{"Classification":"emrfs-site","Properties":{"fs.s3.customAWSCredentialsProvider":"MyAWSCredentialsProviderWithUri"},"Configurations":[]}]'

Now that you have a cluster running with the capability to access data in the research account, you can use various data processing applications available on EMR such as Hive, Pig, Spark or big data processing model like MapReduce on YARN to analyze the data.

There are some applications on EMR — like Presto and Oozie — that do not use EMRFS to interact with S3, so you will not be able to use these applications in this scenario.

Summary

In this post, I explained how EMRFS obtains credentials to sign API calls to S3. I covered how you can implement a custom credentials provider for EMRFS to access objects in an S3 bucket that otherwise could not be accessed using the default credentials provider. I also demonstrated how to configure cross-account S3 API access and use EMRFS to provide custom credentials. This enables your big data applications running on EMR to access data stored in an S3 bucket belonging to a different account.

For more information about using the EMRFS custom credentials provider, see Create an AWSCredentialsProvider for EMRFS.

If you have questions or suggestions, please leave a comment below.

About the Author

Jigar Mistry is a Big Data Support Engineer with Amazon Web Services. He works with customers to provide them architectural guidance and technical support for processing large datasets in the cloud using open-source applications. In his spare time, he catches up with developments regarding humanity’s efforts to colonize Mars.

Encrypt and Decrypt Amazon Kinesis Records Using AWS KMS

The Top 20 Most Viewed AWS IAM Documentation Pages in 2016

2017-01-04 Dave Bishop

Post Syndicated from Dave Bishop original https://aws.amazon.com/blogs/security/the-top-20-most-viewed-aws-iam-documentation-pages-in-2016/

The following 20 pages were the most viewed AWS Identity and Access Management (IAM) documentation pages in 2016. I have included a brief description with each link to give you a clearer idea of what each page covers. Use this list to see what other people have been viewing and perhaps to pique your own interest about a topic you’ve been meaning to research.

What Is IAM?
IAM is a web service that helps you securely control access to AWS resources for your users. You use IAM to control who can use your AWS resources (authentication) and what resources they can use and in what ways (authorization).
Creating an IAM User in Your AWS Account
You can create one or more IAM users in your AWS account. You might create an IAM user when someone joins your organization, or when you have a new application that needs to make API calls to AWS.
The IAM Console and the Sign-in Page
This page provides information about the IAM-enabled AWS Management Console sign-in page and explains how to create a unique sign-in URL for your account.
How Users Sign In to Your Account
After you create IAM users and passwords for each, users can sign in to the AWS Management Console for your AWS account with a special URL.
IAM Best Practices
To help secure your AWS resources, follow these recommendations for IAM.
IAM Policy Elements Reference
This page describes the elements that you can use in an IAM policy. The elements are listed here in the general order you use them in a policy.
Managing Access Keys for IAM Users
Users need their own access keys to make programmatic calls to AWS from the AWS Command Line Interface (AWS CLI), Tools for Windows PowerShell, the AWS SDKs, or direct HTTP calls using the APIs for individual AWS services. To fill this need, you can create, modify, view, or rotate access keys (access key IDs and secret access keys) for IAM users.
Working with Server Certificates
Some AWS services can use server certificates that you manage with IAM or AWS Certificate Manager (ACM). In many cases, we recommend that you use ACM to provision, manage, and deploy your SSL/TLS certificates.
Your AWS Account ID and Its Alias
Learn how to find your AWS account ID number and its alias.
Overview of IAM Policies
This page provides an overview of IAM policies. A policy is a document that formally states one or more permissions.
Using Multi-Factor Authentication (MFA) in AWS
For increased security, we recommend that you configure MFA to help protect your AWS resources. MFA adds extra security because it requires users to enter a unique authentication code from an approved authentication device or SMS text message when they access AWS websites or services.
Example Policies for Administering AWS Resources
This page shows some examples of policies that control access to resources in AWS services.
Using an IAM Role to Grant Permissions to Applications Running on Amazon EC2 Instances
Use an IAM role to manage temporary credentials for applications that run on an EC2 instance. When you use a role, you do not have to distribute long-term credentials to an EC2 instance. Instead, the role supplies temporary permissions that applications can use when they make calls to other AWS resources.
IAM Roles
An IAM role is similar to a user, in that it is an AWS identity with permission policies that determine what the identity can and cannot do in AWS. However, instead of being uniquely associated with one person, a role is intended to be assumable by anyone who needs it.
Enabling a Virtual MFA Device
A virtual MFA device uses a software application to generate a six-digit authentication code that is compatible with the time-based one-time password (TOTP) standard, as described in RFC 6238. The app can run on mobile hardware devices, including smartphones.
Creating Your First IAM Admin User and Group
This procedure describes how to create an IAM group named Administrators, grant the group full permissions for all AWS services, and then create an administrative IAM user for yourself by adding the user to the Administrators group.
Using Instance Profiles
An instance profile is a container for an IAM role that you can use to pass role information to an EC2 instance when the instance starts.
Working with Server Certificates
After you obtain or create a server certificate, you upload it to IAM so that other AWS services can use it. You might also need to get certificate information, rename or delete a certificate, or perform other management tasks.
Temporary Security Credentials
You can use the AWS Security Token Service (AWS STS) to create and provide trusted users with temporary security credentials that can control access to your AWS resources. Temporary security credentials work almost identically to the long-term access key credentials that your IAM users can use.
Setting an Account Password Policy for IAM Users
You can set a password policy on your AWS account to specify complexity requirements and mandatory rotation periods for your IAM users’ passwords.

In the “Comments” section below, let us know if you would like to see anything on these or other IAM documentation pages expanded or updated to make the documentation more useful for you.

– Dave

SAML Identity Federation: Follow-Up Questions, Materials, Guides, and Templates from an AWS re:Invent 2016 Workshop (SEC306)

2016-12-28 Quint Van Deman

Post Syndicated from Quint Van Deman original https://aws.amazon.com/blogs/security/saml-identity-federation-follow-up-questions-materials-guides-and-templates-from-an-aws-reinvent-2016-workshop-sec306/

As part of the re:Source Mini Con for Security Services at AWS re:Invent 2016, we conducted a workshop focused on Security Assertion Markup Language (SAML) identity federation: Choose Your Own SAML Adventure: A Self-Directed Journey to AWS Identity Federation Mastery. As part of this workshop, attendees were able to submit their own federation-focused questions to a panel of AWS experts. In this post, I share the questions and answers from that workshop because this information can benefit any AWS customer interested in identity federation.

I have also made available the full set of workshop materials, lab guides, and AWS CloudFormation templates. I encourage you to use these materials to enrich your exploration of SAML for use with AWS.

Q: SAML assertions are limited to 50,000 characters. We often hit this limit by being in too many groups. What can AWS do to resolve this size-limit problem?

A: Because the SAML assertion is ultimately part of an API call, an upper bound must be in place for the assertion size.

On the AWS side, your AWS solution architect can log a feature request on your behalf to increase the maximum size of the assertion in a future release. The AWS service teams use these feature requests, in conjunction with other avenues of customer feedback, to plan and prioritize the features they deliver. To facilitate this process you need two things: the proposed higher value to which you’d like to see the maximum size raised, and a short written description that would help us understand what this increased limit would enable you to do.

On the AWS customer’s side, we often find that these cases are most relevant to centralized cloud teams that have broad, persistent access across many roles and accounts. This access is often necessary to support troubleshooting or simply as part of an individual’s job function. However, in many cases, exchanging persistent access for just-in-time access enables the same level of access but with better levels of visibility, reduced blast radius, and better adherence to the principle of least privilege. For example, you might implement a fast, efficient, and monitored workflow that allows you to provision a user into the necessary backend directory group for a short duration when needed in lieu of that user maintaining all of those group memberships on a persistent basis. This approach could effectively resolve the limit issue you are facing.

Q: Can we use OpenID Connect (OIDC) for federated authentication and authorization into the AWS Management Console? If so, does it have a similar size limit?

A: Currently, AWS support for OIDC is oriented around providing access to AWS resources from mobile or web applications, not access to the AWS Management Console. This is possible to do, but it requires the construction of a custom identity broker. In this solution, this broker would consume the OIDC identity, use its own logic to authenticate and authorize the user (thus being subject only to any size limits you enforce on the OIDC side), and use the sts:GetFederatedToken call to vend the user an AWS Security Token Service (STS) token for either AWS Management Console use or API/CLI use. During this sts:GetFederatedToken call, you attach a scoping policy with a limit of 2,048 characters. See Creating a URL that Enables Federated Users to Access the AWS Management Console (Custom Federation Broker) for additional details about custom identity brokers.

Q: We want to eliminate permanent AWS Identity and Access Management (IAM) access keys, but we cannot do so because of third-party tools. We are contemplating using HashiCorp Vault to vend permanent keys. Vault lets us tie keys to LDAP identities. Have you seen this work elsewhere? Do you think it will work for us?

A: For third-party tools that can run within Amazon EC2, you should use EC2 instance profiles to eliminate long-term credentials and their associated management (distribution, rotation, etc.). For third-party tools that cannot run within EC2, most customers opt to leverage their existing secrets-management tools and processes for the long-lived keys. These tools are often enhanced to make use of AWS APIs such as iam:GetCredentialReport (rotation information) and iam:{Create,Update,Delete}AccessKey (rotation operations). HashiCorp Vault is a popular tool with an available AWS Quick Start Reference Deployment, but any secrets management platform that is able to efficiently fingerprint the authorized resources and is extensible to work with the previously mentioned APIs will fill this need for you nicely.

Q: Currently, we use an Object Graph Navigation Library (OGNL) script in our identity provider (IdP) to build role Amazon Resource Names (ARNs) for the role attribute in the SAML assertion. The script consumes a list of distribution list display names from our identity management platform of which the user is a member. There is a 60-character limit on display names, which leaves no room for IAM pathing (which has a 512-character limit). We are contemplating a change. The proposed solution would make AWS API calls to get role ARNs from the AWS APIs. Have you seen this before? Do you think it will work? Does the AWS SAML integration support full-length role ARNs that would include up to 512 characters for the IAM path?

A: In most cases, we recommend that you use regular expression-based transformations within your IdP to translate a list of group names to a list of role ARNs for inclusion in the SAML assertion. Without pathing, you need to know the 12-digit AWS account number and the role name in order to be able to do so, which is accomplished using our recommended group-naming convention (AWS-Acct#-RoleName). With pathing (because “/” is not a valid character for group names within most directories), you need an additional source from which to pull this third data element. This could be as simple as an extra dimension in the group name (such as AWS-Acct#-Path-RoleName); however, that would multiply the number of groups required to support the solution. Instead, you would most likely derive the path element from a user attribute, dynamic group information, or even an external information store. It should work as long as you can reliably determine all three data elements for the user.

We do not recommend drawing the path information from AWS APIs, because the logic within the IdP is authoritative for the user’s authorization. In other words, the IdP should know for which full path role ARNs the user is authorized without asking AWS. You might consider using the AWS APIs to validate that the role actually exists, but that should really be an edge case. This is because you should integrate any automation that builds and provisions the roles with the frontend authorization layer. This way, there would never be a case in which the IdP authorizes a user for a role that doesn’t exist. AWS SAML integration supports full-length role ARNs.

Q: Why do AWS STS tokens contain a session token? This makes them incompatible with third-party tools that only support permanent keys. Is there a way to get rid of the session token to make temporary keys that contain only the access_key and the secret_key components?

A: The session token contains information that AWS uses to confirm that the AWS STS token is valid. There is no way to create a token that does not contain this third component. Instead, the preferred AWS mechanisms for distributing AWS credentials to third-party tools are EC2 instance profiles (in EC2), IAM cross-account trust (SaaS), or IAM access keys with secrets management (on-premises). Using IAM access keys, you can rotate the access keys as often as the third-party tool and your secrets management platform allow.

Q: How can we use SAML to authenticate and authorize code instead of having humans do the work? One proposed solution is to use our identity management (IDM) platform to generate X.509 certificates for identities, and then present these certificates to our IdP in order to get valid SAML assertions. This could then be included in an sts:AssumeRoleWithSAML call. Have you seen this working before? Do you think it will work for us?

A: Yes, when you receive a SAML assertion from your IdP by using your desired credential form (user name/password, X.509, etc.), you can use the sts:AssumeRoleWithSAML call to retrieve an AWS STS token. See How to Implement a General Solution for Federated API/CLI Access Using SAML 2.0 for a reference implementation.

Q: As a follow-up to the previous question, how can we get code using multi-factor authentication (MFA)? There is a gauth project that uses NodeJS to generate virtual MFAs. Code could theoretically get MFA codes from the NodeJS gauth server.

A: The answer depends on your choice of IdP and MFA provider. Generically speaking, you need to authenticate (either web-based or code-based) to the IdP using all of your desired factors before the SAML assertion is generated. This assertion can then include details of the authentication mechanism used as an additional attribute in role-assumption conditions within the trust policy in AWS. This lab guide from the workshop provides further details and a how-to guide for MFA-for-SAML.

This blog post clarifies some re:Invent 2016 attendees’ questions about SAML-based federation with AWS. For more information presented in the workshop, see the full set of workshop materials, lab guides, and CloudFormation templates. If you have follow-up questions, start a new thread in the IAM forum.

– Quint

AWS CloudTrail Now Tracks Cross-Account Activity to Its Origin

2016-11-17 Kai Zhao

Post Syndicated from Kai Zhao original https://aws.amazon.com/blogs/security/aws-cloudtrail-now-tracks-cross-account-activity-to-its-origin/

You can use AWS Identity and Access Management (IAM) roles and AWS Security Token Service (STS) to set up cross-account access between AWS accounts. When you assume an IAM role in another AWS account to obtain cross-account access to services and resources in that account, AWS CloudTrail logs the cross-account activity. Starting today, CloudTrail logs AssumeRole calls in the role-owning account (the account being accessed), including the unique ID of the IAM entity (a user or role) assuming the role in the account being accessed. This additional information helps you identify the entity that requested cross-account access and then trace its subsequent cross-account activity.

In this blog post, I show how you can use the new AssumeRole log file in the role owner’s account to trace unexpected cross-account activity to its origin.

Example of AWS cross-account AWS access

Before I get into the walkthrough, let’s briefly review the basics of AWS cross-account access. The following diagram shows a typical cross-account access scenario.

In this diagram, IAM user Alice in the Dev account (the role-assuming account) needs to access the Prod account (the role-owning account). Here’s how it works:

Alice in the Dev account assumes an IAM role (WriteAccess) in the Prod account by calling AssumeRole.
STS returns a set of temporary security credentials.
Alice uses the temporary security credentials to access services and resources in the Prod account. Alice could, for example, make calls to Amazon S3 and Amazon EC2, which are granted by the WriteAccess role.

With the new CloudTrail logging behavior, you can look in the CloudTrail bucket of the role-owning account (Prod account) and see a log file of the AssumeRole call. Previously, CloudTrail only logged this in the role-assuming account (Dev account).

How do I use the new log?

Let’s walk through an example of how to trace cross-account actions back to the user or role that initiated cross-account access (note that the new logging behavior appears in Step 2 below). In this walkthrough, I show how a security administrator who notices unusual cross-account activity can trace it back to its origin.

Note: This walkthrough assumes that the administrator has access to both AWS accounts involved in the cross-account activity, which is a common use case. However, if you granted cross-account access to a third-party AWS account (such as a business partner or an ISV’s SaaS product), tracing cross-account activity back to an account that you don’t own requires voluntary participation from the third party.

Step 1: In the role-owning account, identify the cross-account action you want to investigate
The security administrator notices that someone has deleted an S3 bucket in the Prod account that no user or role should delete. The administrator looks in CloudTrail and finds the following log file of the S3 DeleteBucket action. It shows that AssumedRole credentials were used for the DeleteBucket action, which indicates that this could have been cross-account activity. Note that AssumedRole credentials don’t necessarily mean that activity was cross-account—users or roles in the same account can also assume a role.

The administrator now wants to know who assumed this IAM role used to make this request and proceeds to search their logs for an AssumeRole event with a matching accessKeyId (highlighted in the following userIdentity block).

{
    "eventVersion": "1.04",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "AROAIDPPEZS35WEXAMPLE:AliceSession",
        "arn": "arn:aws:sts::111122223333:assumed-role/WriteAccess/AliceSession",
        "accountId": "111122223333",
        "accessKeyId": "ASIAIVOTP66PNEXAMPLE",
        "sessionContext": {
            "attributes": {
                "mfaAuthenticated": "false",
                "creationDate": "2016-11-14T17:25:26Z"
            },
            "sessionIssuer": {
                "type": "Role",
                "principalId": "AROAIDPPEZS35WEXAMPLE",
                "arn": "arn:aws:iam::111122223333:role/WriteAccess",
                "accountId": "111122223333",
                "userName": "WriteAccess"
            }
        }
    },
    "eventTime": "2016-11-14T17:25:45Z",
    "eventSource": "s3.amazonaws.com",
    "eventName": "DeleteBucket",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "72.21.198.66",
    "userAgent": "[aws-cli/1.11.10 Python/2.7.8 Linux/3.2.45-0.6.wd.865.49.315.metal1.x86_64 botocore/1.4.67]",
    "requestParameters": {
        "bucketName": "ProdAccountExampleBucket"
    },
    "responseElements": null,
    "requestID": "10B2252463D56D4C",
    "eventID": "42493e77-75e5-41fa-a2e1-766555c30611",
    "eventType": "AwsApiCall",
    "recipientAccountId": "111122223333"
}

Step 2: In the role-owning account, identify the AssumeRole event that provided credentials used for the action
The administrator searches the Prod account’s CloudTrail logs and finds the log file of the AssumeRole API call that yielded the temporary security credentials used for the action in question (see the following log). This is the new CloudTrail log type available with today’s release. The log record includes who assumed the role, when the role was assumed, the credentials that were returned by the AssumeRole call, and other metadata (all of which was previously logged only in the role assumer’s account).

{
    "eventVersion": "1.05",
    "userIdentity": {
        "type": "AWSAccount",
        "principalId": "AIDAJ45Q7YFFAREXAMPLE",
        "accountId": "123456789012"
    },
    "eventTime": "2016-11-14T17:25:26Z",
    "eventSource": "sts.amazonaws.com",
    "eventName": "AssumeRole",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "72.21.198.66",
    "userAgent": "aws-cli/1.11.10 Python/2.7.8 Linux/3.2.45-0.6.wd.865.49.315.metal1.x86_64 botocore/1.4.67",
    "requestParameters": {
        "roleArn": "arn:aws:iam::111122223333:role/WriteAccess",
        "roleSessionName": "AliceSession"
    },
    "responseElements": {
        "credentials": {
            "accessKeyId": "ASIAIVOTP66PNEXAMPLE",
            "expiration": "Nov 14, 2016 6:25:26 PM",
            "sessionToken": "<encoded_session_token_blob>"
        },
        "assumedRoleUser": {
            "assumedRoleId": "AROAIDPPEZS35WEXAMPLE:AliceSession",
            "arn": "arn:aws:sts::111122223333:assumed-role/WriteAccess/AliceSession"
        }
    },
    "requestID": "54ead46a-aa8f-11e6-bb4e-dd0a0d824ead",
    "eventID": "b590c927-3148-4d8e-ad7c-8d516fdae801",
    "resources": [
        {
            "ARN": "arn:aws:iam::111122223333:role/WriteAccess",
            "accountId": "111122223333",
            "type": "AWS::IAM::Role"
        }
    ],
    "eventType": "AwsApiCall",
    "recipientAccountId": "111122223333",
    "sharedEventID": "9fe2fddd-a338-4380-b64d-ca74b7b828e5"
}

In this new log:

The green-highlighted userIdentity block contains the principalId of the IAM user or IAM role in the Dev account that called AssumeRole.
The yellow-highlighted accessKeyId contains the temporary access key ID returned by the AssumeRole call (note that CloudTrail omits the temporary secret key).
The blue-highlighted sharedEventID connects this AssumeRole log file in the role-owning account to the corresponding AssumeRole log file in the role-assuming account.

The administrator uses this temporary accessKeyId to correlate this initial AssumeRole call with subsequent cross-account activity identified in Step 1. If the administrator comes across more log files in the Prod account with a matching accessKeyId, the administrator knows that the same principalId made those API calls using temporary security credentials obtained from this AssumeRole call.

Now the administrator is ready to identify the entity in the Dev account responsible for deleting the S3 bucket in the Prod account.

Step 3: In the role-assuming account, find a matching AssumeRole event that identifies the original caller
The administrator refers to the sharedEventID value from the preceding example log to identify the original role assumer. The administrator searches in the Dev account’s CloudTrail logs for an AssumeRole log file with the same sharedEventID. When the administrator finds the corresponding AssumeRole event (see the following log for an example), she looks at the the userIdentity block to learn more details about the caller, including the friendly name of the IAM user or IAM role, and its Amazon Resource Name (ARN).

In this case, the administrator discovers that an IAM user named Alice in the Dev account (123456789012) was responsible for deleting the S3 bucket in the Prod account (111122223333).

{
    "eventVersion": "1.05",
    "userIdentity": {
        "type": "IAMUser",
        "principalId": "AIDAJ45Q7YFFAREXAMPLE",
        "arn": "arn:aws:iam::123456789012:user/Alice",
        "accountId": "123456789012",
        "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
        "userName": "Alice"
    },
    "eventTime": "2016-11-14T17:25:26Z",
    "eventSource": "sts.amazonaws.com",
    "eventName": "AssumeRole",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "72.21.198.66",
    "userAgent": "aws-cli/1.11.10 Python/2.7.8 Linux/3.2.45-0.6.wd.865.49.315.metal1.x86_64 botocore/1.4.67",
    "requestParameters": {
        "roleArn": "arn:aws:iam::111122223333:role/WriteAccess",
        "roleSessionName": "AliceSession"
    },
    "responseElements": {
        "credentials": {
            "accessKeyId": "ASIAIVOTP66PNEXAMPLE",
            "expiration": "Nov 14, 2016 6:25:26 PM",
            "sessionToken": "<encoded_session_token_blob>"
        },
        "assumedRoleUser": {
            "assumedRoleId": "AROAIDPPEZS35WEXAMPLE:AliceSession",
            "arn": "arn:aws:sts::111122223333:assumed-role/WriteAccess/AliceSession"
        }
    },
    "requestID": "54ead46a-aa8f-11e6-bb4e-dd0a0d824ead",
    "eventID": "cf1929cf-7123-4186-b85d-791c9d0974a9",
    "resources": [
        {
            "ARN": "arn:aws:iam::111122223333:role/WriteAccess",
            "accountId": "111122223333",
            "type": "AWS::IAM::Role"
        }
    ],
    "eventType": "AwsApiCall",
    "recipientAccountId": "123456789012",
    "sharedEventID": "9fe2fddd-a338-4380-b64d-ca74b7b828e5"
}

The AWS Security Blog published a blog post earlier this year about a solution for auditing cross-account activity that uses CloudTrail, Amazon CloudWatch Events, and AWS Lambda. This automated log-tracing solution works with the new logging behavior made available today.

To learn more about STS logging, see the IAM user guide. If you have comments about auditing cross-account activity or this blog post, start a new thread in the IAM forum.

– Kai

Noise

Tag Archives: AWS STS

AWS Single Sign-On Now Enables Command Line Interface Access for AWS Accounts Using Corporate Credentials

The Top 20 Most Viewed AWS IAM Documentation Pages in 2016

The collective thoughts of the interwebz

Overview of the credentials provider workflow

Outline of the steps to retrieve and use security token

Deploy the solution

1. Create an AWS IoT thing

2. Register a certificate

3. Configure an IAM role

Configure the PassRole permissions

4. Create a role alias

5. Attach a policy

Using thing attributes as policy variables

Thing substitution variables in AWS IoT policies

Thing substitution variables in IAM policies

6. Request a security token

7. Use the security token to sign a request

Conclusion

Sample application

Setup

Challenge 1: Cross-account deployment using CodePipeline

Challenge 2: Cross-account stack lookup using custom resources

Using a Lambda function as a custom resource to look up the booking SNS topic

Step1: Deploying the custom resource

Step2: Custom resource IAM role

Step3: The CloudFormation stack uses the custom resource

Step4: The custom resource assumes a role in another account

Step5: The custom resource obtains the stack exports from all stacks in the other account

Step 6: The custom resource returns the stack exports to the calling CloudFormation stack

Step7: The custom resource handles all the event types sent by the calling CloudFormation stack

Challenge 3: Cross-account SNS subscription

Permissions on booking SNS topic

Permissions on the Airmiles Lambda function

Summary

Acknowledgments

Prerequisites

Configure the maximum session duration for an existing IAM role to 4 hours

Access AWS resources with temporary security credentials using AWS CLI/SDK

Conclusion

The case for creating a segregated data loading account

Loading OpenFDA data into Amazon Redshift

Opening up the security groups

Recap

Additional Reading

About the Author

Introductory level

Advanced level

Expert level

Process overview

The account creation process

1. Sign in to AWS Organizations

2. Create a new member account

3. Verify account creation

4. Assume a role

5. Configure the new account

6. Move the new account into an OU

7. Reduce the IAM role permissions

Automating the entire end-to-end process

Download the script and CloudFormation templates

Summary

Amazon Cognito Concepts

Amazon Cognito Federated Identities

EMRFS and EC2 instance profiles

Custom credentials providers

Configuring a custom credentials provider for EMRFS

Walkthrough

Configure IAM inside account [email protected]

Configure IAM inside account [email protected]

Implement the custom credential provider

Summary

About the Author

Related

Example of AWS cross-account AWS access

How do I use the new log?

The collective thoughts of the interwebz