Tag Archives: AWS STS

How to Use AWS Organizations to Automate End-to-End Account Creation

2017-07-24 David Schonbrun

Post Syndicated from David Schonbrun original https://aws.amazon.com/blogs/security/how-to-use-aws-organizations-to-automate-end-to-end-account-creation/

AWS Organizations offers new capabilities for managing AWS accounts, including automated account creation via the Organizations API. For example, you can bring new development teams onboard by using the Organizations API to create an account, AWS CloudFormation templates to configure the account (such as for AWS Identity and Access Management [IAM] and networking), and service control policies (SCPs) to help enforce corporate policies.

In this blog post, I demonstrate the step-by-step process for end-to-end account creation in Organizations as well as how to automate the entire process. I also show how to move a new account into an organizational unit (OU).

Process overview

The following process flow diagram illustrates the steps required to create an account, configure the account, and then move it into an OU so that the account can take advantage of the centralized SCP functionality in Organizations. The tasks in the blue nodes occur in the master account in the organization in question, and the task in the orange node occurs in the new member account I create. In this post, I provide a script (in both Bash/CLI and Python) that you can use to automate this account creation process, and I walk through each step shown in the diagram to explain the process in detail. For the purposes of this post, I use the AWS CLI in combination with CloudFormation to create and configure an account.

The account creation process

Follow the steps in this section to create an account, configure it, and move it into an OU. I am also providing a script and CloudFormation templates that you can use to automate the entire process.

1. Sign in to AWS Organizations

In order to create an account, you must sign in to your organization’s master account with a minimum of the following permissions:

organizations:DescribeOrganization
organizations:CreateAccount

2. Create a new member account

After signing in to your organization’s master account, create a new member account. Before you can create the member account, you need three pieces of information:

An account name – The friendly name of the member account, which you can find on the Accounts tab in the master account.
An email address – The email address of the owner of the new member account. This email address is used by AWS when we need to contact the account owner.
An IAM role name – The name of an IAM role that Organizations automatically preconfigures in the new member account. This role trusts the master account, allowing users in the master account to assume the role, as permitted by the master account administrator. The role also has administrator permissions in the new member account. If you do not change the role’s name, the name defaults to OrganizationAccountAccessRole.

The following AWS CLI command creates a new member account.

aws organizations create-account --email <newAccEmail> --account-name "<newAccName>" --role-name <roleName>

To explain the placeholder values in the preceding command that you must update with your own values:

newAccEmail – The email address of the owner of the new member account. This email address must not already be associated with another AWS account.
newAccName – The friendly name of the new member account.
roleName – The name of an IAM role that Organizations automatically preconfigures in the new member account. The default name is OrganizationAccountAccessRole.

This CLI command returns a request_id that uniquely identifies the request, a value that is required for in Step 3.

Important: When you create an account using Organizations, you currently cannot remove this account from your organization. This, in turn, can prevent you from later deleting the organization.

3. Verify account creation

Account creation may take a few seconds to complete, so before doing anything with the newly created account, you must first verify the account creation status. To check the status, you must have at least the following permission:

organizations:DescribeCreateAccountStatus

The following CLI command, with the request_id returned in the previous step as an input parameter, verifies that the account was created:

aws organizations describe-create-account-status --create-account-request-id <request_id>

The command returns the state of your account creation request and can have three different values: IN_PROGRESS, SUCCEEDED, and FAILED.

4. Assume a role

After you have verified that the new account has been created, configure the account. In order to configure the newly created account, you must sign in with a user who has permission to assume the role submitted in the createAccount API call. In the example in Step 1, I named the role OrganizationAccountAccessRole; however, if you revised the name of the role, you must use that revised name when assuming the role. Note that when an account is created from within an organization, cross-account trust between the master and programmatically created accounts is automatically established.

The following CLI command assumes a role.

aws sts assume-role --role-arn <role-arn> --role-session-name <"role-session-name">

To explain the placeholder values in the preceding command that you must update with your own values:

role-arn – The Amazon Resource Name (ARN) of the role to assume.
role-session-name – An identifier for the assumed role session.

5. Configure the new account

After you assume the role, build the new account’s networking, IAM, and governance resources as explained in this section. Again, to learn more about and download the account creation script and the templates that can automate this process, see “Automating the entire end-to-end process” later in this post.

Networking – Amazon VPC, web access control lists (ACLs), and Internet gateway:
1. Create a new Amazon VPC to enable you to launch AWS resources in a virtual network that you define.
2. Run the script at the end of this post to create a VPC with two subnets (one public subnet and one private subnet) in each of two Availability Zones.
3. Set up web ACLs to control traffic in and out of the subnets. You can set up network ACLs with rules similar to your security groups in order to add an additional layer of security to your VPC.
4. Connect your VPC to remote networks by using a VPN connection.
5. If the resources in the VPC require access to the Internet, create an Internet gateway to allow communication between instances in your VPC and the Internet.
IAM – Identity provider (IdP), IAM policies, and IAM roles:
1. Many customers use enterprise federated authentication (such as Active Directory) to manage users and permissions centrally outside AWS. If you use federated authentication, set up an IdP.
2. After you set up the IdP, author the customer managed IAM policies you will use.
3. Use AWS managed policies or your customer managed policies to manage access to your AWS resources.
Governance – AWS Config Rules:
1. Create AWS Config rules to help manage and enforce standards for resources deployed on AWS.
2. Develop a tagging strategy that specifies a minimum set of tags required on every taggable resource. A tagging rule checks that all resources created or edited fulfill this requirement. A noncompliance report is created to document resources that do not meet the AWS Config rule. AWS Lambda scripts can also be launched as a result of AWS Config rules.

6. Move the new account into an OU

Before allowing your development teams to access the new member account that you configured in the previous steps, apply an SCP to the account to limit the API calls that can be made by all users. To do this, you must move the member account into an OU that has an SCP attached to it.

An OU is a container for accounts. It can contain other OUs, allowing you to create a hierarchy that resembles an upside-down tree with a “root” at the top and OU “branches” that reach down, ending with accounts that are the “leaves” of the tree. When you attach a policy to one of the nodes in the hierarchy, it affects all the branches (OUs) and leaves (accounts) under it. An OU can have exactly one parent, and currently, each account can be a member of exactly one OU.

The following CLI command moves an account into an OU.

aws organizations move-account --account-id <account_id> --source-parent-id <source_parent_id> --destination-parent-id <destination_parent_id>

To explain the placeholder values in the preceding command that you must update with your own values:

account_id – The unique identifier (ID) of the account you want to move.
source_parent_id – The unique ID of the root or OU from which you want to move the account.
destination_parent_id – The unique ID of the root or OU to which you want to move the account.

7. Reduce the IAM role permissions

The OrganizationAccountAccessRole is created with full administrative permissions to enable the creation and development of the new member account. After you complete the development process and you have moved the member account into an OU, reduce the permissions of OrganizationAccountAccessRole to match your anticipated use of this role going forward.

Automating the entire end-to-end process

To help you fully automate the process of creating new member accounts, setting up those accounts, and moving new member accounts into an OU, I am providing a script in both Bash/CLI and Python. You can modify or call additional CloudFormation templates as needed.

Download the script and CloudFormation templates

Download the script and CloudFormation templates to help you automate this end-to-end process. The global variables in the script are set in the opening lines of code. Update these variables’ values, and they will flow as input parameters to the API commands when the script is executed. I have prepopulated the roleName by using AWS best practices nomenclature, but you can use a custom name.

I am including the following descriptions of the elements of the script to give you a better idea of how the script works.

Bash/CLI:

Organization-new-acc.sh – An example shell script that includes parameters, account creation, and a call to the JSON sample templates for each of three subtasks in Step 5 earlier in this post.
CF-VPC.json – An example Cloud Formation template that creates and configures a VPC in the new member account. Each AWS account must have at least one VPC as a networking construct where you can deploy customer resources. Though AWS does create a default VPC when an account is created, you must configure that VPC to meet your needs. This includes creating subnets with specific IP Classless Inter-Domain Routing (CIDR) blocks, creating gateways (including an Internet gateway, a customer gateway, a VPN tunnel, AWS Storage Gateway, Amazon API Gateway, and a NAT gateway), and VPC peering connections. Web ACLs are also part of this process to limit the source IP addresses and ports that can access the VPC. The VPC created by this script includes four subnets across two Availability Zones. Two of the subnets are public and two are private.
CF-IAM.json – An example Cloud Formation template that creates IAM roles in the new member account. As part of a security baseline, you should develop a standard set of IAM roles and related policies. Update this template with the IAM role definitions and policies you want to create in the member account to controls privilege and access.
CF-ConfigRules.json – An example Cloud Formation template that creates an AWS Config rule to enforce tagging standards on resources created in the new account.
Organization_Output.docx – Example output of the results from running Organization-new-acc.sh.

Python:

Create_account_with_iam.py – An example Python template that creates an account, moves it into an OU, applies an SCP, and then calls additional templates to deploy resources. CF-VPC.JSON can be called by this template if you first customize the .json file.

Baseline.yml – An example CloudFormation template for creating a new IAM administrative user, IAM user group, IAM role, and IAM policy in the account.

Summary

In this post, I have demonstrated the step-by-step process for end-to-end account creation in Organizations as well as how to automate the entire process. I also showed how to move a new account into an OU. This solution should save you some time and help you avoid common issues that tend to crop up in the manual account-creation process. To learn more about the features of Organizations, see the AWS Organizations User Guide. For more information about the APIs used in this post, see the Organizations API Reference.

If you have comments about this blog post, submit them in the “Comments” section below. If you have implementation or troubleshooting questions, start a new thread on the Organizations forum.

– David

Secure API Access with Amazon Cognito Federated Identities, Amazon Cognito User Pools, and Amazon API Gateway

2017-06-08 Ed Lima

Post Syndicated from Ed Lima original https://aws.amazon.com/blogs/compute/secure-api-access-with-amazon-cognito-federated-identities-amazon-cognito-user-pools-and-amazon-api-gateway/

Ed Lima, Solutions Architect

Our identities are what define us as human beings. Philosophical discussions aside, it also applies to our day-to-day lives. For instance, I need my work badge to get access to my office building or my passport to travel overseas. My identity in this case is attached to my work badge or passport. As part of the system that checks my access, these documents or objects help define whether I have access to get into the office building or travel internationally.

This exact same concept can also be applied to cloud applications and APIs. To provide secure access to your application users, you define who can access the application resources and what kind of access can be granted. Access is based on identity controls that can confirm authentication (AuthN) and authorization (AuthZ), which are different concepts. According to Wikipedia:

“The process of authorization is distinct from that of authentication. Whereas authentication is the process of verifying that “you are who you say you are,” authorization is the process of verifying that “you are permitted to do what you are trying to do.” This does not mean authorization presupposes authentication; an anonymous agent could be authorized to a limited action set.”

Amazon Cognito allows building, securing, and scaling a solution to handle user management and authentication, and to sync across platforms and devices. In this post, I discuss the different ways that you can use Amazon Cognito to authenticate API calls to Amazon API Gateway and secure access to your own API resources.

Amazon Cognito Concepts

It’s important to understand that Amazon Cognito provides three different services:

Today, I discuss the use of the first two. One service doesn’t need the other to work; however, they can be configured to work together.

Amazon Cognito Federated Identities

To use Amazon Cognito Federated Identities in your application, create an identity pool. An identity pool is a store of user data specific to your account. It can be configured to require an identity provider (IdP) for user authentication, after you enter details such as app IDs or keys related to that specific provider.

After the user is validated, the provider sends an identity token to Amazon Cognito Federated Identities. In turn, Amazon Cognito Federated Identities contacts the AWS Security Token Service (AWS STS) to retrieve temporary AWS credentials based on a configured, authenticated IAM role linked to the identity pool. The role has appropriate IAM policies attached to it and uses these policies to provide access to other AWS services.

Amazon Cognito Federated Identities currently supports the IdPs listed in the following graphic.

Continue reading Secure API Access with Amazon Cognito Federated Identities, Amazon Cognito User Pools, and Amazon API Gateway →

Securely Analyze Data from Another AWS Account with EMRFS

2017-04-07 Jigar Mistry

Post Syndicated from Jigar Mistry original https://aws.amazon.com/blogs/big-data/securely-analyze-data-from-another-aws-account-with-emrfs/

Sometimes, data to be analyzed is spread across buckets owned by different accounts. In order to ensure data security, appropriate credentials management needs to be in place. This is especially true for large enterprises storing data in different Amazon S3 buckets for different departments. For example, a customer service department may need access to data owned by the research department, but the research department needs to provide that access in a secure manner.

This aspect of securing the data can become quite complicated. Amazon EMR uses an integrated mechanism to supply user credentials for access to data stored in S3. When you use an application (Hive, Spark, etc.) on EMR to read or write a file to or from an S3 bucket, the S3 API call needs to be signed by proper credentials to be authenticated.

Usually, these credentials are provided by the EC2 instance profile that you specify during cluster launch. What if the EC2 instance profile credentials are not enough to access an S3 object, because that object requires a different set of credentials?

This post shows how you can use a custom credentials provider to access S3 objects that cannot be accessed by the default credentials provider of EMRFS.

EMRFS and EC2 instance profiles

When an EMR cluster is launched, it needs an IAM role to be specified as the Amazon EC2 instance profile. An instance profile is a container that is used to pass the permissions contained in an IAM role when the EC2 instance is starting up. The IAM role essentially defines the permissions for anyone who assumes the role.

In the case of EMR, the IAM role contained in the instance profile has permissions to access other AWS services such as Amazon S3, Amazon CloudWatch, Amazon Kinesis, etc. This role obtains temporary credentials via the EC2 instance metadata service and provides them to the application that needs to access other AWS services.

For example, when a Hive application on EMR needs to read input data from an S3 bucket (where the S3 bucket path is specified by the s3:// URI), it invokes a default credentials provider function of EMRFS. The provider in turn obtains the temporary credentials from the EC2 instance profile and uses those credentials to sign the S3 GET request.

Custom credentials providers

In certain cases, the credentials obtained by the default credentials provider might not be enough to sign requests to an S3 bucket to which your IAM user does not have permissions to access. Maybe the bucket has a different owner, or restrictive bucket policies that allow access only to a specific IAM user or role.

In situations like this, you have other options that allow your IAM user to access the data. You could modify the S3 bucket policy to allow access to your IAM user but this might be a security risk. A better option is to implement a custom credentials provider for EMRFS to ensure that your S3 requests are signed by the correct credentials. A custom credentials provider ensures that only a configured EMR cluster has access to the data in S3. It provides much better control over who can access the data.

Configuring a custom credentials provider for EMRFS

Create a credentials provider by implementing both the AWSCredentialsProvider (from the AWS Java SDK) and the Hadoop Configurable classes for use with EMRFS when it makes calls to Amazon S3.

Each implementation of AWSCredentialsProvider can choose its own strategy for loading credentials depending on the use case. You can either load credentials using the AWS STS AssumeRole API action or from a Java properties file if you would like to make API calls using the credentials of a specific IAM user. Then, package your custom credentials provider in a JAR file, upload the JAR file to your EMR cluster, and specify the class name by setting fs.s3.customAWSCredentialsProvider in the emrfs-site configuration classification.

Walkthrough

Suppose you would like to analyze data stored in an S3 bucket owned by the research department of your company which has its own AWS account. You can launch an EMR cluster in your account and leverage EMRFS to access data stored in the bucket owned by the research department.

For this example, the two accounts of your company are:

Research: [email protected] (Account ID: 123456789012)
Your department: [email protected] (Account ID: 111222333444)

Also, you have an IAM user called “data_analyst” in [email protected] This user should be granted cross-account access to read data from the bucket called “research-data” in [email protected] In order to enable cross-account access, follow these procedures:

Configure IAM inside account [email protected]
Configure IAM inside account [email protected]

Configure IAM inside account [email protected]

Sign in to the IAM console.
Choose Roles, Create New Role
Enter a name for the role, such as “demo-role”
Expand the Role for Cross-Account Access section and select role type Provide access between AWS accounts you own.
Add [email protected] as the account from which IAM users can access [email protected] This can be done by specifying the AWS account ID for [email protected] The account ID can be obtained from the My Account page in the AWS Management Console.
On the Attach Policy page, choose Next Step
Review the details and choose Create Role
In the left navigation, choose Policies, Create Policy

Choose Create Your Own Policy and name the policy as demo-role-policy, where the policy document is:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::research-data",
                "arn:aws:s3:::research-data/*"
            ]
        }
    ]
}

In the left navigation pane, choose Roles page, open the demo-role role, and choose Attach Policy. From the list of displayed policies, open demo-role-policy (which you just created in Step 9) and attach the policy
To configure the trust relationship for the role, choose Trust Relationships, Edit Trust Relationship

On the Edit Trust Relationship page, the policy document should specify the IAM user data_analyst who can assume this role

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam:: 111222333444:data_analyst"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Configure IAM inside account [email protected]

Attach an IAM policy to the user “data_analyst” which allows the user to assume the IAM role “demo-role” created in the account [email protected]

Sign in to the IAM console
Choose Policies, Create Policy

Choose Create Your Own Policy and name the policy as “data-analyst-policy”, where the policy document is:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam:: 123456789012:role/demo-role"
        }
    ]
}

Choose Create Policy
Select the policy just created, choose Policy actions, Attach
For Principal entity, select the user “data_analyst” and choose Attach policy

Implement the custom credential provider

After the IAM configurations are finished, you implement a custom credentials provider to enable the EMR cluster to access objects stored in the S3 bucket “research-data”.

The following is sample code for the custom credentials provider that reads the IAM user data_analyst credentials from a Java properties file. If the bucket URI points to a bucket in the [email protected] account, the provider then assumes the role to obtain temporary credentials. On the other hand, if the bucket URI points to a bucket in the user’s own account, the credentials are read from the EC2 instance profile for the EMR cluster.

import java.io.IOException;
import java.net.URI;
import org.apache.hadoop.conf.Configurable;
import org.apache.hadoop.conf.Configuration;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.BasicSessionCredentials;
import com.amazonaws.auth.InstanceProfileCredentialsProvider;
import com.amazonaws.auth.PropertiesCredentials;
import com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient;
import com.amazonaws.services.securitytoken.model.AssumeRoleRequest;
import com.amazonaws.services.securitytoken.model.AssumeRoleResult;

final class MyAWSCredentialsProviderWithUri implements AWSCredentialsProvider, Configurable {

	private Configuration configuration;
	private static AWSCredentials credentials, iamUserCredentials;
	private static final String role_arn =
			"arn:aws:iam::123456789012:role/demo-role";
	
	//Specifying the S3 bucket URI for the other account
	private static final String bucket_URI = "s3://research-data/";

	public MyAWSCredentialsProviderWithUri(URI uri, Configuration conf) {
		this.configuration = conf;
		if (uri.toString().startsWith(bucket_URI)) {
			try {
				//Reading the credentials of the IAM users from Java properties file
				iamUserCredentials = new PropertiesCredentials
						(MyAWSCredentialsProviderWithUri.class.getResourceAsStream("Credentials.properties"));
			} catch (IOException e) {
				e.printStackTrace();
			}
			AWSSecurityTokenServiceClient stsClient = new
					AWSSecurityTokenServiceClient(iamUserCredentials);
			//Assuming the role in the other account to obtain temporary credentials
			AssumeRoleRequest assumeRequest = new AssumeRoleRequest()
			.withRoleArn(role_arn)
			.withDurationSeconds(3600)
			.withRoleSessionName("demo");
			AssumeRoleResult assumeResult =
					stsClient.assumeRole(assumeRequest);
			BasicSessionCredentials temporaryCredentials =
					new BasicSessionCredentials(
							assumeResult.getCredentials().getAccessKeyId(),
							assumeResult.getCredentials().getSecretAccessKey(),
							assumeResult.getCredentials().getSessionToken());
			credentials = temporaryCredentials;
		} else {
			//Extracting the credentials from EC2 metadata service
			Boolean refreshCredentialsAsync = true;
			InstanceProfileCredentialsProvider creds = new InstanceProfileCredentialsProvider
					(refreshCredentialsAsync);
			credentials = creds.getCredentials();
		}
	}

	@Override
	public AWSCredentials getCredentials() {
		//Returning the credentials to EMRFS to make S3 API calls
		return credentials;
	}

	@Override
	public void refresh() {}

	@Override
	public void setConf(Configuration conf) {
	}

	@Override
	public Configuration getConf() {
		return configuration;
	}
}

As you might have noticed in the above code you are using the file “Credentials.properties” to read the IAM user data_analyst credentials, where the contents of that file are:

accessKey=AKIAxxxxxxxxxxxxxxxx
secretKey=iXbXxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Now, compile and package the code in to a JAR file to be used by your EMR cluster for cross-account S3 access. Use the following steps:

SSH in to the master node of a running EMR cluster, navigate to the home directory of the Hadoop user, create a folder called MyAWSCredentialsProvider, and add the following files in the folder:

MyAWSCredentialsProviderWithUri.java
Credentials.properties

In the folder, execute the following commands to compile and package the Java code in to a JAR file:

$ javac -cp $(hadoop classpath) MyAWSCredentialsProviderWithURI.java
$ jar -cvf MyAWSCredentialsProviderWithUri.jar MyAWSCredentialsProviderWithUri.class Credentials.properties

Upload the generated JAR file to your S3 bucket:

$ aws s3 cp MyAWSCredentialsProviderWithUri.jar s3://<YOUR-BUCKET>/

After you have executed the steps above, create a bootstrap action script (configure_emrfs_lib.sh) whose contents are as follows:

sudo aws s3 cp s3://<YOUR-BUCKET>/MyAWSCredentialsProviderWithUri.jar /usr/share/aws/emr/emrfs/auxlib/

Now, launch a new EMR cluster. Specify the full class name of the credentials provider by setting fs.s3.customAWSCredentialsProvider in the emrfs-site configuration classification. Add a custom bootstrap action (configure_emrfs_lib.sh), which copies the JAR file to the auxiliary library folder of EMRFS on all nodes of the cluster.

Using AWS CLI, run the following command to launch an EMR cluster configured with a custom credentials provider. The configuration also includes a bootstrap action to place your custom JAR file in the relevant directory.

aws emr create-cluster --applications Name=Hive --bootstrap-actions '[{"Path":"s3://<YOUR-BUCKET>/configure_emrfs_lib.sh","Name":"Custom action"}]' --ec2-attributes '{"KeyName":"<YOUR-KEY-PAIR>","InstanceProfile":"EMR_EC2_DefaultRole","SubnetId":"subnet-xxxxxxxx","EmrManagedSlaveSecurityGroup":"sg-xxxxxxxx","EmrManagedMasterSecurityGroup":"sg-xxxxxxxx"}' --service-role EMR_DefaultRole --enable-debugging --release-label emr-5.4.0 --log-uri 's3n://<YOUR-BUCKET/' --name 'test-awscredentialsprovider-emrfs' --instance-type=m3.xlarge --instance-count 3  --configurations '[{"Classification":"emrfs-site","Properties":{"fs.s3.customAWSCredentialsProvider":"MyAWSCredentialsProviderWithUri"},"Configurations":[]}]'

Now that you have a cluster running with the capability to access data in the research account, you can use various data processing applications available on EMR such as Hive, Pig, Spark or big data processing model like MapReduce on YARN to analyze the data.

There are some applications on EMR — like Presto and Oozie — that do not use EMRFS to interact with S3, so you will not be able to use these applications in this scenario.

Summary

In this post, I explained how EMRFS obtains credentials to sign API calls to S3. I covered how you can implement a custom credentials provider for EMRFS to access objects in an S3 bucket that otherwise could not be accessed using the default credentials provider. I also demonstrated how to configure cross-account S3 API access and use EMRFS to provide custom credentials. This enables your big data applications running on EMR to access data stored in an S3 bucket belonging to a different account.

For more information about using the EMRFS custom credentials provider, see Create an AWSCredentialsProvider for EMRFS.

If you have questions or suggestions, please leave a comment below.

About the Author

Jigar Mistry is a Big Data Support Engineer with Amazon Web Services. He works with customers to provide them architectural guidance and technical support for processing large datasets in the cloud using open-source applications. In his spare time, he catches up with developments regarding humanity’s efforts to colonize Mars.

Encrypt and Decrypt Amazon Kinesis Records Using AWS KMS

The Top 20 Most Viewed AWS IAM Documentation Pages in 2016

2017-01-04 Dave Bishop

Post Syndicated from Dave Bishop original https://aws.amazon.com/blogs/security/the-top-20-most-viewed-aws-iam-documentation-pages-in-2016/

The following 20 pages were the most viewed AWS Identity and Access Management (IAM) documentation pages in 2016. I have included a brief description with each link to give you a clearer idea of what each page covers. Use this list to see what other people have been viewing and perhaps to pique your own interest about a topic you’ve been meaning to research.

What Is IAM?
IAM is a web service that helps you securely control access to AWS resources for your users. You use IAM to control who can use your AWS resources (authentication) and what resources they can use and in what ways (authorization).
Creating an IAM User in Your AWS Account
You can create one or more IAM users in your AWS account. You might create an IAM user when someone joins your organization, or when you have a new application that needs to make API calls to AWS.
The IAM Console and the Sign-in Page
This page provides information about the IAM-enabled AWS Management Console sign-in page and explains how to create a unique sign-in URL for your account.
How Users Sign In to Your Account
After you create IAM users and passwords for each, users can sign in to the AWS Management Console for your AWS account with a special URL.
IAM Best Practices
To help secure your AWS resources, follow these recommendations for IAM.
IAM Policy Elements Reference
This page describes the elements that you can use in an IAM policy. The elements are listed here in the general order you use them in a policy.
Managing Access Keys for IAM Users
Users need their own access keys to make programmatic calls to AWS from the AWS Command Line Interface (AWS CLI), Tools for Windows PowerShell, the AWS SDKs, or direct HTTP calls using the APIs for individual AWS services. To fill this need, you can create, modify, view, or rotate access keys (access key IDs and secret access keys) for IAM users.
Working with Server Certificates
Some AWS services can use server certificates that you manage with IAM or AWS Certificate Manager (ACM). In many cases, we recommend that you use ACM to provision, manage, and deploy your SSL/TLS certificates.
Your AWS Account ID and Its Alias
Learn how to find your AWS account ID number and its alias.
Overview of IAM Policies
This page provides an overview of IAM policies. A policy is a document that formally states one or more permissions.
Using Multi-Factor Authentication (MFA) in AWS
For increased security, we recommend that you configure MFA to help protect your AWS resources. MFA adds extra security because it requires users to enter a unique authentication code from an approved authentication device or SMS text message when they access AWS websites or services.
Example Policies for Administering AWS Resources
This page shows some examples of policies that control access to resources in AWS services.
Using an IAM Role to Grant Permissions to Applications Running on Amazon EC2 Instances
Use an IAM role to manage temporary credentials for applications that run on an EC2 instance. When you use a role, you do not have to distribute long-term credentials to an EC2 instance. Instead, the role supplies temporary permissions that applications can use when they make calls to other AWS resources.
IAM Roles
An IAM role is similar to a user, in that it is an AWS identity with permission policies that determine what the identity can and cannot do in AWS. However, instead of being uniquely associated with one person, a role is intended to be assumable by anyone who needs it.
Enabling a Virtual MFA Device
A virtual MFA device uses a software application to generate a six-digit authentication code that is compatible with the time-based one-time password (TOTP) standard, as described in RFC 6238. The app can run on mobile hardware devices, including smartphones.
Creating Your First IAM Admin User and Group
This procedure describes how to create an IAM group named Administrators, grant the group full permissions for all AWS services, and then create an administrative IAM user for yourself by adding the user to the Administrators group.
Using Instance Profiles
An instance profile is a container for an IAM role that you can use to pass role information to an EC2 instance when the instance starts.
Working with Server Certificates
After you obtain or create a server certificate, you upload it to IAM so that other AWS services can use it. You might also need to get certificate information, rename or delete a certificate, or perform other management tasks.
Temporary Security Credentials
You can use the AWS Security Token Service (AWS STS) to create and provide trusted users with temporary security credentials that can control access to your AWS resources. Temporary security credentials work almost identically to the long-term access key credentials that your IAM users can use.
Setting an Account Password Policy for IAM Users
You can set a password policy on your AWS account to specify complexity requirements and mandatory rotation periods for your IAM users’ passwords.

In the “Comments” section below, let us know if you would like to see anything on these or other IAM documentation pages expanded or updated to make the documentation more useful for you.

– Dave

SAML Identity Federation: Follow-Up Questions, Materials, Guides, and Templates from an AWS re:Invent 2016 Workshop (SEC306)

2016-12-28 Quint Van Deman

Post Syndicated from Quint Van Deman original https://aws.amazon.com/blogs/security/saml-identity-federation-follow-up-questions-materials-guides-and-templates-from-an-aws-reinvent-2016-workshop-sec306/

As part of the re:Source Mini Con for Security Services at AWS re:Invent 2016, we conducted a workshop focused on Security Assertion Markup Language (SAML) identity federation: Choose Your Own SAML Adventure: A Self-Directed Journey to AWS Identity Federation Mastery. As part of this workshop, attendees were able to submit their own federation-focused questions to a panel of AWS experts. In this post, I share the questions and answers from that workshop because this information can benefit any AWS customer interested in identity federation.

I have also made available the full set of workshop materials, lab guides, and AWS CloudFormation templates. I encourage you to use these materials to enrich your exploration of SAML for use with AWS.

Q: SAML assertions are limited to 50,000 characters. We often hit this limit by being in too many groups. What can AWS do to resolve this size-limit problem?

A: Because the SAML assertion is ultimately part of an API call, an upper bound must be in place for the assertion size.

On the AWS side, your AWS solution architect can log a feature request on your behalf to increase the maximum size of the assertion in a future release. The AWS service teams use these feature requests, in conjunction with other avenues of customer feedback, to plan and prioritize the features they deliver. To facilitate this process you need two things: the proposed higher value to which you’d like to see the maximum size raised, and a short written description that would help us understand what this increased limit would enable you to do.

On the AWS customer’s side, we often find that these cases are most relevant to centralized cloud teams that have broad, persistent access across many roles and accounts. This access is often necessary to support troubleshooting or simply as part of an individual’s job function. However, in many cases, exchanging persistent access for just-in-time access enables the same level of access but with better levels of visibility, reduced blast radius, and better adherence to the principle of least privilege. For example, you might implement a fast, efficient, and monitored workflow that allows you to provision a user into the necessary backend directory group for a short duration when needed in lieu of that user maintaining all of those group memberships on a persistent basis. This approach could effectively resolve the limit issue you are facing.

Q: Can we use OpenID Connect (OIDC) for federated authentication and authorization into the AWS Management Console? If so, does it have a similar size limit?

A: Currently, AWS support for OIDC is oriented around providing access to AWS resources from mobile or web applications, not access to the AWS Management Console. This is possible to do, but it requires the construction of a custom identity broker. In this solution, this broker would consume the OIDC identity, use its own logic to authenticate and authorize the user (thus being subject only to any size limits you enforce on the OIDC side), and use the sts:GetFederatedToken call to vend the user an AWS Security Token Service (STS) token for either AWS Management Console use or API/CLI use. During this sts:GetFederatedToken call, you attach a scoping policy with a limit of 2,048 characters. See Creating a URL that Enables Federated Users to Access the AWS Management Console (Custom Federation Broker) for additional details about custom identity brokers.

Q: We want to eliminate permanent AWS Identity and Access Management (IAM) access keys, but we cannot do so because of third-party tools. We are contemplating using HashiCorp Vault to vend permanent keys. Vault lets us tie keys to LDAP identities. Have you seen this work elsewhere? Do you think it will work for us?

A: For third-party tools that can run within Amazon EC2, you should use EC2 instance profiles to eliminate long-term credentials and their associated management (distribution, rotation, etc.). For third-party tools that cannot run within EC2, most customers opt to leverage their existing secrets-management tools and processes for the long-lived keys. These tools are often enhanced to make use of AWS APIs such as iam:GetCredentialReport (rotation information) and iam:{Create,Update,Delete}AccessKey (rotation operations). HashiCorp Vault is a popular tool with an available AWS Quick Start Reference Deployment, but any secrets management platform that is able to efficiently fingerprint the authorized resources and is extensible to work with the previously mentioned APIs will fill this need for you nicely.

Q: Currently, we use an Object Graph Navigation Library (OGNL) script in our identity provider (IdP) to build role Amazon Resource Names (ARNs) for the role attribute in the SAML assertion. The script consumes a list of distribution list display names from our identity management platform of which the user is a member. There is a 60-character limit on display names, which leaves no room for IAM pathing (which has a 512-character limit). We are contemplating a change. The proposed solution would make AWS API calls to get role ARNs from the AWS APIs. Have you seen this before? Do you think it will work? Does the AWS SAML integration support full-length role ARNs that would include up to 512 characters for the IAM path?

A: In most cases, we recommend that you use regular expression-based transformations within your IdP to translate a list of group names to a list of role ARNs for inclusion in the SAML assertion. Without pathing, you need to know the 12-digit AWS account number and the role name in order to be able to do so, which is accomplished using our recommended group-naming convention (AWS-Acct#-RoleName). With pathing (because “/” is not a valid character for group names within most directories), you need an additional source from which to pull this third data element. This could be as simple as an extra dimension in the group name (such as AWS-Acct#-Path-RoleName); however, that would multiply the number of groups required to support the solution. Instead, you would most likely derive the path element from a user attribute, dynamic group information, or even an external information store. It should work as long as you can reliably determine all three data elements for the user.

We do not recommend drawing the path information from AWS APIs, because the logic within the IdP is authoritative for the user’s authorization. In other words, the IdP should know for which full path role ARNs the user is authorized without asking AWS. You might consider using the AWS APIs to validate that the role actually exists, but that should really be an edge case. This is because you should integrate any automation that builds and provisions the roles with the frontend authorization layer. This way, there would never be a case in which the IdP authorizes a user for a role that doesn’t exist. AWS SAML integration supports full-length role ARNs.

Q: Why do AWS STS tokens contain a session token? This makes them incompatible with third-party tools that only support permanent keys. Is there a way to get rid of the session token to make temporary keys that contain only the access_key and the secret_key components?

A: The session token contains information that AWS uses to confirm that the AWS STS token is valid. There is no way to create a token that does not contain this third component. Instead, the preferred AWS mechanisms for distributing AWS credentials to third-party tools are EC2 instance profiles (in EC2), IAM cross-account trust (SaaS), or IAM access keys with secrets management (on-premises). Using IAM access keys, you can rotate the access keys as often as the third-party tool and your secrets management platform allow.

Q: How can we use SAML to authenticate and authorize code instead of having humans do the work? One proposed solution is to use our identity management (IDM) platform to generate X.509 certificates for identities, and then present these certificates to our IdP in order to get valid SAML assertions. This could then be included in an sts:AssumeRoleWithSAML call. Have you seen this working before? Do you think it will work for us?

A: Yes, when you receive a SAML assertion from your IdP by using your desired credential form (user name/password, X.509, etc.), you can use the sts:AssumeRoleWithSAML call to retrieve an AWS STS token. See How to Implement a General Solution for Federated API/CLI Access Using SAML 2.0 for a reference implementation.

Q: As a follow-up to the previous question, how can we get code using multi-factor authentication (MFA)? There is a gauth project that uses NodeJS to generate virtual MFAs. Code could theoretically get MFA codes from the NodeJS gauth server.

A: The answer depends on your choice of IdP and MFA provider. Generically speaking, you need to authenticate (either web-based or code-based) to the IdP using all of your desired factors before the SAML assertion is generated. This assertion can then include details of the authentication mechanism used as an additional attribute in role-assumption conditions within the trust policy in AWS. This lab guide from the workshop provides further details and a how-to guide for MFA-for-SAML.

This blog post clarifies some re:Invent 2016 attendees’ questions about SAML-based federation with AWS. For more information presented in the workshop, see the full set of workshop materials, lab guides, and CloudFormation templates. If you have follow-up questions, start a new thread in the IAM forum.

– Quint

AWS CloudTrail Now Tracks Cross-Account Activity to Its Origin

2016-11-17 Kai Zhao

Post Syndicated from Kai Zhao original https://aws.amazon.com/blogs/security/aws-cloudtrail-now-tracks-cross-account-activity-to-its-origin/

You can use AWS Identity and Access Management (IAM) roles and AWS Security Token Service (STS) to set up cross-account access between AWS accounts. When you assume an IAM role in another AWS account to obtain cross-account access to services and resources in that account, AWS CloudTrail logs the cross-account activity. Starting today, CloudTrail logs AssumeRole calls in the role-owning account (the account being accessed), including the unique ID of the IAM entity (a user or role) assuming the role in the account being accessed. This additional information helps you identify the entity that requested cross-account access and then trace its subsequent cross-account activity.

In this blog post, I show how you can use the new AssumeRole log file in the role owner’s account to trace unexpected cross-account activity to its origin.

Example of AWS cross-account AWS access

Before I get into the walkthrough, let’s briefly review the basics of AWS cross-account access. The following diagram shows a typical cross-account access scenario.

In this diagram, IAM user Alice in the Dev account (the role-assuming account) needs to access the Prod account (the role-owning account). Here’s how it works:

Alice in the Dev account assumes an IAM role (WriteAccess) in the Prod account by calling AssumeRole.
STS returns a set of temporary security credentials.
Alice uses the temporary security credentials to access services and resources in the Prod account. Alice could, for example, make calls to Amazon S3 and Amazon EC2, which are granted by the WriteAccess role.

With the new CloudTrail logging behavior, you can look in the CloudTrail bucket of the role-owning account (Prod account) and see a log file of the AssumeRole call. Previously, CloudTrail only logged this in the role-assuming account (Dev account).

How do I use the new log?

Let’s walk through an example of how to trace cross-account actions back to the user or role that initiated cross-account access (note that the new logging behavior appears in Step 2 below). In this walkthrough, I show how a security administrator who notices unusual cross-account activity can trace it back to its origin.

Note: This walkthrough assumes that the administrator has access to both AWS accounts involved in the cross-account activity, which is a common use case. However, if you granted cross-account access to a third-party AWS account (such as a business partner or an ISV’s SaaS product), tracing cross-account activity back to an account that you don’t own requires voluntary participation from the third party.

Step 1: In the role-owning account, identify the cross-account action you want to investigate
The security administrator notices that someone has deleted an S3 bucket in the Prod account that no user or role should delete. The administrator looks in CloudTrail and finds the following log file of the S3 DeleteBucket action. It shows that AssumedRole credentials were used for the DeleteBucket action, which indicates that this could have been cross-account activity. Note that AssumedRole credentials don’t necessarily mean that activity was cross-account—users or roles in the same account can also assume a role.

The administrator now wants to know who assumed this IAM role used to make this request and proceeds to search their logs for an AssumeRole event with a matching accessKeyId (highlighted in the following userIdentity block).

{
    "eventVersion": "1.04",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "AROAIDPPEZS35WEXAMPLE:AliceSession",
        "arn": "arn:aws:sts::111122223333:assumed-role/WriteAccess/AliceSession",
        "accountId": "111122223333",
        "accessKeyId": "ASIAIVOTP66PNEXAMPLE",
        "sessionContext": {
            "attributes": {
                "mfaAuthenticated": "false",
                "creationDate": "2016-11-14T17:25:26Z"
            },
            "sessionIssuer": {
                "type": "Role",
                "principalId": "AROAIDPPEZS35WEXAMPLE",
                "arn": "arn:aws:iam::111122223333:role/WriteAccess",
                "accountId": "111122223333",
                "userName": "WriteAccess"
            }
        }
    },
    "eventTime": "2016-11-14T17:25:45Z",
    "eventSource": "s3.amazonaws.com",
    "eventName": "DeleteBucket",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "72.21.198.66",
    "userAgent": "[aws-cli/1.11.10 Python/2.7.8 Linux/3.2.45-0.6.wd.865.49.315.metal1.x86_64 botocore/1.4.67]",
    "requestParameters": {
        "bucketName": "ProdAccountExampleBucket"
    },
    "responseElements": null,
    "requestID": "10B2252463D56D4C",
    "eventID": "42493e77-75e5-41fa-a2e1-766555c30611",
    "eventType": "AwsApiCall",
    "recipientAccountId": "111122223333"
}

Step 2: In the role-owning account, identify the AssumeRole event that provided credentials used for the action
The administrator searches the Prod account’s CloudTrail logs and finds the log file of the AssumeRole API call that yielded the temporary security credentials used for the action in question (see the following log). This is the new CloudTrail log type available with today’s release. The log record includes who assumed the role, when the role was assumed, the credentials that were returned by the AssumeRole call, and other metadata (all of which was previously logged only in the role assumer’s account).

{
    "eventVersion": "1.05",
    "userIdentity": {
        "type": "AWSAccount",
        "principalId": "AIDAJ45Q7YFFAREXAMPLE",
        "accountId": "123456789012"
    },
    "eventTime": "2016-11-14T17:25:26Z",
    "eventSource": "sts.amazonaws.com",
    "eventName": "AssumeRole",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "72.21.198.66",
    "userAgent": "aws-cli/1.11.10 Python/2.7.8 Linux/3.2.45-0.6.wd.865.49.315.metal1.x86_64 botocore/1.4.67",
    "requestParameters": {
        "roleArn": "arn:aws:iam::111122223333:role/WriteAccess",
        "roleSessionName": "AliceSession"
    },
    "responseElements": {
        "credentials": {
            "accessKeyId": "ASIAIVOTP66PNEXAMPLE",
            "expiration": "Nov 14, 2016 6:25:26 PM",
            "sessionToken": "<encoded_session_token_blob>"
        },
        "assumedRoleUser": {
            "assumedRoleId": "AROAIDPPEZS35WEXAMPLE:AliceSession",
            "arn": "arn:aws:sts::111122223333:assumed-role/WriteAccess/AliceSession"
        }
    },
    "requestID": "54ead46a-aa8f-11e6-bb4e-dd0a0d824ead",
    "eventID": "b590c927-3148-4d8e-ad7c-8d516fdae801",
    "resources": [
        {
            "ARN": "arn:aws:iam::111122223333:role/WriteAccess",
            "accountId": "111122223333",
            "type": "AWS::IAM::Role"
        }
    ],
    "eventType": "AwsApiCall",
    "recipientAccountId": "111122223333",
    "sharedEventID": "9fe2fddd-a338-4380-b64d-ca74b7b828e5"
}

In this new log:

The green-highlighted userIdentity block contains the principalId of the IAM user or IAM role in the Dev account that called AssumeRole.
The yellow-highlighted accessKeyId contains the temporary access key ID returned by the AssumeRole call (note that CloudTrail omits the temporary secret key).
The blue-highlighted sharedEventID connects this AssumeRole log file in the role-owning account to the corresponding AssumeRole log file in the role-assuming account.

The administrator uses this temporary accessKeyId to correlate this initial AssumeRole call with subsequent cross-account activity identified in Step 1. If the administrator comes across more log files in the Prod account with a matching accessKeyId, the administrator knows that the same principalId made those API calls using temporary security credentials obtained from this AssumeRole call.

Now the administrator is ready to identify the entity in the Dev account responsible for deleting the S3 bucket in the Prod account.

Step 3: In the role-assuming account, find a matching AssumeRole event that identifies the original caller
The administrator refers to the sharedEventID value from the preceding example log to identify the original role assumer. The administrator searches in the Dev account’s CloudTrail logs for an AssumeRole log file with the same sharedEventID. When the administrator finds the corresponding AssumeRole event (see the following log for an example), she looks at the the userIdentity block to learn more details about the caller, including the friendly name of the IAM user or IAM role, and its Amazon Resource Name (ARN).

In this case, the administrator discovers that an IAM user named Alice in the Dev account (123456789012) was responsible for deleting the S3 bucket in the Prod account (111122223333).

{
    "eventVersion": "1.05",
    "userIdentity": {
        "type": "IAMUser",
        "principalId": "AIDAJ45Q7YFFAREXAMPLE",
        "arn": "arn:aws:iam::123456789012:user/Alice",
        "accountId": "123456789012",
        "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
        "userName": "Alice"
    },
    "eventTime": "2016-11-14T17:25:26Z",
    "eventSource": "sts.amazonaws.com",
    "eventName": "AssumeRole",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "72.21.198.66",
    "userAgent": "aws-cli/1.11.10 Python/2.7.8 Linux/3.2.45-0.6.wd.865.49.315.metal1.x86_64 botocore/1.4.67",
    "requestParameters": {
        "roleArn": "arn:aws:iam::111122223333:role/WriteAccess",
        "roleSessionName": "AliceSession"
    },
    "responseElements": {
        "credentials": {
            "accessKeyId": "ASIAIVOTP66PNEXAMPLE",
            "expiration": "Nov 14, 2016 6:25:26 PM",
            "sessionToken": "<encoded_session_token_blob>"
        },
        "assumedRoleUser": {
            "assumedRoleId": "AROAIDPPEZS35WEXAMPLE:AliceSession",
            "arn": "arn:aws:sts::111122223333:assumed-role/WriteAccess/AliceSession"
        }
    },
    "requestID": "54ead46a-aa8f-11e6-bb4e-dd0a0d824ead",
    "eventID": "cf1929cf-7123-4186-b85d-791c9d0974a9",
    "resources": [
        {
            "ARN": "arn:aws:iam::111122223333:role/WriteAccess",
            "accountId": "111122223333",
            "type": "AWS::IAM::Role"
        }
    ],
    "eventType": "AwsApiCall",
    "recipientAccountId": "123456789012",
    "sharedEventID": "9fe2fddd-a338-4380-b64d-ca74b7b828e5"
}

The AWS Security Blog published a blog post earlier this year about a solution for auditing cross-account activity that uses CloudTrail, Amazon CloudWatch Events, and AWS Lambda. This automated log-tracing solution works with the new logging behavior made available today.

To learn more about STS logging, see the IAM user guide. If you have comments about auditing cross-account activity or this blog post, start a new thread in the IAM forum.

– Kai

Noise

Tag Archives: AWS STS

The Top 20 Most Viewed AWS IAM Documentation Pages in 2016

The collective thoughts of the interwebz

Process overview

The account creation process

1. Sign in to AWS Organizations

2. Create a new member account

3. Verify account creation

4. Assume a role

5. Configure the new account

6. Move the new account into an OU

7. Reduce the IAM role permissions

Automating the entire end-to-end process

Download the script and CloudFormation templates

Summary

Amazon Cognito Concepts

Amazon Cognito Federated Identities

EMRFS and EC2 instance profiles

Custom credentials providers

Configuring a custom credentials provider for EMRFS

Walkthrough

Configure IAM inside account [email protected]

Configure IAM inside account [email protected]

Implement the custom credential provider

Summary

About the Author

Related

Example of AWS cross-account AWS access

How do I use the new log?

The collective thoughts of the interwebz