All posts by Tracy Pierce

How to use AWS Config to determine compliance of AWS KMS key policies to your specifications

Post Syndicated from Tracy Pierce original https://aws.amazon.com/blogs/security/how-to-use-aws-config-to-determine-compliance-of-aws-kms-key-policies-to-your-specifications/

One of the top security methodologies is the principle of least privilege, which is the practice of limiting user, application, and service permissions to only those necessary to perform a function or task. In this post, I will describe how you can use AWS Config to create compliance rules that will scan AWS Key Management Service (AWS KMS) key policies to determine whether they follow your company’s guidelines for least privilege. You can use the AWS Config Rules Development Kit (RDK) on GitHub (aws-config-rdk) to quickly create and save custom AWS Config rule sets as AWS Lambda functions in your account(s), and test the rules against sample configuration items.

In this solution, you use a Gherkin syntax file, which is good method for determining the requirements of your rule, as well as how it should react to input. A Gherkin syntax file gives you the ability to create a parameter file and a feature file. The parameter file lists information like RuleName, SourceRuntime, CodeKey, InputParameters, Trigger, SourcePeriodic, and so on. The feature file includes a description of the rule, detailed information about the parameters, what the feature is meant to accomplish, and the scenarios you want to evaluate your rule against.

In this post, I will show you how to take a parameter file and feature file, and use the requirements in them to create your AWS Config rule and testing scenarios using the AWS Config RDK. I include an example key policy file for you to use for your test scenarios. You can download all the code snippets used in this post from the aws-config-aws-kms-policy-rule GitHub repository. I will explain how to use the code examples to create the AWS Config rule as a Lambda function, and how to use the AWS Config RDK to test locally and deploy the rule to your account(s).

Overview

The solution described in this post uses custom AWS Config rules to scan AWS KMS key policies every 24 hours. It checks these rules against a set of parameters that you determine beforehand to meet your organization’s security standards. Based on the rule checks, AWS Config determines the key policy to be either compliant or noncompliant when compared against your company’s specifications. Noncompliant resources are noted as such, so that an administrator can review and determine if the policy should be modified, or if it will be allowed as an exception. The following figure shows the overview of the process.
 

Figure 1: Overview

Figure 1: Overview

The way the process works is as follows:

  1. The AWS Config rule is triggered by a configuration change and scans your key policy.
  2. The key policy is evaluated against your custom AWS Config rule scenarios.
  3. The compliance results of the key policies are presented in the AWS Config console.

The files in the GitHub repository you will use for this post are:

  • AWSConfigRuleKMS.feature – This is the Gherkin file used to determine the scenarios you will test against.
  • AWSConfigRuleKMSPolicy.py – This file is used to create the policy formatting that the AWSConfigRuleKMS.py script uses to parse AWS KMS key policies. Because the AWS KMS key policies are JSON objects, you have to parse them before inputting them as data so you can retrieve comparison results.
  • AWSConfigRuleKMS.py – This is the actual script that does the comparisons of the policies retrieved and the scenarios laid out in the Gherkin file.
  • AWSConfigRuleKMS_test.py – This is the testing file that takes an example policy, runs it against the multiple test scenarios, and outputs the test results. This lets you know if your script is running as expected and producing the proper outcomes.
  • parameters.json – This is the parameters file that tells AWS Config which parameters to check for in the rules.

Prerequisites

This solution has the following prerequisites:

In addition, this solution uses the following services:

Deploying the solution

From the Gherkin file in my repo, you can see you will be testing for eight different scenarios regarding least privilege access to your AWS KMS customer master keys (CMKs). I decided to whitelist any CMKs that have an alias beginning with the word “Otter*”, and any UserID that begins with “AROAOTTER*”. The parameters are set in the parameters.json file. This is how AWS Config knows what to check for in the rules. You will use these same parameters in the AWSConfigRuleKMS_test.py file when performing your tests. Be sure to modify these parameters when creating your own rules.

Scenario 1: Checks to determine if a CMK is marked DISABLED.

Scenario 2: Checks to determine if a CMK is marked ENABLED and is in the list of whitelisted CMKs.

Scenario 3: Checks to determine if a CMK is marked ENABLED, is not in the list of whitelisted CMKs, and has an action equal to kms:*.

Scenario 4: Checks to determine if a CMK is marked ENABLED, is not in the list of whitelisted CMKs, includes a condition for users, does not have kms:*, and has a policy allowing the following actions: kms:Encrypt, kms:Decrypt, kms:Create*, kms:Delete*, and kms:Put* together.

Scenario 5: Checks to determine if a CMK is marked ENABLED, is not in the list of whitelisted CMKs, includes a condition for users, does not have kms:*, and does not have a policy allowing the following actions: kms:Encrypt, kms:Decrypt, kms:Create*, kms:Delete*, and kms:Put* together.

Scenario 6: Checks to determine if a CMK is marked ENABLED, is not in the list of whitelisted CMKs, includes a condition for users, user is listed in the whitelisted users, does not have kms:*, and has a policy allowing only the following actions: kms:Create*, kms:Delete*, and kms:Put*.

Scenario 7: Checks to determine if a CMK is marked ENABLED, is not in the list of whitelisted CMKs, includes a condition for users, user is not listed in the whitelisted users, does not have kms:*, and has a policy allowing only the following actions: kms:Create*, kms:Delete*, and kms:Put*.

Scenario 8: Checks to determine if a CMK is marked ENABLED, is not in the list of whitelisted CMKs, includes a condition for users, user is listed in the whitelisted users, does not have kms:*, and has a policy allowing only the following actions: kms:Encrypt, kms:Decrypt, kms:Create*, kms:Delete*, and kms:Put* together.

You will be using the AWS Config RDK to create and test your rules, and also to deploy them to your account.

To create your AWS Config rule (RDK CLI)

  1. Open a terminal window.
  2. Use the cd command to move to the directory in which you want to create your rule.
  3. Use the create command to create your rule, using the following example. Replace all variables in italics with your inputs.
    
    $ rdk create AWSConfigRuleKMS --runtime python3.6 --resource-types AWS::KMS::Key ---input-parameters '{"CMK_Whitelist":"Otter*","Admin_User_Id":"AROAOTTER*"}'
    

  4. You should see output similar to the following:
    
    Running create!
    Local Rule files created.
    

In your directory, you will now see the following files.

  • AWSConfigRuleKMS.py: This is a skeleton file for you to create your Lambda function. It has some base code and is commented to assist you with building your functions.
  • AWSConfigRuleKMS_test.py: This is a skeleton file for you to create your testing scenario script. It has some base code and helpers in place to make this easier for you.
  • parameters.json: This is a parameter file, based on the inputs from the create command.

For this solution, I have already created the code snippets you will need. Download the files from my GitHub repository. Make sure to include the AWSConfigRuleKMSPolicy.py from my repository in the directory as well.

To download the code snippets from the GitHub repository (CLI)

  1. Open a terminal.
  2. Use the cd command to change to the directory where you created your AWS Config rule.
  3. Run the following command:
    
    git clone https://github.com/aws-samples/aws-config-aws-kms-policy-rule
    

Now that you have the code snippets, you need to place them into the skeleton files to complete the rule.

To complete the Python scripts

  1. In a text editor, open your local copies of both AWSConfigRuleKMS.py and AWSConfigRuleKMS_test.py.
  2. Copy the contents of the AWSConfigRuleKMS.py from my GitHub repository.
  3. In the AWSConfigRuleKMS.py skeleton file, delete the code from the line import json down to and including the line above the section starting with # Helper Functions #. Paste the copied code in its place and save the file.
  4. Copy the contents of the AWSConfigRuleKMS_test.py from my GitHub repository.
  5. In the AWSConfigRuleKMS_test.py skeleton file, delete the code from the line import sys down to and including the line above the section starting with # Helper Functions #. Paste the copied code in its place and save the file.

With all the files updated, you will now use the AWS Config RDK to test the scenarios. For testing, you use the AWSConfigRuleKMS_test.py file, which is the file housing the test scenarios to ensure that your rules work as expected.

To test your AWS Config rule (RDK CLI)

  1. Open a terminal window.
  2. Use the cd command to change to the directory one level above where you created your AWS Config rule. For example, if your AWS Config rule is in C://User/Documents/Config/AWSConfigRuleKMS, then change to the C://User/Documents/Config directory.
  3. Use the test-local command to test your rule, using the following example:
    $ rdk test-local AWSConfigRuleKMS_test
  4. You should see output similar to the following:
    
    Running local test!
    Testing AWSConfigRuleKMS
    Looking for tests in /User/Documents/Config/AWSConfigRuleKMS
    AWSConfigRuleKMS_test.py
    Debug!
    <unittest.suite.TestSuite tests=[<unittest.suite.TestSuite tests=[<AWSConfigRuleKMS_test.TestKMSKeyPolicy testMethod=test__scenario_7_admin_role_not_in_whitelist_sep_of_duty>, <AWSConfigRuleKMS_test.TestKMSKeyPolicy testMethod=test_is_not_cmk>, <AWSConfigRuleKMS_test.TestKMSKeyPolicy testMethod=test_scenario_1_disabled_status>, <AWSConfigRuleKMS_test.TestKMSKeyPolicy testMethod=test_scenario_2_cmk_in_whitelist>, <AWSConfigRuleKMS_test.TestKMSKeyPolicy testMethod=test_scenario_3_kms_star_in_policy>, <AWSConfigRuleKMS_test.TestKMSKeyPolicy testMethod=test_scenario_4_no_sep_of_duty>, <AWSConfigRuleKMS_test.TestKMSKeyPolicy testMethod=test_scenario_5_sep_of_duty_actions>, <AWSConfigRuleKMS_test.TestKMSKeyPolicy testMethod=test_scenario_6_admin_role_in_whitelist_sep_of_duty>, <AWSConfigRuleKMS_test.TestKMSKeyPolicy testMethod=test_scenario_8_admin_role_in_whitelist_no_sep_of_duty>, <AWSConfigRuleKMS_test.TestKMSKeyPolicy testMethod=test_scenario_no_conditions>]>]>
    test__scenario_7_admin_role_not_in_whitelist_sep_of_duty (AWSConfigRuleKMS_test.TestKMSKeyPolicy) ... in Key Policy for alias/testkey, statement does have separation of duties, CMK is not whitelisted, and user id is not whitelisted
    ok
    test_is_not_cmk (AWSConfigRuleKMS_test.TestKMSKeyPolicy) ... ok
    test_scenario_1_disabled_status (AWSConfigRuleKMS_test.TestKMSKeyPolicy) ... CMK alias/testkey is disabled
    ok
    test_scenario_2_cmk_in_whitelist (AWSConfigRuleKMS_test.TestKMSKeyPolicy) ... CMK alias/Otter* is in whitelist for CMK Key Policy check
    ok
    test_scenario_3_kms_star_in_policy (AWSConfigRuleKMS_test.TestKMSKeyPolicy) ... in Key Policy for alias/testkey, statement does have open KMS permissions and CMK is not whitelisted
    ok
    test_scenario_4_no_sep_of_duty (AWSConfigRuleKMS_test.TestKMSKeyPolicy) ... in Key Policy for alias/testkey, statement does not have separation of duties and CMK is not whitelisted
    ok
    test_scenario_5_sep_of_duty_actions (AWSConfigRuleKMS_test.TestKMSKeyPolicy) ... in Key Policy for alias/testkey, statement does have separation of duties and CMK is not whitelisted
    ok
    test_scenario_6_admin_role_in_whitelist_sep_of_duty (AWSConfigRuleKMS_test.TestKMSKeyPolicy) ... in Key Policy for alias/testkey, statement does have separation of duties, CMK is not whitelisted, and user id is whitelisted
    ok
    test_scenario_8_admin_role_in_whitelist_no_sep_of_duty (AWSConfigRuleKMS_test.TestKMSKeyPolicy) ... In Key Policy for alias/testkey, statement does not have separation of duties, CMK is not whitelisted, and user id is whitelisted
    ok
    test_scenario_no_conditions (AWSConfigRuleKMS_test.TestKMSKeyPolicy) ... ok
    
    ----------------------------------------------------------------------
    Ran 10 tests in 0.013s
    
    OK
    <unittest.runner.TextTestResult run=10 errors=0 failures=0>
    

If you encounter errors during testing, they could be caused by:

  • Incorrect code in the AWSConfigRuleKMS.py file
  • Incorrect code in the AWSConfigRuleKMS_test.py file
  • Invalid formatting in the parameters.json file
  • Invalid names on the files – they must match the exact formatting I have.

You should go back through your code to make sure that you used proper syntax, and that the test cases match your requirements. If you run into syntax issues, you can find the full list of AWS supported SDKs on the Tools to Build on AWS page. You can match your code formatting against the code I supplied in my GitHub repository to ensure proper spacing/tabs.

When testing is complete, deploy your AWS Config rule into your account(s). The file to deploy the rule itself is the AWSConfigRuleKMS.py file.

To deploy your AWS Config rule (RDK CLI)

  1. Open a terminal window.
  2. Use the cd command to change to the directory one level above where you created your AWS Config rule. For example, if your rule is in C://User/Documents/Config/AWSConfigRuleKMS, then change to the C://User/Documents/Config directory.
  3. Use the deploy command to deploy your rule, using the following example:
    $ rdk deploy AWSConfigRuleKMS
  4. You should see output similar to the following:
    
    Running deploy!
    Zipping AWSConfigRuleKMS
    Uploading AWSConfigRuleKMS
    Creating CloudFormation Stack for AWSConfigRuleKMS
    Waiting for CloudFormation stack operation to complete...
    ...
    Waiting for CloudFormation stack operation to complete...
    AWS Config deploy complete.
    

There are two ways you can verify that your deployment was successful. You can use either the AWS CloudFormation console, or the AWS Config console. Let’s look at it in the AWS Config console.

To verify deployment in the AWS Config console

  1. Open the AWS Config console.
  2. In the navigation pane, choose Rules.
  3. Choose the AWSConfigRuleKMS rule (or what you named it).
  4. In the Rule details section, you will see the name, trigger type, resource type, rule ARN, parameters, and status, as shown in the following screenshot.
     
    Figure 2: AWSConfigRuleKMS in the AWS Config console

    Figure 2: AWSConfigRuleKMS in the AWS Config console

After the deployment is complete, you must modify the AWS Config role that the AWS Config recorder assumes. Although typically this role would be AWSServiceRoleForConfig, in this solution you need to have your own AWS Config role with the Managed Policy arn:aws:iam::aws:policy/service-role/AWSConfigRole attached. The reason for this is that you need to modify the Trust Policy of the AWS Config role to trust the newly created AWS Lambda role. The AWS Lambda role ARN should look similar to the following:


arn:aws:iam::111122223333:role/rdk/AWSConfigRuleKMS-rdkLambdaRole-RANDOMCHARACTERS

To create your AWS Config recorder role in the IAM console

  1. Open the AWS IAM console.
  2. In the navigation pane, select Roles.
  3. At the top, choose Create Role.
  4. Select Config from the list of services.
  5. For Select your use case, select Config – Customizable.
  6. Choose Next: Permissions.
  7. Choose Next: Tags.
  8. Choose Next: Review.
  9. Enter a descriptive name for the role. For my example, I used CustomConfigRole.
  10. Choose Create role.

To modify the AWS Config recorder role’s trust policy

  1. Open the AWS IAM console.
  2. In the navigation pane, select Roles.
  3. Choose the role created by the RDK for Lambda, which is named AWSConfigRuleKMS-rdkLambdaRole-RANDOMCHARACTERS, or whatever you named it.
  4. Next to the Role ARN, choose the copy icon.
  5. Go back to the Roles screen.
  6. Choose the role you just created.
  7. Choose the Trust relationships tab.
  8. Choose Edit trust relationship.
  9. Copy the following trust policy, and paste it over the existing trust policy. (You need to change the AWS Account ARN in italics to your own AWS Account number and role name):
    
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "",
          "Effect": "Allow",
          "Principal": {
            "AWS": "arn:aws:iam::111122223333:role/rdk/AWSConfigRuleKMS-rdkLambdaRole-132PD765KU42Z",
            "Service": "config.amazonaws.com"
          },
          "Action": "sts:AssumeRole"
        }
      ]
    }
    

  10. Choose Update Trust Policy.

To modify the AWS Config recorder role in the AWS Config console

  1. Open the AWS Config console.
  2. In the navigation pane, select Settings.
  3. Scroll down to AWS Config role*.
  4. Select the radio button next to Choose a role from your account.
  5. Choose the role you created for AWS Config.
  6. Choose Save.

After you have done this, you can use the AWS Config console to force an evaluation of your resources.

To force an AWS Config rule evaluation in the AWS Config console

  1. Open the AWS Config console.
  2. In the navigation pane, select Rules.
  3. Choose the AWSConfigRuleKMS rule (or what you named it).
  4. At the top right-hand side of the console, choose Re-evaluate. This can take a few minutes for results to populate.

To have the key policy correctly evaluated, you need to add permissions to the key policy for the root ARN or the AWS Lambda role ARN to perform kms:GetKeyPolicy.

To update key policies to include the root or AWS Lambda role ARN

  1. Open the AWS KMS console.
  2. In the navigation pane, select Customer managed keys.
  3. Choose the Alias or Key ID you want to modify.
  4. Under Key policy, choose Edit.
  5. Copy the following policy snippet and paste it at the bottom of your current policy. (Replace the values in italics with your actual values).
    
    {
        "Sid": "Enable IAM User Permissions",
        "Effect": "Allow",
        "Principal": {
            "AWS": [
                "arn:aws:iam::111122223333:root",
                "arn:aws:iam::111122223333:role/rdk/AWSConfigRuleKMS-rdkLambdaRole-132PD765KU42Z"
            ]
        },
        "Action": "kms:GetKeyPolicy",
        "Resource": "*"
    }
    

  6. Choose Save changes.

In the following example screenshot of my AWS Config console, you can see that I have quite a few CMKs that don’t meet my compliance policies. The key policy either does not have the required permissions defined in my scenarios, or does not have the permissions for the root ARN or my Lambda role to perform kms:GetKeyPolicy. Both can result in a noncompliant status.
 

Figure 3: Viewing noncompliant CMKs in the AWS Config console

Figure 3: Viewing noncompliant CMKs in the AWS Config console

For example, the key policy for alias/SecurityOtterCMK that follows is missing the condition for aws:userid, as well as mixed permission sets. So this CMK is marked as Noncompliant.


{
    "Version": "2012-10-17",
    "Id": "key-consolepolicy-3",
    "Statement": [
        {
            "Sid": "Allow access for Key Administrators",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::111122223333:root",
                    "arn:aws:iam::444455556666:root"
                ]
            },
            "Action": [
                "kms:Create*",
                "kms:Describe*",
                "kms:Enable*",
                "kms:List*",
                "kms:Put*",
                "kms:Update*",
                "kms:Revoke*",
                "kms:Disable*",
                "kms:Get*",
                "kms:Delete*",
                "kms:TagResource",
                "kms:UntagResource",
                "kms:ScheduleKeyDeletion",
                "kms:CancelKeyDeletion"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow use of the key",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111122223333:role/Admin"
            },
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:DescribeKey"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow attachment of persistent resources",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111122223333:role/Admin"
            },
            "Action": [
                "kms:CreateGrant",
                "kms:ListGrants",
                "kms:RevokeGrant"
            ],
            "Resource": "*",
            "Condition": {
                "Bool": {
                    "kms:GrantIsForAWSResource": "true"
                }
            }
        }
    ]
}

On the other hand, the CMK marked as alias/Otter-EBS is in my whitelisted keys based upon my Gherkin, so it shows as Compliant.
 

Figure 4: The CMK with a status of Compliant

Figure 4: The CMK with a status of Compliant

You can now monitor your KMS keys for least privilege based on your required parameters.

Conclusion

In this post, I showed you how to use the AWS Config RDK to create, test, and deploy a custom AWS Config rule that scans your KMS key policies to look for rules designed to implement a least privilege concept. I supplied all scripts necessary to create your rule and test locally. With this AWS Config rule, you can use the AWS Config console to see whether your AWS KMS key policies are compliant with your company standards, so that you can react accordingly.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on the AWS KMS forum or contact AWS Support.

Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.

Author

Tracy Pierce

Tracy is a Senior Consultant, Security Specialty, for Remote Consulting Services. She enjoys the peculiar culture of Amazon and uses that to ensure every day is exciting for her fellow engineers and customers alike. Customer Obsession is her highest priority and she shows this by improving processes, documentation, and building tutorials. She has her AS in Computer Security & Forensics from SCTD, SSCP certification, AWS Developer Associate certification, and AWS Security Specialist certification. Outside of work, she enjoys time with friends, her Great Dane, and three cats. She keeps work interesting by drawing cartoon characters on the walls at request.

How to use AWS RAM to share your ACM Private CA cross-account

Post Syndicated from Tracy Pierce original https://aws.amazon.com/blogs/security/how-to-use-aws-ram-to-share-your-acm-private-ca-cross-account/

In this post, I use the new Cross-Account feature of AWS Certificate Manager (ACM) Private Certificate Authority (CA) to create a CA in one account and then use ACM in a second account to issue a private certificate that automatically renews the following year. This newly available workflow expands the usability of ACM Private CA so that your organization can build a centralized CA hierarchy and allow issuance across many accounts, which fits the needs of how customers build on AWS today. A central CA hierarchy can enable centralized management of the CA and creates cost savings, because you no longer need one CA per account. Your organization can also simplify the access the PKI team needs to administer the CA but having the CA in one account and sharing the issuance across multiple accounts. This covers use cases like SSL/TLS everywhere and Internet of Things (IoT) production where many production-line systems need to generate certificates. Support for microservice meshes (like AppMesh and Managed Kafka) will be coming soon.

The newly launched feature, Private CA Cross-Account Sharing, gives you the ability to grant permissions for other accounts to use a centralized CA to generate and issue certificates by using AWS Resource Access Manager (RAM) to manage the permissions. This removes the need for a Private CA in every account, saving you $400 for each CA created, which is a more cost-effective way of deployment. Also, each account the Private CA is shared with that creates a private certificate in its own account, remains in full control of the certificate because the key lives in the certificate creation account and is fully protected there. The certificate creation account can associate the certificate to any resource in their account or export it for further use. Each certificate that is created also has the complete managed renewal capability of ACM.

For this walkthrough, I share my ACM Private CA with a single account. This feature also works with AWS Organizations. To give you a couple of examples of creating certificates in a secondary account from the shared Private CA, I show you how to accomplish this through the AWS Command Line Interface (CLI) and through the ACM console.

Solution overview

The solution is simple to both use and configure. It does require you to have an ACM Private CA already created in a single account. If you don’t, you can follow the steps outlined in the ACM Private CA User Guide to create one. After you’ve selected your CA to share, you can create a resource share and include your private CA by using AWS RAM. You can either share this with your full AWS Organizations structure, AWS organizational units, or individual accounts both inside and outside of AWS Organizations. When you share your CA, and the sharing accounts accept the resource share, they have the ability to create certificates through the ACM console or through CLI, API, or AWS CloudFormation. You are only sharing the ability to create, manage, bind, and export certificates from the CA. You are not sharing any of the admin functionality. This enables you to provide a strong separation between admins and users of the Private CA. The workflow for sharing your ACM Private CA is as follows, also shown in Figure 1.

  1. Identify which Private CA(s) you want to share, and which accounts you want to share with.
  2. Create a resource share and then add your ACM Private CA to the share.
  3. Share the resource with a single account or with your AWS Organizations structure.
  4. In the shared account(s), create a certificate through the ACM console (You can choose to share with a single account, or with your entire AWS Organizations structure; you don’t have to do both).
    • Share your Private CA with your AWS Organizations accounts.
    • Share your Private CA with individual accounts.

 

Figure 1: Workflow diagram for sharing your ACM Private CA

Figure 1: Workflow diagram for sharing your ACM Private CA

Prerequisites

For this walkthrough, you should have the following prerequisites:

Deploying the solution through the AWS Management Console

In this section, you can find all the steps to complete this tutorial. I walk you step-by-step through the process for sharing this Private CA and verifying success by creating a private certificate through the AWS Management Console.

To deploy the solution through the AWS Management Console

  1. First, create your shared resource in the AWS RAM console. This is completed in the Private CA OWNING account.
    1. Sign in to the AWS Management Console. For Services, select the Resource Access Manager console.
    2. In the left-hand pane, choose Resource shares, and then choose Create resource share.
    3. For Name, enter Shared_Private_CA.
    4. For Resources, select your ACM Private CA.

      Figure 2: Creating your resource share

      Figure 2: Creating your resource share

    5. For Principals, select either AWS Organizations or an individual account.
    6. Choose Create resource share.
  2. Next, accept the shared resource in your shared account. Note: If you choose to share with AWS Organizations, there is no need for the acceptance step. By sharing with an organization or organizational units, all accounts in that container will have access without going through the acceptance step. Accepting a resource share into your account enables you to control which shared resources are displayed in your account when you list resources. You can reject unwanted shares to prevent the system from displaying unwanted resources that are shared from accounts you don’t know or trust.
    1. In your shared account, sign in to the AWS Management Console. For Services, select the Resource Access Manager console.
    2. On the left-hand pane, under Shared with me, select Resource shares. (You will see the share invite pending.)

      Figure 3: Pending resource share

      Figure 3: Pending resource share

    3. Select the name of the shared resource, and then choose Accept resource share.
    4. After the share is accepted, under Resource shares, you will see that the Shared_Private_CA is now listed as Active.

      Figure 4: Active share

      Figure 4: Active share

  3. Next, create a certificate from the Shared_Private_CA in the shared account.
    1. In the same account, go to the Certificate Manager console
    2. Choose Request a certificate.
    3. Select the option Request a private certificate, then choose Request a certificate.
    4. For CA, select Shared_Private_CA, and then choose Next.
    5. For Add domain names, add the domain www.example2.com, and then choose Next.
    6. Choose Review and request, confirm the information, then choose Confirm and request.
    7. You can now see your new ACM certificate, issued by the Shared_Private_CA in your account.

      Figure 5: Certificate issued by shared ACM Private CA

      Figure 5: Certificate issued by shared ACM Private CA

Deploying the solution through the AWS CLI

You’ve completed this tutorial using the AWS Management Console. Now, I will walk you through the same step-by-step process of sharing your Private CA and creating a private certificate to verify success using the AWS CLI.

To deploy the solution by using the AWS CLI

  1. First, create your shared resource in the AWS RAM console. With credentials from your ACM Private CA OWNING account, run the following command (make sure to replace values in italics with your own values).
    
    aws ram create-resource-share –-name Shared_Private_CA --resource-arn arn:aws:acm-pca:region:111122223333:certificate-authority/fb149362-7de8-47be-8149-example --principals 444455556666
    

  2. Next, accept the shared resource in your shared account. With credentials from your shared account, run the following command (make sure to replace values in italics with your own values).
    
    aws ram accept-resource-share-invitation --resource-share-invitation-arn arn:aws:ram:region:111122223333:resource-share-invitation/ce4b7501-c93d-4477-a19b-example
    

  3. Next, create a certificate from the Shared_Private_CA (make sure to replace values in italics with your own values).
    
    aws acm request-certificate –-domain-name www.example2.com --certificate-authority-arn arn:aws:acm-pca:region:111122223333:certificate-authority/12345678-1234-1234-1234-example --validation-method DNS
    

  4. Finally, verify the certificate by running describe-certificate (make sure to replace values in italics with your own values).
    
    $ aws acm describe-certificate --certificate-arn arn:aws:acm:region:444455556666:certificate/523ffc50-824a-492e-ac11-example
    

Example output is shown as follows.


{
    "Certificate": {
        "CertificateArn": "arn:aws:acm:region:444455556666:certificate/523ffc50-824a-492e-ac11-example",
        "DomainName": "www.example2.com",
        "SubjectAlternativeNames": [
            "www.example2.com"
        ],
        "DomainValidationOptions": [
            {
                "DomainName": "www.example2.com",
                "ValidationEmails": [],
                "ValidationDomain": "www.example2.com",
                "ValidationStatus": "SUCCESS",
                "ValidationMethod": "DNS"
            }
        ],
        "Serial": "54:e6:ee:06:2b:35:d4:c6:53:88:1d:c8:47:f0:5a:1e",
        "Subject": "CN=www.example2.com",
        "Issuer": "Example.com",
        "CreatedAt": "2020-07-20T09:37:51-05:00",
        "IssuedAt": "2020-07-20T09:37:56-05:00",
        "Status": "ISSUED",
        "NotBefore": "2020-07-20T08:37:54-05:00",
        "NotAfter": "2021-08-20T09:37:54-05:00",
        "KeyAlgorithm": "RSA-2048",
        "SignatureAlgorithm": "SHA256WITHRSA",
        "InUseBy": [],
        "Type": "PRIVATE",
        "KeyUsages": [
            {
                "Name": "DIGITAL_SIGNATURE"
            },
            {
                "Name": "KEY_ENCIPHERMENT"
            }
        ],
        "ExtendedKeyUsages": [
            {
                "Name": "TLS_WEB_SERVER_AUTHENTICATION",
                "OID": "1.3.6.1.5.5.7.3.1"
            },
            {
                "Name": "TLS_WEB_CLIENT_AUTHENTICATION",
                "OID": "1.3.6.1.5.5.7.3.2"
            }
        ],
        "CertificateAuthorityArn": "arn:aws:acm-pca:region:111122223333:certificate-authority/f1d590ea-e14a-4c92-8de9-example",
        "RenewalEligibility": "INELIGIBLE",
        "Options": {
            "CertificateTransparencyLoggingPreference": "ENABLED"
        }
    }
}

Conclusion

In this post, I showed you how to share an ACM Private CA with a single account or AWS Organization and then create a certificate from that shared Private CA. We went through steps to do both these tasks through the AWS Management Console and AWS CLI. You now have the option to centralize your ACM Private CA, and share it with your other AWS accounts to issue private certificates. This lowers cost, management overhead, and makes it easier to implement separation of PKI administrators from users of the CA, freeing up time to focus on your AWS infrastructure and security. You can read about more ACM Private CA Best Practices in our ACM User Guide.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on the AWS Certificate Manager forum or contact AWS Support.

Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.

Author

Tracy Pierce

Tracy is a Senior Consultant, Security Specialty, for Remote Consulting Services. She enjoys the peculiar culture of Amazon and uses that to ensure every day is exciting for her fellow engineers and customers alike. Customer Obsession is her highest priority and she shows this by improving processes, documentation, and building tutorials. She has her AS in Computer Security & Forensics from SCTD, SSCP certification, AWS Developer Associate certification, and AWS Security Specialist certification. Outside of work, she enjoys time with friends, her Great Dane, and three cats. She keeps work interesting by drawing cartoon characters on the walls at request.

How to use resource-based policies in the AWS Secrets Manager console to securely access secrets across AWS accounts

Post Syndicated from Tracy Pierce original https://aws.amazon.com/blogs/security/how-to-use-resource-based-policies-aws-secrets-manager-console-to-securely-access-secrets-aws-accounts/

AWS Secrets Manager now enables you to create and manage your resource-based policies using the Secrets Manager console. With this launch, we are also improving your security posture by both identifying and preventing creation of resource policies that grant overly broad access to your secrets across your Amazon Web Services (AWS) accounts. To achieve this, we use the Zelkova engine to mathematically analyze access granted by your resource policy and alert you if such permissions are found. The analysis verifies access across all resource-policy statements, actions, and the set of condition keys used in your policies. To be considered non-public, the resource policy must grant access only to fixed values (values that don’t contain a wildcard) of one or more of the following: aws:SourceArn, aws:SourceVpc, aws:SourceVpce, aws:SourceAccount, aws:SourceIP, and ensure the Principal does not include a “*” entry.

If the policy grants Public or overly broad access to your secrets across AWS accounts, Secrets Manager will block you from applying the policy in the console and alert you with a dashboard message. This prevents your policy from accidentally granting broader access to your secrets, instead ensuring you are restricting it to the intended AWS accounts, AWS services, and AWS Identity and Access Management (IAM) entities. Access to AWS Secrets Manager requires AWS credentials. Those credentials must contain permission to access the AWS resources you want to access, such as your Secrets Manager secrets. In this blog post, we use Public or broad access to refer to values (or a combination of values) in the resource policy that result in a wide access across AWS accounts and principals.

With AWS Secrets Manager, you have the option to store, rotate, manage, and retrieve many types of secrets. These can be database usernames and passwords, API keys, string values, and binary data. AWS supports the ability to share these secrets cross-account by applying resource policies via the AWS Command Line Interface (AWS CLI) and now via the Secrets Manager console.

Why would you need to share a secret? There are many reasons. Perhaps you have database credentials managed in a central account that are needed by applications in your production account. Maybe you have the binary stored for an encryption key that other accounts will use to create AWS Key Management Service (AWS KMS) keys in their accounts. To achieve this goal while ensuring a secure transfer of information and least privilege permissions, you will need a resource-based policy on your secret, a resource-based policy on your AWS KMS Customer Managed Key (CMK) used for encrypting the secret, and a user-based policy on your IAM principal.

You can still create a policy using AWS CLI or AWS SDK permitting access to a broader scope of entities if your business needs dictate. If you do permit this type of broader access, AWS Secrets Manager will show a notification in your dashboard, as shown in Figure 2, below.

Figure 1. This shows the warning when you try to create a resource policy that grants broad access to your secrets via the AWS Secrets Manager console.

Figure 1. This shows the warning when you try to create a resource policy that grants broad access to your secrets via the AWS Secrets Manager console.

 

Figure 2. This alert pops up when you click on a secret that has a resource policy attached via the CLI that grants broad access to the secret.

Figure 2. This alert pops up when you click on a secret that has a resource policy attached via the CLI that grants broad access to the secret.

In the example below, you’ll see how to use the AWS Secrets Manager console to attach a resource-based policy and allow access to your secret from a secondary account. A secret in the CENTRAL_SECURITY_ACCOUNT will be set to allow it to be accessed by an IAM role in the PRODUCTION_ACCOUNT.

In this example:

  • SECURITY_SECRET = The secret created in the CENTRAL_SECURITY_ACCOUNT.
  • SECURITY_CMK = The AWS KMS CMK used to encrypt the SECURITY_SECRET.
  • PRODUCTION_ROLE = The AWS IAM role used to access the SECURITY_SECRET.
  • PRODUCTION_ACCOUNT = The AWS account that owns the AWS IAM role used for cross-account access.

Overview of solution

The architecture of the solution can be broken down into four steps, which are outlined in Figure 3. The four main steps are:

  1. Create the resource-based policy via the AWS Secrets Manager console on the SECURITY_SECRET in the CENTRAL_SECURITY_ACCOUNT.
  2. Update the SECURITY_CMK policy in the CENTRAL_SECURITY_ACCOUNT to allow the role from the PRODUCTION account access.
  3. Grant the AWS IAM role in the PRODUCTION_ACCOUNT permissions to access the secret.
  4. Test and verify access from the PRODUCTION_ACCOUNT.

 

Figure 3. A visual overview of the four steps to use the AWS Secrets Manager console to attach a resource-based policy, allow access to your secret from a secondary account, and test and verify the process.

Figure 3. A visual overview of the four steps to use the AWS Secrets Manager console to attach a resource-based policy, allow access to your secret from a secondary account, and test and verify the process.

Prerequisites

To use the example in this post, you need:

  • An AWS account.
  • An IAM Role with permissions to make modifications in both the CENTRAL_SECURITY_ACCOUNT and the PRODUCTION_ACCOUNT.
  • An IAM role in the PRODUCTION_ACCOUNT you wish to grant permissions to access the SECURITY_SECRET.

Deploying the solution

Step 1: Create a resource-based policy in your CENTRAL_SECURITY account on the SECURITY_SECRET secret

  1. Log in to the AWS Secrets Manager console in the CENTRAL_SECURITY_ACCOUNT.
  2. Choose SECURITY_SECRET.
  3. Choose Edit Permissions next to Resource Permissions (optional).

    Figure 4. The dashboard view where you edit permissions.

    Figure 4. The dashboard view where you edit permissions.

  4. This will bring you to the page to add the resource policy. It will give you a basic template as shown in Figure 5, below.

    Figure 5. The basic template to add the resource policy.

    Figure 5. The basic template to add the resource policy.

  5. Since the full policy is provided for you in this example, delete the template from the text box.
  6. Copy the policy below and paste it in the text box. Make sure to replace PRODUCTION with your AWS account ID. You can also adjust the permissions you grant if needed. This policy allows a specific role in the PRODUCTION_ACCOUNT account to retrieve the current version of your secret. In the example, my IAM Role is called PRODUCTION_ROLE. Note you do not need to replace AWSCURRENT with any other value.
    
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "AWS": "arn:aws:iam::PRODUCTION:role/PRODUCTION_ROLE"
                },
                "Action": "secretsmanager:GetSecretValue",
                "Resource": "*",
                "Condition": {
                    "ForAnyValue:StringEquals": {
                        "secretsmanager:VersionStage": "AWSCURRENT"
                    }
                }
            }
        ]
    }
    

    As shown in Figure 6, below, you’ll see this in the resource policy text area (with your AWS account ID in place of PRODUCTION).

    Figure 6. The example resource policy shown in the console.

    Figure 6. The example resource policy shown in the console.

  7. Choose Save.

Step 2: Update the resource-based policy in your CENTRAL_SECURITY account on the SECURITY_CMK

Note: Secrets in AWS Secrets Manager are encrypted by default. However, it is important for you to provide authorization for IAM Principals that need to access your secrets. Complete authorization requires access to the secret and the KMS CMK used to encrypt it, which prevents accidental public permissions on the secret. It is important to maintain both sets of authorization to provide appropriate access to secrets.

  1. Log in to the AWS KMS console in the CENTRAL_SECURITY_ACCOUNT.
  2. Choose SECURITY_CMK.
  3. Next to Key policy choose the Edit button.
  4. Paste the below code snippet into your key policy to allow the PRODUCTION_ACCOUNT access:
    
    {
        "Sid": "AllowUseOfTheKey",
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::PRODUCTION:role/PRODUCTION_ROLE"
        },
        "Action": [
            "kms:Decrypt",
            "kms:DescribeKey"
        ],
        "Resource": "arn:aws:kms:us-east-1:CENTRAL_SECURITY:key/SECURITY_CMK"
    }
    

    You will need to replace PRODUCTION with your production account ID, PRODUCTION_ROLE with your production Role name, CENTRAL_SECURITY with your security account ID, and SECURITY_CMK with the CMK key ID of your security CMK. If you forget to swap out the account IDs in the policy with your own, you’ll see an error message similar to the one shown in Figure 7, below.

    Figure 7. Error message that appears if you don’t swap out your account number correctly.

    Figure 7. Error message that appears if you don’t swap out your account number correctly.

  5. Choose Save changes.

Step 3: Add permissions to the PRODUCTION_ROLE in the PRODUCTION account

  1. Log in to the AWS IAM console in the PRODUCTION_ACCOUNT account.
  2. In the left navigation pane, choose Roles.
  3. Select PRODUCTION_ROLE.
  4. Under the Permissions tab, choose Add inline policy.
  5. Choose the JSON tab and paste the below policy:
    
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": "secretsmanager:GetSecretValue",
                "Resource": " arn:aws:secretsmanager:us-east-1:CENTRAL_SECURITY:secret:SECURITY_SECRET"
            },
            {
                "Effect": "Allow",
                "Action": [
                    "kms:Decrypt",
                    "kms:DescribeKey"
                ],
                "Resource": "arn:aws:kms:us-east-1:CENTRAL_SECURITY:key/SECURITY_CMK"
            }
        ]
    }
    

    You will need to replace CENTRAL_SECURITY with your security account id, SECURITY_SECRET with the secret id, and SECURITY_CMK with the CMK key id of your security CMK.

  6. Choose Review policy.
  7. Name the policy Central_Security_Account_Security-Secret-Access, and choose Create policy.

Step 4: Test access to the SECURITY_SECRET from the PRODUCTION account

Verification of access via AWS CLI

  1. From the AWS CLI, use the PRODUCTION_ROLE credentials to run the get-secret-value command.
  2. Returned output should look like the example, below, in Figure 8.
    
    $aws secretsmanager get-secret-value --secret-id SECURITY_SECRET --version-stage AWSCURRENT
    
    {
        “ARN”: “arn:aws:secretsmanager:us-east-1:CENTRAL_SECURITY:secret:SECURITY_SECRET”,
        “Name”: “SECURITY_SECRET”,
        “SecretString”: “TheSecretString”,
        “CreatedDate”: 123456789,
        “VersionId”: “64c4250d-0b81-42e0-9a0c-e189d3c9aea8”,
        “VersionsStages”: [
            “AWSCURRENT”
        ]
    }
    

You can also verify the policy was attached from the CENTRAL_SECURITY_ACCOUNT by following the steps below.

Verification of policy via console

  1. Log into the AWS Secrets Manager console in the CENTRAL_SECURITY_ACCOUNT.
  2. Choose SECURITY_SECRET.
  3. Scroll down to where it shows Resource Permissions (optional), and you’ll see your resource policy stored in the console, as shown in Figure 8, below.

    Figure 8. What the example resource policy looks like in the console.

    Figure 8. What the example resource policy looks like in the console.

Conclusion

In this post, you saw how to add a resource-based policy on a secret in AWS Secrets Manager using the console, and how to update your AWS KMS CMK resource-based policy to enable access. The example showed setting up cross-account access, and allowing a role from the PRODUCTION_ACCOUNT to use the secret in the CENTRAL_SECURITY_ACCOUNT. By using the AWS Secrets Manager console to set up the resource-based policy, you now have a straight-forward, visual way to add and manage resource-based policies for your secrets and receive notifications if that policy is too broad.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on the AWS Secrets Manager forum or contact AWS Support.

Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.

Author

Tracy Pierce

Tracy is a Senior Consultant, Security Specialty, for Remote Consulting Services. She enjoys the peculiar culture of Amazon and uses that to ensure every day is exciting for her fellow engineers and customers alike. Customer Obsession is her highest priority and she shows this by improving processes, documentation, and building tutorials. She has her AS in Computer Security and Forensics from SCTD, SSCP certification, AWS Developer Associate certification, and AWS Security Specialist certification. Outside of work, she enjoys time with friends, her Great Dane, and three cats. She keeps work interesting by drawing cartoon characters on the walls at request.

How to migrate a digital signing workload to AWS CloudHSM

Post Syndicated from Tracy Pierce original https://aws.amazon.com/blogs/security/how-to-migrate-a-digital-signing-workload-to-aws-cloudhsm/

Is your on-premises Hardware Security Module (HSM) at end-of-life? Does continued maintenance of your on-premises hardware take a lot of time and cost a lot of money? Do you want or need all of your workloads to be performed on AWS? By migrating these workloads to AWS CloudHSM, you receive automated backups, low cost HSMs, managed maintenance, automatic recovery in event of a hardware failure, integrated fault tolerance, and high-availability. One such workload you might consider migrating is secret key material used for digital signing operations.

Enterprise certificate authority (CA) or public key infrastructure (PKI) applications use the private portion of an asymmetric key pair generated and stored in a hardware security module (HSM) to perform signing operations. Examples of such operations include the creation of digital certificates for web-servers or IoT devices, file signatures, or when negotiating a TLS session. Migrating this type of workload to AWS may save you time and money. If your HSM is at end of life and you need an alternative, you can migrate the digital signing workload to AWS CloudHSM in just a few steps.

This post will focus on a workload that allows you to create and use a digital certificate to digitally sign an arbitrary file. I’ll show you how to create a new asymmetric key pair and generate the corresponding certificate signing request (CSR) on AWS CloudHSM. This CSR, once signed by the appropriate issuing CA, allows your new key pair and the associated certificate to be trusted in the same way as the key pairs in your original HSM. You could then move traffic related to signing operations or issuing certificates to your AWS CloudHSM cluster.

Background

Before I walk you through the steps of migrating a certificate signing workload into CloudHSM, I’ll provide a little background information so you’ll know how CloudHSM, PKI, and CAs work together. Every certificate is associated with a key pair made up of a private (secret) key and a public key. The private key associated with a certificate needs to be kept confidential, so it typically resides on a hardware security module (HSM). The public portion of the key pair is not confidential, is included in the certificate, and can be shared with anyone who wants to verify a digital signature made with the corresponding private key. In a PKI, a CA is the trusted entity that issues digital certificates on behalf of end-entities. At the top of the trust hierarchy is a root CA, which is implicitly trusted when it is established because it acts as the root of trust for intermediate CAs and end-entity certificates that may be issued underneath it. Intermediate CAs are trusted because their certificates are signed by the root CA. Intermediate CAs in turn sign end-entity certificates, which are used to authenticate identities of various actors across the data transfer process. A common use case for end-entity certificates is for web servers so that connecting clients can verify the server’s identity. Generally, end-entity certificates are valid for 1-3 years, intermediate CA certificates are valid for 5-10 years, and root CAs are valid for 30 years or more.

Beyond solving for the non-repudiation of objects signed by end-entity certificates to ensure the owner of the private key performed the signing operation, there is still the problem of trusting that the owner of the private key is the identity they claim to be. When evaluating trust in this way, there are generally two options; relying on public CAs or private CAs. Public CAs widely distribute the public keys of their root certificates into popular client trust stores (for example, browsers and operating systems). This allows users to verify that the identity of the end-entity has been attested to by a publicly trusted CA. This helps when the signer and the verifier of the digital asset don’t know each other and haven’t shared cryptographic material with each other in advance to perform future validations. Private CAs are those for which there are no widely distributed copies of their associated public keys. The verifier has to retrieve the public key from the private CA and has to explicitly trust the cert without any third-party attestation of the signer’s identity. This is appropriate for cases when signers and verifiers are in the same company or know each other. Examples of when to use a private CA are securing virtual private networks, data or file replication between internal servers, remote backups, file-sharing, email, or other personal accounts.

Regardless of the certificate trust model you need, AWS CloudHSM can be used to create the initial key pair and CSR for both public and private CA requests. Note that AWS offers some alternatives for certificate management that may simplify your workloads without having to use AWS CloudHSM directly. AWS Certificate Manager (ACM) automatically creates key pairs and issues public or private certificates to identify resources within your organization. For use cases that need capabilities not yet supported by ACM, or in unusual situations in which a single-tenant HSM under your control is required for compliance reasons, you can use AWS CloudHSM directly for key generation and signing operations.

Organizations currently using an on-premises HSM for the creation of asymmetric keys used in digital certificates often use a vendor-proprietary mechanism to replicate key material across multiple HSMs for resiliency. However, this method prevents the key material from ever being transferred to an HSM offered by a different vendor. Consider it “vendor lock-in’ by design. So, the private key corresponding to the certificates you use for signing and authentication are locked inside that HSM. But if they are locked inside, how do you move to AWS CloudHSM? The answer is that you don’t have to rely on these inaccessible keys: you can create a new key pair and use it within AWS CloudHSM to begin issuing end-entity certificates.

Solution overview

I will go over creating a new private key in AWS CloudHSM using the Windows client and using Microsoft certreq to generate a corresponding CSR. You provide this CSR to your private or public CA to receive a signed certificate in return. This certificate and its public key then needs to be propagated to wherever your signatures are verified. At the end of this post, I will show you how to verify your digital signatures using Microsoft SignTool. SignTool is provided by Microsoft to allow Windows users to digitally sign files, verify file signatures, and file timestamps.
 

Figure 1: Procedural diagram

Figure 1: Procedural diagram

As shown in the diagram above, the steps followed in this post are:

  1. Create a new RSA private key using KSP/CNG through the AWS CloudHSM Windows client.
  2. Using Microsoft certreq, create your CSR.
  3. Provide the CSR to your CA for signing.
  4. Use Microsoft SignTool to sign files in your environment.

Note: You may have to register this new certificate with any partners that do not automatically verify the entire certificate chain. This could be 3rd party applications, vendors, or outside entities that utilize your certificates to determine trust.

Prerequisites

In this walkthrough, I assume that you already have an AWS CloudHSM cluster set up and initialized with at least one HSM device, and an Amazon Elastic Compute Cloud (EC2) Windows-based instance with the AWS CloudHSM client, PowerShell, and Windows SDK with Microsoft SignTool installed. You must have a crypto user (CU) on the HSM to perform the steps in this post.

Deploying the solution

Step 1: Create a new private key using KSP/CNG using the AWS CloudHSM Windows client

On your Windows server where the AWS CloudHSM Windows client is installed, use a text editor to create a certificate request file named IISCertRequest.inf. For the purpose of this post, I have filled out an example file below.


[Version]
Signature = "$Windows NT$"
[NewRequest]
Subject = "CN=example.com,C=US,ST=Washington,L=Seattle,O=ExampleOrg,OU=WebServer"
HashAlgorithm = SHA256
KeyAlgorithm = RSA
KeyLength = 2048
ProviderName = "Cavium Key Storage Provider"
KeyUsage = "CERT_DIGITAL_SIGNATURE_KEY_USAGE"
MachineKeySet = True    

Step 2: Using Microsoft certreq, create your CSR

On the same server, open PowerShell and, at the PowerShell prompt, create a CSR from the IISCertRequest.inf file by using the Windows certreq command. Here’s an example of the command. Remember to change out the text in red italics with your own file name.


PS C:\>certreq -new <IISCertRequest.inf IISCertRequest.csr> 
	SDK Version: 2.03
CertReq: Request Created

If successful, you’ll see the “Request Created” message above, as well as the new file <IISCertRequest.csr> on your server. This certificate will be provided to your choice of public CA for certificate issuance. This will need to be completed manually via your public CAs suggested method of certificate request.

Step 3: Provide the CSR to your CA for signing

The CA that had been signing your existing end-entity certificates with keys generated by your original HSM is the one you use to sign the new certificates with keys generated by AWS CloudHSM, as well. There are many CAs to choose from, such as Digicert, Trustwave, GoDaddy, and so on. You will want to follow their steps for submitting your CSR to receive your certificate in return.

Step 4: Use Microsoft SignTool to sign files in your environment

When you receive your signed certificate back from your chosen CA, save a copy locally on your Windows server. Then, move the certificate file to the Personal Certificate Store in Windows so it can be used by other applications, such as Microsoft SignTool. Here’s an example of the command. Be sure to replace the value in <red italics> with your actual certificate name.
PS C:\certreq -accept <signedCertificate.cer>

Now, the certificate is ready for use, and I’ll show you how to use it to sign a file. First, you have to get the thumbprint of your certificate. To do this, open PowerShell as an Administrator (right-click the app and choose Run as Administrator). Type this command:
PS C:\>Get-ChildItem -path cert:\LocalMachine\My

If successful, you should see an output similar to this. Copy the thumbprint that is returned. You’ll need it when you perform the actual signing operation on a file.


Thumbprint				                Subject
---------------						-----------
49DF7HDJT84723FDKCURLSXYRF9830568CXHSUB2		CN=WINDOWS-CA
VJFU57E6DI9DKMCHAKLDFJA8E73739Q04730QU7A		CN=www.example.com, OU=Certif….

To open the SignTool application, navigate to the app’s directory within PowerShell. By default, this is typically:
C:\Program Files (x86)\Windows Kits\<SDK Version> \bin\<version number> \<CPU architecture>

For example, if you had downloaded the Microsoft Windows SDK 10 version, the application would be stored in:

C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64

When you’ve located the directory, sign your file by running the command below. Remember to replace the values in <red italics> with your own values. The test.exe file in this example can be any valid executable file in your directory.
PS C:\>.\signtool.exe sign /v /fd sha256 /sha1 <thumbprint> /sm /as C:\Users\Administrator\Desktop\<test.exe>

You should see a message like this:


Done Adding Additional Store
Successfully signed C:\User\Administrator\Desktop\<test.exe>

Number of files successfully Signed: 1
Number of warnings: 0
Number of errors: 0

One last optional item you can do is verify the signature on the file using the command below. Again, replace your values for those in red italics.
PS C:\>.\signtool.exe verify /v /pa C:\Users\Administrators\Desktop\<test.exe>

You’ve now successfully migrated your file signing workload to AWS CloudHSM. If your signing certificate was not issued by a publicly trusted CA but instead by a private CA, make sure to deploy a copy of the root CA certificate and any intermediate certs from the private CA on any systems you want to verify the integrity of your signed file.

Conclusion

In this post, I walked you through creating a new RSA asymmetric key pair to create a CSR. After supplying the CSR to your chosen CA and receiving a signing certificate in return, I then showed you a how to use Microsoft SignTool with AWS CloudHSM to sign files in your environment. You can now use AWS CloudHSM to sign code, documents, or other certificates in the same method of your original HSMs.

If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread on the AWS CloudHSM forum.

Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.

Author

Tracy Pierce

Tracy Pierce is a Senior Consultant, Security Specialty, for Remote Consulting Services. She enjoys the peculiar culture of Amazon and uses that to ensure every day is exciting for her fellow engineers and customers alike. Customer Obsession is her highest priority and she shows this by improving processes, documentation, and building tutorials. She has her AS in Computer Security & Forensics from SCTD, SSCP certification, AWS Developer Associate certification, and AWS Security Specialist certification. Outside of work, she enjoys time with friends, her Great Dane, and three cats. She keeps work interesting by drawing cartoon characters on the walls at request.

How to BYOK (bring your own key) to AWS KMS for less than $15.00 a year using AWS CloudHSM

Post Syndicated from Tracy Pierce original https://aws.amazon.com/blogs/security/how-to-byok-bring-your-own-key-to-aws-kms-for-less-than-15-00-a-year-using-aws-cloudhsm/

Note: BYOK is helpful for certain use cases, but I recommend that you familiarize yourself with KMS best practices before you adopt this approach. You can review best practices in the AWS Key Management Services Best Practices (.pdf) whitepaper.

May 14, 2019: We’ve updated a sentence to clarify that this solution does not include instructions for creating an AWS CloudHSM cluster. For information about how to do this, please refer to our documentation.


Back in 2016, AWS Key Management Service (AWS KMS) announced the ability to bring your own keys (BYOK) for use with KMS-integrated AWS services and custom applications. This feature allows you more control over the creation, lifecycle, and durability of your keys. You decide the hardware or software used to generate the customer-managed customer master keys (CMKs), you determine when or if the keys will expire, and you get to upload your keys only when you need them and delete them when you’re done.

The documentation that walks you through how to import a key into AWS KMS provides an example that uses OpenSSL to create your master key. Some customers, however, don’t want to use OpenSSL. While it’s a valid method of creating the key material associated with a KMS CMK, security best practice is to perform key creation on a hardware security module (HSM) or a hardened key management system.

However, using an on-premises HSM to create and back up your imported keys can become expensive. You have to plan for factors like the cost of the device itself, its storage in a datacenter, electricity, maintenance of the device, and network costs, all of which can add up. An on-premises HSM device could run upwards of $10K annually even if used sparingly, in addition to the cost of purchasing the device in the first place. Even if you’re only using the HSM for key creation and backup and don’t need it on an ongoing basis, you might still need to keep it running to avoid complex re-initialization processes. This is where AWS CloudHSM comes in. CloudHSM offers HSMs that are under your control, in your virtual private cloud (VPC). You can spin up an HSM device, create your key material, export it, import it into AWS KMS for use, and then terminate the HSM (since CloudHSM saves your HSM state using secure backups). Because you’re only billed for the time your HSM instance is active, you can perform all of these steps for less than $15.00 a year!

Solution pricing

In this post, I’ll show you how to use an AWS CloudHSM cluster (which is a group that CloudHSM uses to keep all HSMs inside in sync with one another) with one HSM to create your key material. Only one HSM is needed, as you’ll use this HSM just for key generation and storage. Ongoing crypto operations that use the key will all be performed by KMS. AWS CloudHSM comes with an hourly fee that changes based on your region of choice; be sure to check the pricing page for updates prior to use. AWS KMS has a $1.00/month charge for all customer-managed CMKs, including those that you import yourself. In the following chart, I’ve calculated annual costs for some regions. These assume that you want to rotate your imported key annually and that you perform this operation once per year by running a single CloudHSM for one hour. I’ve included 12 monthly installments of $1.00/mo for your CMK. Depending on the speed of your application, additional costs may apply for KMS usage, which can be found on the AWS KMS pricing page.

REGIONCLOUDHSM PRICING STRUCTUREKMS PRICING STRUCTUREANNUAL TOTAL COST
US-EAST-1$1.60/HR$1/MO$13.60/YR
US-WEST-2$1.45/HR$1/MO$13.45/YR
EU-WEST-1$1.47/HR$1/MO$13.47/YR
US-GOV-EAST-1$2.08/HR$1/MO$14.08/YR
AP-SOUTHEAST-1$1.86/HR$1/MO$13.86/YR
EU-CENTRAL-1$1.92/HR$1/MO$13.92/YR
EU-NORTH-1$1.40/HR$1/MO$13.40/YR
AP-SOUTHEAST-2$1.99/HR$1/MO$13.99/YR
AP-NORTHEAST-1$1.81/HR$1/MO$13.81/YR

Solution overview

I’ll walk you through the process of creating a CMK in AWS KMS with no key material associated and then downloading the public key and import token that you’ll need in order to import the key material later on. I’ll also show you how to create, securely wrap, and export your symmetric key from AWS CloudHSM. Finally, I’ll show you how to import your key material into AWS KMS and then terminate your HSM to save on cost. The following diagram illustrates the steps covered in this post.
 

Figure 1: The process of creating a CMK in AWS KMS

Figure 1: The process of creating a CMK in AWS KMS

  1. Create a customer master key (CMK) in AWS KMS that has no key material associated.
  2. Download the import wrapping key and import token from KMS.
  3. Import the wrapping key provided by KMS into the HSM.
  4. Create a 256 bit symmetric key on AWS CloudHSM.
  5. Use the imported wrapping key to wrap the symmetric key.
  6. Import the symmetric key into AWS KMS using the import token from step 2.
  7. Terminate your HSM, which triggers a backup. Delete or leave your cluster, depending on your needs.

Prerequisites

In this walkthrough, I assume that you already have an AWS CloudHSM cluster set up and initialized with at least one HSM device, and an Amazon Elastic Compute Cloud (EC2) instance running Amazon Linux OS with the AWS CloudHSM client installed. You must have a crypto user (CU) on the HSM to perform the key creation and export functions. You’ll also need an IAM user or role with permissions to both AWS KMS and AWS CloudHSM, and with credentials configured in your AWS Command Line Interface (AWS CLI). Make sure to store your CU information and IAM credentials (if a user is created for this activity) in a safe location. You can use AWS Secrets Manager for secure storage.

Deploying the solution

For my demonstration, I’ll be using the AWS CLI. However, if you prefer working with a graphical user interface, step #1 and the important portion of step #3 can be done in the AWS KMS Console. All other steps are via AWS CLI only.

Step 1: Create the CMK with no key material associated

Begin by creating a customer master key (CMK) in AWS KMS that has no key material associated. The CLI command to create the CMK is as follows:

$ aws kms create-key –origin EXTERNAL –region us-east-1

If successful, you’ll see an output on the CLI similar to below. The KeyState will be PendingImport and the Origin will be EXTERNAL.


{
    "KeyMetadata": {
        "AWSAccountId": "111122223333",
        "KeyId": "1234abcd-12ab-34cd-56ef-1234567890ab",
        "Arn": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab",
        "CreationDate": 1551119868.343,
        "Enabled": false,
        "Description": "",
        "KeyUsage": "ENCRYPT_DECRYPT",
        "KeyState": "PendingImport",
        "Origin": "EXTERNAL",
        "KeyManager": "CUSTOMER"
    }
}

Step 2: Download the public key and import token

With the CMK ID created, you now need to download the import wrapping key and the import token. Wrapping is a method of encrypting the key so that it doesn’t pass in plaintext over the network. You need both the wrapping key and the import token in order to import a key into AWS KMS. You’ll use the public key to encrypt your key material, which ensures that your key is not exposed as it’s imported. When AWS KMS receives the encrypted key material, it will use the corresponding private component of the import wrapping key to decrypt it. The public and private key pair used for this action will always be a 2048-bit RSA key that is unique to each import operation. The import token contains metadata to ensure the key material is imported to the correct CMK ID. Both the import token and import wrapping key are on a time limit, so if you don’t import your key material within 24 hours of downloading them, the import wrapping key and import token will expire and you’ll need to download a new set from KMS.

  1. (OPTIONAL) In my example command, I’ll use the wrapping algorithm RSAES_OAEP_SHA_256 to encrypt and securely import my key into AWS KMS. If you’d like to choose a different algorithm, you may use any of the following: RSAES_OAEP_SHA_256, RSAES_OAEP_SHA_1, or RSAES_PKCS1_V1_5.
  2. In the command below, replace the example key ID (shown in red) with the key ID you received in step 1. If you’d like, you can also modify the wrapping algorithm. Then run the command.

    $ aws kms get-parameters-for-import –key-id <1234abcd-12ab-34cd-56ef-1234567890ab> –wrapping-algorithm <RSAES_OAEP_SHA_256> –wrapping-key-spec RSA_2048 –region us-east-1

    If the command is successful, you’ll see an output similar to what’s below. Your output will contain the import token and import wrapping key. Copy this output to a safe location, as you’ll need it in the next step. Keep in mind, these items will expire after 24 hours.

    
        {
            "KeyId": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab ",
            "ImportToken": "AQECAHjS7Vj0eN0qmSLqs8TfNtCGdw7OfGBfw+S8plTtCvzJBwAABrMwggavBgkqhkiG9w0BBwagggagMIIGnAIBADCCBpUGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMpK6U0uSCNsavw0OJAgEQgIIGZqZCSrJCf68dvMeP7Ah+AfPEgtKielFq9zG4aXAIm6Fx+AkH47Q8EDZgf16L16S6+BKc1xUOJZOd79g5UdETnWF6WPwAw4woDAqq1mYKSRtZpzkz9daiXsOXkqTlka5owac/r2iA2giqBFuIXfr2RPDB6lReyrhAQBYTwefTHnVqosKVGsv7/xxmuorn2iBwMfyqoj2gmykmoxrmWmCdK+jRdWKBHtriwplBTEHpUTdHmQnQzGVThfr611XFCQDZ+uk/wpVFY4jZ/Z/…"
            "PublicKey": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4KCiW36zofngmyVTOpMuFfHAXLWTfl2BAvaeq5puewoMt1zPwBP23qKdG10L7ChTVt4H9PCs1vzAWm2jGeWk5fO…",
            "ParametersValidTo": 1551207484.208
        }
        

  3. Because the output of the command is base64 encoded, you must base64 decode both components into binary data before use. To do this, use your favorite text editor to save the output into two separate files. Name one ImportToken.b64—it will have the output of the “ImportToken” section from above. Name the other PublicKey.b64—it will have the output of the “PublicKey” section from above. You will find example commands to save both below.
    
        echo 'AQECAHjS7Vj0eN0qmSLqs8TfNtCGdw7OfGBfw+S8plTtCvzJBwAABrMwggavBgkqhkiG9w0BBwagggagMIIGnAIBADCCBpUGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMpK6U0uSCNsavw0OJAgEQgIIGZqZCSrJCf68dvMeP7Ah+AfPEgtKielFq9zG4aXAIm6Fx+AkH47Q8EDZgf16L16S6+BKc1xUOJZOd79g5UdETnWF6WPwAw4woDAqq1mYKSRtZpzkz9daiXsOXkqTlka5owac/r2iA2giqBFuIXfr2RPDB6lReyrhAQBYTwefTHnVqosKVGsv7/xxmuorn2iBwMfyqoj2gmykmoxrmWmCdK+jRdWKBHtriwplBTEHpUTdHmQnQzGVThfr611XFCQDZ+uk/wpVFY4jZ/Z' > /ImportToken.b64
    
        echo 'MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4KCiW36zofngmyVTOpMuFfHAXLWTfl2BAvaeq5puewoMt1zPwBP23qKdG10L7ChTVt4H9PCs1vzAWm2jGeWk5fO' > /PublicKey.b64 
        

  4. On both saved files, you must then run the following commands, which will base64 decode them and save them in their binary form:
    
        $ openssl enc -d -base64 -A -in PublicKey.b64 -out PublicKey.bin
    
        $ openssl enc -d -base64 -A -in ImportToken.b64 -out ImportToken.bin
        

Step 3: Import the import wrapping key provided by AWS KMS into your HSM

Now that you’ve created the CMK ID and prepared for the key material import, you’re going to move to AWS CloudHSM to create and export your symmetric encryption key. You’re going to import the import wrapping key provided by AWS KMS into your HSM. Before completing this step, be sure you’ve set up the prerequisites I listed at the start of this post.

  1. Log into your EC2 instance with the AWS CloudHSM client package installed, and launch the Key Management Utility. You will launch the utility using the command below:

    $ /opt/cloudhsm/bin/key_mgmt_util

  2. After launching the key_mgmt_util, you’ll need to log in as the CU user. Do so with the command below, being sure to replace <ExampleUser> and <Password123> for your own CU user and password.

    Command: loginHSM -u CU -p <Password123> -s <ExampleUser>

    You should see an output similar to the example below, letting you know that you’ve logged in correctly:

        
        Cfm3LoginHSM returned: 0x00 : HSM Return: SUCCESS
        Cluster Error Status
        Node id 1 and err state 0x00000000 : HSM Return: SUCCESS
        

    Next, you’ll import the import wrapping key provided by KMS into the HSM, so that you can use it to wrap the symmetric key you’re going to create.

  3. Open the file PublicKey.b64 in your favorite text editor and add the line —–BEGIN PUBLIC KEY—– at the very top of the file and the line —–END PUBLIC KEY—— at the very end. (An example of how this should look is below.) Save this new file as PublicKey.pem.
       
        -----BEGIN PUBLIC KEY-----
        MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA0UnLXHbP/jC/QykQQUroaB0xQYYVaZge//NrmIdYtYvaw32fcEUYjTHovWkYkmifUFVYkNaWKNRGuICo1oAY5cgowgxYcsBXZ4Pk2h3v43tIsxG63ZDLKDFju/dtGaa8+CwR8mR…
        -----END PUBLIC KEY-----
        

  4. Use the importPubKey command from the key_mgmt_util to import the key file. An example of the command is below. For the -l flag (label), I’ve named my example <wrapping-key> so that I’ll know what it’s for. You can name yours whatever you choose. Also, remember to replace <PublicKey.pem> with the actual filename of your public key.

    Command: importPubKey -l <wrapping-key> -f <PublicKey.pem>

    If successful, your output should look similar to the following example. Make note of the key handle, as this is the identifying ID for your wrapping key:

          
        Cfm3CreatePublicKey returned: 0x00 : HSM Return: SUCCESS
    
        Public Key Handle: 30 
        
        Cluster Error Status
        Node id 2 and err state 0x00000000 : HSM Return: SUCCESS
        

Step 4: Create a symmetric key on AWS CloudHSM

Next, you’ll create a symmetric key that you want to export from CloudHSM, so that you can import it into AWS KMS.

The first parameter you’ll use for this action is -t with the value 31. This is the key-type flag and 31 equals an AES key. KMS only accepts the AES key type. The second is -s with a value of 32. This is the key-size flag, and 32 equals a 32-byte key (256 bits). The last flag is the -l flag with the value BYOK_Key. This flag is simply a label to name your key, so you may alter this value however you wish.

Here’s what my example looks like:

Command: genSymKey -t 31 -s 32 -l BYOK-KMS

If successful, your output should look similar to what’s below. Make note of the key handle, as this will be the identifying ID of the key you wish to import into AWS KMS.

  
Cfm3LoginHSM returned: 0x00 : HSM Return: SUCCESS

Symmetric Key Created. Key Handle: 20

Cluster Error Status
Node id 1 and err state 0x00000000 : HSM Return: SUCCESS

Step 5: Use the imported import wrapping key to wrap the symmetric key

Now that you’ve imported the import wrapping key into your HSM in the PublicKey.pem file, I’ll show you how to use the PublicKey.pem file and the key_mgmt_util to wrap your symmetric key out of the HSM.

An example of the command is below. Here’s how to customize the parameters:

  • -k refers to the key handle of your symmetric key. Replace the variable that follows -k with your own key handle ID from step 4.
  • -w refers to the key handle of your wrapping key. Replace the variable with your own ID from step 3.4
  • -out refers to the file name of your exported key. Replace the variable with a file name of your choice.
  • -m refers to the wrapping-mechanism. In my example, I’ve used RSA-OAEP, which has a value of 8.
  • -t refers to the hash-type. In my example, I’ve used SHA256, which has a value of 3.
  • -noheader refers to the -noheader flag. This omits the header that specifies CloudHSM-specific key attributes.

Command: wrapKey -k <20> -w <30> -out <KMS-BYOK-March2019.bin> -m 8 -t 3 -noheader

If successful, you should see an output similar to what’s below:

 
Cfm2WrapKey5 returned: 0x00 : HSM Return: SUCCESS

	Key Wrapped.

	Wrapped Key written to file "KMS-BYOK-March2019.bin length 256

	Cfm2WrapKey returned: 0x00 : HSM Return: SUCCESS

With that, you’ve completed the AWS CloudHSM steps. You now have a symmetric key, securely wrapped for transport into AWS KMS. You can log out of the Key Management Utility by entering the word exit.

Step 6: Import the wrapped symmetric key into AWS KMS using the key import function

You’re ready to import the wrapped symmetric key into AWS KMS using the key import function. Make the following updates to the sample command that I provide below:

  • Replace the key-id value and file names with your own.
  • Leave the expiration model parameter blank, or choose from
    KEY_MATERIAL_EXPIRES or KEY_MATERIAL_DOES_NOT_EXPIRE. If you leave the parameter blank, it will default to KEY_MATERIAL_EXPIRES. This expiration model requires you to set the –valid-to parameter, which can be any time of your choosing so long as it follows the format in the example. If you choose KEY_MATERIAL_DOES_NOT_EXPIRE, you may leave the –valid-to option out of the command. To enforce an expiration and rotation of your key material, best practice is to use the KEY_MATERIAL_EXPIRES option and a date of 1-2 years.

Here’s my sample command:

$ aws kms import-key-material –key-id <1234abcd-12ab-34cd-56ef-1234567890ab> –encrypted-key-material fileb://<KMS-BYOK-March2019.bin> –import-token fileb://<ImportToken.bin> –expiration-model KEY_MATERIAL_EXPIRES –valid-to 2020-03-01T12:00:00-08:00 –region us-east-1

You can test that the import was successful by using the key ID to encrypt a small (under 4KB) file. An example command is below:

$ aws kms encrypt –key-id 1234abcd-12ab-34cd-56ef-1234567890ab –plaintext file://test.txt –region us-east-1

A successful call will output something similar to below:

 
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab ", 
    "CiphertextBlob": "AQICAHi4Pj/Jhk6aHMZNegOpYFnnuiEKAtjRWwD33TdDnKNY5gHbUVdrwptz…"
}

Step 7: Terminate your HSM (which triggers a backup)

The last step in the process is to Terminate your HSM (which triggers a backup). Since you’ve imported the key material into AWS KMS, you no longer need to run the HSM. Terminating the HSM will automatically trigger a backup, which will take an exact copy of the key material and the users on your HSM and store it, encrypted, in an Amazon Simple Storage Service (Amazon S3) bucket. If you need to re-import your key into KMS or if your company’s security policy requires an annual rotation of your KMS master key(s), when the time comes, you can spin up an HSM from your CloudHSM backup and be back to where you started. You can re-import your existing key material or create key materials for your new CMK, either way, you’ll need to request a fresh import wrapping key and import token from KMS.

Deleting an HSM can be done with the command below. Replace <cluster-example> with your actual cluster ID and <hsm-example> with your HSM ID:

$ aws cloudhsmv2 delete-hsm –cluster-id <cluster-example> –hsm-id <hsm-example> –region us-east-1

Following these steps, you will have successfully defined a KMS CMK, created your own key material on a CloudHSM device, and then imported it into AWS KMS for use. Dependent upon region, all of this can be done for less than $15.00 a year.

Summary

In this post, I walked you through creating a CMK ID without encryption key material associated, creating and then exporting a symmetric key from AWS CloudHSM, and then importing it into AWS KMS for use with KMS-integrated AWS services. This process allows you to maintain ownership and management over your master keys, create them on FIPS 140-2 Level 3 validated hardware, and maintain a copy for disaster recovery scenarios. This saves not only the time and personnel required to maintain your own HSMs on-premises, but the cost of hardware, electricity, and housing of the device.

If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread on the AWS CloudHSM or AWS KMS forums.

Want more AWS Security news? Follow us on Twitter.

Author

Tracy Pierce

Tracy Pierce is a Senior Cloud Support Engineer at AWS. She enjoys the peculiar culture of Amazon and uses that to ensure every day is exciting for her fellow engineers and customers alike. Customer Obsession is her highest priority and she shows this by improving processes, documentation, and building tutorials. She has her AS in Computer Security & Forensics from SCTD, SSCP certification, AWS Developer Associate certification, and AWS Security Specialist certification. Outside of work, she enjoys time with friends, her Great Dane, and three cats. She keeps work interesting by drawing cartoon characters on the walls at request.

How to migrate your EC2 Oracle Transparent Data Encryption (TDE) database encryption wallet to CloudHSM

Post Syndicated from Tracy Pierce original https://aws.amazon.com/blogs/security/how-to-migrate-your-ec2-oracle-transparent-data-encryption-tde-database-encryption-wallet-to-cloudhsm/

In this post, I’ll show you how to migrate an encryption wallet for an Oracle database installed on Amazon EC2 from using an outside HSM to using AWS CloudHSM. Transparent Data Encryption (TDE) for Oracle is a common use case for Hardware Security Module (HSM) devices like AWS CloudHSM. Oracle TDE uses what is called “envelope encryption.” Envelope encryption is when the encryption key used to encrypt the tables of your database is in turn encrypted by a master key that resides either in a software keystore or on a hardware keystore, like an HSM. This master key is non-exportable by design to protect the confidentiality and integrity of your database encryption. This gives you a more granular encryption scheme on your data.

An encryption wallet is an encrypted container used to store the TDE master key for your database. The encryption wallet needs to be opened manually after a database startup and prior to the TDE encrypted data being accessed, so the master key is available for data decryption. The process I talk about in this post can be used with any non-AWS hardware or software encryption wallet, or a hardware encryption wallet that utilizes AWS CloudHSM Classic. For my examples in this tutorial, I will be migrating from a CloudHSM Classic to a CloudHSM cluster. It is worth noting that Gemalto has announced the end-of-life for Luna 5 HSMs, which our CloudHSM Classic fleet uses.

Note: You cannot migrate from an Oracle instance in
Amazon Relational Database Service (Amazon RDS) to AWS CloudHSM. You must install the Oracle database on an Amazon EC2 instance. Amazon RDS is not currently integrated with AWS CloudHSM.

When you move from one type of encryption wallet to another, new TDE master keys are created inside the new wallet. To ensure that you have access to backups that rely on your old HSM, consider leaving the old HSM running for your normal recovery window period. The steps I discuss will perform the decryption of your TDE keys and then re-encrypt them with the new TDE master key for you.

Once you’ve migrated your Oracle databases to use AWS CloudHSM as your encryption wallet, it’s also a good idea to set up cross-region replication for disaster recovery efforts. With copies of your database and encryption wallet in another region, you can be back in production quickly should a disaster occur. I’ll show you how to take advantage of this by setting up cross-region snapshots of your Oracle database Amazon Elastic Block Store (EBS) volumes and copying backups of your CloudHSM cluster between regions.

Solution overview

For this solution, you will modify the Oracle database’s encryption wallet to use AWS CloudHSM. This is completed in three steps, which will be detailed below. First, you will switch from the current encryption wallet, which is your original HSM device, to a software wallet. This is done by reverse migrating to a local wallet. Second, you’ll replace the PKCS#11 provider of your original HSM with the CloudHSM PKCS#11 software library. Third, you’ll switch the encryption wallet for your database to your CloudHSM cluster. Once this process is complete, your database will automatically re-encrypt all data keys using the new master key.

To complete the disaster recovery (DR) preparation portion of this post, you will perform two more steps. These consist of copying over snapshots of your EC2 volumes and your CloudHSM cluster backups to your DR region. The following diagram illustrates the steps covered in this post.
 

Figure 1: Steps to migrate your EC2 Oracle TDE database encryption wallet to CloudHSM

Figure 1: Steps to migrate your EC2 Oracle TDE database encryption wallet to CloudHSM

  1. Switch the current encryption wallet for the Oracle database TDE from your original HSM to a software wallet via a reverse migration process.
  2. Replace the PKCS#11 provider of your original HSM with the AWS CloudHSM PKCS#11 software library.
  3. Switch your encryption wallet to point to your AWS CloudHSM cluster.
  4. (OPTIONAL) Set up cross-region copies of the EC2 instance housing your Oracle database
  5. (OPTIONAL) Set up a cross-region copy of your recent CloudHSM cluster backup

Prerequisites

This process assumes you have the below items already set up or configured:

Deploy the solution

Now that you have the basic steps, I’ll go into more detail on each of them. I’ll show you the steps to migrate your encryption wallet to a software wallet using a reverse migration command.

Step 1: Switching the current encryption wallet for the Oracle database TDE from your original HSM to a software wallet via a reverse migration process.

To begin, you must configure the sqlnet.ora file for the reverse migration. In Oracle databases, the sqlnet.ora file is a plain-text configuration file that contains information like encryption, route of connections, and naming parameters that determine how the Oracle server and client must use the capabilities for network database access. You will want to create a backup so you can roll back in the event of any errors. You can make a copy with the below command. Make sure to replace </path/to/> with the actual path to your sqlnet.ora file location. The standard location for this file is “$ORACLE_HOME/network/admin“, but check your setup to ensure this is correct.

cp </path/to/>sqlnet.ora </path/to/>sqlnet.ora.backup

The software wallet must be created before you edit this file, and it should preferably be empty. Then, using your favorite text editor, open the sqlnet.ora file and set the below configuration. If an entry already exists, replace it with the below text.


ENCRYPTION_WALLET_LOCATION=
  (SOURCE=(METHOD=FILE)(METHOD_DATA=
    (DIRECTORY=<path_to_keystore>)))

Make sure to replace the <path_to_keystore> with the directory location of your destination wallet. The destination wallet is the path you choose for the local software wallet. You will notice in Oracle the words “keystore” and “wallet” are interchangeable for this post. Next, you’ll configure the wallet for the reverse migration. For this, you will use the ADMINISTER KEY MANAGEMENT statement with the SET ENCRYPTION KEY and REVERSE MIGRATE clauses as shown in the example below.

By using the REVERSE MIGRATE USING clause in your statement, you ensure the existing TDE table keys and tablespace encryption keys are decrypted by the hardware wallet TDE master key and then re-encrypted with the software wallet TDE master key. You will need to log into the database instance as a user that has been granted the ADMINISTER KEY MANAGEMENT or SYSKM privileges to run this statement. An example of the login is below. Make sure to replace the <sec_admin> and <password> with your administrator user name and password for the database.


sqlplus c##<sec_admin> syskm
Enter password: <password> 
Connected.

Once you’re connected, you’ll run the SQL statement below. Make sure to replace <password> with your own existing wallet password and <username:password> with your own existing wallet user ID and password. We are going to run this statement with the WITH BACKUP parameter, as it’s always ideal to take a backup in case something goes incorrectly.

ADMINISTER KEY MANAGEMENT SET ENCRYPTION KEY IDENTIFIED BY <password> REVERSE MIGRATE USING “<username:password>” WITH BACKUP;

If successful, you will see the text keystore altered. When complete, you do not need to restart your database or manually re-open the local wallet as the migration process loads this into memory for you.

With the migration complete, you’ll now move onto the next step of replacing the PKCS#11 provider of your original HSM with the CloudHSM PKCS#11 software library. This library is a PKCS#11 standard implementation that communicates with the HSMs in your cluster and is compliant with PKCS#11 version 2.40.

Step 2: Replacing the PKCS#11 provider of your original HSM with the AWS CloudHSM PKCS#11 software library.

You’ll begin by installing the software library with the below two commands.

wget https://s3.amazonaws.com/cloudhsmv2-software/CloudHsmClient/EL6/cloudhsm-client-pkcs11-latest.el6.x86_64.rpm

sudo yum install -y ./cloudhsm-client-pkcs11-latest.el6.x86_64.rpm

When installation completes, you will be able to find the CloudHSM PKCS#11 software library files in the directory, the default directory for AWS CloudHSM’s software library installs. To ensure processing speed and throughput capabilities of the HSMs, I suggest installing a Redis cache as well. This cache stores key handles and attributes locally, so you may access them without making a call to the HSMs. As this step is optional and not required for this post, I will leave the link for installation instructions here. With the software library installed, you want to ensure the CloudHSM client is running. You can check this with the command below.

sudo start cloudhsm-client

Step 3: Switching your encryption wallet to point to your AWS CloudHSM cluster.

Once you’ve verified the client is running, you’re going to create another backup of the sqlnet.ora file. It’s always a good idea to take backups before making any changes. The command would be similar to below, replacing </path/to/> with the actual path to your sqlnet.ora file.

cp </path/to/>sqlnet.ora </path/to/>sqlnet.ora.backup2

With this done, again open the sqlnet.ora file with your favorite text editor. You are going to edit the line encryption_wallet_location to resemble the below text.


ENCRYPTION_WALLET_LOCATION=
  (SOURCE=(METHOD=HSM))

Save the file and exit. You will need to create the directory where your Oracle database will expect to find the library file for the AWS CloudHSM PKCS#11 software library. You do this with the command below.

sudo mkdir -p /opt/oracle/extapi/64/hsm

With the directory created, you next copy over the CloudHSM PKCS#11 software library from the original installation directory to this one. It is important this new directory only contain the one library file. Should any files exist in the directory that are not directly related to the way you installed the CloudHSM PKCS#11 software library, remove them. The command to copy is below.

sudo cp /opt/cloudhsm/lib/libcloudhsm_pkcs11_standard.so /opt/oracle/extapi/64/hsm

Now, modify the ownership of the directory and everything inside. The Oracle user must have access to these library files to run correctly. The command to do this is below.

sudo chown -R oracle:dba /opt/oracle

With that done, you can start your Oracle database. This completes the migration of your encryption wallet and TDE keys from your original encryption wallet to a local wallet, and finally to CloudHSM as the new encryption wallet. Should you decide you wish to create new TDE master encryption keys on CloudHSM, you can follow the steps here to do so.

These steps are optional, but helpful in the event you must restore your database to production quickly. For customers that leverage DR environments, we have two great blog posts here and here to walk you through each step of the cross-region replication process. The first uses a combination of AWS Step Functions and Amazon CloudWatch Events to copy your EBS snapshots to your DR region, and the second showcases how to copy your CloudHSM cluster backups to your DR region.

Summary

In this post, I walked you through how to migrate your Oracle TDE database encryption wallet to point it to CloudHSM for secure storage of your TDE. I showed you how to properly install the CloudHSM PKCS#11 software library and place it in the directory for Oracle to find and use. This process can be used to migrate most on-premisis encryption wallet to AWS CloudHSM to ensure security of your TDE keys and meet compliance requirements.

If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread on the AWS CloudHSM forum.

Want more AWS Security news? Follow us on Twitter.

Author

Tracy Pierce

Tracy Pierce is a Senior Cloud Support Engineer at AWS. She enjoys the peculiar culture of Amazon and uses that to ensure every day is exciting for her fellow engineers and customers alike. Customer Obsession is her highest priority and she shows this by improving processes, documentation, and building tutorials. She has her AS in Computer Security & Forensics from SCTD, SSCP certification, AWS Developer Associate certification, and AWS Security Specialist certification. Outside of work, she enjoys time with friends, her Great Dane, and three cats. She keeps work interesting by drawing cartoon characters on the walls at request.

How to eliminate EC2 keypairs from password retrieval of provisioned Windows instances using Secrets Manager and CloudFormation

Post Syndicated from Tracy Pierce original https://aws.amazon.com/blogs/security/how-to-eliminate-ec2-keypairs-password-retrieval-provisioned-windows-instances-secrets-manager-cloudformation/

In my previous post, I showed you how you can increase the durability of your applications and prepare for disaster recovery by using AWS Secrets Manager to replicate your secrets across AWS regions. This is just one of many security best practices you can implement in your AWS environment. Another would be removing the need to share the SSH Private Key to retrieve the password for your Amazon Elastic Compute Cloud (EC2) Windows instances. Currently, to retrieve the Administrator password for your EC2 Windows instance, the instance requires the SSH Private Key to decode the administrator password via the Console or CLI. When you have multiple Administrators that require access, this could result in sharing the SSH Private Key, or even the decoded Administrator password.

To increase security of your environment and remove the requirement of SSH Private Key sharing, I’ll show how you can use AWS Secrets Manager to retrieve the Administrator password from EC2 Windows instances, eliminating the need to share the SSH Private Key. By removing the need to share the SSH Private Key, Administrators no longer need to spend time securing the key or putting mechanisms in place to prevent employees from sharing the key.

I will show you how to use AWS CloudFormation to quickly set up resources in AWS Secrets Manager and EC2. I’ll show you how to use Instance user data to set the local Administrator password, which will enable you to retrieve the password securely without using a shared SSH Private Key. User data is data passed to the instance and is used to perform common automated configuration tasks or run scripts. This could be credential information, shell scripts, or cloud-init directives. This also allows for easy scheduling of password rotations for the Administrator password.

Solution overview

The solution described in this post uses a combination of AWS CloudFormation, AWS Secrets Manager, and Amazon EC2. The AWS CloudFormation template creates a new secret in AWS Secrets Manager with a random value, and then provisions the Windows instance in EC2 using that secret value configured in the EC2 userdata. This userdata sets the secret as the Administrator password for RDP access to the instance. The permissions on the secret created by this process also permit it to rotate the local Administrator password, allowing you to meet best security practices.

Prerequisites

This process assumes you already have an IAM user or role set up in your AWS Account that has mutable permissions to AWS Secrets Manager, AWS Identity and Access Management (IAM), CloudFormation, and EC2. This will be necessary to launch the CloudFormation stack from the template located here. You will also want to have a Security Group set up to permit RDP access to the Windows EC2 instance from allowed IP addresses. You will want to ensure you have your IAM user or role credentials configured on your CLI, if you choose to use that method for launch. Configuring the setting of your CLI with security credentials, default output format, and region are what permits the CLI to interact with AWS APIs.

The following diagram illustrates the process covered in this post.
 

Figure 1: Architectural diagram

Figure 1: Architectural diagram

Once you have your IAM user or role set up, launching the CloudFormation stack will create resources in the following order.

  1. Create a secret in AWS Secrets Manager that contains a random string value.
  2. Create an IAM role and instance profile for the Windows instance with permissions to access the secret.
  3. Create the instance, referencing the secret’s value in the user data, which will be used to set the Administrator password.

Deploy the solution

Now that you know the steps being performed, I’ll walk you through how to use both the AWS Management Console or the AWS CLI to complete this setup. I’ll go over the Console setup and then follow it with the CLI option.

Launch the template using the AWS Management Console

  1. Log in to the CloudFormation Console and select your Region. For my examples, I use the EU-WEST-1 Region, Ireland.
  2. Select Create stack and under Choose a template, select Upload a template to Amazon S3, and then, from your local machine, select the template you downloaded above.
  3. Next, select a unique Stack name, supply the AMI of the EC2 Windows image you want to use, and then select Next. Keep in mind these are unique per Region. For my Stack, I have chosen the name SecretsManager-Windows-Blog and the EU-WEST-1 Windows AMI ami-01776b82784323238.
     
    Figure 2: Select a unique "Stack name" and supply the AMI of the EC2 Windows image you want to use

    Figure 2: Select a unique “Stack name” and supply the AMI of the EC2 Windows image you want to use

  4. You now have the option to add some tags to your stack. I chose to tag it with the key/value pair Name/SecretsManager-Windows-Blog. On this page, you can also choose an IAM role already created for the CloudFormation Stack to run as, or leave it empty.
     
    Figure 3: Add tags to your stack

    Figure 3: Add tags to your stack

    Note: Should you choose not to select an IAM role, CloudFormation will require you to accept that it might create IAM resources. In this case, it will create an IAM role named in the following format: StackName-InstanceRole-RandomString, where StackName is the name you chose for the CloudFormation stack, InstanceRole is the IAM role selected or created to launch the EC2 Instance with (this IAM role is what gives the EC2 instance permission to access AWS APIs), and RandomString is a random alphanumeric string to make the IAM role name unique.

  5. On the Review page, verify your stack information is correct, and then hit Create. CloudFormation will launch your EC2 Windows instance, create your Secret in Secrets Manager, and use the Secret value to set your Administrator password.

Launch the template using the AWS CLI

Make sure to replace the values in <red, italic font> in the examples with values from your own account. You will need to download the template referenced above and upload it to your own S3 Bucket. The S3 URL of the template will be necessary for the following steps.

Run this command:


$ aws cloudformation create-stack --stack-name <SecretsManager-Windows-Blog> --template-url <S3_URL> --parameters ParameterKey=AMI,ParameterValue=<ami-01776b82784323238> --tags Key=Name,Value=<SecretsManager-Windows-Blog> --capabilities CAPABILITY_NAMED_IAM --region <eu-west-1>

If the command ran successfully, you’ll see output similar to this:


$ {
    "StackId": "arn:aws:cloudformation:<eu-west-1:111122223333>:stack/<SecretsManager-Windows-Blog>/<Example_Additional_ID_0123456789>"
}        

Review of the resources your stack creates

Now that your Stack is beginning to create resources, I’ll go over each resource creation event in more detail. The first resource created is the secret stored inside AWS Secrets Manager. The secret is created with the name formatting LocalAdminPassword-RandomString, where RandomString is unique to the secret and the EC2 Windows instance. The key/value pairs of this secret are Username/Administrator and Password/RandomString, where RandomString is unique to the secret and the EC2 Windows instance.

Once the secret is created, the stack creates the IAM role and EC2 Instance Profile. These are required for the EC2 Windows instance to communicate with AWS Secrets Manager and retrieve the stored password. The Trust Policy of the role will list ec2.amazonaws.com as the principal entity, meaning the EC2 instance can assume this IAM role. The permission policy comes in the inline format:

  • An inline policy noted in the template. This policy gives the necessary permissions to retrieve the password from the secret created in AWS Secrets Manager The ARN of the secret created earlier by the CloudFormation template in Secrets Manager is used as the value for the inline policy’s Resource attribute. This is accomplished by using the Reference attribute in the CloudFormation template. This way, the instance can only access the value of its own specific secret.

The last bit for the stack to create is the actual EC2 Windows instance. In my examples, I chose to use the template in its original state. This launches a t2.large instance type. Should you want a different instance type, edit the portion of the template named “InstanceType”: “t2.large” to have the instance type you want to launch. The most important part of the template is the UserData section because this is what retrieves the secret value and sets it as the Administrator password on the instance. For reference, here’s the code:


"UserData": {
    "Fn::Base64": {
        "Fn::Join": [
            "\n",
            [
                "",
                "Import-Module AWSPowerShell",
                {
                    "Fn::Join": [
                        "",
                        [
                            "$password = ((Get-SECSecretValue -SecretId '",
                            {
                                "Ref": "LocalAdminPassword"
                            },
                            "').SecretString | ConvertFrom-Json).Password"
                        ]
                    ]
                },
                "net.exe user Administrator $password",
                ""
            ]
        ]
    }
}        

Once the instance has completed the launch process, your stack will move into the CREATE_COMPLETE status. You can check this in the Console by selecting the StackName and then selecting the Resources tab. I prefer to use the Resources tab as it shows the Physical ID of all resources created by the stack. Here’s an example:
 

Figure 4: Check the status on the "Resources" tab

Figure 4: Check the status on the “Resources” tab

To verify that resources are marked with the CREATE_COMPLETE status with the CLI, run this command (don’t forget to replace the <red> placeholders with your stack informations.


$ aws cloudformation describe-stacks --stack-name <SecretsManager-Windows-Blog> --region <eu-west-1>

You’ll see the “StackStatus”: “CREATE_COMPLETE” and you’ll have an EC2 Windows instance launched, it’s password stored in AWS Secrets Manager, and the instance role giving the instance permissions to retrieve it’s password. You will no longer need to share the SSH Private Key, thus removing another potential security issue.

To verify the secret in the AWS Secrets Manager console is the same one used for your EC2 Windows instance, you can look at the name of the secret itself and the tags listed on the EC2 instance. For example, in the screenshots below, you can see that the secret is named LocalAdminPassword-RandomString. You can then match this to the tag value on your instance with the tag key LocalAdminSecretARN.
 

Figure 5: Verify the secret

Figure 5: Verify the secret

 

Figure 6: Match it to the tag value on your instance with the tag key

Figure 6: Match it to the tag value on your instance with the tag key “LocalAdminSecretARN”

You’ve now launched your EC2 Windows instance, generated a random string password, and will no longer require the SSH Private Key to retrieve the Administrator password for login.

Summary

In this post, I showed you a method to set up a custom Administrator password on a Windows EC2 instance using Instance user data. This password is securely encrypted and stored in AWS Secrets Manager, which will also rotate the password for you. By using this method, you won’t have to share SSH Private Keys to retrieve the Administrator passwords.

If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread on the AWS Secrets Manager forum.

Want more AWS Security news? Follow us on Twitter.

Author

Tracy Pierce

Tracy Pierce is a Senior Cloud Support Engineer at AWS. She enjoys the peculiar culture of Amazon and uses that to ensure every day is exciting for her fellow engineers and customers alike. Customer Obsession is her highest priority and she shows this by improving processes, documentation, and building tutorials. She has her AS in Computer Security & Forensics from SCTD, SSCP certification, AWS Developer Associate certification, and AWS Security Specialist certification. Outside of work, she enjoys time with friends, her Great Dane, and three cats. She keeps work interesting by drawing cartoon characters on the walls at request.

How to automate replication of secrets in AWS Secrets Manager across AWS Regions

Post Syndicated from Tracy Pierce original https://aws.amazon.com/blogs/security/how-to-automate-replication-of-secrets-in-aws-secrets-manager-across-aws-regions/

Assume that you make snapshot copies or read-replicas of your RDS databases in a secondary or backup AWS Region as a best practice. By using AWS Secrets Manager, you can store your RDS database credentials securely using AWS KMS Customer Master Keys, otherwise known as CMKs. AWS Key Management Service (AWS KMS) ensures secrets are encrypted at rest. With the integration of AWS Lambda, you can now more easily rotate these credentials regularly and replicate them for disaster recovery situations. This automation keeps credentials stored in AWS Secrets Manager for Amazon Relational Database Service (Amazon RDS) in sync between the origin Region, where your AWS RDS database lives, and the replica Region where your read-replicas live. While using the same credentials for all databases is not ideal, in the instance of disaster recovery, it can be useful for a quicker recovery.

In this post, I show you how to set up secret replication using an AWS CloudFormation template to create an AWS Lambda Function. By replicating secrets across AWS Regions, you can reduce the time required to get back up and running in production in a disaster recovery situation by ensuring your credentials are securely stored in the replica Region as well.

Solution overview

The solution described in this post uses a combination of AWS Secrets Manager, AWS CloudTrail, Amazon CloudWatch Events, and AWS Lambda. You create a secret in Secrets Manager that houses your RDS database credentials. This secret is encrypted using AWS KMS. Lambda automates the replication of the secret’s value in your origin AWS Region by performing a PUT operation on a secret of the same name in the same AWS Region as your read-replica. CloudWatch Events ensures that each time the secret housing your AWS RDS database credentials is rotated, it triggers the Lambda function to copy the secret’s value to your read-replica Region. By doing this, your RDS database credentials are always in sync for recovery.

Note: You might incur charges for using the services used in this solution, including Lambda. For information about potential costs, see the AWS pricing page.

The following diagram illustrates the process covered in this post.
 

Figure 1: Process diagram

Figure 1: Process diagram

This process assumes you have already created a secret housing your RDS database credentials in your main AWS Region and configured your CloudTrail Logs to send to CloudWatch Logs. Once this is complete, the steps to replicate are here:

  1. Secrets Manager rotates a secret in your original AWS Region.
  2. CloudTrail receives a log with “eventName”: “RotationSuceeded”.
  3. CloudTrail passes this log to CloudWatch Events.
  4. A filter in CloudWatch Events for this EventName triggers a Lambda function.
  5. The Lambda function retrieves the secret value from the origin AWS Region.
  6. The Lambda function then performs PutSecretValue on a secret with the same name in the replica AWS Region.

The Lambda function is triggered by a CloudWatch Event passed by CloudTrail. The triggering event is raised whenever a secret successfully rotates, which creates a CloudTrail log with the EventName property set to RotationSucceeded. You will know the secret rotation was successful when it has the label AWSCURRENT. You can read more about the secret labels and how they change during the rotation process here. The Lambda function retrieves the new secret, then calls PutSecretValue on a secret with the same name in the replica AWS Region. This AWS Region is specified by an environment variable inside the Lambda function.

Note: If the origin secret uses a customer-managed Customer Master Key (CMK), then the cloned secret must, as well. If the origin secret uses an AWS-managed CMK, then the cloned secret must, as well. You can’t mix them or the Lambda function will fail. AWS recommends you use customer-managed CMKs because you have full control of the permissions regarding which entities can use the CMK.

The CloudFormation template also creates an AWS Identity and Access Management (IAM) role with the required permissions to create and update secret replicas. Next, you’ll launch the CloudFormation template by using the AWS CloudFormation CLI.

Deploy the solution

Now that you understand how the Lambda function copies your secrets to the replica AWS Region, I’ll explain the commands to launch your CloudFormation stack. This stack creates the necessary resources to perform the automation. It includes the Lambda function, an IAM role, and the CloudWatch Event trigger.

First, make sure you have credentials for an IAM user or role that can launch all resources included in this template configured on your CLI. To launch the template, run the command below. You’ll choose a unique Stack Name to easily identify its purpose and the URL of the provided template you uploaded to your own S3 Bucket. For the following examples, I will use US-EAST-1 as my origin Region, and EU-WEST-1 as my replica Region. Make sure to replace these values and the other variables (identified by red font) with actual values from your own account.


$ aws cloudformation create-stack --stack-name Replication_Stack --template-url S3_URL --parameters ParameterKey=ReplicaKmsKeyArn,ParameterValue=arn:aws:kms:eu-west-1:111122223333:key/Example_Key_ID_12345 ParameterKey=TargetRegion,ParameterValue=eu-west-1 --capabilities CAPABILITY_NAMED_IAM –-region us-east-1

After the previous command is successful, you will see an output similar to the following with your CloudFormation Stack ARN:


$ {
    "StackId": "arn:aws:cloudformation:us-east-1:111122223333:stack/Replication_Stack/Example_additional_id_0123456789"
}

You can verify that the stack has completed successfully by running the following command:


$ aws cloudformation describe-stacks --stack-name Replication_Stack --region us-east-1

Verify that the StackStatus shows CREATE_COMPLETE. After you verify this, you’ll see three resources created in your account. The first is the IAM role, which allows the Lambda function to perform its API calls. The name of this role is SecretsManagerRegionReplicatorRole, and can be found in the IAM console under Roles. There are two policies attached to this role. The first policy is the managed permissions policy AWSLambdaBasicExecutionRole, which grants permissions for the Lambda function to write to AWS CloudWatch Logs. These logs will be used further on in the Event Rule creation, which will trigger the cloning of the origin secret to the replication region.

The second policy attached to the SecretsManagerRegionReplicatorRole role is an inline policy that grants permissions to decrypt and encrypt the secret in both your original AWS Region and in the replica AWS Region. This policy also grants permissions to retrieve the secret from the original AWS Region, and to store the secret in the replica AWS Region. You can see an example of this policy granting access to specific secrets below. Should you choose to use this policy, please remember to place your parameters into the placeholder values.


{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "KMSPermissions",
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt",
                "kms:Encrypt",
                "kms:GenerateDataKey"
            ],
            "Resource": [
          "arn:aws:kms:us-east-1:111122223333:key/Example_Key_ID_12345",
     "arn:aws:kms:eu-west-1:111122223333:key/Example_Key_ID_12345"
      ]
        },
        {
            "Sid": "SecretsManagerOriginRegion",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:DescribeSecret",
                "secretsmanager:GetSecretValue"
            ],
            "Resource": "arn:aws:secretsmanager:us-east-1:111122223333:secret:replica/myexamplereplica*"
        },
        {
            "Sid": "SecretsManagerReplicaRegion",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:CreateSecret",
                "secretsmanager:UpdateSecretVersionStage",
                "secretsmanager:PutSecretValue",
                "secretsmanager:DescribeSecret"
            ],
            "Resource": "arn:aws:secretsmanager:eu-west-1:111122223333:secret:replica/myexamplereplica*"
        }
    ]
}

The next resource created is the CloudWatch Events rule SecretsManagerCrossRegionReplicator. You can find this rule by going to the AWS CloudWatch console, and, under Events, selecting Rules. Here’s an example Rule for the EventName I described earlier that’s used to trigger the Lambda function:


{
    "detail-type": [
        "AWS Service Event via CloudTrail"
    ],
    "source": [
        "aws.secretsmanager"
    ],
    "detail": {
        "eventSource": [
            "secretsmanager.amazonaws.com"
        ],
        "eventName": [
            "RotationSucceeded"
        ]
    }
}

The last resource created by the CloudFormation template is the Lambda function, which will do most of the actual work for the replication. After it’s created, you’ll be able to find this resource in your Lambda console with the name SecretsManagerRegionReplicator. You can download a copy of the Python script here, and you can see the full script below. In the function’s environment variables, you’ll also notice the parameter names and values you entered when launching your stack.


import boto3
from os import environ

targetRegion = environ.get('TargetRegion')
if targetRegion == None:
    raise Exception('Environment variable "TargetRegion" must be set')

smSource = boto3.client('secretsmanager')
smTarget = boto3.client('secretsmanager', region_name=targetRegion)

def lambda_handler(event, context):
    detail = event['detail']

    print('Retrieving SecretArn from event data')
    secretArn = detail['additionalEventData']['SecretId']

    print('Retrieving new version of Secret "{0}"'.format(secretArn))
    newSecret = smSource.get_secret_value(SecretId = secretArn)

    secretName = newSecret['Name']
    currentVersion = newSecret['VersionId']

    replicaSecretExists = True
    print('Replicating secret "{0}" (Version {1}) to region "{2}"'.format(secretName, currentVersion, targetRegion))
    try:
        smTarget.put_secret_value(
            SecretId = secretName,
            ClientRequestToken = currentVersion,
            SecretString = newSecret['SecretString']
        )
        pass
    except smTarget.exceptions.ResourceNotFoundException:
        print('Secret "{0}" does not exist in target region "{1}". Creating it now with default values'.format(secretName, targetRegion))
        replicaSecretExists = False
    except smTarget.exceptions.ResourceExistsException:
        print('Secret version "{0}" has already been created, this must be a duplicate invocation'.format(currentVersion))
        pass

    if replicaSecretExists == False:
        secretMeta = smSource.describe_secret(SecretId = secretArn)
        if secretMeta['KmsKeyId'] != None:
            replicaKmsKeyArn = environ.get('ReplicaKmsKeyArn')
            if replicaKmsKeyArn == None:
                raise Exception('Cannot create replica of a secret that uses a custom KMS key unless the "ReplicaKmsKeyArn" environment variable is set. Alternatively, you can also create the key manually in the replica region with the same name')

            smTarget.create_secret(
                Name = secretName,
                ClientRequestToken = currentVersion,
                KmsKeyId = replicaKmsKeyArn,
                SecretString = newSecret['SecretString'],
                Description = secretMeta['Description']
            )
        else:
            smTarget.create_secret(
                Name = secretName,
                ClientRequestToken = currentVersion,
                SecretString = newSecret['SecretString'],
                Description = secretMeta['Description']
            )
    else:
        secretMeta = smTarget.describe_secret(SecretId = secretName)
        for previousVersion, labelList in secretMeta['VersionIdsToStages'].items():
            if 'AWSCURRENT' in labelList and previousVersion != currentVersion:
                print('Moving "AWSCURRENT" label from version "{0}" to new version "{1}"'.format(previousVersion, currentVersion))
                smTarget.update_secret_version_stage(
                    SecretId = secretName,
                    VersionStage = 'AWSCURRENT',
                    MoveToVersionId = currentVersion,
                    RemoveFromVersionId = previousVersion
                )
                break

    print('Secret {0} replicated successfully to region "{1}"'.format(secretName, targetRegion))

Now that your CloudFormation Stack is completed, and all necessary resources set up, you are ready to begin secret replication. To verify the setup works, you can modify the secret in the origin region that houses your RDS database credentials. It will take a few moments for the secret label to return to AWSCURRENT, and then roughly another 5-15 minutes for the CloudTrail Logs to populate and then send the Events to CloudWatch Logs. Once received, the Event will trigger the Lambda function to complete the replication process. You will be able to verify the secret has correctly replicated by going to the Secrets Manager Console in your replication Region, selecting the replicated secret, and viewing the values. If the values are the same in the replicated secret as they are the origin secret, and both labels are AWSCURRENT, you know replication completed successfully and your credentials will be ready if needed.

Summary

In this post, you learned how you can use AWS Lambda and Amazon CloudWatch Events to automate replication of your secrets in AWS Secrets Manager across AWS Regions. You used a CloudFormation template to create the necessary resources for the replication setup. The CloudWatch Event will watch for any CloudTrail Logs which would trigger the Lambda function that then pulls the secret name and value and replicates it to the AWS Region of your choice. Should a disaster occur, you’ll have increased your chances for a smooth recovery of your databases, and you’ll be back in production quicker.

If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread on the AWS Secrets Manager forum.

Want more AWS Security news? Follow us on Twitter.

Author

Tracy Pierce

Tracy Pierce is a Senior Cloud Support Engineer at AWS. She enjoys the peculiar culture of Amazon and uses that to ensure every day is exciting for her fellow engineers and customers alike. Customer Obsession is her highest priority and she shows this by improving processes, documentation, and building tutorials. She has her AS in Computer Security & Forensics from SCTD, SSCP certification, AWS Developer Associate certification, and AWS Security Specialist certification. Outside of work, she enjoys time with friends, her Great Dane, and three cats. She keeps work interesting by drawing cartoon characters on the walls at request.

How to clone an AWS CloudHSM cluster across regions

Post Syndicated from Tracy Pierce original https://aws.amazon.com/blogs/security/how-to-clone-an-aws-cloudhsm-cluster-across-regions/

You can use AWS CloudHSM to generate, store, import, export, and manage your cryptographic keys. It also permits hash functions to compute message digests and hash-based message authentication codes (HMACs), as well as cryptographically sign data and verify signatures. To help ensure redundancy of data and simplification of the disaster recovery process, you’ll typically clone your AWS CloudHSM cluster into a different AWS region. This then allows you to synchronize keys, including non-exportable keys, across regions. Non-exportable keys are keys that can never leave the CloudHSM device in plaintext. They reside on the CloudHSM device and are encrypted for security purposes.

You clone a cluster to another region in a two-step process. First, you copy a backup to the destination region. Second, you create a new cluster from this backup. In this post, I’ll show you how to set up one cluster in region 1, and how to use the new CopyBackupToRegion feature to clone the cluster and hardware security modules (HSMs) to a virtual private cloud (VPC) in region 2.

Note: This post doesn’t include instructions on how to set up a cross-region VPC to synchronize HSMs across the two cloned clusters. If you need to do that, read this article.

Solution overview

To complete this solution, you can use either the AWS Command Line Interface (AWS CLI)
or the AWS CloudHSM API. For this post, I’ll use the AWS CLI to copy the cluster backup from region 1 to region 2, and then I’ll launch a new cluster from that copied backup.

The following diagram illustrates the process covered in the post.
 

Figure 1: Architecture diagram

Figure 1: Architecture diagram

Here’s how the process works:

  1. AWS CloudHSM creates a backup of the cluster and stores it in an S3 bucket owned by AWS CloudHSM.
  2. You run the CLI/API command to copy the backup to another AWS region.
  3. When the backup is completed, you use that backup to then create a cluster and HSMs.
  4. Note: Backups can’t be copied into or out of AWS GovCloud (US) because it’s a restricted region.

As with all cluster backups, when you copy the backup to a new AWS region, it’s stored in an Amazon S3 bucket owned by an AWS CloudHSM account. AWS CloudHSM manages the security and storage of cluster backups for you. This means the backup in both regions will also have the durability of Amazon S3, which is 99.999999999%. The backup in region 2 will also be encrypted and secured in the same way as your backup in region 1. You can read more about the encryption process of your AWS CloudHSM backups here.

Any HSMs created in this cloned cluster will have the same users and keys as the original cluster at the time the backup was taken. From this point on, you must manually keep the cloned clusters in sync. Specifically:

  • If you create users after creating your new cluster from the backup, you must create them on both clusters manually.
  • If you change the password for a user in one cluster, you must change the password on the cloned clusters to match.
  • If you create more keys in one cluster, you must sync them to at least one HSM in the cloned cluster. Note that after you sync the key from cluster 1 to cluster 2, the CloudHSM automated cluster synchronization will take care of syncing the keys within the 2nd cluster.

Prerequisites

Some items that will need to be in place for this to work are:

Important note: Syncing keys across clusters in more than one region will only work if all clusters are created from the same backup. This is because synchronization requires the same secret key, called a masking key, to be present on the source and destination HSM. The masking key is specific to each cluster. It can’t be exported, and can’t be used for any purpose other than synchronizing keys across HSMs in a cluster.

Step 1: Create your first cluster in region 1

Follow the links in each sub-step below to the documentation page for more information and setup requirements:

  1. Create the cluster. To do this, you will run the command below via CLI. You will want to replace the placeholder <SUBNET ID 1> with one of your private subnets.
    $ aws cloudhsmv2 create-cluster –hsm-type hsm1.medium –subnet-ids <SUBNET ID 1>
  2. Launch your Amazon Elastic Compute Cloud (Amazon EC2) client (in the public subnet). You can follow the steps here to launch an EC2 Instance.
  3. Create the first HSM (in the private subnet). To do this, you will run the command below via CLI. You will want to replace the placeholder <CLUSTER ID> with the ID given from the ‘Create the cluster’ command above. You’ll replace <AVAILABILITY ZONE> with the AZ matching your private subnet. For example, us-east-1a.
    $ aws cloudhsmv2 create-hsm –cluster-id <CLUSTER ID> –availability-zone <AVAILABILITY ZONE>
  4. Initialize the cluster. Initializing your cluster requires creating a self-signed certificate and using that to sign the cluster’s Certificate Signing Request (CSR). You can view an example here of how to create and use a self-signed certificate. Once you have your certificate, you will run the command below to initialize the cluster with it. You will want to replace the placeholder <CLUSTER ID> with your cluster id from step 1.
    $ aws cloudhsmv2 initialize-cluster –cluster-id <CLUSTER ID> –signed-cert file://<CLUSTER ID>_CustomerHsmCertificate.crt –-trust-anchor file://customerCA.crt

    Note: Don’t forget to place a copy of the certificate used to sign your cluster’s CSR into the /opt/cloudhsm/etc directory to ensure a continued secure connection.

  5. Install the cloudhsm-client software. Once the Amazon EC2 client is launched, you’ll need to download and install the cloudhsm-client software. You can do this by running the command below from the CLI:
    wget https://s3.amazonaws.com/cloudhsmv2-software/CloudHsmClient/EL6/cloudhsm-client-latest.el6.x86_64.rpm

    Once downloaded, you’ll install by running this command:
    sudo yum install -y ./cloudhsm-client-latest.el6.x86_64.rpm

  6. The last step in initializing the cluster requires you to configure the cloudhsm-client to point to the ENI IP of your first HSM. You do this on your EC2 client by running this command:
    $ sudo /opt/cloudhsm/bin/configure -a <IP ADDRESS>
    Replace the <IP ADDRESS> placeholder with your HSM’s ENI IP. The cloudhsm-client comes pre-installed with a Python script called “configure” located in the /opt/cloudhsm/bin/ directory. This will update your /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg and
    /opt/cloudhsm/etc/cloudhsm_client.cfg files with your HSM’s IP address. This ensures your client can connect to your cluster.
  7. Activate the cluster. To activate, you must launch the cloudhsm-client by running this command, which logs you into the cluster:

    $ /opt/cloudhsm/bin/cloudhsm_mgmt_util /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg

    Then, you need to enable the secure communication by running this command:

    aws-cloudhsm>enable_e2e

    If you’ve placed the certificate in the correct directory, you should see a response like this on the command line:

    E2E enabled on server 0(server1)

    If you run the command listUsers you’ll see a PRECO user:

    
    aws-cloudhsm>listUsers
    Users on server 0(server1):
    Number of users found:2
    	User ID		User Type	User Name
    	     1		PRECO		admin
    	     2		AU		app_user
    
    

    Change the password for this user to complete the activation process. You do this by first logging in using the command below:

    aws-cloudhsm>loginHSM PRECO admin password

    Once logged in, change the password using this command:

    
    aws-cloudhsm>changePswd PRECO admin <NEW PASSWORD>
    ***************************CAUTION******************************
    This is a CRITICAL operation, should be done on all nodes in the
    cluster. Cav server does NOT synchronize these changes with the 
    nodes on which this operation is not executed or failed, please
    ensure this operation is executed on all nodes in the cluster.
    ****************************************************************
    
    Do you want to continue(y/n)?Y
    Changing password for admin(PRECO) on 1 nodes
    

    Once completed, log out using the command logout, then log back in with the new password, using the command loginHSM PRECO admin <NEW PASSWORD>.

    Doing this allows you to create the first crypto user (CU). You create the user by running the command:
    aws-cloudhsm>createUser <USERTYPE (ex: CO, CU)> <USERNAME> <PASSWORD>
    Replace the red values in this command. The <USERTYPE> can be a CO (crypto officer) or a CU (crypto user). You can find more information about usertypes here. You’ll replace the placeholders <USERNAME> <PASSWORD> with a real user and password combo. Crypto Users are permitted to create and share keys on the CloudHSM.

    Run the command quit to exit this tool.

Step 2: Trigger a backup of your cluster

To trigger a backup that will be copied to region 2 to create your new cluster, add an HSM to your cluster in region 1. You can do this via the console or CLI. The backup that is created will contain all users (COs, CUs, and appliance users), all key material on the HSMs, and the configurations and policies associated with them. The user portion is extremely important because keys can only be synced across clusters to the same user. Make a note of the backup ID because you will need it later. You can find this by logging into the AWS console and navigating to the CloudHSM console, then selecting Backups. There will be a list of backup IDs, cluster IDs, and creation times. Make sure to select the backup ID specifically created for the cross-region copy.

Step 3: Create a key on your cluster in Region 1

There are many ways to create a key. I’m using key_mgmt_util because it’s an easy and straightforward method using CLI commands instead of SDK libraries. Start by connecting to the EC2 client instance that you launched above and ensuring the cloudhsm-client is running. If you aren’t sure, run this command:

$ sudo start cloudhsm-client

Now, launch the key_mgmt_util by running this command:

$ /opt/cloudhsm/bin/key_mgmt_util

When you see the prompt, log in as a CU to create your key, replacing <USERNAME> and <PASSWORD> with an actual CU user’s username and password:

Command: loginHSM -u CU -s <USERNAME> -p <PASSWORD>

To create the key for this example, we’re going to use the key_mgmt_util to generate a symmetric key. Note the -nex parameter is what makes this key non-exportable. An example command is below:

Command: genSymKey -t 31 -s 32 -l aes256 -nex

In the above command:

  1. genSymKey creates the Symmetric key
  2. -t chooses the key type, which in this case is AES
  3. -s states the key size, which in this case is 32 bytes
  4. -l creates a label to easily recognize the key by
  5. -nex makes the key non-exportable

The HSM will return a key handle. This is used as an identifier to reference the key in future commands. Make a note of the key handle because you will need it later. Here’s an example of the full output in which you can see the key handle provided is 37:


Command:genSymKey -t 31 -s 32 -l aes256 -nex
	Cfm3GenerateSymmetricKey returned: 0x00 : HSM Return: SUCCESS
	Symmetric Key Created.   Key Handle: 37
	Cluster Error Status
	Node id 0 and err state 0x00000000 : HSM Return: SUCCESS

Step 4: Copy your backup from region 1 to region 2 and create a cluster from the backup

To copy your backup from region 1 to region 2, from your EC2 client you’ll need to run the command that appears after these important notes:

  • Make sure the proper permissions are applied for the IAM role or user configured for the CLI. You’ll want to be a CloudHSM administrator for these actions. The instructions here show you how to create an admin user for this process, and here is an example of the permissions policy:
    
    {
       "Version":"2018-06-12",
       "Statement":{
          "Effect":"Allow",
          "Action":[
             "cloudhsm:*",
             "ec2:CreateNetworkInterface",
             "ec2:DescribeNetworkInterfaces",
             "ec2:DescribeNetworkInterfaceAttribute",
             "ec2:DetachNetworkInterface",
             "ec2:DeleteNetworkInterface",
             "ec2:CreateSecurityGroup",
             "ec2:AuthorizeSecurityGroupIngress",
             "ec2:AuthorizeSecurityGroupEgress",
             "ec2:RevokeSecurityGroupEgress",
             "ec2:DescribeSecurityGroups",
             "ec2:DeleteSecurityGroup",
             "ec2:CreateTags",
             "ec2:DescribeVpcs",
             "ec2:DescribeSubnets",
             "iam:CreateServiceLinkedRole"
          ],
          "Resource":"*"
       }
    }
    
    

  • To copy the backup over, you need to know the destination region, the source cluster ID, and/or the source backup ID. You can find the source cluster ID and/or the source backup ID in the CloudHSM console.
  • If you use only the cluster ID, the most recent backup of the associated cluster will be chosen for copy. If you specify the backup ID, that associated backup will be copied. If you don’t know these IDs, run the describe-clusters or describe-backups commands.

Here’s the example command:


$ aws cloudhsmv2 copy-backup-to-region --destination-region <DESTINATION REGION> --cluster-id <CLUSTER ID> --backup-id <BACKUP ID>
{
    “DestinationBackup”: {
	“BackupId”: “backup-gdnekhcxf4n”,
	“CreateTimestamp”: 1531742400,
	“BackupState”: “CREATE_IN_PROGRESS”,
	“SourceCluster”: “cluster-kzlczlspnho”,
	“SourceBackup”: “backup-4kuraxsqetz”,
	“SourceRegion”: “us-east-1”
     }
}

Once the backup has been copied to region 2, you’ll see a new backup ID in your console. This is what you’ll use to create your new cluster. You can follow the steps here to create your new cluster from this backup. This cluster will launch already initialized for you, but it will still need HSMs added into it to complete the activation process. Make sure you copy over the cluster certificate from the original cluster to the new region. You can do this by opening two terminal sessions, one for each HSM. Open the certificate file on the HSM in cluster 1 and copy it. On the HSM in cluster 2, create a file and paste the certificate over. You can use any text editor you like to do this. This certificate is required to establish the encrypted connection between your client and HSM instances.

You should also make sure you’ve added the cloned cluster’s Security Group to your EC2 client instance to allow connectivity. You do this by selecting the Security Group for your EC2 client in the EC2 console, and selecting Add rules. You’ll add a rule allowing traffic, with the source being the Security Group ID of your cluster.

Finally, take note of the ENI IP for the HSM because you’ll need it later. You can find this in your CloudHSM Console by clicking on the cluster for more information.

Step 5: Create a new configuration file with one ENI IP from both clusters

To sync a key from a cluster in region 1 to a cluster in region 2, you must create a configuration file that contains at least one ENI IP of an HSM in both clusters. This is required to allow the cloudhsm-client to communicate with both clusters at the same time. This is where the masking key we mentioned earlier comes into play as the syncKey command uses that to copy keys between clusters. This is why the cluster in region 2 must be created from a backup of the cluster in region 1. For the new configuration file, I’m going to copy over the original file /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg to a new file. Name this SyncClusters.cfg. You’re going to edit this new configuration file to have the ENI IP of the HSM in the cluster of region 1 and the ENI IP of the HSM in the cluster of region 2. It should look something like this:


{
    "scard": {
        "certificate": "cert-sc",
        "enable": "no",
        "pkey": "pkey-sc",
        "port": 2225
    },
    "servers": [
        {
            "CAfile": "",
            "CApath": "/opt/cloudhsm/etc/certs",
            "certificate": "/opt/cloudhsm/etc/client.crt",
            "e2e_encryption": {
                "enable": "yes",
                "owner_cert_path": "/opt/cloudhsm/etc/customerCA.crt"
            },
            "enable": "yes",
            "hostname": "",
            "name": "",
            "pkey": "/opt/cloudhsm/etc/client.key",
            "port": 2225,
            "server_ssl": "yes",
            "ssl_ciphers": ""
        },
        {
            "CAfile": "",
            "CApath": "/opt/cloudhsm/etc/certs",
            "certificate": "/opt/cloudhsm/etc/client.crt",
            "e2e_encryption": {
                "enable": "yes",
                "owner_cert_path": "/opt/cloudhsm/etc/customerCA.crt"
            },
            "enable": "yes",
            "hostname": "",
            "name": "",
            "pkey": "/opt/cloudhsm/etc/client.key",
            "port": 2225,
            "server_ssl": "yes",
            "ssl_ciphers": ""
        }
    ]
}

To verify connectivity to both clusters, start the cloudhsm-client using the modified configuration file. The command will look similar to this:

$ /opt/cloudhsm/bin/cloudhsm_mgmt_util.cfg /opt/cloudhsm/etc/SyncClusters.cfg

After connection, you should see something similar to this, with one IP from cluster 1 and one IP from cluster 2:


Connecting to the server(s), it may take time
depending on the server(s) load, please wait...
Connecting to server '<CLUSTER-1-IP>': hostname '<CLUSTER-1-IP>', port 2225...
Connected to server '<CLUSTER-1-IP>': hostname '<CLUSTER-1-IP>', port 2225.
Connecting to server '<CLUSTER-2-IP>': hostname '<CLUSTER-2-IP>', port 2225...
Connected to server '<CLUSTER-2-IP>': hostname '<CLUSTER-2-IP>', port 2225.

If you run the command info server from the prompt, you’ll see a list of servers your client is connected to. Make note of these because they’ll be important when syncing your keys. Typically, you’ll see server 0 as your first HSM in cluster 1 and server 1 as your first HSM in cluster 2.

Step 6: Sync your key from the cluster in region 1 to the cluster in region 2

You’re ready to sync your keys. Make sure you’ve logged in as the Crypto Officer (CO) user. Only the CO user can perform management functions on the cluster (for example, syncing keys).

Note: These steps are all performed at the server prompt, not the aws-cloudhsm prompt.

First, run the command listUsers to get the user IDs of the user that created the keys. Here’s an example:


server0>listUsers
Users on server 0(<CLUSTER-A-IP>):
Number of users found:3

User Id    User Type	User Name     MofnPubKey     LoginFailureCnt	 2FA
1		CO   	admin         NO               0	 	 NO
2		AU   	app_user      NO               0	 	 NO
3		CU   	<USERNAME>    NO               0	 	 NO

Make note of the user ID because you’ll need it later; in this case, it’s 3. Now, you need to see the key handles that you want to sync. You either noted this from earlier, or you can find this by running the findAllKeys command with the parameter for user 3. Here’s an example:


server0>findAllKeys 3 0
Keys on server 0(<CLUSTER-1-IP<):
Number of keys found 1
number of keys matched from start index 0::1
37
findAllKeys success

In this case, the key handle I want to sync is 37. When running the command syncKey, you’ll input the key handle and the server you want to sync it to (the destination server). Here’s an example:

server0>syncKey 37 1

In this example, 37 is the key handle, and 1 is the destination HSM. You’ll run the exit command to back out to the cluster prompt, and from here you can run findAllKeys again, which should show the same key handle on both clusters.


aws-cloudhsm>findAllKeys 3 0
Keys on server 0(<CLUSTER-1-IP>):
Number of keys found 1
number of keys matched from start index 0::1
37
findAllKeys success on server 0(<CLUSTER-1-IP>)
Keys on server 0(<CLUSTER-2-IP>):
Number of keys found 1
number of keys matched from start index 0::1
37
findAllKeys success on server 0(<CLUSTER-2-IP>)

Repeat this process with all keys you want to sync between clusters.

Summary

I walked you through how to create a cluster, trigger a backup, copy that backup to a new region, launch a new cluster from that backup, and then sync keys across clusters. This will help reduce disaster recovery time, while helping to ensure that your keys are secure in multiple regions should a failure occur.

Remember to always manually update users across clusters after the initial backup copy and cluster creation because these aren’t automatic. You must also run the syncKey command on any keys created after this, as well.

You’re now set up for fault tolerance in your AWS CloudHSM environment.

If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread on the AWS CloudHSM forum.

Want more AWS Security news? Follow us on Twitter.

Author

Tracy Pierce

Tracy Pierce is a Senior Cloud Support Engineer at AWS. She enjoys the peculiar culture of Amazon and uses that to ensure every day is exciting for her fellow engineers and customers alike. Customer Obsession is her highest priority and she shows this by improving processes, documentation, and building tutorials. She has her AS in Computer Security & Forensics from SCTD, SSCP certification, AWS Developer Associate certification, and AWS Security Specialist certification. Outside of work, she enjoys time with friends, her Great Dane, and three cats. She keeps work interesting by drawing cartoon characters on the walls at request.

How to Update AWS CloudHSM Devices and Client Instances to the Software and Firmware Versions Supported by AWS

Post Syndicated from Tracy Pierce original https://aws.amazon.com/blogs/security/how-to-update-aws-cloudhsm-devices-and-client-instances-to-the-software-and-firmware-versions-supported-by-aws/

As I explained in my previous Security Blog post, a hardware security module (HSM) is a hardware device designed with the security of your data and cryptographic key material in mind. It is tamper-resistant hardware that prevents unauthorized users from attempting to pry open the device, plug in any extra devices to access data or keys such as subtokens, or damage the outside housing. The HSM device AWS CloudHSM offers is the Luna SA 7000 (also called Safenet Network HSM 7000), which is created by Gemalto. Depending on the firmware version you install, many of the security properties of these HSMs will have been validated under Federal Information Processing Standard (FIPS) 140-2, a standard issued by the National Institute of Standards and Technology (NIST) for cryptography modules. These standards are in place to protect the integrity and confidentiality of the data stored on cryptographic modules.

To help ensure its continued use, functionality, and support from AWS, we suggest that you update your AWS CloudHSM device software and firmware as well as the client instance software to current versions offered by AWS. As of the publication of this blog post, the current non-FIPS-validated versions are 5.4.9/client, 5.3.13/software, and 6.20.2/firmware, and the current FIPS-validated versions are 5.4.9/client, 5.3.13/software, and 6.10.9/firmware. (The firmware version determines FIPS validation.) It is important to know your current versions before updating so that you can follow the correct update path.

In this post, I demonstrate how to update your current CloudHSM devices and client instances so that you are using the most current versions of software and firmware. If you contact AWS Support for CloudHSM hardware and application issues, you will be required to update to these supported versions before proceeding. Also, any newly provisioned CloudHSM devices will use these supported software and firmware versions only, and AWS does not offer “downgrade options.

Note: Before you perform any updates, check with your local CloudHSM administrator and application developer to verify that these updates will not conflict with your current applications or architecture.

Overview of the update process

To update your client and CloudHSM devices, you must use both update paths offered by AWS. The first path involves updating the software on your client instance, also known as a control instance. Following the second path updates the software first and then the firmware on your CloudHSM devices. The CloudHSM software must be updated before the firmware because of the firmware’s dependencies on the software in order to work appropriately.

As I demonstrate in this post, the correct update order is:

  1. Updating your client instance
  2. Updating your CloudHSM software
  3. Updating your CloudHSM firmware

To update your client instance, you must have the private SSH key you created when you first set up your client environment. This key is used to connect via SSH protocol on port 22 of your client instance. If you have more than one client instance, you must repeat this connection and update process on each of them. The following diagram shows the flow of an SSH connection from your local network to your client instances in the AWS Cloud.

Diagram that shows the flow of an SSH connection from your local network to your client instances in the AWS Cloud

After you update your client instance to the most recent software (5.3.13), you then must update the CloudHSM device software and firmware. First, you must initiate an SSH connection from any one client instance to each CloudHSM device, as illustrated in the following diagram. A successful SSH connection will have you land at the Luna shell, denoted by lunash:>. Second, you must be able to initiate a Secure Copy (SCP) of files to each device from the client instance. Because the software and firmware updates require an elevated level of privilege, you must have the Security Officer (SO) password that you created when you initialized your CloudHSM devices.

Diagram illustrating the initiation of an SSH connection from any one client instance to each CloudHSM device

After you have completed all updates, you can receive enhanced troubleshooting assistance from AWS, if you need it. When new versions of software and firmware are released, AWS performs extensive testing to ensure your smooth transition from version to version.

Detailed guidance for updating your client instance, CloudHSM software, and CloudHSM firmware

1.  Updating your client instance

Let’s start by updating your client instances. My client instance and CloudHSM devices are in the eu-west-1 region, but these steps work the same in any AWS region. Because Gemalto offers client instances in both Linux and Windows, I will cover steps to update both. I will start with Linux. Please note that all commands should be run as the “root” user.

Updating the Linux client

  1. SSH from your local network into the client instance. You can do this from Linux or Windows. Typically, you would do this from the directory where you have stored your private SSH key by using a command like the following command in a terminal or PuTTY This initiates the SSH connection by pointing to the path of your SSH key and denoting the user name and IP address of your client instance.
    ssh –i /Users/Bob/Keys/CloudHSM_SSH_Key.pem [email protected].1.1.1

  1. After the SSH connection is established, you must stop all applications and services on the instance that are using the CloudHSM device. This is required because you are removing old software and installing new software in its place. After you have stopped all applications and services, you can move on to remove the existing version of the client software.
    /usr/safenet/lunaclient/bin/uninstall.sh

This command will remove the old client software, but will not remove your configuration file or certificates. These will be saved in the Chrystoki.conf file of your /etc directory and your usr/safenet/lunaclient/cert directory. Do not delete these files because you will lose the configuration of your CloudHSM devices and client connections.

  1. Download the new client software package: cloudhsm-safenet-client. Double-click it to extract the archive.
    SafeNet-Luna-client-5-4-9/linux/64/install.sh

    Make sure you choose the Luna SA option when presented with it. Because the directory where your certificates are installed is the same, you do not need to copy these certificates to another directory. You do, however, need to ensure that the Chrystoki.conf file, located at /etc/Chrystoki.conf, has the same path and name for the certificates as when you created them. (The path or names should not have changed, but you should still verify they are same as before the update.)

  1. Check to ensure that the PATH environment variable points to the directory, /usr/safenet/lunaclient/bin, to ensure no issues when you restart applications and services. The update process for your Linux client Instance is now complete.

Updating the Windows client

Use the following steps to update your Windows client instances:

  1. Use Remote Desktop Protocol (RDP) from your local network into the client instance. You can accomplish this with the RDP application of your choice.
  2. After you establish the RDP connection, stop all applications and services on the instance that are using the CloudHSM device. This is required because you will remove old software and install new software in its place or overwrite If your client software version is older than 5.4.1, you need to completely remove it and all patches by using Programs and Features in the Windows Control Panel. If your client software version is 5.4.1 or newer, proceed without removing the software. Your configuration file will remain intact in the crystoki.ini file of your C:\Program Files\SafeNet\Lunaclient\ directory. All certificates are preserved in the C:\Program Files\SafeNet\Lunaclient\cert\ directory. Again, do not delete these files, or you will lose all configuration and client connection data.
  3. After you have completed these steps, download the new client software: cloudhsm-safenet-client. Extract the archive from the downloaded file, and launch the SafeNet-Luna-client-5-4-9\win\64\Lunaclient.msi Choose the Luna SA option when it is presented to you. Because the directory where your certificates are installed is the same, you do not need to copy these certificates to another directory. You do, however, need to ensure that the crystoki.ini file, which is located at C:\Program Files\SafeNet\Lunaclient\crystoki.ini, has the same path and name for the certificates as when you created them. (The path and names should not have changed, but you should still verify they are same as before the update.)
  4. Make one last check to ensure the PATH environment variable points to the directory C:\Program Files\SafeNet\Lunaclient\ to help ensure no issues when you restart applications and services. The update process for your Windows client instance is now complete.

2.  Updating your CloudHSM software

Now that your clients are up to date with the most current software version, it’s time to move on to your CloudHSM devices. A few important notes:

  • Back up your data to a Luna SA Backup device. AWS does not sell or support the Luna SA Backup devices, but you can purchase them from Gemalto. We do, however, offer the steps to back up your data to a Luna SA Backup device. Do not update your CloudHSM devices without backing up your data first.
  • If the names of your clients used for Network Trust Link Service (NTLS) connections has a capital “T” as the eighth character, the client will not work after this update. This is because of a Gemalto naming convention. Before upgrading, ensure you modify your client names accordingly. The NTLS connection uses a two-way digital certificate authentication and SSL data encryption to protect sensitive data transmitted between your CloudHSM device and the client Instances.
  • The syslog configuration for the CloudHSM devices will be lost. After the update is complete, notify AWS Support and we will update the configuration for you.

Now on to updating the software versions. There are actually three different update paths to follow, and I will cover each. Depending on the current software versions on your CloudHSM devices, you might need to follow all three or just one.

Updating the software from version 5.1.x to 5.1.5

If you are running any version of the software older than 5.1.5, you must first update to version 5.1.5 before proceeding. To update to version 5.1.5:

  1. Stop all applications and services that access the CloudHSM device.
  2. Download the Luna SA software update package.
  3. Extract all files from the archive.
  4. Run the following command from your client instance to copy the lunasa_update-5.1.5-2.spkg file to the CloudHSM device.
    $ scp –I <private_key_file> lunasa_update-5.1.5-2.spkg [email protected]<hsm_ip_address>:

    <private_key_file> is the private portion of your SSH key pair and <hsm_ip_address> is the IP address of your CloudHSM elastic network interface (ENI). The ENI is the network endpoint that permits access to your CloudHSM device. The IP address was supplied to you when the CloudHSM device was provisioned.

  1. Use the following command to connect to your CloudHSM device and log in with your Security Officer (SO) password.
    $ ssh –I <private_key_file> [email protected]<hsm_ip_address>
    
    lunash:> hsm login

  1. Run the following commands to verify and then install the updated Luna SA software package.
    lunash:> package verify lunasa_update-5.1.5-2.spkg –authcode <auth_code>
    
    lunash:> package update lunasa_update-5.1.5-2.spkg –authcode <auth_code>

    The value you will use for <auth_code> is contained in the lunasa_update-5.1.5-2.auth file found in the 630-010165-018_reva.tar archive you downloaded in Step 2.

  1. Reboot the CloudHSM device by running the following command.
    lunash:> sysconf appliance reboot

When all the steps in this section are completed, you will have updated your CloudHSM software to version 5.1.5. You can now move on to update to version 5.3.10.

Updating the software to version 5.3.10

You can update to version 5.3.10 only if you are currently running version 5.1.5. To update to version 5.3.10:

  1. Stop all applications and services that access the CloudHSM device.
  2. Download the v 5.3.10 Luna SA software update package.
  3. Extract all files from the archive.
  4. Run the following command to copy the lunasa_update-5.3.10-7.spkg file to the CloudHSM device.
    $ scp –i <private_key_file> lunasa_update-5.3.10-7.spkg [email protected]<hsm_ip_address>:

    <private_key_file> is the private portion of your SSH key pair and <hsm_ip_address> is the IP address of your CloudHSM ENI.

  1. Run the following command to connect to your CloudHSM device and log in with your SO password.
    $ ssh –i <private_key_file> [email protected]<hsm_ip_address>
    
    lunash:> hsm login

  1. Run the following commands to verify and then install the updated Luna SA software package.
    lunash:> package verify lunasa_update-5.3.10-7.spkg –authcode <auth_code>
    
    lunash:> package update lunasa_update-5.3.10-7.spkg –authcode <auth_code>

The value you will use for <auth_code> is contained in the lunasa_update-5.3.10-7.auth file found in the SafeNet-Luna-SA-5-3-10.zip archive you downloaded in Step 2.

  1. Reboot the CloudHSM device by running the following command.
    lunash:> sysconf appliance reboot

When all the steps in this section are completed, you will have updated your CloudHSM software to version 5.3.10. You can now move on to update to version 5.3.13.

Note: Do not configure your applog settings at this point; you must first update the software to version 5.3.13 in the following step.

Updating the software to version 5.3.13

You can update to version 5.3.13 only if you are currently running version 5.3.10. If you are not already running version 5.3.10, follow the two update paths mentioned previously in this section.

To update to version 5.3.13:

  1. Stop all applications and services that access the CloudHSM device.
  2. Download the Luna SA software update package.
  3. Extract all files from the archive.
  4. Run the following command to copy the lunasa_update-5.3.13-1.spkg file to the CloudHSM device.
    $ scp –i <private_key_file> lunasa_update-5.3.13-1.spkg [email protected]<hsm_ip_address>

<private_key_file> is the private portion of your SSH key pair and <hsm_ip_address> is the IP address of your CloudHSM ENI.

  1. Run the following command to connect to your CloudHSM device and log in with your SO password.
    $ ssh –i <private_key_file> [email protected]<hsm_ip_address>
    
    lunash:> hsm login

  1. Run the following commands to verify and then install the updated Luna SA software package.
    lunash:> package verify lunasa_update-5.3.13-1.spkg –authcode <auth_code>
    
    lunash:> package update lunasa_update-5.3.13-1.spkg –authcode <auth_code>

The value you will use for <auth_code> is contained in the lunasa_update-5.3.13-1.auth file found in the SafeNet-Luna-SA-5-3-13.zip archive that you downloaded in Step 2.

  1. When updating to this software version, the option to update the firmware also is offered. If you do not require a version of the firmware validated under FIPS 140-2, accept the firmware update to version 6.20.2. If you do require a version of the firmware validated under FIPS 140-2, do not accept the firmware update and instead update by using the steps in the next section, “Updating your CloudHSM FIPS 140-2 validated firmware.”
  2. After updating the CloudHSM device, reboot it by running the following command.
    lunash:> sysconf appliance reboot

  1. Disable NTLS IP checking on the CloudHSM device so that it can operate within its VPC. To do this, run the following command.
    lunash:> ntls ipcheck disable

When all the steps in this section are completed, you will have updated your CloudHSM software to version 5.3.13. If you don’t need the FIPS 140-2 validated firmware, you will have also updated the firmware to version 6.20.2. If you do need the FIPS 140-2 validated firmware, proceed to the next section.

3.  Updating your CloudHSM FIPS 140-2 validated firmware

To update the FIPS 140-2 validated version of the firmware to 6.10.9, use the following steps:

  1. Download version 6.10.9 of the firmware package.
  2. Extract all files from the archive.
  3. Run the following command to copy the 630-010430-010_SPKG_LunaFW_6.10.9.spkg file to the CloudHSM device.
    $ scp –i <private_key_file> 630-010430-010_SPKG_LunaFW_6.10.9.spkg [email protected]<hsm_ip_address>:

<private_key_file> is the private portion of your SSH key pair, and <hsm_ip_address> is the IP address of your CloudHSM ENI.

  1. Run the following command to connect to your CloudHSM device and log in with your SO password.
    $ ssh –i <private_key_file> manager#<hsm_ip_address>
    
    lunash:> hsm login

  1. Run the following commands to verify and then install the updated Luna SA firmware package.
    lunash:> package verify 630-010430-010_SPKG_LunaFW_6.10.9.spkg –authcode <auth_code>
    
    lunash:> package update 630-010430-010_SPKG_LunaFW_6.10.9.spkg –authcode <auth_code>

The value you will use for <auth_code> is contained in the 630-010430-010_SPKG_LunaFW_6.10.9.auth file found in the 630-010430-010_SPKG_LunaFW_6.10.9.zip archive that you downloaded in Step 1.

  1. Run the following command to update the firmware of the CloudHSM devices.
    lunash:> hsm update firmware

  1. After you have updated the firmware, reboot the CloudHSM devices to complete the installation.
    lunash:> sysconf appliance reboot

Summary

In this blog post, I walked you through how to update your existing CloudHSM devices and clients so that they are using supported client, software, and firmware versions. Per AWS Support and CloudHSM Terms and Conditions, your devices and clients must use the most current supported software and firmware for continued troubleshooting assistance. Software and firmware versions regularly change based on customer use cases and requirements. Because AWS tests and validates all updates from Gemalto, you must install all updates for firmware and software by using our package links described in this post and elsewhere in our documentation.

If you have comments about this blog post, submit them in the “Comments” section below. If you have questions about implementing this solution, please start a new thread on the CloudHSM forum.

– Tracy

How to Remove Single Points of Failure by Using a High-Availability Partition Group in Your AWS CloudHSM Environment

Post Syndicated from Tracy Pierce original https://blogs.aws.amazon.com/security/post/Tx7VU4QS5RCK7Q/How-to-Remove-Single-Points-of-Failure-by-Using-a-High-Availability-Partition-Gr

A hardware security module (HSM) is a hardware device designed with the security of your data and cryptographic key material in mind. It is tamper-resistant hardware that prevents unauthorized users from attempting to pry open the device, plug any extra devices in to access data or keys such as subtokens, or damage the outside housing. If any such interference occurs, the device wipes all information stored so that unauthorized parties do not gain access to your data or cryptographic key material. A high-availability (HA) setup could be beneficial because, with multiple HSMs kept in different data centers and all data synced between them, the loss of one HSM does not mean the loss of your data.

In this post, I will walk you through steps to remove single points of failure in your AWS CloudHSM environment by setting up an HA partition group. Single points of failure occur when a single CloudHSM device fails in a non-HA configuration, which can result in the permanent loss of keys and data. The HA partition group, however, allows for one or more CloudHSM devices to fail, while still keeping your environment operational.

Prerequisites

You will need a few things to build your HA partition group with CloudHSM:

  • 2 CloudHSM devices. AWS offers a free two-week trial. AWS will provision the trial for you and send you the CloudHSM information such as the Elastic Network Interface (ENI) and the private IP address assigned to the CloudHSM device so that you may begin testing. If you have used CloudHSM before, another trial cannot be provisioned, but you can set up production CloudHSM devices on your own. See Provisioning Your HSMs.
  • A client instance from which to access your CloudHSM devices. You can create this manually, or via an AWS CloudFormation template. You can connect to this instance in your public subnet, and then it can communicate with the CloudHSM devices in your private subnets.
  • An HA partition group, which ensures the syncing and load balancing of all CloudHSM devices you have created.

The CloudHSM setup process takes about 30 minutes from beginning to end. By the end of this how-to post, you should be able to set up multiple CloudHSM devices and an HA partition group in AWS with ease. Keep in mind that each production CloudHSM device you provision comes with an up-front fee of $5,000. You are not charged for any CloudHSM devices provisioned for a trial, unless you decide to move them to production when the trial ends.

If you decide to move your provisioned devices to your production environment, you will be billed $5,000 per device. If you decide to stop the trial so as not to be charged, you have up to 24 hours after the trial ends to let AWS Support know of your decision.

Solution overview

How HA works

HA is a feature of the Luna SA 7000 HSM hardware device AWS uses for its CloudHSM service. (Luna SA 7000 HSM is also known as the “SafeNet Network HSM” in more recent SafeNet documentation. Because AWS documentation refers to this hardware as “Luna SA 7000 HSM,” I will use this same product name in this post.) This feature allows more than one CloudHSM device to be placed as members in a load-balanced group setup. By having more than one device on which all cryptographic material is stored, you remove any single points of failure in your environment.

You access your CloudHSM devices in this HA partition group through one logical endpoint, which distributes traffic to the CloudHSM devices that are members of this group in a load-balanced fashion. Even though traffic is balanced between the HA partition group members, any new data or changes in data that occur on any CloudHSM device will be mirrored for continuity to the other members of the HA partition group. A single HA partition group is logically represented by a slot, which is physically composed of multiple partitions distributed across all HA nodes. Traffic is sent through the HA partition group, and then distributed to the partitions that are linked. All partitions are then synced so that data is persistent on each one identically.

The following diagram illustrates the HA partition group functionality.

  1. Application servers send traffic to your HA partition group endpoint.
  2. The HA partition group takes all requests and distributes them evenly between the CloudHSM devices that are members of the HA partition group.
  3. Each CloudHSM device mirrors itself to each other member of the HA partition group to ensure data integrity and availability.

Automatic recovery

If you ever lose data, you want a hands-off, quick recovery. Before autoRecovery was introduced, you could take advantage of the redundancy and performance HA partition groups offer, but you were still required to manually intervene when a group member was lost.

HA partition group members may fail for a number of reasons, including:

  • Loss of power to a CloudHSM device.
  • Loss of network connectivity to a CloudHSM device. If network connectivity is lost, it will be seen as a failed device and recovery attempts will be made.

Recovery of partition group members will only work if the following are true:

  • HA autoRecovery is enabled.
  • There are at least two nodes (CloudHSM devices) in the HA partition group.
  • Connectivity is established at startup.
  • The recover retry limit is not reached (if reached or exceeded, the only option is manual recovery).

HA autoRecovery is not enabled by default and must be explicitly enabled by running the following command, which is found in Enabling Automatic Recovery.

>vtl haAdmin –autoRecovery –retry <count>

When enabling autoRecovery, set the –retry and –interval parameters. The –retry parameter can be a value between 0 and 500 (or -1 for infinite retries), and equals the number of times the CloudHSM device will attempt automatic recovery. The –interval parameter is in seconds and can be any value between 60 and 1200. This is the amount of time between automatic recovery tries that the CloudHSM will attempt.

Setting up two Production CloudHSM devices in AWS

Now that I have discussed how HA partition groups work and why they are useful, I will show how to set up your CloudHSM environment and the HA partition group itself. To create an HA partition group environment, you need a minimum of two CloudHSM devices. You can have as many as 16 CloudHSM devices associated with an HA partition group at any given time. These must be associated with the same account and region, but can be spread across multiple Availability Zones, which is the ideal setup for an HA partition group. Automatic recovery is great for larger HA partition groups because it allows the devices to quickly attempt recovery and resync data in the event of a failure, without requiring manual intervention.

Set up the CloudHSM environment

To set up the CloudHSM environment, you must have a few things already in place:

  • An Amazon VPC.
  • At least one public subnet and two private subnets.
  • An Amazon EC2 client instance (m1.small running Amazon Linux x86 64-bit) in the public subnet, with the SafeNet client software already installed. This instance uses the key pair that you specified during creation of the CloudFormation stack. You can find a ready-to-use Amazon Machine Image (AMI) in our Community AMIs. Simply log into the EC2 console, choose Launch Instance, click Community AMIs, and search for CloudHSM. Because we regularly release new AMIs with software updates, searching for CloudHSM will show all available AMIs for a region. Select the AMI with the most recent client version.
  • Two security groups, one for the client instance and one for the CloudHSM devices. The security group for the client instance, which resides in the public subnet, will allow SSH on port 22 from your local network. The security group for the CloudHSM devices, which resides in the private subnet, will allow SSH on port 22 and NTLS on port 1792 from your public subnet. These will both be ingress rules (egress rules allow all traffic).
  • An Elastic IP address for the client instance.
  • An IAM role that delegates AWS resource access to CloudHSM. You can create this role in the IAM console:

    1. Click Roles and then click Create New Role.
    2. Type a name for the role and then click Next Step.
    3. Under AWS Service Roles, click Select next to AWS CloudHSM.
    4. In the Attach Policy step, select AWSCloudHSMRole as the policy. Click Next Step.
    5. Click Create Role.

We have a CloudFormation template available that will set up the CloudHSM environment for you:

  1. Go to the CloudFormation console.
  2. Choose Create Stack. Specify https://cloudhsm.s3.amazonaws.com/cloudhsm-quickstart.json as the Amazon S3 template URL.
  3. On the next two pages, specify parameters such as the Stack name, SSH Key Pair, Tags, and SNS Topic for alerts. You will find SNS Topic under the Advanced arrow. Then, click Create.

When the new stack is in the CREATION_COMPLETE state, you will have the IAM role to be used for provisioning your CloudHSM devices, the private and public subnets, your client instance with Elastic IP (EIP), and the security groups for both the CloudHSM devices and the client instance. The CloudHSM security group will already have its necessary rules in place to permit SSH and NTLS access from your public subnet; however, you still must add the rules to the client instance’s security group to permit SSH access from your allowed IPs. To do this:

  1. In the VPC console, make sure you select the same region as the region in which your HSM VPC resides.
  2. Select the security group in your HSM VPC that will be used for the client instance.
  3. Add an inbound rule that allows TCP traffic on port 22 (SSH) from your local network IP addresses.
  4. On the Inbound tab, from the Create a new rule list, select SSH, and enter the IP address range of the local network from which you will connect to your client instance.
  5. Click Add Rule, and then click Apply Rule Changes.

After adding the IP rules for SSH (port 22) to your client instance’s security group, test the connection by attempting to make a SSH connection locally to your client instance EIP. Make sure to write down all the subnet and role information, because you will need this later.

Create an SSH key pair

The SSH key pair that you will now create will be used by CloudHSM devices to authenticate the manager account when connecting from your client instance. The manager account is simply the user that is permitted to SSH to your CloudHSM devices. Before provisioning the CloudHSM devices, you create the SSH key pair so that you can provide the public key to the CloudHSM during setup. The private key remains on your client instance to complete the authentication process. You can generate the key pair on any computer, as long as you ensure the client instance has the private key copied to it. You can also create the key pair on Linux or Windows. I go over both processes in this section of this post.

In Linux, you will use the ssh-keygen command. By typing just this command into the terminal window, you will receive output similar to the following.

$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user/.ssh/id_rsa.
Your public key has been saved in /home/user/.ssh/id_rsa.pub.
The key fingerprint is: df:c4:49:e9:fe:8e:7b:eb:28:d5:1f:72:82:fb:f2:69
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|             .   |
|            o    |
|           + .   |
|        S   *.   |
|         . =.o.o |
|          ..+ +..|
|          .o Eo .|
|           .OO=. |
+-----------------+
$

In Windows, use PuTTYgen to create your key pair:

  1. Start PuTTYgen. For Type of key to generate, select SSH-2 RSA.
  2. In the Number of bits in a generated key field, specify 2048.
  3. Click Generate.
  4. Move your mouse pointer around in the blank area of the Key section below the progress bar (to generate some randomness) until the progress bar is full.
  5. A private/public key pair has now been generated.
  6. In the Key comment field, type a name for the key pair that you will remember.
  7. Click Save public key and name your file.
  8. Click Save private key and name your file. It is imperative that you do not lose this key, so make sure to store it somewhere safe.
  9. Right-click the text field labeled Public key for pasting into OpenSSH authorized_keys file and choose Select All.
  10. Right-click again in the same text field and choose Copy.

The following screenshot shows what the PuTTYgen output will look like after you have created the key pair.

You must convert the keys created by PuTTYgen to OpenSSH format for use with other clients by using the following command.

ssh-keygen –i –f puttygen_key > openssh_key

The public key will be used to provision the CloudHSM device and the private key will be stored on the client instance to authenticate the SSH sessions. The public SSH key will look something like the following. If it does not, it is not in the correct format and must be converted using the preceding procedure.

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6bUsFjDSFcPC/BZbIAv8cAR5syJMB GiEqzFOIEHbm0fPkkQ0U6KppzuXvVlc2u7w0mg
PMhnkEfV6j0YBITu0Rs8rNHZFJs CYXpdoPxMMgmCf/FaOiKrb7+1xk21q2VwZyj13GPUsCxQhRW7dNidaaYTf14sbd9A qMUH4UOUjs
27MhO37q8/WjV3wVWpFqexm3f4HPyMLAAEeExT7UziHyoMLJBHDKMN7 1Ok2kV24wwn+t9P/Va/6OR6LyCmyCrFyiNbbCDtQ9JvCj5
RVBla5q4uEkFRl0t6m9 XZg+qT67sDDoystq3XEfNUmDYDL4kq1xPM66KFk3OS5qeIN2kcSnQ==

Whether you are saving the private key on your local computer or moving it to the client instance, you must ensure that the file permissions are correct. You can do this by running the following commands (throughout this post, be sure to replace placeholder content with your own values). The first command sets the necessary permissions; the second command adds the private key to the authentication agent.

$ chmod 600 ~/.ssh/<private_key_file>
$ ssh-add ~/.ssh/<private_key_file>

Set up the AWS CLI tools

Now that you have your SSH key pair ready, you can set up the AWS CLI tools so that you may provision and manage your CloudHSM devices. If you used the CloudFormation template or the CloudHSM AMI to set up your client instance, you already have the CLI installed. You can check this by running at the command prompt: CloudHSM version. The resulting output should be “Version”: “3.0.5”. If you chose to use your own AMI and install the Luna SA software, you can install the CloudHSM CLI Tools with the following steps. The current version in use is 3.0.5.

$ wget https://s3.amazon.com/cloudhsm-software/CloudHsmCLI.egg
$ sudo easy_install-2.7 –s /usr/local/bin CloudHsmCLI.egg
$ cloudhsm version
{
      “Version”: “<version>”
}

You must also set up file and directory ownership for your user on the client instance and the Chrystoki.conf file. The Chrystoki.conf file is the configuration file for the CloudHSM device. By default, CloudHSM devices come ready from the factory for immediate housing of cryptographic keys and performing cryptographic processes on data, but must be configured to connect to your client instances:

  1. On the client instance, set the owner and write permission on the Chrystoki.conf file.
$ sudo chown <owner> /etc/Chrystoki.conf
$ sudo chmod +w /etc/Chrystoki.conf

The <owner> can be either the user or a group the user belongs to (for example, ec2-user).

  1. On the client instance set the owner of the Luna client directory:
$ sudo chown <owner> -R <luna_client_dir>

The <owner> should be the same as the <owner> of the Chrystoki.conf file. The <luna_client_dir> differs based on the version of the LunaSA client software installed. If these are new setups, use version 5.3 or newer; however, if you have older clients with version 5.1 installed, use version 5.1:

  • Client software version 5.3: /usr/safenet/lunaclient/
  • Client software version 5.1: /usr/lunasa/

You also must configure the AWS CLI tools with AWS credentials to use for the API calls. These can be set by config files, passing the credentials in the commands, or by instance profile. The most secure option, which eliminates the need to hard-code credentials in a config file, is to use an instance profile on your client instance. All CLI commands in this post are performed on a client instance launched with a IAM role that has CloudHSM permissions. If you want to set your credentials in a config file instead, you must remember that each CLI command should include –profile <profilename>, with <profilename> being the name you assigned in the config file for these credentials.See Configuring the AWS CloudHSM CLI Tools for help with setting up the AWS CLI tools.

You will then set up a persistent SSH tunnel for all CloudHSM devices to communicate with the client instance. This is done by editing the ~/.ssh/config file. Replace <CloudHSM_ip_address> with the private IP of your CloudHSM device, and replace <private_key_file> with the file location of your SSH private key created earlier (for example, /home/user/.ssh/id_rsa).

Host <CloudHSM_ip_address>
User manager
IdentityFile <private_key_file>

Also necessary for the client instance to authenticate with the CloudHSM partitions or partition group are client certificates. Depending on the LunaSA client software you are using, the location of these files can differ. Again, if these are new setups, use version 5.3 or newer; however, if you have older clients with version 5.1 installed, use version 5.1:

  • Linux clients

    • Client software version 5.3: /usr/safenet/lunaclient/cert
    • Client software version 5.1: /usr/lunasa/cert
  • Windows clients

    • Client software version 5.3: %ProgramFiles%\SafeNet\LunaClient\cert
    • Client software version 5.1: %ProgramFiles%\LunaSA\cert

To create the client certificates, you can use the OpenSSL Toolkit or the LunaSA client-side vtl commands. The OpenSSL Toolkit is a program that allows you to manage TLS (Transport Layer Security) and SSL (Secure Sockets Layer) protocols. It is commonly used to create SSL certificates for secure communication between internal network devices. The LunaSA client-side vtl commands are installed on your client instance along with the Luna software. If you used either CloudFormation or the CloudHSM AMI, the vtl commands are already installed for you. If you chose to launch a different AMI, you can download the Luna software. After you download the software, run the command linux/64/install.sh as root on a Linux instance and install the Luna SA option. If you install the softtware on a Windows instance, run the command windows\64\LunaClient.msi to install the Luna SA option. I show certificate creation in both OpenSSL Toolkit and LunaSA in the following section.

OpenSSL Toolkit

      $ openssl genrsa –out <luna_client_cert_dir>/<client_name>Key.pem 2048
      $ openssl req –new –x509 –days 3650 –key <luna_client_cert_dir>/<client_cert>Key.pem –out <client_name>.pem

The <luna_client_cert_dir> is the LunaSA Client certificate directory on the client and the <client_name> can be whatever you choose.

LunaSA

      $ sudo vtl createCert –n <client_name>

The output of the preceding LunaSA command will be similar to the following.

Private Key created and written to:

<luna_client_cert_dir>/<client_name>Key.pem

Certificate created and written to:

<luna_client_cert_dir>/<client_name>.pem

You will need these key file locations later on, so make sure you write them down or save them to a file. One last thing to do at this point is create the client Amazon Resource Name (ARN), which you do by running the following command.

$ CloudHSM create-client –certificate-file <luna_client_cert_dir>/<client_name>.pem

{
      “ClientARN”: “<client_arn>”,
      “RequestId”: “<request_id>”
}

Also write down in a safe location the client ARN because you will need it when registering your client instances to the HA partition group.

Provision your CloudHSM devices

Now for the fun and expensive part. Always remember that for each CloudHSM device you provision to your production environment, there is an upfront fee of $5,000. Because you need more than one CloudHSM device to set up an HA partition group, provisioning two CloudHSM devices to production will cost an upfront fee of $10,000.

If this is your first time trying out CloudHSM, you can have a two-week trial provisioned for you at no cost. The only cost will occur if you decide to keep your CloudHSM devices and move them into production. If you are unsure of the usage in your company, I highly suggest doing a trial first. You can open a support case requesting a trial at any time. You must have a paid support plan to request a CloudHSM trial.

To provision the two CloudHSM devices, SSH into your client instance and run the following CLI command.

$ CloudHSM create-CloudHSM \
--subnet-id <subnet_id> \
--ssh-public-key-file <public_key_file> \
--iam-role-arn <iam_role_arn> \
--syslog-ip <syslog_ip_address>

The response should resemble the following.

{
      “CloudHSMArn”: “<CloudHSM_arn>”,
      “RequestId”: “<request_id>”
}

Make note of each CloudHSM ARN because you will need them to initialize the CloudHSM devices and later add them to the HA partition group.

Initialize the CloudHSM devices

Configuring your CloudHSM devices, or initializing them as the process is formally called, is what allows you to set up the configuration files, certificate files, and passwords on the CloudHSM itself. Because you already have your CloudHSM ARNs from the previous section, you can run the describe-hsm command to get the EniId and the EniIp of the CloudHSM devices. Your results should be similar to the following.

$ CloudHSM describe-CloudHSM --CloudHSM-arn <CloudHSM_arn>
{    
     "EniId": "<eni_id>",   
     "EniIp": "<eni_ip>",    
     "CloudHSMArn": "<CloudHSM_arn>",    
     "IamRoleArn": "<iam_role_arn>",    
     "Partitions": [],    
     "RequestId": "<request_id>",    
     "SerialNumber": "<serial_number>",    
     "SoftwareVersion": "5.1.3-1",    
     "SshPublicKey": "<public_key_text>",    
     "Status": "<status>",    
     "SubnetId": "<subnet_id>",    
     "SubscriptionStartDate": "2014-02-05T22:59:38.294Z",    
     "SubscriptionType": "PRODUCTION",    
     "VendorName": "SafeNet Inc."
}

Now that you know the EniId of each CloudHSM, you need to apply the CloudHSM security group to them. This ensures that connection can occur from any instance with the client security group assigned. When a trial is provisioned for you, or you provision CloudHSM devices yourself, the default security group of the VPC is automatically assigned to the ENI.You must change this to the security group that permits ingress ports 22 and 1792 from your client instance.

To apply a CloudHSM security group to an EniId:

  1. Go to the EC2 console, and choose Network Interfaces in the left pane.
  2. Select the EniId of the CloudHSM.
  3. From the Actions drop-down list, choose Change Security Groups. Choose the security group for your CloudHSM device, and then click Save.

To complete the initialization process, you must ensure a persistent SSH connection is in place from your client to the CloudHSM. Remember the ~/.ssh/config file you edited earlier? Now that you have the IP address of the CloudHSM devices and the location of the private SSH key file, go back and fill in that config file’s parameters by using your favorite text editor.

Now, initialize using the initialize-hsm command with the information you gathered from the provisioning steps. The values in red in the following example are meant as placeholders for your own naming and password conventions, and should be replaced with your information during the initialization of the CloudHSM devices.

$CloudHSM initialize-CloudHSM \
--CloudHSM-arn <CloudHSM_arn> \
--label <label> \
--cloning-domain <cloning_domain> \
--so-password <so_password>

The <label> is a unique name for the CloudHSM device that should be easy to remember. You can also use this name as a descriptive label that tells what the CloudHSM device is for. The <cloning_domain> is a secret used to control cloning of key material from one CloudHSM to another. This can be any unique name that fits your company’s naming conventions. Examples could be exampleproduction or exampledevelopment. If you are going to set up an HA partition group environment, the <cloning_domain> must be the same across all CloudHSMs. The <so_password> is the security officer password for the CloudHSM device, and for ease of remembrance, it should be the same across all devices as well. It is important you use passwords and cloning domain names that you will remember, because they are unrecoverable and the loss of them means loss of all data on a CloudHSM device. For your use, we do supply a Password Worksheet if you want to write down your passwords and store the printed page in a secure place.

Configure the client instance

Configuring the client instance is important because it is the secure link between you, your applications, and the CloudHSM devices. The client instance opens a secure channel to the CloudHSM devices and sends all requests over this channel so that the CloudHSM device can perform the cryptographic operations and key storage. Because you already have launched the client instance and mostly configured it, the only step left is to create the Network Trust Link (NTL) between the client instance and the CloudHSM. For this, we will use the LunaSA vtl commands again.

  1. Copy the server certificate from the CloudHSM to the client.
$ scp –i ~/.ssh/<private_key_file> [email protected]<CloudHSM_ip_address>:server.pem
  1. Register the CloudHSM certificate with the client.
$ sudo vtl addServer –n <CloudHSM_ip_address> -c server.pem
New server <CloudHSM_ip_address> successfully added to server list.
  1. Copy the client certificate to the CloudHSM.
$ scp –i ~/.ssh/<private_key_file> <client_cert_directory>/<client_name>.pem [email protected]<CloudHSM_ip_address>:
  1. Connect to the CloudHSM.
$ ssh –i ~/.ssh/<private_key_file> [email protected]<CloudHSM_ip_address>
lunash:>
  1. Register the client.
lunash:> client register –client <client_id> -hostname <client_name>

The <client_id> and <client_name> should be the same for ease of use, and this should be the same as the name you used when you created your client certificate.

  1. On the CloudHSM, log in with the SO password.
lunash:> hsm login
  1. Create a partition on each CloudHSM (use the same name for ease of remembrance).
lunash:> partition create –partition <name> -password <partition_password> -domain <cloning_domain>

The <partition_password> does not have to be the same as the SO password, and for security purposes, it should be different.

  1. Assign the client to the partition.
lunash:> client assignPartition –client <client_id> -partition <partition_name>
  1. Verify that the partition assigning went correctly.
lunash:> client show –client <client_id>
  1. Log in to the client and verify it has been properly configured.
$ vtl verify
The following Luna SA Slots/Partitions were found:
Slot    Serial #         Label
====    =========        ============
1      <serial_num1>     <partition_name>
2      <serial_num2>     <partition_name>

You should see an entry for each partition created on each CloudHSM device. This step lets you know that the CloudHSM devices and client instance were properly configured.

The partitions created and assigned via the previous steps are for testing purposes only and will not be used in the HA parition group setup. The HA partition group workflow will automatically create a partition on each CloudHSM device for its purposes. At this point, you have created the client and at least two CloudHSM devices. You also have set up and tested for validity the connection between the client instance and the CloudHSM devices. The next step in to ensure fault tolerance by setting up the HA partition group.

Set up the HA partition group for fault tolerance

Now that you have provisioned multiple CloudHSM devices in your account, you will add them to an HA partition group. As I explained earlier in this post, an HA partition group is a virtual partition that represents a group of partitions distributed over many physical CloudHSM devices for HA. Automatic recovery is also a key factor in ensuring HA and data integrity across your HA partition group members. If you followed the previous procedures in this post, setting up the HA partition group should be relatively straightforward.

Create the HA partition group

First, you will create the actual HA partition group itself. Using the CloudHSM CLI on your client instance, run the following command to create the HA partition group and name it per your company’s naming conventions. In the following command, replace <label> with the name you chose.

$ CloudHSM create-hapg –group-label <label>

Register the CloudHSM devices with the HA partition group

Now, add the already initialized CloudHSM devices to the HA partition group. You will need to run the following command for each CloudHSM device you want to add to the HA partition group.

$ CloudHSM add-CloudHSM-to-hapg \
--CloudHSM-arn <CloudHSM_arn> \
--hapg-arn <hapg_arn> \
--cloning-domain <cloning_domain> \
--partition-password <partition_password> \
--so-password <so_password>

You should see output similar to the following after each successful addition to the HA partition group.

{
      “Status”: “Addition of CloudHSM <CloudHSM_arn> to HA partition group <hapg_arn> successful”
}

Register the client with the HA partition group

The last step is to register the client with the HA partition group. You will need the client ARN from earlier in the post, and you will use the CloudHSM CLI command register-client-to-hapg to complete this process.

$ CloudHSM register-client-to-hapg \
--client-arn <client_arn> \
--hapg-arn <hapg_arn>
{
      “Status”: “Registration of the client <client_arn> to the HA partition group <hapg_arn> successful”
}

After you register the client with the HA partition group, you have the client configuration file and the server certificates. You have already registered the client with the HA partition group, but now you have to actually assign it as well, which you do by using the get-client-configuration AWS CLI command.

$ CloudHSM get-client-configuration \
--client-arn <client_arn> \
--hapg-arn <hapg_arn> \
--cert-directory <server_cert_location> \
--config-directory /etc/

The configuration file has been copied to /etc/
The server certificate has been copied to <server_cert_location>

The <server_cert_location> will differ depending on the LunaSA client software you are using:

  • Client software version 5.3: /usr/safenet/lunaclient/cert/server
  • Client software version 5.1: /usr/lunasa/cert/server

Lastly, to verify the client configuration, run the following LunaSA vtl command.

$ vtl haAdmin show

In the output, you will see a heading, HA Group and Member Information. Ensure that the number of group members equals the number of CloudHSM devices you added to the HA partition group. If the number does not match what you have provisioned, you might have missed a step in the provisioning process. Going back through the provisioning process usually repairs this. However, if you still encounter issues, opening a support case is the quickest way to get assistance.

Another way to verify the HA partition group setup is to check the /etc/Chrystoki.conf file for output similar to the following.

VirtualToken = {
   VirtualToken00Label = hapg1;
   VirtualToken00SN = 1529127380;
   VirtualToken00Members = 475256026,511541022;
}
HASynchronize = {
   hapg1 = 1;
}
HAConfiguration = {
   reconnAtt = -1;
   AutoReconnectInterval = 60;
   HAOnly = 1;

Summary

You have now completed the process of provisioning CloudHSM devices, the client instance for connection, and your HA partition group for fault tolerance. You can begin using an application of your choice to access the CloudHSM devices for key management and encryption. By accessing CloudHSM devices via the HA partition group, you ensure that all traffic is load balanced between all backing CloudHSM devices. The HA partition group will ensure that each CloudHSM has identical information so that it can respond to any request issued.

Now that you have an HA partition group set up with automatic recovery, if a CloudHSM device fails, the device will attempt to recover itself, and all traffic will be rerouted to the remaining CloudHSM devices in the HA partition group so as not to interrupt traffic. After recovery (manual or automatic), all data will be replicated across the CloudHSM devices in the HA partition group to ensure consistency.

If you have questions about any part of this blog post, please post them on the IAM forum.

– Tracy

How to Set Up Uninterrupted, Federated User Access to AWS Using AD FS

Post Syndicated from Tracy Pierce original https://blogs.aws.amazon.com/security/post/Tx1MHOLKFJESWBS/How-to-Set-Up-Uninterrupted-Federated-User-Access-to-AWS-Using-AD-FS

Microsoft Active Directory Federation Services (AD FS) is a common identity provider that many AWS customers use to give federated users access to the AWS Management Console. AD FS uses multiple certificates to ensure secure communication between servers and to act as authentication mechanisms. One such mechanism is called the token-signing certificate.

When the token-signing certificate expires, or is changed, the trust relationship between the claim provider, AD FS, and the relying party, AWS Security Token Service (AWS STS), is broken. Without a valid certificate to prove the calling server’s identity, the receiving party cannot verify the certificate, which terminates the request and thus prevents federated users from being able to access the AWS Management Console. Luckily, this can be avoided!

In this blog post, I explain how you can use the AutoCertificateRollover feature in AD FS to enable uninterrupted connections between your claim provider and your relying trust. I also show how to set up a secondary certificate manually in AD FS to avoid service interruption when a server certificate expires.

This post assumes that you have a working AD FS configuration. If you do not, you can follow the steps in the following blog posts to get you up and running:

Let’s start by taking a quick look at how AD FS uses the token-signing certificate.

Background

The token-signing certificate is used by AD FS to sign the Security Assertion Markup Language (SAML) assertion—also known as an AuthN response—that AD FS sends to a relying party to authenticate to Active Directory (AD) its information, such as Role, RoleSessionName, and X509 certificates. For this post’s use case, the relying party is AWS STS, which AD FS uses to provide federated users access to the AWS Management Console and AWS APIs. The following diagram illustrates the authentication process.

  1. The flow is initiated when a user—let’s call him Bob—browses to the AD FS sample site (https://Fully.Qualified.Domain.Name.Here/adfs/ls/IdpInitiatedSignOn.aspx) inside his domain. When Bob installed AD FS, he was given a new virtual directory named adfs for his default website, which includes this sample site.
  2. The sign-on page authenticates Bob against Active Directory. Depending on the browser Bob is using, he might be prompted for his Active Directory user name and password.
  3. Bob’s browser receives an AuthN response in the form of an authentication response from AD FS.
  4. Bob’s browser posts the AuthN response to the AWS sign-in endpoint for SAML (https://signin.aws.amazon.com/saml). Behind the scenes, sign-in uses the AssumeRoleWithSAML API to request temporary security credentials and then constructs a sign-in URL for the AWS Management Console.
  5. Bob’s browser receives the sign-in URL and is redirected to the AWS Management Console.

For a more complete description of the certificates that AD FS uses, see Understanding Certificates Used by AD FS.

Key AD FS settings and how to change them

AD FS has default settings to help ensure that your certificates never expire. You can see these default settings by opening a Windows PowerShell window with administrative rights on your AD FS server, and then running Get-ADFSProperties.

If you are using AD FS 2.0, you first need to import the AD FS cmdlets for Windows by running the following commands. If you are using AD FS 3.0, these cmdlets will already be installed for you.

Add-PSSnapin Microsoft.Adfs.PowerShell
Get-ADFSProperties

The output should resemble what the following screenshot shows.

The output shows you the settings for your AD FS properties, and it tells you whether or not AutoCertificateRollover is enabled, the host name of your server, timeouts, and so on. These settings are handy to know for AD FS server administration tasks, such as checking certificate settings, timeouts, domain, and containers. However, for the purposes of this post, the settings and values I will focus on in the output are the following:

AutoCertificateRollover                    : True
CertificateGenerationThreshold             : 20
CertificatePromotionThreshold              : 5
CertificateRolloverInterval                : 720

To explain these settings:

  • When set to True, AutoCertificateRollover enables the automatic generation of a new self-signed certificate to sign your AuthN response.
  •  There is also a default CertificateGenerationThreshold of 20 days that creates the secondary certificate 20 days before the original certificate expires. This certificate will not be used until the CertificatePromotionThreshold is reached.
  • The default setting for CertificatePromotionThreshold is 5 days. 5 days after the secondary certificate is created, it will automatically be promoted to primary. This gives your relying parties a 5-day window to update the federation metadata so that zero downtime is experienced. After this 5-day window, the primary certificate is set to secondary and will then expire on the original expiration date. You must download the metadata document again so that it contains the newly created certificate.

Because the relying parties count on having a valid certificate to verify identity, they need a copy of this new metadata document to validate requests. This allows you to minimize the concern of the certificate expiring and having downtime. The relying trusts still need to upload the new metadata document to ensure that a copy of the new certificate is readily available for authentication purposes. If the relying trust does not update the metadata document, downtime with connection to that service could be experienced after the original primary certificate expires.

  • AD FS checks the certificate status every CertificateRolloverInterval, which is in minutes. The default value is 720 minutes (12 hours).

You can change the aforementioned AD FS properties by using the Set-ADFSProperties cmdlet. For example, you can change the CertificatePromotionThreshold from 5 to 10 days by running the following command.

Set-ADFSProperties -CertificatePromotionThreshold 10

This change gives you more time to deal with the relying party update.

If certificate rotation is something you would like to handle manually because you want to have full control, you can disable the rollover feature by running the following command.

Set-ADFSProperties –AutoCertificateRollover:$false

To avoid service interruption, you must download the new metadata document and upload it to the relying parties that use it, before the primary certificate expires.

Next, you will change permissions on a certificate so that AD FS has access to it.

Change permissions on the private key of the certificate

A prerequisite for completing this section is that you acquire a new, unexpired certificate. You can do this by creating a self-signed certificate as explained in Enabling Federation to AWS Using Windows Active Directory, ADFS, and SAML 2.0, or you can purchase a Secure Sockets Layer (SSL) certificate from the third-party provider of your choice. Manual rotation is required if you purchase an SSL certificate from a trusted third party because AD FS does not have a rollover feature for these certificates.

After you have installed a new, unexpired certificate on your AD FS server, you must ensure that the AD FS service account has access to the private key for this new server certificate.

To change the permissions on the private key of the certificate:

  1. On your AD FS server, open the MMC Console. Click Start, type MMC, and then press Enter.
  2. In the MMC Console, click File and then click Snap-in Add/Remove. Then click Certificate, Local Computer, and then OK.
  3. Under Certificates (Local Computer), expand Personal, and then click Certificates (as shown in the following screenshot).

  1. Right-click the new certificate you just installed, and then click All Tasks – Manage Private Key.
  2. On the Permissions tab, select Add User and grant the AD FS service account full control. Do this by searching for the service account name you selected when setting up your AD FS server, clicking OK, and then selecting the check box for Full Control under Allow (as shown in the following screenshot). Then click Apply.


     

  3. If you are not sure about which account name to run as, you can retrieve this information by clicking Start and Run, and then typing services.msc. As shown in the following screenshot, check the Log On As column for Active Directory Federation Service. In my case, the account is ADFSSVC$. Windows adds a $ to the end of managed service accounts, such as accounts that are used to run services like AD FS. User accounts do not have such a suffix added to them.

Now that AD FS has access to the private key of the new certificate, you can open the AD FS console and configure the server to add this new certificate as a secondary certificate when signing the SAML AuthN response. This is to ensure that a secondary certificate is available for identity authentication when the primary certificate expires.

Manually add the new certificate as the secondary certificate

As explained previously in this post, the only certificate that AWS needs from AD FS is the token-signing certificate.

To add the new certificate manually for AD FS server authentication:

  1. Open the AD FS console, click the Service folder, and then click the Certificates folder, as shown in the following screenshot.

  1. Click Add Token-Signing Certificate and select the certificate you wish to use as the secondary certificate. After you upload it, this certificate is listed as Secondary.

Note: In order to complete this process, you must disable any self-signed, autorotating certificates you may have configured.

  1. Open Windows PowerShell as an Administrator and run the following commands. If you are using AD FS 3.0, you can skip the first command, which adds the PSSnapin that is already installed in AD FS 3.0.
Add-PSSnapin Microsoft.ADFS.PowerShell
Set-ADFSProperties –AutoCertificateRollover $false
  1. Download the new copy of the metadata document from the following link (placed in a browser on your AD FS server): https://<yourservername>/FederationMetadata/2007-06/FederationMetadata.xml.

You are still using the soon-to-expire certificate (the original primary certificate), but the new certificate is set as a secondary certificate. If you look at your FederationMetadata.xml file now, you will see that both certificates are included. The trust with your relying party is based on the information shared in the FederationMetadata.xml file, so the final step is to update your relying parties with this new metadata document.

To update your relying parties with the new metadata document:

  1. Sign in to the AWS Management Console.
  2. Click Services and then click IAM to go to the IAM console.
  3. Click Identity Providers in the left pane.
  4. Select the name of the identity provider (IdP) you created for your SAML SSO.
  5. Click Upload Metadata and select the file you downloaded from your AD FS server via the federation metadata link just provided in the previous section.
  6. Click Upload. The IdP now has an updated FederationMetadata.xml document to validate authentication requests from your claims provider

Updating the metadata document in AWS after creating and adding the secondary certificate ensures there is no downtime later when transitioning from the primary certificate to the secondary certificate.

Update the AWS configuration before the primary certificate expires

When you have the new federation metadata with the soon-to-expire (primary) certificate and the new (secondary) certificate that was either automatically generated by AD FS or one that you installed, you must update the relying party configuration before the primary certificate expires.

To update the AWS configuration:

  1. Sign in to the AWS Management Console as an IAM user that has access to update IdPs.
  2. In the IAM console in the Identity Providers section, select the IdP you want to update.
  3. Click Upload Metadata and then click Choose File. Navigate to the directory into which you downloaded the new FederationMetadata.xml file and choose the file. Click Upload.

Test it!

Now that you have uploaded the new metadata document with both certificates listed, you can test a sign-in to the AWS Management Console through your normal means of federation. This sign-in should complete without issue. However, if you experience any errors, you can check a few things:

  1. Is the primary certificate still listed in the AD FS console?
  2. Did the relying party upload the certificate correctly?
  3. Check the FederationMetadata.xml file to ensure all security information is still being passed as before (Role, RoleSessionName, X509 certificate, and so on).

If you are using autorollover, the process is complete. No further action is required on your part to ensure a valid certificate is used for identity validation between your claims provider and relying trust. If you manually set the certificate, to ensure zero downtime you must rotate the secondary certificate to become the primary certificate before it expires.

To rotate the secondary certificate to be the primary certificate:

  1. Open the AD FS console and click Certificates.
  2. Right-click the new certificate you uploaded, and then click Primary.

To help keep your setup “clean,” follow these steps to remove the expired certificates from your server:

  1. Open the AD FS console and click Certificates.
  2. Select the old certificate under Token-Signing Certificate, and then click Delete.

Going forward, server certificate expiration should not affect your ability to connect with AWS via your SAML setup.

If you have comments about this blog post, please add them to the “Comments” section below. If you have questions about this blog post, start a new thread on the IAM forum.

– Tracy

How to Set Up Uninterrupted, Federated User Access to AWS Using AD FS

Post Syndicated from Tracy Pierce original https://blogs.aws.amazon.com/security/post/Tx1MHOLKFJESWBS/How-to-Set-Up-Uninterrupted-Federated-User-Access-to-AWS-Using-AD-FS

Microsoft Active Directory Federation Services (AD FS) is a common identity provider that many AWS customers use to give federated users access to the AWS Management Console. AD FS uses multiple certificates to ensure secure communication between servers and to act as authentication mechanisms. One such mechanism is called the token-signing certificate.

When the token-signing certificate expires, or is changed, the trust relationship between the claim provider, AD FS, and the relying party, AWS Security Token Service (AWS STS), is broken. Without a valid certificate to prove the calling server’s identity, the receiving party cannot verify the certificate, which terminates the request and thus prevents federated users from being able to access the AWS Management Console. Luckily, this can be avoided!

In this blog post, I explain how you can use the AutoCertificateRollover feature in AD FS to enable uninterrupted connections between your claim provider and your relying trust. I also show how to set up a secondary certificate manually in AD FS to avoid service interruption when a server certificate expires.

This post assumes that you have a working AD FS configuration. If you do not, you can follow the steps in the following blog posts to get you up and running:

Enabling Federation to AWS Using Windows Active Directory, AD FS, and SAML 2.0

How to Set Up SSO to the AWS Management Console for Multiple Accounts by Using AD FS and SAML 2.0

Let’s start by taking a quick look at how AD FS uses the token-signing certificate.

Background

The token-signing certificate is used by AD FS to sign the Security Assertion Markup Language (SAML) assertion—also known as an AuthN response—that AD FS sends to a relying party to authenticate to Active Directory (AD) its information, such as Role, RoleSessionName, and X509 certificates. For this post’s use case, the relying party is AWS STS, which AD FS uses to provide federated users access to the AWS Management Console and AWS APIs. The following diagram illustrates the authentication process.

The flow is initiated when a user—let’s call him Bob—browses to the AD FS sample site (https://Fully.Qualified.Domain.Name.Here/adfs/ls/IdpInitiatedSignOn.aspx) inside his domain. When Bob installed AD FS, he was given a new virtual directory named adfs for his default website, which includes this sample site.

The sign-on page authenticates Bob against Active Directory. Depending on the browser Bob is using, he might be prompted for his Active Directory user name and password.

Bob’s browser receives an AuthN response in the form of an authentication response from AD FS.

Bob’s browser posts the AuthN response to the AWS sign-in endpoint for SAML (https://signin.aws.amazon.com/saml). Behind the scenes, sign-in uses the AssumeRoleWithSAML API to request temporary security credentials and then constructs a sign-in URL for the AWS Management Console.

Bob’s browser receives the sign-in URL and is redirected to the AWS Management Console.

For a more complete description of the certificates that AD FS uses, see Understanding Certificates Used by AD FS.

Key AD FS settings and how to change them

AD FS has default settings to help ensure that your certificates never expire. You can see these default settings by opening a Windows PowerShell window with administrative rights on your AD FS server, and then running Get-ADFSProperties.

If you are using AD FS 2.0, you first need to import the AD FS cmdlets for Windows by running the following commands. If you are using AD FS 3.0, these cmdlets will already be installed for you.

Add-PSSnapin Microsoft.Adfs.PowerShell
Get-ADFSProperties

The output should resemble what the following screenshot shows.

The output shows you the settings for your AD FS properties, and it tells you whether or not AutoCertificateRollover is enabled, the host name of your server, timeouts, and so on. These settings are handy to know for AD FS server administration tasks, such as checking certificate settings, timeouts, domain, and containers. However, for the purposes of this post, the settings and values I will focus on in the output are the following:

AutoCertificateRollover : True
CertificateGenerationThreshold : 20
CertificatePromotionThreshold : 5
CertificateRolloverInterval : 720

To explain these settings:

When set to True, AutoCertificateRollover enables the automatic generation of a new self-signed certificate to sign your AuthN response.

 There is also a default CertificateGenerationThreshold of 20 days that creates the secondary certificate 20 days before the original certificate expires. This certificate will not be used until the CertificatePromotionThreshold is reached.

The default setting for CertificatePromotionThreshold is 5 days. 5 days after the secondary certificate is created, it will automatically be promoted to primary. This gives your relying parties a 5-day window to update the federation metadata so that zero downtime is experienced. After this 5-day window, the primary certificate is set to secondary and will then expire on the original expiration date. You must download the metadata document again so that it contains the newly created certificate.

Because the relying parties count on having a valid certificate to verify identity, they need a copy of this new metadata document to validate requests. This allows you to minimize the concern of the certificate expiring and having downtime. The relying trusts still need to upload the new metadata document to ensure that a copy of the new certificate is readily available for authentication purposes. If the relying trust does not update the metadata document, downtime with connection to that service could be experienced after the original primary certificate expires.

AD FS checks the certificate status every CertificateRolloverInterval, which is in minutes. The default value is 720 minutes (12 hours).

You can change the aforementioned AD FS properties by using the Set-ADFSProperties cmdlet. For example, you can change the CertificatePromotionThreshold from 5 to 10 days by running the following command.

Set-ADFSProperties -CertificatePromotionThreshold 10

This change gives you more time to deal with the relying party update.

If certificate rotation is something you would like to handle manually because you want to have full control, you can disable the rollover feature by running the following command.

Set-ADFSProperties –AutoCertificateRollover:$false

To avoid service interruption, you must download the new metadata document and upload it to the relying parties that use it, before the primary certificate expires.

Next, you will change permissions on a certificate so that AD FS has access to it.

Change permissions on the private key of the certificate

A prerequisite for completing this section is that you acquire a new, unexpired certificate. You can do this by creating a self-signed certificate as explained in Enabling Federation to AWS Using Windows Active Directory, ADFS, and SAML 2.0, or you can purchase a Secure Sockets Layer (SSL) certificate from the third-party provider of your choice. Manual rotation is required if you purchase an SSL certificate from a trusted third party because AD FS does not have a rollover feature for these certificates.

After you have installed a new, unexpired certificate on your AD FS server, you must ensure that the AD FS service account has access to the private key for this new server certificate.

To change the permissions on the private key of the certificate:

On your AD FS server, open the MMC Console. Click Start, type MMC, and then press Enter.

In the MMC Console, click File and then click Snap-in Add/Remove. Then click Certificate, Local Computer, and then OK.

Under Certificates (Local Computer), expand Personal, and then click Certificates (as shown in the following screenshot).

Right-click the new certificate you just installed, and then click All Tasks – Manage Private Key.

On the Permissions tab, select Add User and grant the AD FS service account full control. Do this by searching for the service account name you selected when setting up your AD FS server, clicking OK, and then selecting the check box for Full Control under Allow (as shown in the following screenshot). Then click Apply.


 

If you are not sure about which account name to run as, you can retrieve this information by clicking Start and Run, and then typing services.msc. As shown in the following screenshot, check the Log On As column for Active Directory Federation Service. In my case, the account is ADFSSVC$. Windows adds a $ to the end of managed service accounts, such as accounts that are used to run services like AD FS. User accounts do not have such a suffix added to them.

Now that AD FS has access to the private key of the new certificate, you can open the AD FS console and configure the server to add this new certificate as a secondary certificate when signing the SAML AuthN response. This is to ensure that a secondary certificate is available for identity authentication when the primary certificate expires.

Manually add the new certificate as the secondary certificate

As explained previously in this post, the only certificate that AWS needs from AD FS is the token-signing certificate.

To add the new certificate manually for AD FS server authentication:

Open the AD FS console, click the Service folder, and then click the Certificates folder, as shown in the following screenshot.

Click Add Token-Signing Certificate and select the certificate you wish to use as the secondary certificate. After you upload it, this certificate is listed as Secondary.

Note: In order to complete this process, you must disable any self-signed, autorotating certificates you may have configured.

Open Windows PowerShell as an Administrator and run the following commands. If you are using AD FS 3.0, you can skip the first command, which adds the PSSnapin that is already installed in AD FS 3.0.

Add-PSSnapin Microsoft.ADFS.PowerShell
Set-ADFSProperties –AutoCertificateRollover $false

Download the new copy of the metadata document from the following link (placed in a browser on your AD FS server): https://<yourservername>/FederationMetadata/2007-06/FederationMetadata.xml.

You are still using the soon-to-expire certificate (the original primary certificate), but the new certificate is set as a secondary certificate. If you look at your FederationMetadata.xml file now, you will see that both certificates are included. The trust with your relying party is based on the information shared in the FederationMetadata.xml file, so the final step is to update your relying parties with this new metadata document.

To update your relying parties with the new metadata document:

Sign in to the AWS Management Console.

Click Services and then click IAM to go to the IAM console.

Click Identity Providers in the left pane.

Select the name of the identity provider (IdP) you created for your SAML SSO.

Click Upload Metadata and select the file you downloaded from your AD FS server via the federation metadata link just provided in the previous section.

Click Upload. The IdP now has an updated FederationMetadata.xml document to validate authentication requests from your claims provider

Updating the metadata document in AWS after creating and adding the secondary certificate ensures there is no downtime later when transitioning from the primary certificate to the secondary certificate.

Update the AWS configuration before the primary certificate expires

When you have the new federation metadata with the soon-to-expire (primary) certificate and the new (secondary) certificate that was either automatically generated by AD FS or one that you installed, you must update the relying party configuration before the primary certificate expires.

To update the AWS configuration:

Sign in to the AWS Management Console as an IAM user that has access to update IdPs.

In the IAM console in the Identity Providers section, select the IdP you want to update.

Click Upload Metadata and then click Choose File. Navigate to the directory into which you downloaded the new FederationMetadata.xml file and choose the file. Click Upload.

Test it!

Now that you have uploaded the new metadata document with both certificates listed, you can test a sign-in to the AWS Management Console through your normal means of federation. This sign-in should complete without issue. However, if you experience any errors, you can check a few things:

Is the primary certificate still listed in the AD FS console?

Did the relying party upload the certificate correctly?

Check the FederationMetadata.xml file to ensure all security information is still being passed as before (Role, RoleSessionName, X509 certificate, and so on).

If you are using autorollover, the process is complete. No further action is required on your part to ensure a valid certificate is used for identity validation between your claims provider and relying trust. If you manually set the certificate, to ensure zero downtime you must rotate the secondary certificate to become the primary certificate before it expires.

To rotate the secondary certificate to be the primary certificate:

Open the AD FS console and click Certificates.

Right-click the new certificate you uploaded, and then click Primary.

To help keep your setup “clean,” follow these steps to remove the expired certificates from your server:

Open the AD FS console and click Certificates.

Select the old certificate under Token-Signing Certificate, and then click Delete.

Going forward, server certificate expiration should not affect your ability to connect with AWS via your SAML setup.

If you have comments about this blog post, please add them to the “Comments” section below. If you have questions about this blog post, start a new thread on the IAM forum.

– Tracy