All posts by Jeff Levine

Using AWS CloudHSM-backed certificates with Microsoft Internet Information Server

Post Syndicated from Jeff Levine original

SSL/TLS certificates are used to create encrypted sessions to endpoints such as web servers. If you want to get an SSL certificate, you usually start by creating a private key and a corresponding certificate signing request (CSR). You then send the CSR to a certificate authority (CA) and receive a certificate. When a user seeks to securely browse your website, the web server presents the certificate to the user’s browser. The user’s browser verifies that the certificate is associated with a trusted root certificate, and then uses the certificate to encrypt the session data. The web server then uses the private key to decrypt the user’s data. The security of this protocol relies on the private key remaining private.

A common problem with web servers, however, is that the private key and the certificate reside on the web server itself. If an attacker compromises the server and steals both the private key and certificate, the attacker could potentially use them to intercept the session or create an imposter server. If your business requires the use of hardware security modules for key storage, you can use AWS CloudHSM to generate and hold the private keys for web server certificates rather than placing them on the web server directly and reduce your exposure. On Windows, AWS CloudHSM integrates with Microsoft Internet Information Server (IIS) through the Key Storage Provider extensions to the Microsoft Cryptography Next Generation API (KSP/CNG). In this blog post, I will show you how to leverage CloudHSM KSP/CNG capabilities to secure the private key within the hardware security module (HSM), if you are using Microsoft Internet Information Server (IIS) to host your website.

Prerequisites and assumptions

You need the following for this walkthrough.

  • An AWS account and permissions to provision Amazon Virtual Private Cloud (Amazon VPC), Amazon Elastic Compute Cloud (Amazon EC2) instances, Amazon Route 53, and Amazon CloudHSM. I recommend using an AWS user account ID that has full administrative privileges.
  • A region that supports CloudHSM. I’m using the eu-west-1 (Ireland) region.
  • A domain to use for certificate generation. I’m using the domain
  • A trusted public certificate authority (CA).
  • A working knowledge of the Amazon VPC, Amazon EC2, and Amazon CloudHSM services.
  • Familiarity with the administration of Windows Server 2012 R2

Important: You will incur charges for the services used in this example. You can find the cost of each service on that service’s pricing page. You can also use the Simple Monthly Calculator to estimate the cost.

Architectural overview

I’ll go over a few diagrams before I start building. I’ll start with a diagram of the infrastructure.

Figure 1: CloudHSM infrastructure

Figure 1: CloudHSM infrastructure

This diagram shows a VPC in the eu-west-1 (Ireland) region. An Amazon EC2 instance running Windows Server 2012 R2 resides on a public subnet. This instance will run both the CloudHSM client software and IIS to host a website. The instance has an Elastic IP and can be accessed via the Internet Gateway. I’ll also have security groups that enable RDP, HTTP, and HTTPS access. The private subnet hosts the Elastic Network Interface (ENI) for CloudHSM cluster that has a single HSM.

And here’s the certification hierarchy:

Figure 2: Certification hierarchy

Figure 2: Certification hierarchy

The Amazon EC2 instance will run Windows Server 2012 R2 (WS2012R2) and IIS. I’ll use the KSP/CNG capabilities of CloudHSM to request a CSR from the WS2012R2 instance. The KSP/CNG integration will reach out to CloudHSM that will, in turn, create a private/public key pair. The private key will remain within CloudHSM while the public key will be used to create the CSR. I’ll then submit the CSR to a CA. The CA will sign the CSR with the CA‘s private key and return a certificate. I’ll then accept the CSR with the certificate that will make it available to IIS. I’ll then secure the default IIS website with the certificate. Finally, I’ll browse to the website and show how it uses the CNG/KSP integration.

Now that you’ve seen what I’m building, we can get started!

Build the CloudHSM infrastructure

You’ll need to set up a CloudHSM environment. This environment will consist of a VPC, a CloudHSM cluster with a single HSM, and a Windows 2012 R2 EC2 instance that will run both the CloudHSM client and IIS. For each of the steps below, I have provided a link to the relevant documentation, as well as any additional instructions to build out the infrastructure to match my sample environment.

  1. Select a region that offers AWS CloudHSM. I’m using the eu-west-1 (Ireland) region.
  2. Create a Virtual Private Cloud (VPC). Use the following values:
    VPC IPv4 CIDR block:
    VPC Name: cloudhsm-vpc
    Public subnet IPv4 CIDR block:
    Public subnet Availability Zone: eu-west-1b
    Public subnet name: cloudhsm-public
  3. Create a private subnet. Use the following values:
    IPv4 CIDR block:
    Availability Zone: eu-west-1b
    Name: cloudhsm-private
  4. Create a cluster. Use the following values:
    HSM Availability Zone: eu-west-1b
    Subnet: Use the private subnet you just created in eu-west-1b.
    Number of HSMs: 1
  5. Launch an Amazon EC2 client instance. Use the following values:
    AMI: Microsoft Windows Server 2012 R2 Base
    Instance type: t2.medium
    VPC: cloudhsm-vpc
    Subnet: cloudhsm-public
    Auto-assign Public IP: Enable
    Name tag: cloudhsm
    Security group: The CloudHSM cluster security group

    Once the instance is running, create an additional administrative Windows user because the built-in Administrator account has restrictions.

  6. Create an additional security group to allow you to connect to the instance and attach the security group to the instance in addition to the security group of the CloudHSM cluster.
  7. Allocate and associate an Elastic IP address to the instance so that the instance’s IP address persists if you need to stop and start the instance.
  8. Create a DNS “A record” to map the host name to the Elastic IP. In this example, I’m mapping to the Elastic IP that I allocated.
  9. Create an HSM. After the HSM is created, note the HSM Id in the form of hsm- followed be an alphanumeric string; for example, hsm-1234abcd.
  10. Initialize the cluster.
  11. Install and configure the AWS CloudHSM client (Windows).
    You might need to use a command window with administrative privileges to do this.
  12. Activate the cluster.
  13. Create a crypto user. You will need to specify the ID and the password of the crypto user in the environment variables that you will create in the next section.

Define environment variables

We now need to define some Windows system environment variables (not user variables) used by the CNG/KSP integration. You can modify the environment variables by selecting System from the Control Panel, selecting Advanced System Settings, and selecting Environment Variables.

  1. Go into the Windows Control Panel and enter the following definitions:
    liquidsecurity_daemon_id 1
    liquidsecurity_daemon_socket_type TCPSOCKET
    liquidsecurity_daemon_tcp_port 1111
    n3fips_username USER
    n3fips_password USER:PASSWORD
    n3fips_partition hsm-ABCDEFGHIJK
    • Substitute the user ID and password that you specified in step 12 for the USER and PASSWORD values.
    • Substitute the HSM ID value you got from step 8 for hsm-ABCDEFGHIJK.

    Your Windows Control Panel configuration should look like the image below, not including the above substitutions.

    Figure 3: Environment variable settings

    Figure 3: Environment variable settings

  2. Reboot the EC2 instance to ensure that the environment variables are in place for all process.

Launch the CloudHSM client

You must start the CloudHSM client in order to enable the CNG/KSP integration.

  1. Open a command prompt window on the Windows instance.
  2. Enter the following commands:

    cd \ProgramData\Amazon\CloudHSM\data
    \Program Files\Amazon\CloudHSM\cloudhsm_client.exe cloudhsm_client.cfg

    The cloudhsm_client.exe command will continue to run. Important note: Do not close the window! The output at the bottom of the window looks similar to the following:

    Figure 4: CloudHSM client output

    Figure 4: CloudHSM client output

Launch key_mgmt_util

We are now going to launch the key_mgmt_util client and confirm that no keys are present in the CloudHSM cluster.

  1. Open a new command window. Do not use the window that you opened in the previous section.
  2. Enter the following commands.

    cd “\Program files\Amazon\CloudHSM”
    loginHSM –u CU –s USER –p PASSWORD

    (replace USER and PASSWORD with the crypto user and password that you created above)


You should see a message saying that there are no keys present, as shown in the figure below.

Figure 5: findKey output showing no keys present

Figure 5: findKey output showing no keys present

Important note: Do not exit from key_mgmt_util! You’ll return to this window later.

Install IIS

Here’s how:

  1. Follow the steps of this tutorial to install IIS. When installing IIS, make the following changes:
    • Use the Windows Server 2012 instructions as a guideline.
    • Do not install MySQL and PHP because you won’t need these services for this walkthrough.
  2. From your workstation’s web browser, browse the Elastic IP or the DNS “A Record” of the web server using the HTTP protocol. You should see the default Microsoft IIS page as shown below. If you don’t see the page, check the security groups for the server and verify that port 80 (HTTP) and port 443 (HTTPS) are open.
    Figure 6:

    Figure 6:

  3. If you browse to the same IP using the HTTPS protocol, you should not get a response because you haven’t configured HTTPS on the EC2 instance.

Provision a server certificate using the CNG/KSP integration

I’m now going to show how to provision a server certificate. I will do this with the certreq program that’s included with Windows Server 2012 R2. Certreq generates a CSR that I’ll submit to a CA. Certreq supports the KSP/CNG standard and can place certificates into the Windows certificate store.

  1. Create a file named request.inf that contains the lines below. Note that the Subject line may wrap onto the following line. It begins with Subject and ends with Washington and a closing quotation mark (“).

    Signature= $Windows NT$
    Subject = "C=US,,O=Information Technology,OU=Certificate Management,L=Seattle,S=Washington"
    HashAlgorithm = SHA256
    KeyAlgorithm = RSA
    KeyLength = 2048
    ProviderName = Cavium Key Storage Provider
    KeyUsage = 0xf0
    MachineKeySet = True
    SuppressDefaults = True
    SMIME = false
    RequestType = PKCS10
    [Extensions] = "{text}"
    _continue_ = ""
    _continue_ = ""

    Note the presence of the extensions section that allows you to specify SANs (Subject Alternative Names). SANs are now required by some browsers. Remember to substitute your own domain for in the above example.

  2. Create a certificate request with certreq.exe.
    certreq.exe -new request.inf request.csr.pem

    Certreq returns a message saying that the certificate request has been created.

  3. Figure 7: Output from certreq.exe showing that the CSR has been created

    Figure 7: Output from certreq.exe showing that the CSR has been created

  4. In the same command window where you ran key_mgmt_util, run findKey again to see if there are any new keys. Use the findKey command by itself to display all keys. This shows that two keys have been added with handles 7 and 8. Use the -c 2 parameter to display only public keys (handle 262214) and -c 3 to display private keys (handle 262154). The two keys are part of the public/private key pair that were just created with certreq.
    Figure 8: findKey output showing two new keys

    Figure 8: findKey output showing two new keys

  5. Submit the CSR you just created to the CA using the procedures provided by the CA. The CA will return a certificate that is named request.crt.pem (or similar).

    If the certificate authority doesn’t accept the CSR provided by certreq.exe, try the following:

    1. If the certificate authority asks you to provide the domains for the certificate, make sure you enter all of the SANs included in the request.inf file.
    2. Check the first and last lines of the CSR and see if the word “NEW” appears. If so, remove the word in both lines.
  6. Use the certreq command to accept the new certificate and insert it into the certificate store where IIS can access it.

    certreq.exe -accept request.crt.pem

Configure and test the secure website

  1. Start the IIS Manager. You can do this through the Server Manager or by running the following command:


  2. Under Connections, expand the local server, and then expand the Sites folder below. Select the site named Default Web Site.
    Figure 9: IIS Manager settings

    Figure 9: IIS Manager settings

  3. On the right, under Actions, select Edit Site-> Bindings, and then add a binding with the following fields making sure to substitute your domain name in place of

    Type: https
    Host name:
    SSL certificate:

  4. Select OK to save the new binding, and then select Close.
    Figure 10: Adding site bindings

    Figure 10: Adding site bindings

  5. Select Manage Website -> Restart. This will restart the website and ensure the new binding is active.
  6. Browse to, remembering to substitute your own domain. You’ll see the default website with a secure connection as shown below. If you get a warning message from your workstation’s browser, make sure you added the root-level certificate to the trusted certificate store. (Note: This should not be an issue if you’re using a commercial CA supported by major web browsers)
    Figure 11:

    Figure 11:

  7. Examine the certificate information being reported to the browser. For my example, I’m using the Chrome browser so I’ll select the padlock icon, then Certificates, then Detail.
    Figure 12: Server certificate details

    Figure 12: Server certificate details

    Figure 12 shows that the Subject field contains the same information that I provided before. You can check out the other fields for information about the Subject Alternative Names and other fields and verify that the correct certificate is being used.


For projects that have special requirements regarding the storage of keys, AWS CloudHSM supports the Microsoft KSP/CNG standard that enables you to store the private key of an SSL/TLS certificate within an HSM. Since the private key no longer has to reside on the server, it’s no longer at risk if the server itself were to be compromised.

If you have feedback about this blog post, submit comments in the “Comments” section below. If you have questions about this blog post, start a new thread on the Amazon CloudHSM forum or contact AWS Support.

Want more AWS Security news? Follow us on Twitter.

How to Monitor AWS Account Configuration Changes and API Calls to Amazon EC2 Security Groups

Post Syndicated from Jeff Levine original

You can use AWS security controls to detect and mitigate risks to your AWS resources. The purpose of each security control is defined by its control objective. For example, the control objective of an Amazon VPC security group is to permit only designated traffic to enter or leave a network interface. Let’s say you have an Internet-facing ecommerce website, and your security administrator has determined that only HTTP (TCP port 80) and HTTPS (TCP 443) traffic should be allowed access to the public subnet. As a result, your administrator configures a security group to meet this control objective.

What if, though, someone were to inadvertently change this security group’s rules and enable FTP or other protocols to access the public subnet from any location on the Internet? That expanded access could weaken the security posture of your assets. Consequently, your administrator might need to monitor the integrity of your company’s security controls so that the controls maintain their desired effectiveness.

In this blog post, I explore two methods for detecting unintended changes to VPC security groups. The two methods address not only control objectives but also control failures.

The two methods and when to use them

In this post, Method 1 uses AWS Config to monitor changes to a security group’s configuration as part of an organization’s overall compliance auditing program. Method 1 views a change to a VPC security group as a compliance risk. Use this method when you want to bolster your company’s compliance management.

Method 2 uses AWS CloudTrail and Amazon CloudWatch Events to identify AWS API calls that could change the configurations of VPC security groups. Method 2 views a change to a VPC security group as a potential security incident that should be identified in near real time. Use this method when you want to support your company’s monitoring of security operations.

Both of these methods can be effective parts of a defense-in-depth approach to control monitoring.

Method 1: Use AWS Config to check the configuration of a security group

The first method uses AWS Config, a fully managed service that provides you with an AWS resource inventory, configuration history, and configuration change notifications to enable security and governance. You can create AWS Config rules that automatically check the configuration of AWS resources that are recorded by AWS Config. For this example, I use a Config rule that is invoked whenever a change is made to a security group. Attach the Config rule to an AWS Lambda function that examines the ingress rules of a security group to see if the group remains in compliance with the rules.

The following Lambda function code defines a list named REQUIRED_PERMISSIONS with elements that represent a protocol, port range, and IP range that together define a security permission. This JSON notation is identical to what you would use when creating a security group with the AWS EC2 authorize-security-group-ingress command.

    "IpProtocol" : "tcp",
    "FromPort" : 80,
    "ToPort" : 80,
    "UserIdGroupPairs" : [],
    "IpRanges" : [{"CidrIp" : ""}],
    "PrefixListIds" : []
    "IpProtocol" : "tcp",
    "FromPort" : 443,
    "ToPort" : 443,
    "UserIdGroupPairs" : [],
    "IpRanges" : [{"CidrIp" : ""}],
    "PrefixListIds" : []

In this function, the first list entry allows TCP port 80 (HTTP) from anywhere on the internet ( The second list entry does the same for TCP port 443 (HTTPS).

The Lambda function then examines the configuration event that was passed to it as a result of the configuration change. The configuration event contains a list variable named ip_permissions that comes from the IpPermissions field of the Config event payload. The list represents the permissions of the security group after the configuration change is made. The function then examines the new permissions to see if they match the permissions listed in the preceding REQUIRED_PERMISSIONS list, as shown in the following code.

authorize_permissions =
    [item for item in REQUIRED_PERMISSIONS
        if item not in ip_permissions]
revoke_permissions =
    [item for item in ip_permissions
        if item not in REQUIRED_PERMISSIONS]

The authorize_permissions list contains the permissions that are in the REQUIRED_PERMISSIONS list that are not in the configuration event’s permission list. Thus, authorize_permissions represents the permissions that must be authorized (added) to the security group to bring it into alignment with the REQUIRED_PERMISSIONS list.

The revoke_permissions list contains the permissions that are in the configuration event’s permission list but not in the REQUIRED_PERMISSIONS list. Therefore, the revoke_permissions list represents the permissions that need to be revoked (removed) from the security group to bring it into alignment with the REQUIRED_PERMISSIONS list.

The function then performs the necessary authorizations and revocations using the authorize_security_group_ingress() and revoke_security_group_ingress() API calls. All of the results are logged by Amazon CloudWatch Logs.

To implement Method 1, follow these steps:

  1. Create a Lambda execution role and an AWS Config role.
  2. Enable AWS Config to record the configuration of security groups so that AWS Config can monitor security groups for changes to their configurations. The following screenshot shows an example of the settings.
    Screenshot showing an example of the configuration settings
  3. Create a security group that enables ingress TCP ports 80, 443, 465, and 993. You will use this security group only for the purposes of this blog post. The group should appear as the following screenshot shows.
    Screenshot of inbound rules of the security group
  4. Create the Python Lambda function using the code from the AWS Config rules library on GitHub.
  5. Create the AWS Config rule using the Lambda function you created in Step 4. For Trigger type, choose Configuration changes. For Scope of changes, choose EC2: SecurityGroup, and then type the ID of the security group you created in Step 3. The following screenshot shows these configuration settings.
    Screenshot of the trigger's configuration settings
  6. Run the Config rule. This will queue the rule for execution, and the rule should run to completion in about 10 minutes.
  7. Check the security group you created in Step 3. You should see that only ingress TCP ports 80 and 443 remain, as shown in the following screenshot.
    Screenshot showing that only ports 80 and 443 remain

The use of AWS Config in Method 1 allows for the configuration of a security group to be tracked along with other AWS resources. Changes to the security group’s configuration are reported during the next Config compliance evaluation, typically within 10 minutes. The notifications of changes to your security groups can be used to support your organization’s compliance management program.

Method 2: Use CloudTrail and CloudWatch Events to monitor API calls

In the first method, I approached the solution from the perspective of the object being changed—in this case, a security group. In the second method, I will focus on the “actors,” namely the API calls that may attempt to change the security group. By using CloudTrail with CloudWatch Events, you can invoke a Lambda function when specific API calls such as authorize_security_group_ingress() and revoke_security_group_ingress() are made. The API call event payload contains an IpPermission list that you can scan, as you did in Method 1.

When you create the Lambda function for CloudWatch Events, you can create an event selector, which functions as a filter for the Lambda function so that the function is only invoked for specific events. This approach has two advantages: it avoids unnecessary calls to Lambda, and you can simplify your Python code to handle only the desired events. Consider the following event selector.

  "detail-type": [
    "AWS API Call via CloudTrail"
  "detail": {
    "eventSource": [
    "eventName": [
    "requestParameters": {
      "groupId": [

This selector looks for CloudTrail API events (AWS API Call via CloudTrail) that involve the two API calls we previously discussed with the security group you wish to examine (in this case, sg-abc12345).

The function code for Method 2 differs from that of Method 1 in one important way: the Lambda function for Method 2 does not make any changes to the security group. The reason for this difference is that the code would need to use the same APIs to adjust the security group that triggered the Lambda function in the first place, potentially resulting in recursion.

To implement Method 2, follow these steps:

  1. Create a Lambda execution role and policy.
  2. Create the Python Lambda function using the code from AWS Labs.
  3. Create a security group with no ingress permissions (meaning no TCP or UDP ports). Use this security group only for the purposes of this blog post, and do not attach this group to a resource.
  4. Enable CloudTrail.
  5. Create a CloudWatch Events rule that is triggered by API calls. Use an event selector as described previously in this post as well as the Lambda function you created in Step 2.
  6. Add a rule to the security group you created in Step 3 of Method 2 to allow TCP port 445 inbound. TCP port 445 has no special significance; any TCP ports other than 80 and 443 would work.
    Screenshot showing the inbound rule that allows port 445
  7. After a few minutes, you should see a message in CloudWatch Logs telling you that TCP ports 80 and 443 must be authorized and that TCP port 445 must be revoked, as shown in the following screenshot.
    Screenshot of the CloudWatch Logs message saying which ports must be authorized or revoked

The use of CloudTrail and CloudWatch Events in Method 2 allows for the near real-time detection of API calls that could change the configuration of a VPC security group. The notifications of changes are posted to CloudWatch Logs, providing useful information to your organization’s security operations management program.


You can layer Config, CloudTrail, and CloudWatch Events on top of Amazon VPC security groups to provide a defense-in-depth approach to security. Though VPC security groups provide critical filtering capabilities, Config rules, CloudTrail, and CloudWatch Events take the protection to a deeper level by monitoring security groups and notifying you of potentially unintended changes.

Whether you use Method 1 or Method 2 depends on your goals. If you are focused on compliance management, Method 1 enables you to incorporate the configuration of security groups into your organization’s compliance management program. If your concern is more about incident detection, Method 2 offers a faster way to detect changes to a security group’s configuration. Both methods can help you add security to your AWS infrastructure.

If you have comments about this blog post, submit them in the “Comments” section below. If you have questions about implementing this post’s two methods, start a new thread on the VPC forum.

– Jeff