Tag Archives: AWS Key Management Service*

Caching KMS data keys in multi-thread environments: Per-tenant encryption for event-driven systems at scale

2026-08-06 Maria Gutovsky

Post Syndicated from Maria Gutovsky original https://aws.amazon.com/blogs/security/caching-kms-data-keys-in-multi-thread-environments-per-tenant-encryption-for-event-driven-systems-at-scale/

This post assumes familiarity with envelope encryption and the AWS Encryption SDK.

When your encryption system generates millions of duplicate API calls per hour, costs spiral and performance degrades. That’s exactly the challenge NICE Actimize faced while operating their global-scale, event-driven financial crime detection platform on Amazon Web Services (AWS).

NICE Actimize, a leading provider of financial crime, risk, and compliance solutions, processes millions of encrypted messages daily across hundreds of tenants. By rethinking how they cache encryption keys, they reduced their AWS Key Management Service (AWS KMS) costs by 77% while maintaining strict security guarantees and per-tenant encryption isolation.

In this post, we explore the cache stampede problem that emerges when envelope encryption meets high-concurrency, multi-tenant architectures. We walk through two solutions: the AWS-recommended hierarchical keyring pattern and a custom caching approach that NICE Actimize built for their regulated environment. These patterns apply to multi-tenant software as a service (SaaS) environments and high-throughput systems where per-tenant encryption generates significant KMS API volume.

Why per-tenant encryption matters

Financial services systems operate under strict regulatory requirements. You must encrypt data at rest and in transit. For multi-tenant SaaS providers, this requirement might go further: each tenant’s data must be encrypted with separate keys to provide complete cryptographic isolation. If one tenant’s key is compromised, no other tenant’s data is at risk.

Consider an enterprise SaaS environment built on an event-driven architecture using Amazon Managed Streaming for Apache Kafka (Amazon MSK), with many different databases for storing data and Amazon Simple Queue Service (Amazon SQS) for messaging. Messages flow continuously between producers and consumers, and each message must be encrypted with the correct tenant-specific key. At scale with millions of messages daily across hundreds of tenants, this creates a massive volume of encryption and decryption operations.

To handle this volume efficiently, the standard approach is envelope encryption: a two-tier model where an AWS KMS key encrypts short-lived data keys, and those data keys encrypt the actual data. Your application can encrypt large volumes of data locally without calling AWS KMS for every operation, reducing latency and costs.

The cache stampede problem

Envelope encryption reduces AWS KMS calls, but it doesn’t eliminate them. Each encrypt operation still requires a data key, either generated fresh using GenerateDataKey or retrieved from a cache, and each decrypt operation must unwrap an encrypted data key (EDK) by calling Decrypt. In high-throughput systems processing millions of messages, these calls add up quickly.

The AWS Encryption SDK provides a built-in solution for this: the CachingCryptoMaterialsManager. This component caches data encryption materials (data keys) locally, so your application can reuse them across multiple operations without calling AWS KMS each time. You configure a time-to-live (TTL), a maximum message-use limit, and a local cache, and the SDK handles the rest.

This approach works well under moderate load when you partition the cache by tenant AWS KMS key Amazon Resource Name (ARN) so that each tenant’s encryption materials remain cryptographically isolated. However, a critical problem emerges as concurrency scales to hundreds of threads processing millions of encrypted messages in parallel: the cache stampede, also known as the thundering herd problem.

How the stampede occurs

The CachingCryptoMaterialsManager caches the result of the SDK’s internal getMaterialsForEncrypt and decryptMaterials calls at the materials level. The cache stampede, however, happens at the KMS API call level. When a cached data key expires or a new, previously-unseen EDK arrives, the following sequence unfolds:

On encrypt – data key explosion: Multiple threads simultaneously call encrypt() for the same tenant. Each thread finds the cache entry expired and independently calls GenerateDataKey against AWS KMS. Instead of one thread generating a data key while others wait, N threads create N distinct data keys. Each new data key produces a unique EDK, which inflates the EDK cardinality across the system.
On decrypt – redundant unwrap calls: Those extra unique EDKs propagate downstream. When consumers later read encrypted records, each distinct EDK is a separate cache key. Multiple threads encountering the same EDK simultaneously each trigger an independent Decrypt call to AWS KMS because the cache has no coordination mechanism to make competing threads wait for a single in-flight request.
Compounding effect: The encrypt-side stampede creates excess EDK cardinality, which degrades the decrypt-side cache hit ratio, which triggers more KMS calls, which drives up costs further. In the NICE Actimize case, this produced a ratio of 30% unique data keys to data records in DynamoDB tables, meaning nearly one in three records was encrypted with a different data key.

At enterprise SaaS scale, this compounding effect can generate millions of redundant AWS KMS GenerateDataKey and Decrypt calls per hour, even with the SDK’s built-in caching enabled. The following figure shows the pattern leading to a stampede.

Figure 1: Cache stampede – multiple threads independently calling AWS KMS for the same encrypted data key, creating duplicate requests

The stampede follows this sequence on the encrypt side:

Multiple threads call encrypt() for the same tenant concurrently.
Each thread checks the CachingCryptoMaterialsManager and finds the cache entry expired.
With no coordination mechanism, each thread independently calls GenerateDataKey.
AWS KMS returns N distinct data keys (one per thread).
Each data key produces a unique EDK, inflating cardinality across the system.

On the decrypt side, the inflated EDK cardinality compounds the problem:

Consumer threads encounter unique EDKs that were never cached.
Multiple threads hitting the same EDK simultaneously each trigger a separate Decrypt call. AWS KMS returns the same plaintext data key N times, doing redundant work.

Two paths forward

We evaluated two approaches to solve the cache stampede problem. Each fits different architectural requirements and regulatory constraints.

Option A: Hierarchical keyring with DynamoDB (AWS-recommended)

AWS addresses the cache stampede challenge through the hierarchical keyring pattern, which introduces an additional level of key hierarchy that significantly reduces how often cache stampedes occur.

In this architecture, branch keys serve as intermediate wrapping keys stored in a DynamoDB table. This DynamoDB table acts as a shared cache layer that coordinates across all instances in your distributed fleet.

Figure 2: Hierarchical keyring architecture – branch keys in DynamoDB coordinating across distributed instances

The architecture (shown in Figure 2) works as follows:

The application requests encryption through the hierarchical keyring.
The keyring checks the local cache for the tenant’s branch key.
On a cache miss, it queries the DynamoDB Key Store table for the active branch key.
AWS KMS decrypts the branch key (this is the only KMS call in the flow).
The decrypted branch key is returned to the keyring.
The keyring stores the branch key in the local cache for subsequent requests.
The keyring derives a unique wrapping key from the branch key and generates the data key locally.

The key insight is that the cache is thread-aware. When the cache expires, threads coordinate to make a single request to refresh the cache. Only a single thread is used to make a call to the branch key, rather than all the threads acting independently. Additionally, by adding an additional key into the key hierarchy, branch keys don’t live within AWS KMS. This means cache misses and the stampedes they trigger interact with the branch key, and don’t make as many calls to the AWS KMS service at the top of the hierarchy:

Without hierarchical keyrings: Your local cache needs to store all the data encryption keys, and has constant misses as new, unique data keys arrive with each encrypted message. A miss can trigger a stampede.
With hierarchical keyrings: The same branch key wraps thousands or millions of data keys. A cache miss only occurs when a branch key expires or is first requested, which happens orders of magnitude less frequently than without hierarchical keyrings.

The DynamoDB table acts as a coordination point. The first thread to request a missing branch key retrieves it from AWS KMS and stores it in DynamoDB (the Key Store table). Subsequent requests from instances in the fleet retrieve the cached branch key from DynamoDB instead of making duplicate AWS KMS calls.

Beyond reducing cache miss frequency, the hierarchical keyring provides built-in stampede protection within its local cache implementation. The SDK offers multiple cache types, and the Default cache, designed for heavily multi-threaded environments, prevents multiple threads from calling AWS KMS on cache expiry by notifying a single thread that the branch key materials entry is about to expire 10 seconds in advance. That one thread refreshes the cache while all other threads continue serving requests using the still-valid entry.

This solution integrates with the AWS Encryption SDK and requires minimal code changes to existing applications. For event-driven architectures processing encrypted Kafka streams, this approach reduces KMS call volume by orders of magnitude while preserving per-tenant cryptographic isolation.

Option B: Custom KMS client caching – Solving the stampede at the API layer

While the hierarchical keyring (Option A) addresses the stampede by reducing how often cache misses occur, there’s a complementary approach: eliminating the stampede at its source by caching KMS API responses directly, using atomic, single-flight cache loading that prevents concurrent threads from issuing duplicate calls. This is the path NICE Actimize took.

The IClientSupplier extension point in AWS Encryption SDK v3

In the AWS Encryption SDK v2, decorating the AWS KMS client on a per-request basis was possible through the RegionalClientSupplier interface, but it was an advanced and undocumented use case. Without explicit guidance or a supported pattern, caching strategies typically operated above the SDK layer, making it difficult to prevent duplicate KMS calls at their source. The AWS Encryption SDK v3 introduced the IClientSupplier interface, which the AwsKmsMrkMultiKeyring accepts at construction time. This interface is called by the SDK whenever it needs a KMS client for a given AWS Region, and you control what it returns, making it possible to insert a caching layer between the SDK and AWS KMS.

Architecture: A decorated KMS client with two Caffeine caches
The solution is a CachedKmsClient—a decorator that wraps the standard AWS SDK KmsClient and interposes two Caffeine LoadingCache instances between the application and AWS KMS:

Cache	Key	Value	Purpose
GenerateDataKey cache	GenerateDataKeyRequest (tenant KMS key ARN and key spec)	GenerateDataKeyResponse (EDK and plaintext data key)	Ensures encrypt operations on the same node reuse the same data key for a given tenant KMS key during the cache window
Decrypt cache	DecryptRequest (EDK and key ARN)	DecryptResponse (plaintext data key)	Ensures decrypt operations for the same EDK share a single KMS call result

Both caches are configured with refreshAfterWrite (default: 1 hour, configurable), which means:

During the refresh window, concurrent threads receive the cached response instantly resulting in zero KMS calls.
When a cache entry expires, Caffeine’s LoadingCache.get() guarantees that exactly one thread executes the loader function (the actual KMS API call), while all other concurrent threads block and wait for that single result. This is the atomic, single-flight property that eliminates the stampede.

Security consideration: Caching plaintext data keys in memory means the keys exist in process memory for the duration of the cache TTL. The TTL acts as a security control: shorter TTLs reduce the window of exposure in the event of a memory dump, while longer TTLs reduce KMS call volume. Choose a TTL that balances your security requirements with your cost and performance goals. Key rotation at the KMS key level remains unaffected by the cache, because rotated keys produce new data keys on the next cache refresh.

Integration with the AWS Encryption SDK v3

The integration is minimal. The IClientSupplier AWS Lambda function returns a CachedKmsClient singleton for each AWS Region, this singleton is passed into the AwsKmsMrkMultiKeyring at keyring construction time. From that point forward, each GenerateDataKey and Decrypt call the SDK makes flows through the caching decorator transparently, with no changes to the encrypt or decrypt call sites.

The CachedKmsClient is a singleton per Region (managed using a ConcurrentHashMap), so all tenants on the same node share the same caching layer but their data keys remain fully isolated because the cache keys include the tenant-specific AWS KMS key ARN.

Why Caffeine?

Caffeine is a high-performance, near-optimal Java caching library well-suited for this pattern for several reasons:

Atomic loading: LoadingCache.get() guarantees that on a cache miss, only one thread executes the loader while others wait. This is the core property that eliminates the stampede.
refreshAfterWrite semantics: Unlike expireAfterWrite (which blocks all threads during refresh), refreshAfterWrite allows one thread to asynchronously reload the entry while other threads continue to serve the stale-but-valid cached value. This eliminates latency spikes during key rotation.
Observability: Cache eviction listeners and Micrometer metric counters can be wired in to track actual KMS call volume per tenant KMS key, enabling real-time cost monitoring.

Choosing between the two options

The hierarchical keyring with DynamoDB (Option A) is a production-ready, AWS-recommended solution that reduces stampede frequency by introducing longer-lived branch keys. It’s the best choice for most organizations. Particularly when starting fresh or when the operational overhead of an additional data store is acceptable.

NICE Actimize chose the custom caching approach (Option B) for a pragmatic reason: it avoided introducing a new infrastructure dependency into the encryption critical path. Their platform already operated at scale across hundreds of tenants, and adding a DynamoDB table as a key coordination layer would have meant taking on additional operational responsibility: provisioning, monitoring, backup, access control, and ensuring high availability for a component that sits directly in the encrypt/decrypt hot path. In a regulated financial services environment, each new stateful component in the security chain requires its own resilience planning, failure-mode analysis, and compliance review. The Caffeine cache used in Option B, by contrast, is an in-process library (a JAR on the classpath). It is stateless, requires no network calls, no provisioning and no operational overhead. It makes a lighter dependency than a managed cloud resource in the critical path. There is no shared state to lose, no additional infrastructure to protect, and no new failure mode beyond what already exists with AWS KMS itself. If a node restarts, the cache rebuilds on the next KMS call.

Results

By implementing a rotation policy with the optimized caching approach, NICE Actimize achieved the following results:

77% reduction in AWS KMS costs – Eliminating millions of redundant API calls translated directly into significant cost savings.
Maintained strict per-tenant isolation – Per-tenant encryption isolation remained fully intact, with no compromise to their security posture.
Improved system performance – Removing the stampede of duplicate AWS KMS calls reduced latency and freed up system resources for core processing.
Simplified operations – A coordinated caching layer replaced fragmented, per-thread caching, reducing operational complexity.

Conclusion and next steps

The cache stampede problem compounds in multi-tenant encryption systems: excess data key generation on the encrypt side degrades cache hit ratios on the decrypt side, creating a feedback loop of redundant KMS calls. The AWS-recommended hierarchical keyring pattern with DynamoDB provides a production-ready solution that integrates with the AWS Encryption SDK with minimal code changes. For regulated environments requiring additional control, a custom caching approach can deliver similar results.

If you operate a multi-tenant SaaS platform or a high-throughput system with per-tenant encryption requirements, consider these patterns to optimize your encryption costs and performance.

To get started, explore the following resources:

If you have questions or feedback about this post, leave a comment in the Comments section.

AWS KMS or AWS CloudHSM: Choose the right key management solution

2026-07-28 Derek Tumulak

Post Syndicated from Derek Tumulak original https://aws.amazon.com/blogs/security/aws-kms-or-aws-cloudhsm-choose-the-right-key-management-solution/

Choosing the right cryptographic key management service on Amazon Web Services (AWS) starts with understanding the difference between AWS Key Management Service (AWS KMS) and AWS CloudHSM. Both provide key storage backed by a hardware security module (HSM) but serve very different needs. AWS KMS is a fully managed service that integrates with all AWS services and all AWS Regions, making it the right choice for most key management workloads. AWS CloudHSM is a specialized option for use cases where you have strict requirements for dedicated HSM instances or must support legacy applications built around traditional HSM interfaces.

Quick comparison

The following table shows the pricing, AWS Region availability, algorithms, and AWS service integrations as of July 2026.

Criteria	AWS KMS	AWS CloudHSM
Best for	Most cloud-based key management needs	Lift-and-shift from on-premises applications and use of legacy algorithms
Deployment	AWS managed HSMs, accessed through API endpoints	Customer managed HSMs, accessed through an Elastic Network Interface (ENI) in your virtual private cloud (VPC)
Cost	Pay per use (symmetric and RSA 2048 operations): $1 per key plus $0.03 per 10,000 requests per month	Pay by the hour (us-east-1): $1.60 per HSM instance per hour
AWS integration	All AWS services	Custom integration with AWS services
Region coverage	All AWS Regions	32 Regions

Quick decision guide

Choose AWS KMS for most use cases. Choose AWS CloudHSM only if you require:

Direct integration with third-party tools such as Microsoft SignTool, Nginx, and HAProxy that rely on traditional HSM interfaces, including: PKCS#11, Java Cryptographic Extension (JCE), OpenSSL Provider, and Key Storage Provider (KSP). These interfaces are required when your application is built to communicate with an HSM directly rather than through a cloud API.
Deprecated algorithms such as 3DES and PKCS#1 v1.5 with RSA. If you need to run less commonly used operations not supported by AWS KMS such as AES key wrapping and AES with CTR or CBC modes.

Shared benefits

AWS KMS and AWS CloudHSM both provide robust encryption key management capabilities that help organizations meet their security and compliance requirements. While each service offers distinct features tailored to different use cases, they share several core benefits that make them valuable tools for protecting sensitive data in the cloud.

Security

AWS KMS and AWS CloudHSM both provide tamper-resistant, HSM-based key management with physical data center controls. They secure administration and workloads with Transport Layer Security (TLS). Neither service allows AWS employees to access your key material. Both services deliver equivalent security through Federal Information Processing Standard (FIPS) 140-3 Level 3 validated hardware and enforce strict cryptographic isolation of customer keys. Compliance frameworks such as the ones listed below validate security based on cryptographic boundaries rather than hardware or partition dedication. The multi-tenant architecture of AWS KMS provides the same security guarantees as the single-tenant model used by AWS CloudHSM while reducing operational complexity and cost. Customer security teams consistently approve AWS KMS adoption after confirming that cryptographic isolation meets their single-tenant security and compliance requirements.

Regulatory compliance

AWS KMS and AWS CloudHSM meet major compliance certifications, including:

Federal Information Processing Standard (FIPS) 140-3 Level 3
Payment Card Industry Data Security Standard (PCI-DSS)
Health Insurance Portability and Accountability Act (HIPAA)
Federal Risk and Authorization Management Program (FedRAMP)

Both services protect data including personally identifiable information (PII) and Protected Health Information (PHI).

Standard algorithms

AWS KMS and AWS CloudHSM support standard cryptographic operations including AES-256, RSA, ECDSA, Ed25519, ECDH, ML-DSA, SHA-2, and HMAC. Both services are actively investing in post-quantum cryptography (PQC) to help customers prepare for future quantum computing threats and are committed to expanding PQC algorithm support as National Institute of Standards and Technology (NIST) standards are finalized.

Performance

AWS KMS supports a default request rate for cryptographic operations ranging from 10,000 transactions per second (TPS) to 100,000 TPS per account based on Region. You can request quota increases beyond the default limits. AWS CloudHSM requires explicit provisioning of additional instances for higher throughput. Customers typically provision at least one additional HSM instance to handle peak activity, which can be difficult to predict due to lack of utilization metrics.

Operational support

AWS KMS and AWS CloudHSM both support high availability, durability, automatic backup, and software patching. AWS KMS is a Regional service with high availability and durability provided without any customer management required. AWS CloudHSM is a zonal service with customers required to manage high availability and durability.

Given these shared capabilities, the choice of which service to use depends on your specific requirements. The following sections outline decision points to help you choose.

When to choose AWS KMS

AWS KMS offers a fully managed service that simplifies key management operations and reduces operational overhead compared to AWS CloudHSM. Organizations choose AWS KMS when they need seamless integration with AWS services, automatic key rotation, and a cost-effective solution that doesn’t require dedicated HSM management.

AWS integration

AWS KMS integrates with all AWS services across all major categories. These include AI platforms, storage, databases, and compute services. Most of these services support AWS KMS customer managed keys, giving you full control over the key using policies and access controls. For customers that value convenience over control, AWS services provide transparent encryption using AWS owned keys, eliminating the cost and lifecycle management overhead of customer-owned keys. Both customer managed and AWS owned keys are AWS KMS keys. AWS Identity and Access Management (IAM) enables least-privilege access controls, key policies to control access, and auditing all key usage through AWS CloudTrail.

Operational simplicity

AWS KMS handles all operational tasks including HSM instance provisioning and maintenance, automatic key rotation, auto-scaling, disaster recovery, and comprehensive audit logging. This eliminates the operational overhead required to maintain a solution based on AWS CloudHSM.

Cost considerations

AWS KMS costs $1 per month per key plus $0.03 per 10,000 requests (symmetric and RSA 2048 operations). AWS CloudHSM costs approximately $1.60 per hour per HSM (approximately $1,152 per month), excluding the operational overhead for staff to manage the cluster—which further favors AWS KMS for most workloads.

Break-even analysis:

Less than 500 million operations per month: AWS KMS typically costs 35–99% less
500 million–1 billion operations per month: Costs are comparable
More than 1 billion operations per month: AWS CloudHSM might be more cost-effective.

Note: Many AWS services cache Data Encryption Keys (DEKs) locally, significantly reducing the number of AWS KMS API calls. Actual AWS KMS costs at scale are often much lower than raw operation counts suggest. For example: A workload with 100 keys and 100 million monthly operations using two HSMs for high availability:

AWS CloudHSM: Approximately $2,304 per month for two HSMs plus operational costs

AWS KMS: $100 per month (keys) plus $300 per month in operational costs for a total of $400 per month

Savings using AWS KMS: $1904 per month (83% reduction)

Region coverage

AWS KMS operates in every AWS Region, including all commercial Regions, GovCloud, China Regions, and the European Sovereign Cloud Region. AWS CloudHSM operates in 34 Regions, and AWS evaluates each new Region individually for AWS CloudHSM support.

When to choose AWS CloudHSM

AWS CloudHSM provides HSM-specific interfaces and support for legacy cryptographic algorithms that aren’t available from AWS KMS.

Lift-and-shift on-premises workloads

AWS CloudHSM supports traditional HSM interfaces such as PKCS#11, JCE, OpenSSL, and KSP, simplifying migration to AWS with minimal application changes. AWS is actively expanding AWS KMS integration options for these workloads. Contact AWS Support to discuss current alternatives.

Legacy cryptographic algorithms

AWS CloudHSM supports deprecated algorithms such as 3DES and PKCS#1 v1.5 padding with RSA. It also supports less commonly used operations such as AES key wrapping and AES with CTR or CBC modes.

Conclusion

For most organizations, AWS KMS delivers enterprise-grade security with lower costs and zero operational overhead. Choose AWS CloudHSM only if you have specific requirements for traditional HSM interfaces or less commonly used algorithms and can justify the additional cost and operational complexity.

Ready to get started? Use these guides to implement your chosen solution:

If you have feedback about this post, submit comments in the Comments section below.

Deploy modern data platforms in minutes with MDAA

2026-07-02 Sudeshna Dash

Post Syndicated from Sudeshna Dash original https://aws.amazon.com/blogs/big-data/deploy-modern-data-platforms-in-minutes-with-mdaa/

Modern Data Architecture Accelerator (MDAA) is an open source framework that replaces infrastructure code with concise YAML configuration, so your team can deploy a governed, production-ready data architecture, reducing deployment time from months to weeks (depending on complexity and team experience).

Organizations building modern data architecture on AWS face a critical challenge: deploying production-ready, governed infrastructure traditionally requires 6–12 months of custom development, thousands of lines of infrastructure code, and continuous remediation cycles to maintain security and compliance. Governance is often added incrementally, treated as an afterthought that creates compliance gaps and engineering rework.

MDAA addresses this by replacing infrastructure code with concise YAML configuration, achieving up to 97.6 percent code reduction (from approximately 1,800 lines of AWS CloudFormation to 45 lines of MDAA YAML) while embedding governance from the start. The complete Governed Lakehouse Starter Kit deploys 491 AWS resources across 12 stacks from approximately 450 lines of YAML configuration, representing a 66x verbosity ratio where each line automatically expands into production-ready infrastructure.

In this post, we explore how MDAA transforms data architecture development from months of manual coding to production-ready deployment through configuration-driven infrastructure and embedded governance, examine a real customer transformation, and provide a clear implementation pathway for your own data modernization journey.

Customer use case and challenge

A university system office needed to modernize its analytics architecture across 17 campuses while managing sensitive educational data. Their third-party dependency created bottlenecks that slowed feature implementation from weeks to months, and their IT team lacked the cloud skillsets to build modern infrastructure independently.

With MDAA, they achieved:

95 percent reduction in time-to-value for dashboard and feature implementation (from weeks to hours).
17 campuses integrated into a unified, secure architecture.
7.2TB of data and over 8,000 dashboards migrated successfully.
Significant cost savings by removing third-party dependencies and reducing license costs.
Enhanced security posture for external stakeholders accessing sensitive educational data.

The team used MDAA to implement a modernization strategy with continuous integration and continuous delivery (CI/CD) for automated deployment. The architecture now supports rapid response to stakeholder requests while maintaining strict data governance through AWS Lake Formation.

Their transformation demonstrates what becomes possible when governance is embedded from launch rather than added incrementally, moving from months-long manual development to weeks of production-ready deployment through configuration-driven infrastructure.

Solution: MDAA and its value propositions

MDAA’s capabilities stem from its modular, composable architecture. The accelerator provides over 40 pre-built modules that encapsulate AWS best practices for security, governance, and operational excellence. Organizations describe the outcomes they want in MDAA-specific YAML configuration files (not CloudFormation or Terraform YAML) and the accelerator automatically translates these configurations into AWS Cloud Development Kit (AWS CDK) constructs, which then deploy via CloudFormation with embedded governance.

Configuration over code. The MDAA framework takes a fundamentally different approach: describe the outcomes you want in YAML, and the accelerator deploys production-ready infrastructure with embedded governance. Consider deploying a governed data lake where fraud detection teams need write access to transaction data, while marketing analytics teams require read-only access to customer behavior data. Traditional approaches require over 1,800 lines of CloudFormation across Amazon Simple Storage Service (Amazon S3) buckets, AWS Key Management Service (AWS KMS) keys, AWS Identity and Access Management (IAM) policies, and Lake Formation permissions. With MDAA, the same governed data lake is expressed in 45 lines of configuration, a 97.6 percent reduction, while helping you apply encryption, least-privilege access, and cross-account governance as built-in defaults.

The configuration deploys multi-zone S3 storage with KMS encryption, Lake Formation permissions with tag-based access control (TBAC) enabled, Amazon SageMaker Unified Studio for data product discovery, and encrypted AWS Glue Data Catalog with automated crawlers. All permissions flow through Lake Formation rather than individual IAM policies.

Embedded governance from day one. Governance is declared in YAML and deployed alongside infrastructure from the first run. Fine-grained access controls, encrypted data catalogs, data quality validation, audit trails, and sensitive data classification are all part of the same configuration. MDAA’s Governed Lakehouse starter kit defines an entire governed data architecture in roughly 450 lines of YAML, which produces approximately 29,700 lines of CloudFormation across 12 stacks (a 98.5 percent reduction in infrastructure code).

Modular, composable architecture. Each module is purpose-built to handle a specific capability within the data architecture. Modules communicate through AWS Systems Manager Parameter Store, passing resource identifiers (Amazon Resource Names (ARNs), IDs, and names) between stacks. This approach removes hardcoded dependencies. A KMS key created in one module can be referenced by another through parameter resolution, with all dependencies resolved automatically at deployment time.

The diagram illustrates the deployed architecture and team-level access flow that MDAA generates from the 45-line configuration.

Progressive architecture patterns. MDAA provides four reference architecture patterns that align to progressive stages of data infrastructure maturity:

Basic Data Lake deploys a governed data lake with built-in security controls, data quality checks, centralized metadata management using AWS Lake Formation and AWS Glue.
Data Science Platform extends the data lake with Amazon SageMaker notebooks, feature stores, and machine learning (ML) pipelines so data science teams can experiment and train models on governed data.
SageMaker Unified Studio adds a single interface for analytics and ML collaboration, connecting data engineers, analysts, and data scientists in one workspace.
Generative AI Platform layers Amazon Bedrock and Retrieval Augmented Generation (RAG) capabilities on top of your existing data foundation, so teams can build generative AI applications grounded in enterprise data.

Each pattern builds the one before it. You can start with the Basic Data Lake and adopt additional patterns as your team’s needs grow. MDAA’s modular design means you add capabilities without rearchitecting what you already deployed.

The infrastructure is versioned through GitHub, repeatable across environments, and auditable through comprehensive AWS CloudTrail logging. Data engineers focus on data pipelines and business logic while MDAA manages infrastructure complexity and governance integration. This represents the fundamental shift: from writing infrastructure code to describing the outcomes you want through configuration, with governance embedded from the start.

Use case of MDAA: Governed data architecture

DataOps teams spend significant time on governance tasks, including permissions management, compliance validation, and access control, rather than building pipelines and analytics. These aren’t data problems, they’re governance problems that consume engineering capacity meant for higher-value work. MDAA addresses this at the architectural level. Governance is declared in YAML and deployed alongside infrastructure from the first run.

The following sections walk through how each governance module works in practice.

Amazon SageMaker Unified Studio provides a governed data catalog where data producers publish data products, and consumers discover and subscribe to them. Your deployment with MDAA includes a pre-configured domain, blueprints (managed and custom), projects, and environment profiles, all defined in a single configuration file:

# sagemaker.yaml --- 16 lines that deploy 114 CloudFormation resources
domains:
  domain1:
    dataAdminRole:
      id: ssm:/{{org}}/govern1/generated-role/data-admin/id
    description: SMUS Domain 1
    userAssignment: MANUAL

    tooling:
      vpcId: '{{context:vpc_id}}'
      subnetIds:
        - '{{context:private_subnet_id1}}'
        - '{{context:private_subnet_id2}}'

    groups:
      team1:
        ssoId: '{{context:team1-group-sso-id}}'
      team2:
        ssoId: '{{context:team2-group-sso-id}}'

Behind this configuration, MDAA deploys an Amazon SageMaker Unified Studio domain with dedicated KMS keys, execution and provisioning roles, and single sign-on group profiles for team access. Data producers tag and publish assets with metadata, ownership, and classification. Consumers browse a searchable catalog, see only authorized assets, and request access through a governed workflow. Cross-account and cross-business-unit data sharing flows through a subscription model, ensuring every access grant is tracked, auditable, and revocable.

Use case of MDAA: Restricting access to cardholder data using Lake Formation

AWS Lake Formation provides fine-grained access control at database and table levels, removing manual IAM policy management. MDAA deploys AWS Lake Formation with pre-configured settings that disable IAMAllowedPrincipals, the critical governance setting that ensures all permissions flow through centralized governance:

# lakeformation-settings.yaml --- 6 lines that deploy 25 CloudFormation resources
lakeFormationAdminRoles:
  - id: generated-role-id:data-admin
createCdkLFAdmin: true
createDataZoneAdminRole: true
iamAllowedPrincipalsDefault: false

That last flag is the single most important governance setting in the platform. Without it, an IAM principal with glue:GetTable can read tables in the catalog, bypassing the entire access control model. Most manual setups miss this or defer it.

With the data lake configuration, you declare roles and access policies in YAML where admins get full control, engineers get read access to curated data, extract, transform, and load (ETL) roles get scoped write access, and MDAA compiles them into the correct S3 bucket policies and Lake Formation registrations.

Use case of MDAA: Ensuring data integrity with AWS Glue Data Quality

AWS Glue Data Quality runs automated validation rulesets continuously as part of the pipeline, not as periodic batch checks. MDAA’s data quality module supports over 15 built-in rule types, from completeness and uniqueness checks to statistical thresholds and data freshness validation:

# data-quality.yaml
projectName: example-project

rulesets:
  customer-data-quality:
    description: Validate customer data completeness and uniqueness
    targetTable:
      databaseName: project:databaseName/customer-data
      tableName: customers
    ruleset:
      - ruleType: IsComplete
        column: customer_id
      - ruleType: Uniqueness
        column: email
        comparisonOperator: ">"
        threshold: 0.95
      - ruleType: RowCount
        comparisonOperator: ">"
        value: 100

Quality metrics flow into Amazon CloudWatch for real-time alerting. If anomalies are detected, automated workflows quarantine affected records and alert data engineering teams before issues reach downstream consumers.

Protecting metadata at rest: AWS Glue Data Catalog encryption

Table schemas, column names, and partition structures can reveal sensitive information about an organization’s data architecture, even without access to the underlying data. AWS Glue Catalog Encryption secures metadata at rest using AWS KMS-managed keys. MDAA configures catalog encryption by default, so schema definitions and connection passwords are encrypted from initial deployment without requiring manual key management setup. Access to catalog metadata follows the same Lake Formation governance controls applied to the data itself, so teams see only the schemas that they’re authorized to query.

Auditing every data access event: CloudTrail integration

Every data access event must be logged and attributable to a specific identity. Without a complete audit trail, demonstrating compliance during a regulatory review becomes a manual, error-prone process. AWS CloudTrail captures API-level activity across the data infrastructure, recording who accesses what data, when, and from which service. MDAA configures CloudTrail integration by default, so audit logging is active from initial deployment rather than added retroactively. Log data flows into a centralized, tamper-resistant store, giving compliance teams a single location to query access history across all business units and accounts.

Identifying sensitive data automatically: Macie integration

In large environments, sensitive information spreads across dozens of S3 buckets through pipelines, transforms, and ad hoc data drops, and self-reporting data owners consistently produce gaps. Amazon Macie uses machine learning to automatically discover and classify sensitive data in S3, surfacing findings at the object level without manual tagging. MDAA configures Macie across your S3 buckets during deployment, routing findings to Amazon EventBridge where automated workflows can alert owners or trigger remediation.

Together, these controls form a layered defense: Lake Formation governs access to cataloged data, Glue Data Quality validates integrity on arrival, and Macie identifies sensitive data that lands outside governed pipelines to reduce compliance risk.

Multi-account data mesh

MDAA provides extensive support for multi-account data mesh setups, with decentralized data ownership across business units and centralized governance. The data mesh starter kit supports cross-account data product publishing and consumption, allowing organizations to scale data sharing while maintaining consistent security and compliance controls.

Technical implementation

Ready to deploy your modern data architecture? Here are the resources to get started:

MDAA Implementation Guide provides detailed instructions for deploying all starter packages, including architecture patterns, configuration examples, security best practices, and troubleshooting guidance.

MDAA Hands-on Workshop offers step-by-step guided implementation with AWS experts. The workshop covers configuration management best practices, implementation patterns, hands-on labs with real-world scenarios, and cleanup instructions.

GitHub Repository and Documentation provide source code, module reference, and comprehensive documentation.

Organizations approach MDAA from different starting points. Some modernize existing data architectures, migrating from on-premises infrastructure or legacy cloud architectures. Others build new architectures for artificial intelligence and machine learning (AI/ML) initiatives or generative AI applications. Financial services organizations require PCI-DSS compliance from day one. Healthcare organizations need controls that can help support HIPAA. Each journey benefits from MDAA’s configuration-driven approach and embedded governance.

Conclusion

MDAA transforms data architecture development from months of manual coding to production-ready deployment. Configuration-driven infrastructure reduces development time by 40–60 percent while embedding governance from the start. The university system’s 95 percent reduction in time-to-value demonstrates the outcome: organizations deploy secure, compliant, governed data architectures in weeks rather than months.

Financial services organizations can deploy architectures to help them align with PCI-DSS compliance requirements using Lake Formation access controls, Glue Data Quality validation, SageMaker Unified Studio data discovery, comprehensive CloudTrail audit trails, and automated Macie data classification, all inherited from configuration rather than built manually.

Data architecture journeys need not follow six-month timelines with governance added incrementally. MDAA provides an alternative: describe the outcomes you want through YAML configuration, inherit pre-validated security controls, and deploy production-ready infrastructure with comprehensive governance from initial deployment.

Security and compliance is a shared responsibility between AWS and the customer. For more information, see the AWS Shared Responsibility Model.

Need help or have questions? Contact AWS ProServe for personalized guidance on selecting the right package and deployment strategy for your organization.

About the author

Identify unused AWS KMS keys and prevent accidental key deletions

2026-06-02 Andrea Rossi

Post Syndicated from Andrea Rossi original https://aws.amazon.com/blogs/security/identify-unused-aws-kms-keys-and-prevent-accidental-key-deletions/

As you scale your use of Amazon Web Services (AWS), managing KMS keys becomes increasingly important. Whether you manage a handful of keys or thousands across multiple AWS accounts and AWS Regions, there’s often a need to audit key usage to help you meet compliance requirements, evaluate your risk posture, and optimize key management costs. However, determining which keys are actively in use and which have been sitting idle can be a time consuming and complex task.

To help with this, AWS Key Management Service (AWS KMS) has launched the GetKeyLastUsage API, a new feature that you can use to quickly determine when each key was last used for a cryptographic operation, significantly enhancing your audit capabilities and key lifecycle management. For more information, see Determine past usage of a KMS key.

Before this launch, the primary way to audit key usage was through AWS CloudTrail logs. CloudTrail captures every cryptographic operation by default, so the data is available. The difficulty is turning that data into actionable insight. You need to identify which keys to examine, query the right logs, and repeat that process frequently enough to maintain an accurate view. For the most recent 90 days, CloudTrail event history makes this manageable. Beyond that, you need to create a dedicated trail to deliver logs to Amazon Simple Storage Service (Amazon S3) for long-term retention, then query those logs using tools such as Amazon Athena to determine when a key was last used.

Determine when a key was last used

AWS KMS now provides a direct way to see when a key was last used for cryptographic operations. You can also see this information using the AWS Management Console for AWS KMS and the AWS Command Line Interface (AWS CLI).

The GetKeyLastUsage API returns the date and time of the most recent cryptographic operation performed with a KMS key, without requiring you to search through CloudTrail logs. The API returns the date and time of the last key operation, the type of operation performed, CloudTrail event ID, and KMS request ID. You can access this information for all customer-managed keys and AWS managed keys irrespective of key spec, key origin, key store, or key usage type.

In addition, you can restrict a key from being disabled or scheduled for deletion if it was recently used, by incorporating this usage information as a condition within the KMS key policy. See the Preventing accidental key deletion with policy controls section for implementation details.

About the tracking period

One of the important concepts you must understand before relying on the last usage information reported on a KMS key is the tracking period. The tracking period is the date from which AWS KMS began tracking cryptographic activity for the key. Tracking began on April 23, 2026, for most AWS Regions. Understanding the tracking period is critical because it determines whether the absence of usage information means a key has never been used or only hasn’t been used since tracking started.

For example, if you have a key created on January 1, 2026, and you check its usage, any cryptographic operations that occurred between January 1 and April 22 wouldn’t be captured in the usage information. Thus, you can’t conclude that it’s never been used, because it might have been used in the months before tracking began.

Getting started

There’s nothing to enable or additional configuration required to view usage information on last cryptographic operation performed on your KMS keys.

To view KMS key usage:

Go to the AWS KMS console and choose Customer-managed keys in the navigation pane and select a key. Look for Last used on the general configuration.

Figure 1: KMS key general configuration page
Choose the link under Last used to see additional details such as Timestamp, Operation, and the CloudTrail event ID.

Figure 2: View last used details including timestamp, operation, and event ID
The Last used column is also shown when you attempt to schedule key deletion, so that you can make informed decisions.

Figure 3: Scheduled key deletion warning

API reference

See the following examples for ideas on how to use the GetKeyLastUsage API to better understand KMS key usage.

Use case 1: Cost optimization through unused key cleanup

If you manage thousands of AWS KMS keys distributed across multiple AWS accounts, you might have keys that have remained unused since creation or keys that are no longer needed. By cleaning up these keys, you can reduce operational costs and minimize your security footprint. However, without visibility into which keys are actively performing cryptographic operations, it can be difficult to distinguish between keys protecting critical workloads and those that can be safely decommissioned.

Note that there are some precautions that you should take before scheduling key deletion. While the last usage information can help identify unused keys, it shouldn’t be the only factor in deciding whether to delete or disable a key. The last usage information tells you when a key was last used, not whether it will be needed in the future. A key might be unused for months but still required to decrypt files, for compliance scenarios or disaster recovery as shown in figure 4.

When you identify a potentially unused key, first disable it using DisableKey and monitor your applications and services for any encryption or decryption failures.

Figure 4: A use case where GetKeyLastUsage doesn’t accurately reflect whether a KMS key is still required

As an example, Amazon EBS volumes only interact with KMS keys during specific lifecycle events like volume creation, attachment, and detachment. After a volume is attached to an Amazon Elastic Compute Cloud (Amazon EC2) instance, the plaintext data encryption key is cached in the Nitro Card hardware, and all subsequent read/write operations use this cached key without any further AWS KMS API calls. This means a production volume running continuously for months or years will show no KMS activity during that entire period. However, the volume remains completely dependent on that KMS key for any future operations like instance restarts, volume reattachments, or disaster recovery scenarios. If someone deletes the KMS key, the encrypted data key stored with the volume can never be decrypted again, making the volume’s data permanently and irreversibly inaccessible. Before deleting any KMS key, you must verify it has no associated EBS volumes or snapshots, regardless of how long ago the last KMS API call occurred.

AWS provides a mechanism where you can create a CloudWatch alarm that notifies you if a key pending deletion is being accessed, giving you an opportunity to cancel the deletion before data becomes inaccessible.

Solution with GetKeyLastUsage API

Here’s a sample script that scans all customer-managed keys in an account and retrieves each key’s last usage date through the GetKeyLastUsage API. It accepts two optional inputs: a threshold in days and an AWS Region. The script filters and displays only keys that haven’t been used within the specified period, presenting results in a table with the key name, AWS account ID, AWS Region, and last usage date. This can help you identify unused encryption keys.

The following is an example to scan all keys that haven’t been used in the last 180 days in the us-east-1 Region:

./script.sh 180 us-east-1
#!/bin/bash
DAYS=${1:-90}
REGION=${2:-$(aws configure get region)}
CUTOFF=$(date -v-${DAYS}d +%s 2>/dev/null || date -d "-${DAYS} days" +%s)
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
printf "Showing keys not used in the last %s days (Region: %s)\n\n" "$DAYS" "$REGION"
printf "%-50s %-15s %-20s %-15s\n" "Key Name" "Account ID" "Region" "Last Usage Date"
printf "%.0s-" {1..100}
printf "\n"
for key_id in $(aws kms list-keys --region $REGION --query 'Keys[*].KeyId' --output text); do
key_manager=$(aws kms describe-key --region $REGION --key-id $key_id --query 'KeyMetadata.KeyManager' --output text)
if [ "$key_manager" = "CUSTOMER" ]; then
last_usage=$(aws kms get-key-last-usage --region $REGION --key-id $key_id)
timestamp=$(echo $last_usage | jq -r '.KeyLastUsage.TimeStamp // empty')
if [ -z "$timestamp" ]; then
last_epoch=0
else
last_epoch=$(date -jf "%Y-%m-%dT%H:%M:%S" "$(echo $timestamp | cut -d. -f1)" +%s 2>/dev/null || date -d "$timestamp" +%s)
fi
if [ "$last_epoch" -lt "$CUTOFF" ]; then
key_alias=$(aws kms list-aliases --region $REGION --key-id $key_id --query 'Aliases[0].AliasName' --output text)
key_name=${key_alias:-$key_id}
[ "$key_name" = "None" ] && key_name=$key_id
if [ -z "$timestamp" ]; then
tracking_date=$(echo $last_usage | jq -r '.TrackingStartDate' | cut -d'T' -f1)
last_used="${tracking_date}*"
else
last_used=$(echo $timestamp | cut -d'T' -f1)
fi
printf "%-50s %-15s %-20s %-15s\n" "$key_name" "$ACCOUNT_ID" "$REGION" "$last_used"
fi
fi
done
printf "\n* = No operations performed since tracking started\n"

Use case 2: Preventing accidental key deletion with policy controls

Organizations frequently face the risk of accidental key deletions, which can have severe operational consequences. Despite precautions and safety measures, accidents can happen. A key might be deleted because someone believes it’s no longer in use, only to discover that critical applications or workloads depend on it. This results in data access failures, application downtime, and emergency recovery procedures. Without visibility into recent key usage, teams lack the information needed to make safe disable decisions or implement effective safeguards.

Solution with policy based controls

To prevent KMS keys from being accidentally Disabled or Deleted use the kms:TrailingDaysWithoutKeyUsage condition key in key policies to automatically block deletion or disabling of recently used keys:

Open the AWS KMS console and choose Customer managed keys in the navigation pane.
Select the key you want to protect.
In the Key policy tab, choose Edit.
In the policy editor, add the following statement:

{
  "Sid": "PreventDeletionOfRecentlyUsedKeys",
  "Effect": "Deny",
  "Principal": "*",
  "Action": [
    "kms:ScheduleKeyDeletion",
    "kms:DisableKey"
  ],
  "Resource": "*",
  "Condition": {
    "NumericLessThanEquals": {
      "kms:TrailingDaysWithoutKeyUsage": "365"
    }
  }
}

Choose Save changes.

The policy prevents deletion or disabling a key if it was used within the past 365 days. You can adjust the threshold to match your organization’s requirements. For more information about the condition key, see kms:TrailingDaysWithoutKeyUsage.

Important considerations

When reviewing key usage for possible deletion, consider the following:

Key deletion is irreversible and makes encrypted data unrecoverable. AWS enforces a 7–30 day waiting period. During this time, monitor usage attempts and cancel the deletion if necessary. Delete a key only if you’re certain that no data has been encrypted or will be encrypted with it. Consider disabling the key first to test the impact of unavailable keys.
CloudTrail remains authoritative because it provides the full audit trail. GetKeyLastUsage quickly tells you when and what operations occurred, but CloudTrail shows you who made the request and with what parameters. Learn more about logging KMS API calls with CloudTrail.

Conclusion

The GetKeyLastUsage API enhances your KMS key management capabilities by providing immediate access to usage data that was previously only present in CloudTrail logs. Start by opening the AWS KMS console and checking the Last used field for any customer-managed keys and AWS managed keys to see this information in action. For broader key auditing, integrate the API into your existing automation scripts using the AWS CLI examples provided.

If you have feedback about this post, submit comments in the Comments section below.

Security posture improvement in the AI era

2026-05-01 Celeste Bishop

Post Syndicated from Celeste Bishop original https://aws.amazon.com/blogs/security/security-posture-improvement-in-the-ai-era/

It’s only been a few weeks since Anthropic announced the Claude Mythos Preview model and launched Project Glasswing with AWS and other leading organizations. This has generated a lot of discussion about the future of cybersecurity and what the ever-increasing capabilities of foundation models mean to organizations.

As AWS CISO Amy Herzog pointed out in the Project Glasswing announcement, “At AWS, we build defenses before threats emerge, from our custom silicon up through the technology stack. Security isn’t a phase for us; it’s continuous and embedded in everything we do.”

Read more from Amy about this in Building AI defenses at scale: Before the threats emerge.

While the discussion around the future of cybersecurity is important, the only thing we know for certain is that organizations need to be able to react quickly to the rapid changes AI is bringing to technology and business in general. And you can’t react quickly if your security fundamentals aren’t dialed in.

The security hygiene gap

It’s easy to assume you have the foundational security elements covered, or to overlook some completely. Basic security use cases like identity management, threat detection, vulnerability management, data protection, and network security can be inconsistently implemented across cloud environments. While AI is reshaping the security landscape, strong security fundamentals continue to be essential for every organization, regardless of size or industry.

These are the security basics that matter whether or not you’re adopting AI: patching consistently, enforcing least-privilege access, enabling logging and monitoring, encrypting data at rest and in transit, and reviewing security configurations regularly. When these fundamentals are in place, you’re better positioned to take advantage of AI-driven tools and respond to newly discovered vulnerabilities, wherever they come from.

While the concepts that drive security fundamentals are universal, implementing them in your environment is best done with an understanding of the context unique to your organization. That’s why we have a multitude of freely available materials—like the AWS Well-Architected Framework—that you can use to help ask the right questions and implement changes in your environment. We also offer programs like the Security Health Improvement Program (SHIP) to help you improve your security posture through prescriptive guidance and continuous improvement.

What is the Security Health Improvement Program (SHIP)?

SHIP is a no-cost program available to every AWS customer, regardless of support tier. SHIP provides a proven, data-driven methodology to:

Assess your current security posture using data from your AWS environment
Identify specific opportunities to improve across 10 core security use cases
Build a prioritized action plan tailored to your environment
Establish a mechanism for continuous security improvement

The program is led by AWS Solutions Architects and Technical Account Managers who take you through a personalized report, contextualize findings for your environment, and help you build a prioritized action plan.

Why SHIP matters in the AI era

Project Glasswing highlights an important shift: AI-powered tools are accelerating the pace of vulnerability discovery, which means organizations need to be prepared to assess and respond to findings and changing situations faster than before. In addition to external factors, as organizations adopt AI—whether deploying foundation models, building agentic workflows, or using AI-powered services—how they implement their security controls must change as well. A strong security foundation is what makes confident AI adoption possible.

Here’s how SHIP helps:

Address foundational security gaps proactively

SHIP uses a data-driven methodology to identify opportunities to improve and optimize across 10 core security use cases: threat detection, cloud security posture management, application security testing, configuration management, access governance, vulnerability management, application protection, network security, encryption, and secrets management. The program includes a SHIP assessment to identify critical security findings related to your current security posture, so your team can build a prioritized roadmap for improvement tailored to your environment.

Establish the security baseline AI workloads require

Before you deploy your first model on Amazon Bedrock or build agentic workflows with Amazon Bedrock AgentCore, you need confidence that your underlying infrastructure follows security best practices. SHIP uses actual data from your environment to provide prescriptive, specific guidance rather than generic security recommendations. This is especially relevant as AI-driven vulnerability discovery tools become more widely available: organizations with strong baselines will be able to act on new findings quickly and effectively.

Build a mechanism for continuous security improvement

As AI capabilities evolve, organizations benefit from having a repeatable process to assess and strengthen their security posture over time. SHIP establishes the methodology and mechanisms for your team to continuously assess, prioritize, and improve. By building this operational capability, you’re strengthening your organization’s ability to adapt and contributing to broader industry resilience. As the cybersecurity community integrates AI into defense strategies, SHIP helps you maintain foundational best practices so you can adopt these innovations effectively and with confidence.

Getting started is straightforward

SHIP is available today, at no cost, to every AWS customer. Here’s how to get started:

Talk to your AWS account team. Ask about scheduling a SHIP engagement, or request one directly on the SHIP page.
Attend a SHIP Activation Day. AWS regularly hosts hands-on workshops where you can run the SHIP assessment with AWS Solutions Architects and start building your improvement plan.
Explore the prescriptive guidance. Consult the AWS Well-Architected Framework – Security Lens for documentation, reference architectures, and implementation guides you can start using today.

Take the next step together

AWS is committed to being the most secure cloud, from our participation in Project Glasswing to the security embedded in every layer of our infrastructure. Security is a shared responsibility, and programs like SHIP give customers the tools, guidance, and support to strengthen their security foundations so they can build confidently, no matter what comes next.

Ready to improve your security posture? Contact your AWS account team to schedule a SHIP engagement, or visit the SHIP resources page to learn more.

Digital Transformation at Santander: How Platform Engineering is Revolutionizing Cloud Infrastructure

2026-02-26 Julio Bando

Post Syndicated from Julio Bando original https://aws.amazon.com/blogs/architecture/digital-transformation-at-santander-how-platform-engineering-is-revolutionizing-cloud-infrastructure/

This post is cowritten by Julio Bando from Santander.

Santander faced a significant technical challenge in managing an infrastructure that processes billions of daily transactions across more than 200 critical systems. The expansion into diverse financial services, including investment banking, wealth management, insurance, and payment solutions, had created unprecedented technological complexity, requiring a robust, agile, and scalable infrastructure solution. This raised two main issues. Santander needed to ensure that provisioned services followed established architecture definitions, and they needed to reduce infrastructure provisioning time, which took up to 90 days. This situation demanded intensive operational effort. The solution emerged through an innovative platform engineering initiative called Catalyst, which transformed the bank’s cloud infrastructure and development management. This post analyzes the main cases, benefits, and results obtained with this initiative.

The Catalyst solution

Santander is a global financial services company present in more than 10 countries, with over 160 million customers worldwide. They conceived Catalyst in conjunction with the Platform Strategy Program (PSP), an Amazon Web Services (AWS) program specialized in infrastructure platform design. Implemented through a partnership between AWS Professional Services and Santander, the platform was designed to abstract infrastructure provisioning complexity, standardize architectural compliance, and create a framework that enables new technologies in the bank.

The platform’s in-house frontend was developed as an intuitive developer portal, offering a unified interface for all provisioning and resource management needs. At the platform’s core is the control plane cluster, based on Amazon Elastic Kubernetes Service (Amazon EKS). This cluster is the brain of the operation, orchestrating all components and workflows. Within the cluster, Crossplane plays a fundamental role, acting as a universal resource provisioner that Santander uses to manage resources across multiple cloud providers consistently and declaratively.

The control plane cluster has three components:

Data plane claims – Managed by ArgoCD, a continuous delivery tool, the component is responsible for continuous synchronization and deployment of application stacks (integrated sets of cloud resources) and configurations, exploring the GitOps concept.
Policies catalog – A central repository of policies ensuring compliance and security across all operations using Open Policy Agent (OPA).
Stacks catalog – A library of composite resource definitions and Compositions enabling quick and standardized creation of complex environments.

Santander used this innovative architecture to significantly reduce provisioning time from 90 days to only a few hours and in some cases only minutes. Catalyst brought significant benefits in terms of standardization, security, and governance. The provisioning cycle decreased from 30 days to 2 days, and proof of concept preparation time jumped from 90 days to only 1 hour. The consolidation of over 100 pipelines into a single control plane will further simplify infrastructure management. The following diagram shows the Santander catalyst architecture.

This diagram shows the AWS architecture of Santander's Catalyst platform that provides AI capabilities to teams across the company.

Key platform capabilities

Catalyst’s implementation enabled the creation of strategic workloads demonstrating the platform’s versatility and robustness:

Generative AI agents stack – The first success case was implementing a complete stack for AI agents integrating:
- Amazon Bedrock for foundational model consumption
- Amazon Simple Storage Service (Amazon S3) for data storage
- AWS Key Management Service (AWS KMS) for cryptographic key management
- Custom AWS Identity and Access Management (IAM) policies for security
Modern data platform – One of the most complex workloads implemented through Catalyst was the new data platform, including:
- Built-in integration with Databricks
- Data lakes
- Automated extract, transform, and load (ETL) workflows
- Integration with centralized data catalog
- Segregated environments for experimentation. With this implementation, the bank significantly reduces approximately 3,000 monthly tickets related to data experimentation environment provisioning.
Cloud process orchestration – creation of a modern process orchestration environment with significant results:
- Migration of legacy workflows to AWS Step Functions
- Implementation of retry patterns and error handling
- Centralized process monitoring

Overall result

This stack reduced AI agent implementation time from 105 days to only 24 hours, eliminating dozens of provisioning tickets per environment. The success of these workloads demonstrates Catalyst’s technical capability and the solution’s versatility in meeting different business needs. Each implementation brought valuable learnings that were incorporated into the platform, creating a virtuous cycle of continuous improvement. The variety of implemented workloads also shows how Catalyst has the potential to be a universal platform, capable of supporting everything from traditional use cases to the most innovative ones involving AI and legacy system modernization. Catalyst’s success wasn’t limited to operational efficiency. The platform also catalyzed a cultural change within Santander, promoting an automation and self-service mindset among development teams. This resulted in faster overall development velocity, more agile teams, and enhanced capability to respond quickly to market changes.

Conclusion

Catalyst represents more than merely a technological tool—it’s a digital transformation enabler that’s redefining cloud development standards at the bank. With the platform, Santander addressed the challenges of a scaled environment and established a solid foundation for continuous innovation and future growth.

With these practical cases, Santander proves that investment in platform engineering solves technical problems and enables new business possibilities, keeping the bank at the forefront of digital transformation in the financial sector.

About the authors

Secure Apache Spark writes to Amazon S3 on Amazon EMR with dynamic AWS KMS encryption

2026-01-27 Pinxi Tai

Post Syndicated from Pinxi Tai original https://aws.amazon.com/blogs/big-data/secure-apache-spark-writes-to-amazon-s3-on-amazon-emr-with-dynamic-aws-kms-encryption/

When processing data at scale, many organizations use Apache Spark on Amazon EMR to run shared clusters that handle workloads across tenants, business units, or classification levels. In such multi-tenant environments, different datasets often require distinct AWS Key Management Service (AWS KMS) keys to enforce strict access controls and meet compliance requirements. At the same time, operational efficiency might drive these organizations to consolidate their data pipelines. Instead of running separate Spark jobs for each dataset, it could be more efficient to run a single job on Amazon EMR that processes inputs once and writes multiple outputs to Amazon Simple Storage Service (Amazon S3), each encrypted with its own KMS key.

Although consolidating multiple datasets in one Spark job reduces orchestration overhead and simplifies code maintenance, you might encounter challenges with encryption configurations. By default, the EMRFS and S3A file system clients cache their settings, which can cause encryption keys to persist incorrectly across writes. This means if you change the encryption key between writes to Amazon S3 in an Apache Hadoop environment, some output files can end up encrypted with unintended keys, leading to possible security and compliance concerns.

In this post, we show how to securely write data to Amazon S3 from Spark jobs running on Amazon EMR, while dynamically managing different KMS keys for encryption. We discuss three approaches to solve this challenge and how to choose the right solution for your use case.

Amazon S3 server-side encryption options with Amazon EMR

When writing data to Amazon S3 from Amazon EMR, you can choose from multiple server-side encryption options. The two most commonly used options are:

Server-side encryption with Amazon S3 managed keys (SSE-S3) – Amazon S3 manages the encryption keys for you
Server-side encryption with KMS keys (SSE-KMS) – AWS KMS manages the keys, and you can use custom KMS keys with fine-grained access control

When running Spark jobs on Amazon EMR, data writes to Amazon S3 occur through one of the following file system implementations:

EMRFS – The default implementation for Amazon EMR versions below 7.10.0
S3A – The default implementation starting from Amazon EMR 7.10.0

Both implementations provide configuration properties to control server-side encryption. The following tables show how to specify the KMS key for SSE-KMS encryption.

For EMRFS (default in Amazon EMR versions below 7.10.0), refer to the following table.

Property	Description
`fs.s3.enableServerSideEncryption`	Enables server-side encryption. Defaults to SSE-S3 if no KMS key is provided.
`fs.s3.serverSideEncryption.kms.keyId`	Specifies the KMS key ID or ARN for SSE-KMS encryption.

For S3A (default in Amazon EMR starting from 7.10.0), refer to the following table.

Property	Description
`fs.s3a.encryption.algorithm`	Specifies the encryption algorithm.
`fs.s3a.encryption.key`	Specifies the KMS key ID or ARN for SSE-KMS encryption.

Starting from the Amazon EMR 7.10.0 release, the S3A file system has replaced EMRFS as the default EMR S3 connector. For more information, refer to Migration Guide: EMRFS to S3A Filesystem.

Challenges to prevent encryption key reuse due to file system caching

In practice, a unified Spark job might write outputs for multiple tenants or classifications in a single run. In this situation, applying the correct encryption key for each output is critical to maintaining compliance and enforcing isolation in multi-tenant S3 buckets without the complexity of managing separate Spark jobs for each dataset.

When Spark executors write to Amazon S3, they use a file system (EMRFS or S3A) client that is cached and reused for performance optimization. The problem is that each file system instance keeps the encryption settings it was first created with. Each executor’s Java Virtual Machine (JVM) creates and caches a file system (and its underlying S3 client) for a given S3 bucket. This cached instance, along with its encryption configuration, persists throughout the executor’s lifecycle. If you change the encryption key in Spark after some data has been written, the existing cached client can’t pick up the new key.

For example, the following PySpark code first creates a Spark session with S3 server-side encryption enabled. It then writes a DataFrame to two different folders within the same S3 bucket amzn-s3-demo-bucket1, but with different KMS keys. The first write operation writes the data to folder1 using kmsKey1 for encryption, and the second write operation writes to folder2 using kmsKey2.

# Pseudo-code: setting different keys for successive writes to different folders within the same S3 bucket

spark = SparkSession.builder \
.appName("Write data to S3 with KMS") \
.config("spark.hadoop.fs.s3.enableServerSideEncryption", "true") \
.getOrCreate()

df.write.option('fs.s3.serverSideEncryption.kms.keyId', kmsKey1).save("s3://amzn-s3-demo-bucket1/folder1/")
df.write.option('fs.s3.serverSideEncryption.kms.keyId', kmsKey2).save("s3://amzn-s3-demo-bucket1/folder2/")

You might expect files in folder1/ to use kmsKey1 and files in folder2/ to use kmsKey2. But due to caching, the second write can still use the client configured with kmsKey1. This leads to mixed or incorrect encryption key usage across outputs.

Solution overview

Our objective is to achieve correct encryption of each output S3 object with its intended KMS key, even when a single Spark job writes multiple outputs. To implement this, you can use one of the following approaches:

Disable file system cache – Turn off S3 client caching so a new client is created for each write, picking up the current key
Separate Spark applications or sessions – Run a separate Spark application (or session) for each distinct encryption key, so each client is initialized fresh
Use S3 bucket default encryption – Configure bucket-level SSE-KMS with the desired key so Amazon S3 automatically applies the correct encryption key

Each method offers a different balance of implementation complexity, performance, and flexibility. The following sections provide detailed implementation steps and considerations for each approach.

Method 1: Disable file system cache

Disabling the file system cache forces Spark to create a new S3 client for each write, which applies the updated encryption settings. This can be done using a Spark configuration or EMR cluster settings.

The property name for disabling the cache depends on your URI scheme (s3:// or s3a://), not on your choice of file system (EMRFS or S3A). The following table summarizes which configuration property name you should use to disable the cache.

Properties for s3:// URI scheme	Properties for s3a:// URI scheme
`fs.s3.impl.disable.cache`	`fs.s3a.impl.disable.cache`
`spark.hadoop.fs.s3.impl.disable.cache`	`spark.hadoop.fs.s3a.impl.disable.cache`

To use this method, complete the following steps:

Disable the file system cache in Spark configuration.
For s3:// scheme, you can manually set spark.hadoop.fs.s3.impl.disable.cache=true, for example (PySpark):

# PySpark example for "s3://"
# Create Spark session
spark = SparkSession.builder \
.appName("Write data to S3 with KMS") \
.config("spark.hadoop.fs.s3.impl.disable.cache", "true") \
.getOrCreate()

Alternatively, you can use the following spark-defaults configuration classification:

[
  {
    "Classification": "spark-defaults",
    "Properties": {
      "spark.hadoop.fs.s3.impl.disable.cache": "true"
    }
  }
]

For s3a:// scheme, you can manually set spark.hadoop.fs.s3a.impl.disable.cache=true, for example (PySpark):

# PySpark example for "s3a://"
# Create Spark session
spark = SparkSession.builder \
.appName("Write data to S3 with KMS") \ 
.config("spark.hadoop.fs.s3a.impl.disable.cache", "true") \
.getOrCreate()

Alternatively, you can use the following spark-defaults configuration classification:

[
  {
    "Classification": "spark-defaults",
    "Properties": {
      "spark.hadoop.fs.s3a.impl.disable.cache": "true"
    }
  }
]

Instead of disabling the cache specifically for Spark applications, you can optionally configure the EMR cluster’s core-site.xml to disable the file system cache globally at cluster level. You must configure the /etc/hadoop/conf/core-site.xml file on the primary nodes of your EMR cluster. For example, when creating or modifying the cluster, use the following configuration.
For s3:// scheme:
```
[
  {
    "Classification": "core-site",
    "Properties": {
      "fs.s3.impl.disable.cache": "true"
    }
  }
]
```
For s3a:// scheme:
```
[
  {
    "Classification": "core-site",
    "Properties": { 
      "fs.s3a.impl.disable.cache": "true"
    }
  }
]
```
Enable SSE-KMS encryption.
For EMRFS, set spark.hadoop.fs.s3.enableServerSideEncryption=true for Spark applications only or use the following configuration to enable encryption at cluster level:
```
[
  {
    "Classification": "emrfs-site",
    "Properties": {
      "fs.s3.enableServerSideEncryption": "true"
    }
  }
]
```
For S3A, set spark.hadoop.fs.s3a.encryption.algorithm=SSE-KMS for Spark applications only or use the following configuration to enable encryption at cluster level:
```
[
  {
    "Classification": "core-site",
    "Properties": {
      "fs.s3a.encryption.algorithm": "SSE-KMS"
    }
  }
]
```
When using EMRFS with fs.s3.impl.disable.cache=true, you must also disable the EMRFS S3-optimized committer to avoid errors. You can do this by either manually setting spark.sql.parquet.fs.optimized.committer.optimization-enabled=false or using the following spark-defaults configuration classification:
```
[
  {
    "Classification": "spark-defaults",
    "Properties": {
      "spark.sql.parquet.fs.optimized.committer.optimization-enabled": "false"
    }
  }
]
```

For more information about configuring applications on EMR clusters, refer to Configure applications when you create a cluster and Reconfigure an instance group in a running cluster.

Considerations

Use this method when you need to write data to an S3 bucket using multiple KMS keys within a single Spark application. This is a quick, straightforward implementation that works well for the following use cases:

Testing environments and debugging sessions
Proof-of-concept demonstrations
Low-volume or one-time jobs where write performance is not critical
Workloads that frequently switch encryption keys to write data to different S3 prefixes within the same bucket

Before implementation, consider the following performance impacts:

Increased latency for each write
Additional S3 API operations
Extra connection overhead

Although this method provides a pragmatic solution when splitting work into separate Spark applications isn’t feasible, we don’t recommend it for high-throughput or latency-sensitive production workloads. The increased API traffic can lead to higher costs and potential throttling. For production implementations, consider Method 2 and Method 3.

Method 2: Use separate Spark applications or sessions

When writing data with multiple encryption keys, use a separate Spark application (or Spark session) for each distinct key. The file system needs to be initialized with the correct encryption key in a fresh JVM context when writing data with different keys. You can achieve this by either submitting a separate Spark application or starting a new Spark session. This enables the S3 client to be created with the intended encryption key.

Complete the following steps:

Divide the write tasks by KMS key. For example, prepare separate DataFrames or filter logic for each key.

Submit separate jobs. Choose either of the following options:

Use spark-submit commands. For example:

# For EMRFS
spark-submit –conf spark.hadoop.fs.s3.serverSideEncryption.kms.keyId=kmsKey1 job1.py
spark-submit --conf spark.hadoop.fs.s3.serverSideEncryption.kms.keyId=kmsKey2 job2.py

# For S3A
spark-submit –conf spark.hadoop.fs.s3a.server-side-encryption.key=kmsKey1 job1.py
spark-submit --conf spark.hadoop.fs.s3a.server-side-encryption.key=kmsKey2 job2.py

Use Spark sessions in code (PySpark example):

# For EMRFS
for kmsKey in [kmsKey1, kmsKey2]:
    spark = SparkSession.builder \
        .appName("Write data to S3 with KMS") \
        .config("spark.hadoop.fs.s3.enableServerSideEncryption", "true") \
        .config("spark.hadoop.fs.s3.serverSideEncryption.kms.keyId", kmsKey) \
        .getOrCreate()
    write_df_for_key(kmsKey)  # Pseudocode for writing data for this key
    spark.stop()

# For S3A
for kmsKey in [kmsKey1, kmsKey2]:
    spark = SparkSession.builder \
        .appName("Write data to S3 with KMS") \
        .config("spark.hadoop.fs.s3a.encryption.algorithm", "SSE-KMS") \
        .config("spark.hadoop.fs.s3a.encryption.key", kmsKey) \
        .getOrCreate()
    write_df_for_key(kmsKey)  # Pseudocode for writing data for this key
    spark.stop()

Use your preferred workflow (such as AWS Step Functions, Apache Airflow, or a wrapper script) to launch jobs in sequence or parallel.

Considerations

Use this method when you need a production-grade solution for applying different KMS keys at scale. This approach maintains file system caching benefits and works well for the following use cases:

High-throughput or latency-sensitive workloads with frequent write operations
Scenarios requiring strong isolation between different KMS keys
Multi-tenant environments with separate compliance boundaries

This method creates fresh S3 clients with each new Spark application or session. Compared to Method 1, it offers several advantages:

Avoids per-write connection and API overhead
Maintains full compatibility with the EMRFS S3-optimized committer
Enforces credential boundaries between workloads and improves operational and compliance isolation by assigning each Spark application or session a dedicated key

Before implementation, consider the following trade-offs:

Requires orchestration of multiple Spark applications and sessions or clusters
Involves higher resource overhead
Increases operational complexity

Choose this method when performance, cost predictability, and security isolation are more important than single-process simplicity.

Method 3: Use S3 bucket default encryption

Where possible, configure S3 bucket-level default encryption (SSE-KMS) with the desired KMS key to automatically encrypt objects written to that bucket.

Complete the following steps:

On the Amazon S3 console or using the AWS Command Line Interface (AWS CLI), enable default SSE-KMS for the bucket with the desired key. For instructions on enabling SSE-KMS for S3 buckets, refer to Configuring default encryption.
With default encryption enabled for the S3 bucket, you can write without specifying a KMS key in Spark. Amazon S3 can encrypt each object with the bucket’s key. Your Spark code only needs standard write operations.

Considerations

Use this method when your workloads can use a single KMS key per bucket. This approach works well for the following use cases:

Production environments prioritizing operational simplicity
Workloads where all data in a bucket shares the same security requirements
Scenarios where encryption configuration should be managed at the bucket level
Use cases that map naturally to per-bucket separation

This method provides several advantages:

Alleviates the need to configure encryption in Spark applications
Automatically applies the default KMS key for all writes
Simplifies encryption management

Before implementation, consider the following limitations:

You must have all data in one bucket to use the same encryption key
You can’t apply different keys to different prefixes within the same bucket

Choose this method when you need a simple, reliable approach that provides strong security while simplifying operational management.

Choosing the right approach

Choose the method based on your workload’s security requirements, performance needs, and operational constraints:

Method 1 – Use when you need to apply multiple KMS keys within a single Spark job and can accept some performance impact
Method 2 – Use for production workloads that require different encryption keys within the same bucket and need optimal performance
Method 3 – Use when a single KMS key per bucket meets your encryption requirements and you want simplified operations

Conclusion

In this post, we demonstrated how to handle multiple KMS keys when writing to Amazon S3 from Spark jobs on Amazon EMR. When encrypting multiple outputs with different encryption keys in a single Spark application, it’s important to consider the file system caching behavior. We presented several practical solutions with their respective trade-offs. You can start implementing these solutions in your environment by first testing the file system cache-disable method, which provides a straightforward approach to handling multiple encryption keys. As your workload grows, consider evolving to separate Spark sessions or S3 bucket default encryption based on your specific requirements. After implementing a solution, verify that each S3 object’s SSE-KMS key is the intended one (for example, by checking S3 object metadata). We also recommend measuring job performance and S3 API usage, especially for the cache-disable approach.

About the authors

Post-quantum (ML-DSA) code signing with AWS Private CA and AWS KMS

2025-11-17 Panos Kampanakis

Post Syndicated from Panos Kampanakis original https://aws.amazon.com/blogs/security/post-quantum-ml-dsa-code-signing-with-aws-private-ca-and-aws-kms/

Following our recent announcement of ML-DSA support in AWS Key Management Service (AWS KMS), we just introduced post-quantum ML-DSA signature support in AWS Private Certificate Authority (AWS Private CA). Customers can use AWS Private CA to create and manage their own private public key infrastructure (PKI) hierarchies. Through this integration, you can establish and use customer-managed quantum-resistant roots of trust for code signing, device authentication, outside (of AWS) workload authentication with AWS IAM Roles Anywhere, or communication tunnels such as IKEv2/IPsec or Mutual TLS (mTLS) using private PKI.

As outlined in the AWS post-quantum cryptography migration plan, establishing quantum-resistant roots of trust is critical for systems that need to maintain security for extended periods of time. ML-DSA, a signature scheme standardized in FIPS 204, provides quantum resistance while maintaining the performance characteristics needed for deployments at scale.

Previously, we shared how to use AWS Private CA and AWS KMS for code signing. In this post, we show you how to combine the post-quantum signing capability provided by AWS KMS with post-quantum code-signing PKI from AWS Private CA. Consumers of signed code that have been pre-provisioned with the post-quantum PKI roots can rest assured that the software could not have been forged by an adversary with a cryptographically relevant quantum computer (CRQC). For demonstration purposes, we use the diy-code-signing-kms-private-ca sample program, which uses the AWS SDK for Java. This code creates a PKI infrastructure, generates a code-signing certificate, signs binary code, and verifies the signature. Although we break down the steps to demonstrate the functionality in this post, you can run the Runner as-is to see it in action with commands found in the README file.

This post uses the Cryptographic Message Syntax (CMS) standard for encapsulating the signatures generated for input binary data. It stores the signature, X.509 certificate, and chain used to establish trust. The signature, known as a detached signature, doesn’t contain the original data. The detached signature can be used together with the original file, which was signed with standard tools such as OpenSSL natively to validate the authenticity of the file.

Create a post-quantum PKI hierarchy

For this post, we will use AWS Private CA to introduce a code-signing PKI. It will consist of a root CA to sign a subordinate CA, and a code-signing certificate signed by the subordinate CA. The whole chain will consist of quantum-resistant ML-DSA certificates.

CA hierarchy creation

First, the post-quantum CA hierarchy must be created with ML-DSA. In this example, we use the ML-DSA-65 variant of the post-quantum signature algorithm. You can do this with the AWS Java SDK as shown in the Runner.java file:

PrivateCA rootPrivateCA = PrivateCA.builder()
	.withCommonName(ROOT_COMMON_NAME)
	.withType(CertificateAuthorityType.ROOT)
	.withAlgorithmFamily(ML_DSA_65_ALGORITHM_FAMILY)
	.getOrCreate();

PrivateCA subordinatePrivateCA = PrivateCA.builder()
    .withIssuer(rootPrivateCA).withCommonName(SUBORDINATE_COMMON_NAME)
    .withType(CertificateAuthorityType.SUBORDINATE)
	.withAlgorithmFamily(ML_DSA_65_ALGORITHM_FAMILY)
    .getOrCreate();

Code-signer creation

For code signing, you need an asymmetric key pair and a code-signing certificate. The asymmetric ML-DSA key pair is generated in AWS KMS and the code-signing certificate is issued by AWS Private CA.

Create an ML-DSA key pair in AWS KMS

First, you must create an asymmetric key pair for code signing operations. Similar to the creation of the hierarchy, the AWS Java SDK can be used to create that AWS KMS key (key pair). Signing will be taking place with the key pair’s private key in AWS KMS. The corresponding public key will be in the code-signing leaf certificate signed by the subordinate CA. These calls are performed as part of the main method within the Runner.java file:

AsymmetricCMK codeSigningCMK = AsymmetricCMK
    .builder().withAlias(CMK_ALIAS)
	.withAlgorithmFamily(ML_DSA_65_ALGORITHM_FAMILY)
    .getOrCreate();

Alternatively, you can generate the key pair in AWS KMS with the AWS Management Console or the AWS Command Line Interface (AWS CLI) as shown in the ML-DSA KMS security blog.

Issue a code-signing certificate

Creating a certificate signing request (CSR) using AWS Private CA is a two-step process. First, you must create a CSR that contains both an identity (Subject) and the previously created AWS KMS public key. The following code snippet in Runner.java accomplishes this:

String codeSigningCSR = codeSigningCMK
	.generateCSR(END_ENTITY_COMMON_NAME);

OpenSSL 3.5 or later can parse this CSR to view its content with the following command if the CSR contents have been written to disk at csr.pem:

openssl req -in csr.pem -inform pem -text -noout
Certificate Request:
	Data:
		Version: 1 (0x0)
		Subject: CN=CodeSigningCertificate
		Subject Public Key Info:
			Public Key Algorithm: ML-DSA-65
				ML-DSA-65 Public-Key:
				pub:
					<Public Key Data>   
		Attributes:
			Requested Extensions:
				X509v3 Basic Constraints:
					CA:FALSE
	Signature Algorithm: ML-DSA-65
	Signature Value:
		<Signature Data>

You can see that the CSR contains an ML-DSA-65 public key. Its corresponding private key will be used to sign code.

The CSR is used by the subordinate CA to issue the code-signing certificate. Note that the code-signing template is used in the templateArn of the IssueCertificate request in the relevant PrivateCA.java file. The inclusion of this template helps ensure that AWS Private CA will issue a certificate with the correct Key Usage (KU) and Extended Key Usage (EKU) extension values, regardless of the values presented in the CSR.

IssueCertificateRequest issueCertificateRequest = IssueCertificateRequest.builder()
	.idempotencyToken(UUID.randomUUID().toString())
	.certificateAuthorityArn(subordinatePrivateCA.arn())
	.csr(SdkBytes.fromUtf8String(csr))
	.signingAlgorithm(algorithmFamily.getPcaSigningAlgorithm())
	.templateArn("arn:aws:acm-pca:::template/CodeSigningCertificate/V1")
	.validity(validity)
	.build();

IssueCertificateResponse issueCertificateResponse = client
	.issueCertificate(issueCertificateRequest);

String certificateArn = issueCertificateResponse.certificateArn();

GetCertificateRequest getCertificateRequest = GetCertificateRequest.builder()
	.certificateAuthorityArn(ca.arn())
	.certificateArn(certificateArn)
	.build();

The response includes the ML-DSA-65 code-signing certificate. You can use OpenSSL 3.5 or later to inspect the contents of the certificate after you save it to a file named code-signing-cert.pem:

openssl x509 -in code-signing-cert.pem -inform pem -text -noout
Certificate:
	Data:
		Version: 3 (0x2)
		Serial Number:
			1a:15:af:1e:64:8d:cd:29:b4:dc:66:2a:8b:1e:ee:b0
		Signature Algorithm: ML-DSA-65
		Issuer: CN=CodeSigningSubordinate-MLDSA65
		Validity
			Not Before: Sep 24 13:10:38 2025 GMT
			Not After : Sep 24 14:10:38 2026 GMT
		Subject: CN=CodeSigningCertificate
		Subject Public Key Info:
			Public Key Algorithm: ML-DSA-65
				ML-DSA-65 Public-Key:
				pub:
					<Public Key Data>
		X509v3 extensions:
			X509v3 Basic Constraints:
				CA:FALSE
			X509v3 Authority Key Identifier:
B7:EF:2E:C9:7A:A8:7E:B5:D6:2D:9A:3F:C7:A7:F8:9D:74:01:6A:EF
			X509v3 Subject Key Identifier:

7F:63:35:0C:56:F8:ED:F1:2A:DF:B5:2E:7C:F1:2C:D9:A0:0E:63:B6
			X509v3 Key Usage: critical
				Digital Signature
			X509v3 Extended Key Usage: critical
				Code Signing
	Signature Algorithm: ML-DSA-65
	Signature Value:
		<Signature Data>

You can see that the certificate includes the ML-DSA-65 public key of the code-signing key pair and the ML-DSA-65 signature from the subordinate CA. You also see the KU and the EKU values, which represent a code-signing certificate from the AWS Private CA template.

Sign code

At this point, you have set up the code-signing PKI, have a code-signing certificate issued by AWS Private CA and a corresponding ML-DSA key pair residing in KMS.

The Java SDK can be used to generate a CMS signature for a code binary file. In the background, this is accomplished by calling the AWS KMS Sign API with the ML-DSA key pair as shown in Runner.java. The following is part of the Java code. This first snippet involves building a certificate chain and then using it along with the code-signing AWS KMS key, the signer’s certificate, and <DATA_TO_SIGN>, the byte array representation of the code file, to generate the detached signature in a CMS structure.

	// Parse code-signing certificate from PEM
	X509CertificateHolder signerCert = CertificateUtils
		.fromPEM(codeSigningCertificate.certificate());

	Collection<X509CertificateHolder> chainCerts = CertificateUtils
		.toCertificateHolders(codeSigningCertificate.certificateChain());

	// Build certificate chain including code-signing cert and intermediate certs
	Collection<X509CertificateHolder> certChain = new ArrayList<> ();
	certChain.add(signerCert);

	// Parse certificate chain
	for (X509CertificateHolder chainCert : chainCerts) {
		if (!chainCert.equals(signerCert)) {
			certChain.add(chainCert);
		}
	}

	// Create detached CMS signature
	CMSCodeSigningObject cmsCodeSigningObject = CMSCodeSigningObject
		.createDetachedSignature(
			codeSigningCMK,
			ML_DSA_65_ALGORITHM_FAMILY,
			<DATA_TO_SIGN>,
			signerCert,
			certChain);

The code-signing object is written to disk in signature-MLDSA65.p7s. You can inspect it with OpenSSL 3.5 or later:

openssl cms -cmsout -in signature-MLDSA65.p7s -inform DER -print
CMS_ContentInfo:
	contentType: pkcs7-signedData (1.2.840.113549.1.7.2)
	d.signedData:
		version: 1
		digestAlgorithms:
			algorithm: shake256 (2.16.840.1.101.3.4.2.12)
			parameter: <ABSENT>
		encapContentInfo:
			eContentType: pkcs7-data (1.2.840.113549.1.7.1)
			eContent: <ABSENT>
		certificates:
			d.certificate:
				cert_info:
					version: 2
					serialNumber: 0xD0B2937F5BABC80AD55C0A90E1DE7057
					signature:
						algorithm: ML-DSA-65 (2.16.840.1.101.3.4.3.18)
						parameter: <ABSENT>
					issuer:			CN=CodeSigningSubordinate-MLDSA65
					validity:
						notBefore: Oct 28 15:05:27 2025 GMT
						notAfter: Oct 28 16:05:26 2026 GMT
					subject:		CN=CodeSigningCertificate
					key:		X509_PUBKEY:
						algor:
							algorithm: ML-DSA-65 (2.16.840.1.101.3.4.3.18)
							parameter: <ABSENT>
						public_key:(0 unused bits)
							...
						issuerUID: <ABSENT>
						subjectUID: <ABSENT>
						extensions:
							object: X509v3 Basic Constraints (2.5.29.19)
							critical: FALSE
							value:
								0000 - 30 00 0.
                                    
							object: X509v3 Authority Key Identifier (2.5.29.35)
							critical: FALSE
							value:
								0000 - 30 16 80 14 b7 ef 2e c9-7a a8 7e b5 d60.......z.~..
								000d - 2d 9a 3f c7 a7 f8 9d 74-01 6a ef-.?....t.j.

                        	object: X509v3 Subject Key Identifier (2.5.29.14)
							critical: FALSE
							value:
								0000 - 04 14 7f 63 35 0c 56 f8-ed f1 2a df b5...c5.V...*..
								000d - 2e 7c f1 2c d9 a0 0e 63-b6.|.,...c.

                         	object: X509v3 Key Usage (2.5.29.15)
							critical: TRUE
							value:
								0000 - 03 02 07 80....
                                    
							object: X509v3 Extended Key Usage (2.5.29.37)
							critical: TRUE
							value:
								0000 - 30 0a 06 08 2b 06 01 05-05 07 03 030...+.......
					sig_alg:
						algorithm: ML-DSA-65 (2.16.840.1.101.3.4.3.18)
						parameter: <ABSENT>
					signature:(0 unused bits)
						...
		d.certificate:
			cert_info:
			version: 2
			serialNumber: 29577999257397559174219641462943780786
			signature:
				algorithm: ML-DSA-65 (2.16.840.1.101.3.4.3.18)
				parameter: <ABSENT>
				issuer:			CN=CodeSigningRoot-MLDSA65
				[...]
                
		d.certificate:
			cert_info:
			version: 2
			serialNumber: 0xB9419A2C5D2422B3A58A5B449546D74B
			signature:
				algorithm: ML-DSA-65 (2.16.840.1.101.3.4.3.18)
				parameter: <ABSENT>
				issuer:			CN=CodeSigningRoot-MLDSA65
				[...]
	crls:
		<ABSENT>
	signerInfos:
		version: 1
		d.issuerAndSerialNumber:
			issuer:				CN=CodeSigningSubordinate-MLDSA65
			serialNumber: 0xD0B2937F5BABC80AD55C0A90E1DE7057
		digestAlgorithm:
			algorithm: shake256 (2.16.840.1.101.3.4.2.12)
			parameter: <ABSENT>
		signedAttrs:
			object: contentType (1.2.840.113549.1.9.3)
			set:
				OBJECT:pkcs7-data (1.2.840.113549.1.7.1)

			object: signingTime (1.2.840.113549.1.9.5)
			set:
				UTCTIME:Oct 28 16:05:27 2025 GMT

			object: id-aa-CMSAlgorithmProtection (1.2.840.113549.1.9.52)
			set:
				SEQUENCE:
	0:d=0hl=2 l=26 cons: SEQUENCE
	2:d=1hl=2 l=11 cons:SEQUENCE
	4:d=2hl=2 l=9 prim:OBJECT:shake256
	15:d=1hl=2 l=11 cons:cont [ 1 ]
	17:d=2hl=2 l=9 prim:OBJECT:ML-DSA-65

        	object: messageDigest (1.2.840.113549.1.9.4)
			set:
				OCTET STRING:
					...
		signatureAlgorithm:
			algorithm: ML-DSA-65 (2.16.840.1.101.3.4.3.18)
			parameter: <ABSENT>
		signature:
			[...]

The CMS signature object directly encapsulates both the code-signing certificate and the subordinate CA certificate. It’s expected that the root certificate will reside in a customer-managed trust store. In addition to these certificates, the CMS object also contains the digest of the input data within the signedAttrs of the signerInfos in the ASN.1 structure. The digest algorithm is SHAKE256 and the OCTET STRING represents the binary digest itself. The use of ML-DSA in CMS is specified in RFC9882.

Note: Although this example uses one ML-DSA signature, some use cases might include dual signatures, a traditional and a quantum-resistant one. Such signed artifacts can be backwards compatible with legacy verifiers that support and can only verify the traditional signature. Upgraded verifiers can verify both signatures.

Verify signed code

Before loading a signed code artifact, its signature needs to be verified. That includes verifying the signature of the code and validating the certificate chain to the trusted root CA. The following code snippet from the main method within the file Runner.java is used for the certificate chain validation and the signature in the code object:

X509CertificateHolder rootCACertificate = CertificateUtils.fromPEM(rootCACertificatePEM); 
cmsCodeSigningObject.verifyDetachedSignature(<DATA_TO_SIGN>, rootCACertificate);

The preceding code retrieves the ML-DSA public key from the code-signing certificate; AWS access or credentials aren’t needed to validate the signature. Entities that have the root CA certificate loaded in their trust store can verify it without needing access to the AWS KMS verify API.

Note: The Runner.java implementation doesn’t use a certificate trust store that’s either part of a browser or part of a file system within the resident operating system of a device or a server. The trust store is placed in an instance of a Java class object for the purpose of this post. If you’re planning to use this code-signing example in a production system, you must change the implementation to use a trust store on the host. To do so, you can build and distribute a secure trust store that includes the root CA certificate.

Alternatively, OpenSSL 3.5 or later can be used to validate the detached signature of the provided input data file with root-ca-MLDSA65.pem, the provided root CA certificate from AWS Private CA.

openssl cms -verify -in signature-MLDSA65.p7s -content <input-data-file> \
            -CAfile root-ca-MLDSA65.pem -inform DER -purpose any \
            -binary -out /dev/null
CMS Verification successful

Note: Although this post focused on code-signing, AWS Private CA can enable post-quantum ML-DSA authentication for other private PKI use cases. In one scenario, applications outside of AWS can access AWS resources by temporarily using certificate-based authentication and swapping it with AWS credentials with AWS IAM Roles Anywhere. AWS IAM Roles Anywhere now supports ML-DSA PKIs like the one created in this post. In another scenario, an mTLS client or IKEv2/IPsec peer could use an ML-DSA certificate issued by AWS Private CA to be authenticated by a server or peer respectively who has been pre-provisioned with the post-quantum PKI root certificate.

Conclusion

This announcement marks an important milestone for post-quantum authentication. With the introduction of ML-DSA X.509 certificates in AWS Private CA, customers can bring quantum resistance to their private PKI use cases. These use cases include client authentication for mTLS or IKEv2/IPsec tunnels, IAM Roles Anywhere, or applications that use private PKI authentication. ML-DSA certificates with AWS Private CA and signing with AWS KMS also enable post-quantum code-singing and establishing post-quantum long-lived roots of trust for devices designed to operate for a long time even after CRQCs became available. Learn more about post-quantum cryptography in general and the overall AWS plan to migrate to post-quantum cryptography.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on the AWS Security, Identity, & Compliance re:Post or contact AWS Support. For more details regarding AWS PQC efforts, refer to our PQC page.

Migrate encrypted Amazon EC2 instances across AWS Regions without sharing AWS KMS keys

2025-10-16 Rakesh Mannepalli

Post Syndicated from Rakesh Mannepalli original https://aws.amazon.com/blogs/compute/migrate-encrypted-amazon-ec2-instances-across-aws-regions-without-sharing-aws-kms-keys/

At AWS, we’ve designed our global infrastructure with isolated AWS Regions to help you achieve high fault tolerance and stability for your applications. These AWS Regions are organized into partitions, each with distinct network and security boundaries.

As your business evolves, you might need to migrate workloads between AWS Regions. Perhaps you’re looking to reduce latency for users in new geographic areas, meet Region-specific compliance requirements, or you’re an ISV expanding your product’s availability. Whatever your motivation, cross-Region migration needs careful planning, especially when dealing with encrypted resources.

When migrating Amazon Elastic Compute Cloud (Amazon EC2) instances with encrypted Amazon Elastic Block Storage (Amazon EBS) volumes across AWS Regions with in the same account or a different account, you face a particular challenge: AWS Key Management Service (AWS KMS) keys are AWS Region-specific and cannot be shared across AWS Regions. This post provides a step-by-step approach to successfully migrate your encrypted EC2 instances without compromising your security posture by sharing your KMS keys.

Solution overview

The following diagram and steps are an overview of how an EC2 instance can be migrated to a different Region in a different account without sharing the KMS keys.

Figure 1:Design to migrate EC2 between two accounts

Prerequisites

The following prerequisites are necessary to complete this solution:

Create an S3 bucket in both the source and target Region.
Configure the target account Amazon S3 bucket with the following policy to copy the Amazon Machine Image (AMI) file between two accounts:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:sts::1234567891:assumed-role/<rolename>"
            },
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "arn:aws:s3:::<target-bucket-name>/*"
        }
    ]
}

Implementation steps

Now based on the above architecture, you are implementing the follow steps to move your EC2 instance from the source account to the target account

Create an AMI of the server that you want to move to a different Region in the same account or different account.
1. Choose the server
2. Choose Actions, Image and templates, and Create image.
  
  Figure 2: steps to create an AMI
3. Fill the details and choose Create Image.
  
  Figure 3: Confirming AMI creation attributes
Check the status of the AMI by choosing AMI ID or under AMI on your left-hand side menu, wait until the status shows as Available.

Figure 4: AMI availability status

Run the following command using AWS CloudShell from the AWS console.

aws ec2 create-store-image-task --image-id ami-xxxxxxxxx –bucket <bucket_name>

You can check the status of the job using the following command to make sure it is completed.
```
aws ec2 describe-store-image-tasks
```
Figure 5: CloudShell command execution
Now you can see your AMI bin file in the S3 bucket.

Figure 6: .bin file in the source S3 bucket
Copy the AMI bin to the target S3 bucket using the following command from the CloudShell in the source account.
```
aws s3 cp s3://<source_bucket>/ami-000xxxxxxxxx.bin s3://<target_bucket>/
```
When the copy job is completed, validate the AMI’s availability in .bin format in the target AWS account S3 bucket.

Figure 7: .bin file in the target S3 bucket
Now restore the .bin file as an AMI in the target account by running the following command in the target account CloudShell.
```
aws ec2 create-restore-image-task --object-key ami-xxxx.bin --bucket <target_bucket> --name "<AMI_name>"
```
Figure 8: CloudShell command execution
Check the availability of the AMI under the EC2 section in the target. You should find a new AMI ID along with the source Region information.

Figure 9: AMI created in the target account
Launch the instance using the migrated AMI in the target Region.

Figure 10: Launched EC2 instance in the target account

Limitations

Following are the limitations with this process:

To store an AMI, your AWS account must either own the AMI and its snapshots, or the AMI and its snapshots must be shared directly with your account. You can’t store an AMI if it is only publicly shared.
Only Amazon EBS-backed AMIs can be stored using these APIs.
Paravirtual (PV) AMIs are not supported.
The size of an AMI (before compression) that can be stored is limited to 5,000 GB.
Quota on store image requests: 1,200 GB of storage work (snapshot data) in progress.
Quota on restore image requests: 600 GB of restore work (snapshot data) in progress.
For the duration of the store task, the snapshots must not be deleted and the AWS Identity and Access Management (IAM) principal doing the store must have access to the snapshots, otherwise the store process fails.
You can’t create multiple copies of an AMI in the same S3 bucket.
An AMI that is stored in an S3 bucket can’t be restored with its original AMI ID. You can mitigate this by using AMI aliasing.
Currently the store and restore APIs are only supported by using the AWS Command Line Interface (AWS CLI), AWS SDKs, and Amazon EC2 API. You can’t store and restore an AMI using the Amazon EC2 console.

Clean up resources

When you have successfully deployed the server in the target Region you can delete the S3 buckets that were created for this migration. You can also terminate EC2 and delete associated EBS volumes and snapshots if you do not need them to avoid additional cost.

Conclusion

In this post, we showed you how to migrate an Amazon EC2 instances into another Region in a different account without sharing any AWS KMS keys in a secured manner.

AWS IAM Identity Center now supports customer-managed KMS keys for encryption at rest

2025-10-06 Sébastien Stormacq

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/aws-iam-identity-center-now-supports-customer-managed-kms-keys-for-encryption-at-rest/

Starting today, you can use your own AWS Key Management Service (AWS KMS) keys to encrypt identity data, such as user and group attributes, stored in AWS IAM Identity Center organization instances.

Many organizations operating in regulated industries need complete control over encryption key management. While Identity Center already encrypts data at rest using AWS-owned keys, some customers require the ability to manage their own encryption keys for audit and compliance purposes.

With this launch, you can now use customer-managed KMS keys (CMKs) to encrypt Identity Center identity data at rest. CMKs provide you with full control over the key lifecycle, including creation, rotation, and deletion. You can configure granular access controls to keys with AWS Key Management Service (AWS KMS) key policies and IAM policies, helping to ensure that only authorized principals can access your encrypted data. At launch time, the CMK must reside in the same AWS account and Region as your IAM Identity Center instance. The integration between Identity Center and KMS provides detailed AWS CloudTrail logs for auditing key usage and helps meet regulatory compliance requirements.

Identity Center supports both single-Region and multi-Region keys to match your deployment needs. While Identity Center instances can currently only be deployed in a single Region, we recommend using multi-Region AWS KMS keys unless your company policies restrict you to single-Region keys. Multi-Region keys provide consistent key material across Regions while maintaining independent key infrastructure in each Region. This gives you more flexibility in your encryption strategy and helps future-proof your deployment.

Let’s get started
Let’s imagine I want to use a CMK to encrypt the identity data of my Identity Center organization instance. My organization uses Identity Center to give employees access to AWS managed applications, such as Amazon Q Business or Amazon Athena.

As of today, some AWS managed applications cannot be used with Identity Center configured with a customer managed KMS key. See AWS managed applications that you can use with Identity Center to keep you updated with the ever evolving list of compatible applications.

The high-level process requires first to create a symmetric customer managed key (CMK) in AWS KMS. The key must be configured for encrypt and decrypt operations. Next, I configure the key policies to grant access to Identity Center, AWS managed applications, administrators, and other principals who need access the Identity Center and IAM Identity Center service APIs. Depending on your usage of Identity Center, you’ll have to define different policies for the key and IAM policies for IAM principals. The service documentation has more details to help you cover the most common use cases.

This demo is in three parts. I first create a customer managed key in AWS KMS and configure it with permissions that will authorize Identity Center and AWS managed applications to use it. Second, I update the IAM policies for the principals that will use the key from another AWS account, such as AWS applications administrators. Finally, I configure Identity Center to use the key.

Part 1: Create the key and define permissions

First, let’s create a new CMK in AWS KMS.

The key must be in the same AWS Region and AWS account as the Identity Center instance. You must create the Identity Center instance and the key in the management account of your organization within AWS Organization.

I navigate to the AWS Key Management Service (AWS KMS) console in the same Region as my Identity Center instance, then I choose Create a key. This launches me into the key creation wizard.

Under Step 1–Configure key, I select the key type–either Symmetric (a single key used for both encryption and decryption) or Asymmetric (a public-private key pair for encryption/decryption and signing/verification). Identity Center requires symmetric keys for encryption at rest. I select Symmetric.

For key usage, I select Encrypt and decrypt which allows the key to be used only for encrypting and decrypting data.

Under Advanced options, I select KMS – recommended for Key material origin, so AWS KMS creates and manages the key material.

For Regionality, I choose between Single-Region or Multi-Region key. I select Multi-Region key to allow key administrators to replicate the key to other Regions. As explained already, Identity Center doesn’t require this today but it helps to future-proof your configuration. Remember that you can not transform a single-Region key to a multi-Region one after its creation (but you can change the key used by Identity Center).

Then, I choose Next to proceed with additional configuration steps, such as adding labels, defining administrative permissions, setting usage permissions, and reviewing the final configuration before creating the key.

Under Step 2–Add Labels, I enter an Alias name for my key and select Next.

In this demo, I am editing the key policy by adding policy statements using templates provided in the documentation. I skip Step 3 and Step 4 and navigate to Step 5–Edit key policy.

Identity Center requires, at the minimum, permissions allowing Identity Center and its administrators to use the key. Therefore, I add three policy statements, the first and second authorize the administrators of the service, the third one to authorize the Identity Center service itself.

{
	"Version": "2012-10-17",
	"Id": "key-consolepolicy-3",
	"Statement": [
		{
			"Sid": "Allow_IAMIdentityCenter_Admin_to_use_the_KMS_key_via_IdentityCenter_and_IdentityStore",
			"Effect": "Allow",
			"Principal": {
				"AWS": "ARN_OF_YOUR_IDENTITY_CENTER_ADMIN_IAM_ROLE"
			},
			"Action": [
				"kms:Decrypt",
				"kms:Encrypt",
				"kms:GenerateDataKeyWithoutPlaintext"
			],
			"Resource": "*",
			"Condition": {
				"StringLike": {
					"kms:ViaService": [
						"sso.*.amazonaws.com",
						"identitystore.*.amazonaws.com"
					]
				}
			}
		},
		{
			"Sid": "Allow_IdentityCenter_admin_to_describe_the_KMS_key",
			"Effect": "Allow",
			"Principal": {
				"AWS": "ARN_OF_YOUR_IDENTITY_CENTER_ADMIN_IAM_ROLE"
			},
			"Action": "kms:DescribeKey",
			"Resource": "*"
		},
		{
			"Sid": "Allow_IdentityCenter_and_IdentityStore_to_use_the_KMS_key",
			"Effect": "Allow",
			"Principal": {
				"Service": [
					"sso.amazonaws.com",
					"identitystore.amazonaws.com"
				]
			},
			"Action": [
				"kms:Decrypt",
				"kms:ReEncryptTo",
				"kms:ReEncryptFrom",
				"kms:GenerateDataKeyWithoutPlaintext"
			],
			"Resource": "*",
            "Condition": {
    	       "StringEquals": { 
                      "aws:SourceAccount": "<Identity Center Account ID>" 
	           }
            }		
		},
		{
			"Sid": "Allow_IdentityCenter_and_IdentityStore_to_describe_the_KMS_key",
			"Effect": "Allow",
			"Principal": {
				"Service": [
					"sso.amazonaws.com",
					"identitystore.amazonaws.com"
				]
			},
			"Action": [
				"kms:DescribeKey"
			],
			"Resource": "*"
		}		
	]
}

I also have to add additional policy statements to allow my use case: the use of AWS managed applications. I add these two policy statements to authorize AWS managed applications and their administrators to use the KMS key. The document lists additional use cases and their respective policies.

{
    "Sid": "Allow_AWS_app_admins_in_the_same_AWS_organization_to_use_the_KMS_key",
    "Effect": "Allow",
    "Principal": "*",
    "Action": [
        "kms:Decrypt"
    ],
    "Resource": "*",
    "Condition": {
        "StringEquals" : {
           "aws:PrincipalOrgID": "MY_ORG_ID (format: o-xxxxxxxx)"
        },
        "StringLike": {
            "kms:ViaService": [
                "sso.*.amazonaws.com", "identitystore.*.amazonaws.com"
            ]
        }
    }
},
{
   "Sid": "Allow_managed_apps_to_use_the_KMS_Key",
   "Effect": "Allow",
   "Principal": "*",
   "Action": [
      "kms:Decrypt"
    ],
   "Resource": "*",
   "Condition": {
      "Bool": { "aws:PrincipalIsAWSService": "true" },
      "StringLike": {
         "kms:ViaService": [
             "sso.*.amazonaws.com", "identitystore.*.amazonaws.com"
         ]
      },
      "StringEquals": { "aws:SourceOrgID": "MY_ORG_ID (format: o-xxxxxxxx)" }
   }
}

You can further restrict the key usage to a specific Identity Center instance, specific application instances, or specific application administrators. The documentation contains examples of advanced key policies for your use cases.

To help protect against IAM role name changes when permission sets are recreated, use the approach described in the Custom trust policy example.

Part 2: Update IAM policies to allow use of the KMS key from another AWS account

Any IAM principal that uses the Identity Center service APIs from another AWS account, such as Identity Center delegated administrators and AWS application administrators, need an IAM policy statement that allows use of the KMS key via these APIs.

I grant permissions to access the key by creating a new policy and attaching the policy to the IAM role relevant for my use case. You can also add these statements to the existing identity-based policies of the IAM role.

To do so, after the key is created, I locate its ARN and replace the key_ARNin the template below. Then, I attach the policy to the managed application administrator IAM principal. The documentation also covers IAM policies that grants Identity Center delegated administrators permissions to access the key.

Here is an example for managed application administrators:

{
      "Sid": "Allow_app_admins_to_use_the_KMS_key_via_IdentityCenter_and_IdentityStore",
      "Effect": "Allow",
      "Action": 
        "kms:Decrypt",
      "Resource": "<key_ARN>",
      "Condition": {
        "StringLike": {
          "kms:ViaService": [
            "sso.*.amazonaws.com",
            "identitystore.*.amazonaws.com"
          ]
        }
      }
    }

The documentation shares IAM policies template for the most common use cases.

Part 3: Configure IAM Identity Center to use the key

I can configure a CMK either during the enablement of an Identity Center organization instance or on an existing instance, and I can change the encryption configuration at any time by switching between CMKs or reverting to AWS-owned keys.

Please note that an incorrect configuration of KMS key permissions can disrupt Identity Center operations and access to AWS managed applications and accounts through Identity Center. Proceed carefully to this final step and ensure you have read and understood the documentation.

After I have created and configured my CMK, I can select it under Advanced configuration when enabling Identity Center.

To configure a CMK on an existing Identity Center instance using the AWS Management Console, I start by navigating to the Identity Center section of the AWS Management Console. From there, I select Settings from the navigation pane, then I select the Management tab, and select Manage encryption in the Key for encrypting IAM Identity Center data at rest section.

At any time, I can select another CMK from the same AWS Account, or switch back to an AWS-managed key.

After choosing Save, the key change process takes a few seconds to complete. All service functionalities continue uninterrupted during the transition. If, for whatever reasons, Identity Center can not access the new key, an error message will be returned and Identity Center will continue to use the current key, keeping your identity data encrypted with the mechanism it is already encrypted with.

Things to keep in mind
The encryption key you create becomes a crucial component of your Identity Center. When you choose to use your own managed key to encrypt identity attributes at rest, you have to verify the following points.

Have you configured the necessary permissions to use the KMS key? Without proper permissions, enabling the CMK may fail or disrupt IAM Identity Center administration and AWS managed applications.
Have you verified that your AWS managed applications are compatible with CMK keys? For a list of compatible applications, see AWS managed applications that you can use with IAM Identity Center. Enabling CMK for Identity Center that is used by AWS managed applications incompatible with CMK will result in operational disruption for those applications. If you have incompatible applications, do not proceed.
Is your organization using AWS managed applications that require additional IAM role configuration to use the Identity Center and Identity Store APIs? For each such AWS managed application that’s already deployed, check the managed application’s User Guide for updated KMS key permissions for IAM Identity Centre usage and update them as instructed to prevent application disruption.

For brevity, the KMS key policy statements in this post omit the encryption context, which allows you to restrict the use of the KMS key to Identity Center including a specific instance. For your production scenarios, you can add a condition like this for Identity Center:

"Condition": {
   "StringLike": {
      "kms:EncryptionContext:aws:sso:instance-arn": "${identity_center_arn}",
      "kms:ViaService": "sso.*.amazonaws.com"
    }
}

or this for Identity Store:

"Condition": {
   "StringLike": {
      "kms:EncryptionContext:aws:identitystore:identitystore-arn": "${identity_store_arn}",
      "kms:ViaService": "identitystore.*.amazonaws.com"
    }
}

Pricing and availability
Standard AWS KMS charges apply for key storage and API usage. Identity Center remains available at no additional cost.

This capability is now available in all AWS commercial Regions, AWS GovCloud (US), and AWS China Regions. To learn more, visit the IAM Identity Center User Guide.

We look forward to learning how you use this new capability to meet your security and compliance requirements.

— seb

Simplify multi-tenant encryption with a cost-conscious AWS KMS key strategy

2025-08-22 Itay Meller

Post Syndicated from Itay Meller original https://aws.amazon.com/blogs/architecture/simplify-multi-tenant-encryption-with-a-cost-conscious-aws-kms-key-strategy/

Organizations face diverse challenges when it comes to managing encryption keys. While some scenarios demand strict separation, there are compelling use cases where a centralized approach can streamline operations and reduce complexity. In this post, our focus is on a software-as-a-service (SaaS) provider scenario, but the principles we discuss can be adopted by large organization facing similar key management challenges.

Managing encryption across a multi-tenant, multi-service architecture presents a significant challenge. Many organizations find themselves struggling with the complexity and costs associated with provisioning separate AWS Key Management Service (AWS KMS) customer managed keys for each tenant and service. This approach, while secure, often leads to growing operational overhead and increased AWS KMS usage costs over time.

But what if there was a more efficient way?

In this post, we unveil a strategy that uses a single customer managed key (symmetric) per tenant across services. By the end of this post, you’ll learn:

How to implement a scalable, secure, and cost-effective encryption model
Techniques for using one customer managed key per tenant across multiple services and environments
Methods for encrypting tenant data in Amazon DynamoDB and other storage types while maintaining tenant isolation

Multi-tenant encryption requirements for SaaS providers

Data isolation is fundamental to multi-tenant SaaS architectures, serving both compliance requirements and customer confidence. Many SaaS providers need to encrypt sensitive information—from API keys and credentials to personal data—across storage solutions such as DynamoDB and Amazon Simple Storage Service (Amazon S3).

While these storage services provide default encryption at rest, they typically use a single shared key across data items. Consider DynamoDB in a shared pool model, where one table contains data from multiple tenants. In this setup, the tenant data is encrypted using the same AWS KMS Key, regardless of ownership.

KMS key represents a container for top-level key material and is uniquely defined within the KMS, for more information on the different keys involved when encrypting or decrypting data using KMS, see AWS KMS key hierarchy.

This shared-key approach often proves insufficient for SaaS providers operating under strict security and compliance frameworks. Some customers require:

Bring your own key (BYOK) capabilities
Logical isolation of their data through dedicated encryption keys

To meet these requirements, providers can implement customer-specific AWS KMS managed keys, helping to ensure that each customer’s sensitive data remains isolated and inaccessible to other tenants.

Alternatively, providers might consider a silo model with separate tables for each customer. However, this approach introduces its own challenges—as the tenant base grows, managing numerous individual tables becomes increasingly complex and service quota limits might become a constraint.

Managing growth: KMS key management at scale

When scaling a SaaS platform, empowering teams to develop services independently is crucial. A quick way to scale is to have each team develop independently using a dedicated account. This often leads to a decentralized approach where each service manages its own KMS keys per customer. However, this autonomy comes with hidden costs as your customer base and service portfolio expand.

The challenge of key proliferation

As the company grows, the number of keys multiplies with each new customer and service addition. This proliferation creates several organizational challenges:

Cost impact: A single AWS KMS key costs $1 monthly, increasing to a maximum of $3 per month with two or more key rotations.
Operational complexity: Managing many KMS keys across environments and accounts is error-prone and hard to scale.
Organizational waste: Duplicate efforts across teams because each develops and maintains their own code for managing customer key lifecycles.
Governance overhead: It becomes difficult to enforce consistent policies or track KMS key usage across multiple AWS accounts.

A streamlined approach

The solution lies in implementing a centralized key management strategy. One KMS key per tenant, maintained in a central AWS account. This approach effectively addresses the cost, operational, and governance challenges while maintaining security.

In the following sections, we explore how to implement this centralized approach and securely share KMS keys across various services and AWS accounts.

Solution overview: Centralizing tenant key management

At the heart of our solution lies a centralized tenant key management service (shown as Service A in the following figure). This service handles every aspect of customer KMS key lifecycle—from creation during tenant onboarding to managing aliases, access policies and deletion.

The service achieves secure, scalable key usage across the organization through cross-account AWS Identity and Access Management (IAM) access. It grants other services (for example, the customer-facing service in Account B in the following figure) a permission to perform specific encryption operations using tenant-specific KMS keys through role delegation. This implementation follows AWS best practices for cross-account access, utilizing IAM and AWS Security Token Service (AWS STS) role assumption as described in the AWS documentation and this blog post.

Architecture diagram showing centralizing tenant key management flow with JWT authentication, role assumption ,data encryption and saving in DynamoDB

Centralized key management in practice: Encrypting customer data

Let’s examine how this works in practice with a common scenario:

Service A: Our centralized tenant key management service in Account A
Service B: A customer-facing workload running in Account B

When a customer interacts with Service B, it needs to store sensitive information securely, whether that’s secrets, API keys, or license information in a DynamoDB table. Instead of relying on shared KMS keys or default encryption, Service B encrypts data using the customer’s dedicated KMS key managed by Service A. The process works through AWS Identity and Access Management (IAM) role delegation. Service B temporarily assumes a role (ServiceARole) in Account A, receiving fine-grained, scoped down permissions for the specific tenant’s KMS key. With these temporary credentials, Service B can perform client-side encryption operations on sensitive information using the AWS SDK or the AWS Encryption SDK.

In this blog post, we used Boto3. For more advanced use-cases requiring data key caching or keyrings, use the AWS Encryption SDK.

Solution walkthrough

Let’s expand the technical aspects of the solution depicted above. Assumptions and definitions:

Incoming requests include an authentication header with a JSON Web Token (JWT) that includes data identifying the current tenant’s ID. These tokens are signed by an identity provider, making sure that the JWT cannot be modified, and the tenant identity can be trusted.
Account A: Centralized key management service.
Account B: Business service that serves customer requests.
alias/customer-<tenant-id> is the format of the aliases in account A. Each alias points to the KMS key of the corresponding customer identified by value of <tenant-id>. Service A creates these aliases during tenant onboarding and deletes them during tenant offboarding.
ServiceARole: A role in Account A that can encrypt and decrypt a KMS key that has an alias prefixed with alias/customer-*. The permissions are scoped down further using session policies when ServiceBRole assumes ServiceARole.
ServiceBRole: A role in Account B that can assume ServiceARole in Account A to gain access to the customer’s KMS key. This will be the AWS Lambda function’s execution role.

Note that Service B’s compute layer in this case is a Lambda function, but the solution works for other compute architectures. Let’s go over the flow in more detail:

Use service with JWT

A customer who belongs to a tenant signs in to the SaaS solution and is given a JWT that identifies its tenants with a tenant ID (<tenant-id>). The customer makes an action in ServiceB and sends sensitive information.

ServiceB handles the request (in a Lambda function), verifies the JWT token and wants to:

1. Encrypt the customer’s sensitive data
2. Save the encrypted data along with other data in the DynamoDB table

Assume role

In this example, the Lambda function uses its execution role credentials to assume the ServiceA role in the ServiceA account. Another way to grant cross-account access to KMS keys is by using KMS grants, to learn more, see Allowing users in other accounts to use a KMS key.

Let’s review the ServiceRoleA IAM policy:

Grants encrypt and decrypt access to a KMS key using the alias/customer-* pattern.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowKMSByAlias",
      "Effect": "Allow",
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:GenerateDataKey*"
      ],
      "Resource": "*",
      "Condition": {
        "StringLike": {
          "kms:RequestAlias": "alias/customer-*"
        }
      }
    }
  ]
}

To encrypt tenant secrets securely and at scale, we grant application roles cross-account access to KMS keys—but only through their alias, which maps to a tenant identifier present in their JWT authentication token, enforcing strong isolation.

You can control access to KMS keys based on the aliases that are associated with each KMS key. To do so, use the kms:RequestAlias and kms:ResourceAliases condition keys as specified in the Use aliases to control access to KMS keys.

In addition, the trust relationship policy of the ServiceARole allows the ServiceBRole in account B to assume it:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<ACCOUNT_B_ID>:role/ServiceBRole"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Depending on your environment, you can add additional conditions to this trust policy to further reduce the scope of who can assume this role. For more information, see IAM and AWS STS condition context keys.

Then, each KMS customer managed key will have the following policy. For example, a KMS key for a customer with <tenant-id>: 123 will have a policy that restricts access to the key using the specific customer alias and only through ServiceRoleA.

{
  "Version": "2012-10-17",
  "Id": "TenantKeyPolicy",
  "Statement": [
    {
      "Sid": "AllowServiceARoleViaAlias",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<ACCOUNT_A_ID>:role/ServiceARole"
      },
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:GenerateDataKey*"
      ],
      "Resource": "*",
      "Condition": {
        "StringLike": {
          "kms:RequestAlias": "alias/customer-123"
        }
      }
    }
  ]
}

The following is a Python code example demonstrating how Service B dynamically assumes a role in Account A to encrypt data for a specific tenant using a session-scoped IAM policy that allows access only to that tenant’s KMS key alias.

This pattern follows the same principles outlined in Isolating SaaS Tenants with Dynamically Generated IAM Policies. The idea is to generate and attach a tenant-specific IAM policy at runtime, granting the minimum required permissions to operate on tenant-owned resources—in this case, a KMS key alias. The credentials will allow the Lambda function to use only the KMS key that belongs to a customer (identified by tenant_id).

We will call the assume_role_for_tenant for every tenant.

The condition of "StringEquals" - "kms:RequestAlias": alias is the magical AWS STS sauce, it restricts ServiceB to use the current tenant’s alias in its encryption SDK calls and relies on alias authorization

import boto3
def assume_role_for_tenant(tenant_id: str):
    alias = f"alias/customer-{tenant_id}"
    # Session policy scoped to only the specific alias
    session_policy = {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "kms:Encrypt",
                    "kms:Decrypt",
                    "kms:GenerateDataKey*"
                ],
                "Resource": "*",
                "Condition": {
                    "StringEquals": {
                        "kms:RequestAlias": alias
                    }
                }
            }
        ]
    }
    # Assume ServiceARole in Account A with inline session policy
    sts = boto3.client("sts")
    assumed = sts.assume_role(
        RoleArn="arn:aws:iam::<ACCOUNT_A_ID>:role/ServiceARole",
        RoleSessionName=f"Tenant{tenant_id}Session",
        Policy=json.dumps(session_policy)
    )
    return assumed["Credentials"]

Encrypt data and save in DynamoDB

Now, what remains to do is use the assumed role credentials and use AWS SDK to encrypt the sensitive customer data and store it in the DynamoDB table.

# Use temporary credentials to create a KMS client
    creds = assume_role_for_tenant(tenant_id, plaintext)
    kms = boto3.client(
        "kms",
        region_name="us-east-1",
        aws_access_key_id=creds["AccessKeyId"],
        aws_secret_access_key=creds["SecretAccessKey"],
        aws_session_token=creds["SessionToken"]
    )
    # Encrypt using the alias
    response = kms.encrypt(
        KeyId= f"alias/customer-{tenant_id}"
        Plaintext=plaintext
    )
    # store response["CiphertextBlob"] in DynamoDB table

This post doesn’t address isolation between different services, only between tenants. If such service isolation is required, you can use encryption context, an optional set of non-secret key/value pairs that can contain additional contextual information about the data, for example the service identifier. This helps ensure that services can only encrypt or decrypt data using the relevant service encryption context.

Benefits of centralized key management

Let’s examine how this solution addresses our earlier challenges.

Tenant isolation by design

Despite reducing the total number of KMS keys, we maintain strict tenant isolation. Each customer’s sensitive data remains encrypted with their dedicated key, identified by a unique alias (alias/customer-<tenant-id>). Access control to the tenant key is tightly managed through IAM role delegation, following least privilege principles:

Service A exclusively controls the management of the tenants’ KMS keys.
Service B can only assume a role that grants restricted encrypt, decrypt, and GenerateDataKey access for the customer managed key designated by the alias: alias/customer-<tenant-id>.

Optimized cost management

Our approach significantly reduces costs by moving from multiple service-specific KMS keys per tenant to a single KMS key per tenant that is shared securely across services and environments. This behavior introduces a new centralized account (Account A) that provides access to encryption keys under the right circumstances. It is important to understand AWS STS limits, specifically for AssumeRole calls and consider temporary IAM credentials caching mechanisms if those limits become a bottleneck. Additionally, if KMS limits are a bottleneck, consider using data key caching by using the AWS Encryption SDK.

Streamlined operations and governance

By centralizing key management in Service A, you can achieve:

Consistent KMS key lifecycle management across the organization
Improved audit capabilities using AWS CloudTrail to better understand key access patterns by service
Reduced operational overhead
Simplified compliance monitoring

The only additional complexity is the initial cross-account role delegation setup between Service A and other services. After being established, this framework can be scaled to accommodate new tenants and services.

It’s best to encapsulate the assume-role logic, policy generation, and AWS SDK client initialization within a shared organization-wide SDK. This abstraction reduces cognitive load for developers and minimizes the risk of misconfigurations. You can take it a step further by exposing high-level utility functions such as encrypt_tenant_data() and decrypt_tenant_data(), hiding the underlying complexity while promoting secure and consistent usage patterns across teams.

Conclusion

In this post, we explored an efficient approach to managing encryption keys in a multi-tenant SaaS environment through centralization. We examined common challenges faced by growing SaaS providers, including key proliferation, rising costs, and operational complexity across multiple AWS accounts and services. The solution, centralizing key management, uses AWS best practices for IAM role delegation and cross-account access, enabling organizations to maintain security and compliance while reducing operational overhead. By implementing this approach, SaaS providers or large organizations facing similar challenges can effectively manage their encryption infrastructure as they scale, without compromising on security or increasing complexity.

About the authors

How to create post-quantum signatures using AWS KMS and ML-DSA

2025-06-13 Jake Massimo

Post Syndicated from Jake Massimo original https://aws.amazon.com/blogs/security/how-to-create-post-quantum-signatures-using-aws-kms-and-ml-dsa/

As the capabilities of quantum computing evolve, AWS is committed to helping our customers stay ahead of emerging threats to public-key cryptography. Today, we’re announcing the integration of FIPS 204: Module-Lattice-Based Digital Signature Standard (ML-DSA) into AWS Key Management Service (AWS KMS). Customers can now create and use ML-DSA keys through the same familiar AWS KMS APIs they use today for digital signatures, including CreateKey, Sign, and Verify operations. This new feature is generally available and you can use ML-DSA in the following AWS Regions: US West (N. California), and Europe (Milan) with the remaining commercial Regions to follow in the coming days. This launch is part of our broader AWS post-quantum cryptography migration plan, which we covered in our recent blog post. In this post, we guide you through creating ML-DSA keys and post-quantum signatures with AWS KMS.

Many organizations use AWS KMS to cryptographically sign firmware, operating systems, applications, or other artifacts. With ML-DSA support in AWS KMS, you can now generate and use post-quantum keys for signing operations within FIPS-140-3 Level 3 certified HSMs. By implementing ML-DSA signatures now, you can help make sure that your systems remain secure throughout their operational lifetime, even if cryptographically relevant quantum computers become available. This is especially important for manufacturers who install long-lived roots of trust during production—whether embedded directly in hardware or in devices that might remain offline for extended periods. In both cases, cryptographic signatures cannot be easily updated after deployment, making post-quantum readiness critical for the entire operational lifetime of these systems.

What’s new

AWS KMS offers three new AWS KMS key specs: ML_DSA_44, ML_DSA_65, and ML_DSA_87, which you can use with the new post-quantum SigningAlgorithm ML_DSA_SHAKE_256. Like our other signing algorithms, this name includes the hash function that’s used within the signature scheme to digest messages before signing or verification. In this case, the hash function used is SHAKE256—part of the SHA-3 family of hash functions standardized by NIST in FIPS 202.

Table 1 shows the details for each key spec, including their NIST security categories and corresponding key sizes in bytes. Each ML-DSA key spec represents a balance between security strength and resource requirements. ML-DSA-44 is suitable for applications requiring security comparable to classical 128-bit encryption, while ML-DSA-65 and ML-DSA-87 provide progressively stronger security levels equivalent to classical 192-bit and 256-bit encryption, respectively. As you move up in security levels, you’ll notice corresponding increases in key and signature sizes, enabling you to choose the key spec that best matches your security needs and engineering constraints.

Key spec	NIST security Level	Public key (B)	Private key (B)	Signature (B)
ML_DSA_44	1 (equivalent to 128-bit security)	1312	2560	2420
ML_DSA_65	3 (equivalent to 192-bit security)	1952	4032	3309
ML_DSA_87	5 (equivalent to 256-bit security)	2592	4896	4627

When using the AWS KMS Sign API with a RAW MessageType, the message to be signed is limited to 4096 bytes. For messages larger than 4096 bytes, pre-processing the message outside of AWS KMS to create what’s known as µ (mu) is required to generate a smaller-sized message input to the KMS Sign API. This external mu process pre-digests the message using the public key of the ML-DSA signing key pair to create a message size of 64 bytes. To support this launch, we’ve added a new message type in the KMS Sign API—EXTERNAL_MU—that can be used with ML-DSA signing or verification calls to indicate when a message has been pre-processed using µ (mu) before submitted to AWS KMS.

In the following sections, we include more information about constructing external mu and demonstrate basic AWS KMS operations with ML-DSA. We cover key creation, signature generation and verification, and both RAW and EXTERNAL_MU signing modes. Note that the produced RAW or EXTERNAL_MU ML-DSA signatures are identical when the same message and signing key are used.

Creating ML-DSA keys

To start, create an asymmetric AWS KMS key using the AWS Command Line Interface (AWS CLI) example command:

aws kms create-key --key-spec ML_DSA_65 --key-usage SIGN_VERIFY

This command will return a response similar to the following:

{
    "KeyMetadata": {
        "Origin": "AWS_KMS",
        "KeyId": "1234abcd-12ab-34cd-56ef-1234567890ab",
        "MultiRegion": false,
        "Description": "",
        "KeyManager": "CUSTOMER",
        "Enabled": true,
        "SigningAlgorithms": [
            "ML_DSA_SHAKE_256"
        ],
        "CustomerMasterKeySpec": "ML_DSA_65",
        "KeyUsage": "SIGN_VERIFY",
        "KeySpec": "ML_DSA_65",
        "KeyState": "Enabled",
        "CreationDate": 1748371316.734,
        "Arn": "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab",
        "AWSAccountId": "111122223333"
    }
}

Make note of the KeyId or Arn value from the response; you’ll need this to reference your key in subsequent signing operations. The response confirms that the creation of an ML_DSA_65 key configured for SIGN_VERIFY operations, which will use the ML_DSA_SHAKE_256 signing algorithm for signature operations.

Signing

In this section, we include some examples of ML-DSA signing and verifying a JSON Web Token (JWT) commonly used to transfer claims between parties for web authorization. In 2021, we described how to sign and verify JWTs with Elliptic Curve Digital Signature Algorithm (ECDSA), a classic asymmetric cryptographic algorithm (see How to verify AWS KMS signatures in decoupled architectures at scale). In the following examples, the token is instead signed with an ML-DSA private key managed by AWS KMS and verified either within AWS KMS or externally using OpenSSL.

The JWT content to be signed is from section 3.1 of RFC7519. More specifically, the JWT header is:

{"typ":"JWT",
 "alg":"ML-DSA-65"}

And the JWT claim set is:

{"iss":"joe",
 "exp":1748952000,
 "http://example.com/is_root":true}

You can produce the JWT message to be signed by using the Base64URL encoding of the header and payload as:

echo -n -e '{"typ":"JWT",\015\012 "alg":"ML-DSA-65"}' | \
	basenc --base64url -w 0 | \
	sed 's/=//g' ; echo -n "." ; echo -n -e '{"iss":"joe",\015\012 "exp":1748952000,\015\012 "http://example.com/is_root":true}' | \
	basenc --base64url -w 0 | sed 's/=//g' ; echo ""

This command will output the following Base64 to be signed with ML-DSA:

eyJ0eXAiOiJKV1QiLA0KICJhbGciOiJNTC1EU0EtNjUifQ.eyJpc3MiOiJqb2UiLA0KICJleHAiOjE3NDg5NTIwMDAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ

Note that the following examples output the ML-DSA signature produced on the message by using the ML-DSA private key managed by AWS KMS in a binary format. You need to convert them to Base64URL to use them in JWT, but various data encryption and signing formats can use these signatures. These include Cryptographic Message Syntax (CMS), CBOR Object Signing and Encryption (COSE), or image signing encodings for UEFI and Open Titan. While converting between binary and these formats is straightforward, support for the new algorithms might not be available in common cryptographic implementations of these signing formats at the time of this writing.

RAW ML-DSA signing (no external mu)

To sign a message of less than 4096 bytes in AWS KMS with ML-DSA, you can use the AWS CLI:aws kms sign \

aws kms sign \
    --key-id <1234abcd-12ab-34cd-56ef-1234567890ab> \
    --message ' eyJ0eXAiOiJKV1QiLA0KICJhbGciOiJNTC1EU0EtNjUifQ.eyJpc3MiOiJqb2UiLA0KICJleHAiOjE3NDg5NTIwMDAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ' \
    --message-type RAW \
    --signing-algorithm ML_DSA_SHAKE_256 \
    --output text \
    --query Signature | base64 --decode > ExampleSignature.bin

Make sure to replace the target-key-id value of <1234abcd-12ab-34cd-56ef-1234567890ab> with your KeyId. This command will produce a signature and write it to disk as ExampleSignature.bin.

After producing the signature, you can create the complete JWT (consisting of header, payload, and signature) with a single command:

echo -n "eyJ0eXAiOiJKV1QiLA0KICJhbGciOiJNTC1EU0EtNjUifQ.eyJpc3MiOiJqb2UiLA0KICJleHAiOjE3NDg5NTIwMDAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ." ; \
	basenc --base64url -w 0 ExampleSignature.bin | \
	sed 's/=//g' ; echo ""

This command will output a ready-to-use JWT in the format required by RFC 7519 and signed using AWS KMS:

eyJ0eXAiOiJKV1QiLA0KICJhbGciOiJNTC1EU0EtNjUifQ.eyJpc3MiOiJqb2UiLA0KICJleHAiOjE3NDg5NTIwMDAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ.<base64url of the signature as per RFC7519>

External mu ML-DSA signing

Note that AWS KMS imposes a 4096-byte limit on the size of the raw message when using the Sign API to minimize the latency of the response. In cases where the message to be signed is larger than 4096 bytes or if pre-digesting the external mu has performance advantages you need, you must use the EXTERNAL_MU message type instead of RAW in AWS KMS.

Before using the EXTERNAL_MU message type with the AWS KMS Sign API, you must locally perform a pre-hash calculation on your message. So, first, retrieve the public key from AWS KMS, and convert it to DER format using the following command (replace the example key ID with a valid key ID from your AWS account):

aws kms get-public-key \
    --key-id <1234abcd-12ab-34cd-56ef-1234567890ab> \
    --output text \
    --query PublicKey | base64 --decode > public_key.der

To construct the external mu digest:

Construct a message prefix (M`): M` = (domain separator || context length || context || Message).
In this example, set the domain separator value and context length as zero; this sets the context used in the signature as the empty string, which is the default.
Hash the public key then prepend it to the message prefix:
(SHAKE256(pk) || M’).
Hash to produce a 64-byte mu:
Mu = SHAKE256(SHAKE256(pk) || M’)

You can use a single OpenSSL 3.5 command to construct the digest:

{
    openssl asn1parse -inform DER -in public_key.der -strparse 17 -noout -out - 2>/dev/null |
    openssl dgst -provider default -shake256 -xoflen 64 -binary;
    printf '\x00\x00';
    echo -n "eyJ0eXAiOiJKV1QiLA0KICJhbGciOiJNTC1EU0EtNjUifQ.eyJpc3MiOiJqb2UiLA0KICJleHAiOjE3NDg5NTIwMDAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ"
} | openssl dgst -provider default -shake256 -xoflen 64 -binary > mu.bin

Now you can call AWS KMS to sign the 64-byte digest to produce the ML-DSA signature in file ExampleSignature.bin, making sure to set the MessageType to EXTERNAL_MU:

aws kms sign \
    --key-id 1234abcd-12ab-34cd-56ef-1234567890ab \
    --message fileb://mu.bin \
    --message-type EXTERNAL_MU \
    --signing-algorithm ML_DSA_SHAKE_256 \
    --output text \
    --query Signature | base64 --decode > ExampleSignature.bin

The final signed JWT token is identical to the one produced previously in RAW mode.

Signature verification using AWS KMS

In this section, we show you how to verify ML-DSA signatures using AWS KMS or locally in your own environment. We assume that you have an ML-DSA signature in ExampleSignature.bin, produced on the JWT content with the private key in AWS KMS and identified with KEY_ARN.

Note that, although the following examples demonstrate signature verification using public keys directly from AWS KMS, these same principles extend to certificate-based systems, such as a private PKI, in which public keys are embedded in end-entity certificates (of the signer). In such scenarios, verifiers would first verify the identity of the signer by validating the certificate chain ties to a trusted root, then use the public key of the end-entity certificate to verify the ML-DSA signature of the content. The IETF is standardizing ML-DSA for use in X.509 certificates through RFC draft draft-ietf-lamps-dilithium-certificates.

RAW ML-DSA verification

To verify the signature using AWS KMS, you can call the following command, replacing the example key-id with the same one you used to sign.

aws kms verify \
    --key-id <1234abcd-12ab-34cd-56ef-1234567890ab> \
    --message "eyJ0eXAiOiJKV1QiLA0KICJhbGciOiJNTC1EU0EtNjUifQ.eyJpc3MiOiJqb2UiLA0KICJleHAiOjE3NDg5NTIwMDAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ" \
    --message-type RAW \
    --signing-algorithm ML_DSA_SHAKE_256 \
    --signature fileb://ExampleSignature.bin

The response will return:

{
    "KeyId": "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab",
    "SignatureValid": true,
    "SigningAlgorithm": "ML_DSA_SHAKE_256"
}

The verification result is stored in the SignatureValid field.

External mu ML-DSA verification

If you have the external mu digest of the JWT content in mu.bin along with the signature and the corresponding keypair in AWS KMS, you can use the digest without having access to the entire message or calculating the digest again.

aws kms verify \
    --key-id <1234abcd-12ab-34cd-56ef-1234567890ab> \
    --message fileb://mu.bin \
    --message-type EXTERNAL_MU \
    --signing-algorithm ML_DSA_SHAKE_256 \
    --signature fileb://ExampleSignature.bin

To regenerate the external mu mu.bin from the message and the public key, see the External mu ML DSA signing section above.

Local signature verification using OpenSSL 3.5

If you want to reduce AWS KMS API consumption costs and better control the use of API quotas while keeping the security of AWS KMS-generated and stored keys for ML-DSA signature generation, you can verify ML-DSA signatures locally, outside of AWS KMS.

In this example, you use OpenSSL 3.5 to verify the signature in ExampleSignature.bin. You first must fetch the DER-encoded public key from AWS KMS in file public_key.der as shown in the External mu ML DSA signing section. OpenSSL 3.5 can then verify the signature on the message by using the public key.

echo -n "eyJ0eXAiOiJKV1QiLA0KICJhbGciOiJNTC1EU0EtNjUifQ.eyJpc3MiOiJqb2UiLA0KICJleHAiOjE3NDg5NTIwMDAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ" | \
	openssl dgst -verify public_key.der -signature ExampleSignature.bin

Successful verification will output: Verified OK

Conclusion

Today’s launch of ML-DSA support in AWS KMS marks an important milestone in our commitment to post-quantum cryptography. With three different security levels of ML-DSA in both raw and external digest modes, you have flexible options to meet your security requirements while preparing for the quantum computing era. The seamless integration with existing AWS KMS APIs makes it straightforward to incorporate quantum-resistant signatures into your applications today. This implementation is particularly valuable if you need to:

Meet FIPS 140-3 compliance requirements when using post-quantum cryptography.
Sign code, artifacts, documents or other data that need to remain trusted and verifiable for many years into the future, including the period after cryptographically relevant quantum computers exist.
Start post-quantum cryptography testing as part of your application development process using a cryptographic service such as AWS KMS that has previously been approved for use.

Learn more about post-quantum cryptography in general and the overall AWS plan to migrate to post-quantum cryptography.

How to use on-demand rotation for AWS KMS imported keys

2025-06-06 Jeremy Stieglitz

Post Syndicated from Jeremy Stieglitz original https://aws.amazon.com/blogs/security/how-to-use-on-demand-rotation-for-aws-kms-imported-keys/

Today, we’re announcing support for on-demand rotation of symmetric encryption AWS Key Management Service (AWS KMS) keys with imported key material (EXTERNAL origin). This new capability enables you to rotate the cryptographic key material of these keys without changing the key identifier (key ID or Amazon Resource Name (ARN)). Rotating keys helps you meet compliance requirements and security best practices that mandate periodic key rotation.

AWS KMS has long supported automatic key rotation for AWS KMS keys whose key material is generated by AWS KMS (AWS_KMS origin). Until now, AWS KMS customers who imported their own key material could not rotate that material without creating a new KMS key. This process called manual rotation required updating references to the older key identifiers. With today’s launch, the key ID of the imported key remains unchanged after rotation, so existing workloads are not disrupted. In this post, we tell you how the new capability works, look at key material expiry and deletion features unique to imported keys, and review pricing for this new feature.

How it works

When you create an AWS KMS key with EXTERNAL origin, AWS KMS assigns a fixed identifier to the key called the key ID. However, AWS KMS doesn’t generate key material for the cryptographic operations. You must import your own key material using the ImportKeyMaterial operation.

When you import key material, AWS KMS computes a unique key material identifier based on the key ID and the key material. Even if you import the same key material in different keys, AWS KMS will assign distinct key material identifiers. This computation uses a cryptographic hash so the key material identifier doesn’t reveal information about the key material itself. AWS KMS embeds this key material identifier in the ciphertext blob produced by symmetric encryption.

Until now, after you imported key material into an AWS KMS key, you could not import additional key material into that key to rotate the key. With this new feature launch, you can associate multiple imported key materials with a single, symmetric-encryption key. You can use the RotateKeyOnDemand operation to make the most recently imported key material the current key material. AWS KMS uses the current key material to generate new ciphertext. Unless deleted or expired, the other key materials remain available for decryption. When you present ciphertext for decryption, AWS KMS automatically selects the correct key material using the key material identifier embedded in the ciphertext.

To help improve the auditability that keys have rotated, we’ve added new identifiers in KMS API responses for the specific key material used. The KeyMaterialId is a new field that AWS KMS will return in addition to the KeyId. Similarly, the DescribeKey response for these keys now displays the identifier of the current key material as CurrentKeyMaterialId. The inclusion of the KeyMaterialId and CurrentKeyMaterialId in API responses makes key rotation more transparent.

Before we dive into the details, the following is an outline of the overall process to rotate an imported key:

Create a symmetric encryption KMS key with EXTERNAL origin
Import key material into the key using GetParametersForImport and ImportKeyMaterial APIs. The first key material becomes usable immediately. This part is unchanged and maintains backwards compatibility with the current behavior of AWS KMS.
Use the key to create ciphertext and decrypt it. You’ll notice the key material ID matches the CurrentKeyMaterialId displayed in the DescribeKey response.
When you want to rotate this key, import a second key material into the key. The ImportKeyMaterial API now has a new ImportType input parameter which lets you inform AWS KMS whether you are associating new key material with a key (--import-type NEW_KEY_MATERIAL) or re-importing previously associated key material (--import-type EXISTING_KEY_MATERIAL).
Use ListKeyRotations with --include-key-material ALL_KEY_MATERIAL to view both key materials. The key material state of the second key material will be PENDING_ROTATION.
Use the RotateKeyOnDemand operation to initiate on-demand key rotation.
Optionally, you can use the GetKeyRotationStatus operation to monitor the in-progress rotation. The response will contain OnDemandRotationStartDate only while the rotation is in progress.
Use ListKeyRotations with --include-key-material ALL_KEY_MATERIAL after rotation completes to view key materials associated with this key. The KeyMaterialState of the new key material you imported will change from PENDING_ROTATION to CURRENT. The key material state of the first key material will change from CURRENT to NON_CURRENT.
Use the key to create ciphertext and decrypt it. You’ll notice the CurrentKeyMaterialId is used for creating ciphertext, but the key material used for decryption is automatically determined by AWS KMS.

Using the AWS CLI for rotating an imported key

The following is a sample sequence of AWS KMS commands to exercise the import key rotation functionality using the AWS Command Line Interface (AWS CLI). The specific commands that follow work in Linux or MacOS environments and might need to be modified for use in a Windows environment. This functionality can also be exercised through the AWS SDKs. These operations, except for wrapping a key material for import, generate-data-key, and decrypt can be initiated in the AWS Management Console.

Step 1: Create a key and import key material

This section should be familiar to anyone who has used the existing import key functionality in AWS KMS.

Create a symmetric encryption key with EXTERNAL origin and save the key ARN. The initial state of this key is PendingImport.

export EXTERNAL_KEY1_ARN=$(aws kms create-key --origin EXTERNAL | tee /dev/stderr | jq -r .KeyMetadata.Arn)
{
    "KeyMetadata": {
        "AWSAccountId": "111122223333",
        "KeyId": "97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "Arn": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "CreationDate": "2025-05-08T13:55:04.605000-07:00",
        "Enabled": false,
        "Description": "",
        "KeyUsage": "ENCRYPT_DECRYPT",
        "KeyState": "PendingImport",
        "Origin": "EXTERNAL",
        "KeyManager": "CUSTOMER",
        "CustomerMasterKeySpec": "SYMMETRIC_DEFAULT",
        "KeySpec": "SYMMETRIC_DEFAULT",
        "EncryptionAlgorithms": [
            "SYMMETRIC_DEFAULT"
        ],
        "MultiRegion": false
    }
}

Generate a 256-bit (32-byte) key material to be imported. In the following command, we use OpenSSL to generate the imported key material.
```
openssl rand 32 > "KeyMaterial1.bin"
```

Use the get-parameters-for-import command to create the wrapping key and import token and save them to files. AWS KMS supports multiple wrapping algorithms; we use RSAES_OAEP_SHA_256 with a 4096-bit RSA key in the following example. The value of the ImportToken and PublicKey fields in the response has been trimmed for brevity.

export WRAPPING_ALGORITHM="RSAES_OAEP_SHA_256"
export WRAPPING_KEY_SPEC="RSA_4096"
export KMS_PARAMETERS_FOR_IMPORTED_KEY_MATERIAL1=$(aws kms get-parameters-for-import \
    --key-id "${EXTERNAL_KEY1_ARN}" \
    --wrapping-algorithm "${WRAPPING_ALGORITHM}" \
    --wrapping-key-spec "${WRAPPING_KEY_SPEC}" | tee /dev/stderr)
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "ImportToken": "AQECAHie/gkutEg+qLc1mscyCnHfNPS1aKVSxf6xd3PX05ny2wAADWIwgg1eBgkqhkiG9w0BBwaggg1PMIINSwIBADCCDUQGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMn8AovXPs/ 
…
O22maj4+9nDAgjVkxZQWhjA2VHEyORnw8Qb29X8gvjwmE8xhlNWbnM1zZlDcClDzfJriLVuoXAO92HK1Vihs5hiE8/9tu6DegtXfp28WVIVTttnGjkjdVmChC7cg7yZhs4xu1LzN39LLyR9Q1O/9EQYHbwYXgp6tpMt2JyGhH/lRt2kJl+BPUEfKNsWkoj0Cq3Y=",
    "PublicKey": "MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAvXskfFFVoZVaZrEuN8UK3vckGww8tJHvsIxLmqKEbFt7D3dp/Jh8AoaVm2n/kdcGD5Hm8NwOQC40jsRjJKTqiTSr8/y8XEmpB0qi68EtKM8RyEmz7u1R5Vn6uoIZxKb/WMvVME/7tntyU4/uhVTGUlrfZItAV+
…
nuGzOptwcaprtT2iNthNZ38UQ2orETbfwdG6yZPt1qS8Jk/O0H+KtkcsNLHrDUE9XjQX4WDfkNbf/fyqaCLUpSTdONTLXqE16pVieLgrXr1845mECAwEAAQ==",
    "ParametersValidTo": "2025-05-09T13:56:45.632000-07:00"
}

export PUBLIC_KEY_FOR_IMPORTED_KEY_MATERIAL1=$(echo "${KMS_PARAMETERS_FOR_IMPORTED_KEY_MATERIAL1}" | jq -r .PublicKey) 
export IMPORT_TOKEN_FOR_IMPORTED_KEY_MATERIAL1=$(echo "${KMS_PARAMETERS_FOR_IMPORTED_KEY_MATERIAL1}" | jq -r .ImportToken) 

echo "${PUBLIC_KEY_FOR_IMPORTED_KEY_MATERIAL1}" | base64 -d > "WrappingKeyForKeyMaterial1.bin"
echo "${IMPORT_TOKEN_FOR_IMPORTED_KEY_MATERIAL1}" | base64 -d > "ImportTokenForKeyMaterial1.bin"

Wrap the key material using the wrapping key. We use OpenSSL, a popular open source cryptographic library to illustrate this step. For a detailed explanation of this step, see the AWS KMS Developer Guide.

openssl pkeyutl -encrypt -in "KeyMaterial1.bin" -out "WrappedKeyMaterial1.bin" -inkey "WrappingKeyForKeyMaterial1.bin" -keyform DER -pubin -encrypt -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256 -pkeyopt rsa_mgf1_md:sha256

Import the key material. Optionally, you can assign a key material description. The description can be used to keep track of where the key material is durably maintained outside AWS KMS. This key material description is displayed alongside other information for this key material in the console and the ListKeyRotations API response. We also capture the key material ID from the response. In this example, the key material doesn’t expire. Optionally, you can set an expiration time.

export KEY_MATERIAL1_ID=$(aws kms import-key-material --key-id "${EXTERNAL_KEY1_ARN}" --encrypted-key-material fileb://WrappedKeyMaterial1.bin --import-token fileb://ImportTokenForKeyMaterial1.bin --expiration-model KEY_MATERIAL_DOES_NOT_EXPIRE --key-material-description "Q1 2025 key from HSM1" | tee /dev/stderr | jq .KeyMaterialId | tr -d \")
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241"
}

The key is now be enabled for use in cryptographic operations and the CurrentKeyMaterialId in the DescribeKey response should match ${KEY_MATERIAL1_ID}

echo "${KEY_MATERIAL1_ID}"
04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241

aws kms describe-key --key-id "${EXTERNAL_KEY1_ARN}"
{
    "KeyMetadata": {
        "AWSAccountId": "111122223333",
        "KeyId": "97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "Arn": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "CreationDate": "2025-05-08T13:55:04.605000-07:00",
        "Enabled": true,
        "Description": "",
        "KeyUsage": "ENCRYPT_DECRYPT",
        "KeyState": "Enabled",
        "Origin": "EXTERNAL",
        "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE",
        "KeyManager": "CUSTOMER",
        "CustomerMasterKeySpec": "SYMMETRIC_DEFAULT",
        "KeySpec": "SYMMETRIC_DEFAULT",
        "EncryptionAlgorithms": [
            "SYMMETRIC_DEFAULT"
        ],
        "MultiRegion": false,
        "CurrentKeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241"
    }
}

Use ListKeyRotations to view key materials associated with the key. There should only be one key material with the same ID as in ${KEY_MATERIAL1_ID} and with a key material state of CURRENT.

aws kms list-key-rotations --key-id "${EXTERNAL_KEY1_ARN}" --include-key-material ALL_KEY_MATERIAL
{
    "Rotations": [
        {
            "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
            "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241",
            "KeyMaterialDescription": "First key material",
            "ImportState": "IMPORTED",
            "KeyMaterialState": "CURRENT",
            "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE"
        }
    ],
    "Truncated": false
}

Step 2: Use the first key material to create and decrypt ciphertext

This step demonstrates how to verify the key material ID of your imported key in cryptographic operations.

Use the GenerateDataKey operation and capture the ciphertext. This operation returns a data key in both plaintext and ciphertext form. The KeyMaterialId in the response matches the identifier for the first key material.

export KEY1_CIPHERTEXT_BLOB1=$(aws kms generate-data-key --key-id "${EXTERNAL_KEY1_ARN}" --number-of-bytes 32 | tee /dev/stderr | jq -r .CiphertextBlob)
{
    "CiphertextBlob": "AQIBAHgE9c9Q7o8Ff5dLmCCrj6iDfSg2OoWo+O8GpGyTrcWyQQFGzFweTmSjHkrflCzUMvY+AAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMdNV4KMSitkzp9j0pAgEQgDuXnjXR/KvDWjf3KfTAksjAyQJETPoC7zBq6ND2n2c7IUT/EUn+cbmBhXx5P/EI3l9drqsP/6NS5rRYSw==",
    "Plaintext": "3OQAOySTM3/E7QHH3a+GGzEz3HKS30rUcvUep+uueas=",
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241",
    "KeyOrigin": "EXTERNAL"
}

Decrypt the ciphertext and compare it to the plaintext key. The KeyMaterialId in the response matches the identifier for the first key material. The plaintext in decrypt response matches the plaintext data key in the GenerateDataKey response.

aws kms decrypt --ciphertext-blob "${KEY1_CIPHERTEXT_BLOB1}"
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "Plaintext": "3OQAOySTM3/E7QHH3a+GGzEz3HKS30rUcvUep+uueas=",
    "EncryptionAlgorithm": "SYMMETRIC_DEFAULT",
    "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241",
    "KeyOrigin": "EXTERNAL"
}

Step 3: Import a second key material into the key for on-demand rotation

Import key rotations start with importing new key material into this key.

Generate a second key material (also 256 bits in length).
```
openssl rand 32 > "KeyMaterial2.bin"
```

Use the get-parameters-for-import command to create the wrapping key and import token for the second key material to be imported. The value of the ImportToken and PublicKey fields in the response has been trimmed for brevity.

export WRAPPING_ALGORITHM="RSAES_OAEP_SHA_256"
export WRAPPING_KEY_SPEC="RSA_4096"
export KMS_PARAMETERS_FOR_IMPORTED_KEY_MATERIAL2=$(aws kms get-parameters-for-import --key-id "${EXTERNAL_KEY1_ARN}" --wrapping-algorithm "${WRAPPING_ALGORITHM}" --wrapping-key-spec "${WRAPPING_KEY_SPEC}" | tee /dev/stderr)
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "ImportToken": "AQECAHie/gkutEg+qLc1mscyCnHfNPS1aKVSxf6xd3PX05ny2wAADWAwgg1cBgkqhkiG9w0BBwagIINE0aPHP5vUwaR6nukZvp8aGKeKcN3EDCsWtTNEF2RJFLazEeHItB84viBqlNIPpf8gNcCad+ODRrCyZeAisqeyIOPRoNn+vS3KMHIpMjhBUsLF6yys7FJts7P+ncF9n1bmTC4qYln5BaZ9FoI1P4NikEGiDakG8rtVJM7jWKJqVipifZvhlY8EKM8hE8e7zqR3C13TQhJXec0rVaqsFxSyjX/hbKkY55mDw36xMJaK6G9F3Fi9Pg5fKc/oOG4gUGXKSSZBoL3Jt3ssr7ACPuzFfhMOaqQ/
… /mfaCPAJN5dH0cmppKiOYqA7RSMsx2/sPMWekVLh6j38RMqZHHIq67jos0p0h8EseWpEmI2mnbJ/yoWLY/DkOEami+6scIPgvyvVzRM9d2hR7BCd0/9kwZxIN6WX0",
    "PublicKey": "MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAiGzUXf7XWN5nN5GOo/5McrsIYfiLQrDjVdOen0v7QqQcQ04p+ozI5NDGLADYd0bLNn4cm2xrlxqUjTqrtW0U8k9A/5DurYprmNmF6M+Q+zj+EkR5XoLmcuLALifHXDMI0Af20L46NK6uVKYqDl7gP4xXOP+ka57NfHmWGWDGTOKUQNZSS0emBe0UGS
…
7AC0FuwndrShwNxiKEIWPi3NhJ70jkYCrle1mQqKHpsFe6bGKLvp1gdMWByy0fY9lOH9HfoQB0FonEDZjldWBn3YXtP/eUwrgyyCk/Po7i5pjA6cjf/mBTmwmVD/jBAxW45YHFaLaJGPillaQy0Xeu89Mdwq8uEh8AixwTKbQ9Jwk3RG5A33QFj6Qb67sCAwEAAQ==",
    "ParametersValidTo": "2025-05-09T14:10:08.028000-07:00"
}
export PUBLIC_KEY_FOR_IMPORTED_KEY_MATERIAL2=$(echo "${KMS_PARAMETERS_FOR_IMPORTED_KEY_MATERIAL2}" | jq -r .PublicKey)
export IMPORT_TOKEN_FOR_IMPORTED_KEY_MATERIAL2=$(echo "${KMS_PARAMETERS_FOR_IMPORTED_KEY_MATERIAL2}" | jq -r .ImportToken)

echo "${PUBLIC_KEY_FOR_IMPORTED_KEY_MATERIAL2}" | base64 -d > "WrappingKeyForKeyMaterial2.bin"
echo "${IMPORT_TOKEN_FOR_IMPORTED_KEY_MATERIAL2}" | base64 -d > "ImportTokenForKeyMaterial2.bin"

Wrap the second key material using the wrapping key.

openssl pkeyutl -encrypt -in "KeyMaterial2.bin" -out "WrappedKeyMaterial2.bin" -inkey "WrappingKeyForKeyMaterial2.bin" -keyform DER -pubin -encrypt -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256 -pkeyopt rsa_mgf1_md:sha256

Import the second key material. Optionally, you can assign a key material description. We also capture the key material ID from the response. In this example, the key material doesn’t expire. Optionally, you can set an expiration time.

Note: This call will fail if you omit the import-type parameter or set it to EXISTING_KEY_MATERIAL. Specifying import-type as NEW_KEY_MATERIAL allows the API caller to associate additional key material with the imported key.

export KEY_MATERIAL2_ID=$(aws kms import-key-material --key-id "${EXTERNAL_KEY1_ARN}" --encrypted-key-material fileb://WrappedKeyMaterial2.bin --import-token fileb://ImportTokenForKeyMaterial2.bin --expiration-model KEY_MATERIAL_DOES_NOT_EXPIRE --key-material-description "Q2 2025 key from HSM1" --import-type NEW_KEY_MATERIAL | tee /dev/stderr | jq .KeyMaterialId | tr -d \")
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "KeyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e"
}

Use ListKeyRotations to view key materials associated with the key. There should now be two key materials. The key material state of the second key material should be PENDING_ROTATION.

aws kms list-key-rotations --key-id "${EXTERNAL_KEY1_ARN}" --include-key-material ALL_KEY_MATERIAL
{
    "Rotations": [
        {
            "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
            "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241",
            "KeyMaterialDescription": "First key material",
            "ImportState": "IMPORTED",
            "KeyMaterialState": "CURRENT",
            "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE"
        },
        {
            "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
            "KeyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e",
            "KeyMaterialDescription": "Second key material",
            "ImportState": "IMPORTED",
            "KeyMaterialState": "PENDING_ROTATION",
            "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE"
        }
    ],
    "Truncated": false
}

The CurrentKeyMaterialId in DescribeKey response should still be ${KEY_MATERIAL1_ID}.

echo "${KEY_MATERIAL1_ID}"
04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241

aws kms describe-key --key-id "${EXTERNAL_KEY1_ARN}"
{
    "KeyMetadata": {
        "AWSAccountId": "111122223333",
        "KeyId": "97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "Arn": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "CreationDate": "2025-05-08T13:55:04.605000-07:00",
        "Enabled": true,
        "Description": "",
        "KeyUsage": "ENCRYPT_DECRYPT",
        "KeyState": "Enabled",
        "Origin": "EXTERNAL",
        "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE",
        "KeyManager": "CUSTOMER",
        "CustomerMasterKeySpec": "SYMMETRIC_DEFAULT",
        "KeySpec": "SYMMETRIC_DEFAULT",
        "EncryptionAlgorithms": [
            "SYMMETRIC_DEFAULT"
        ],
        "MultiRegion": false,
        "CurrentKeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241"
    }
}

Step 4: Use on-demand rotation to update the current key material

This step moves the current key material for this key to the newly imported key material.

Use the RotateKeyOnDemand operation to initiate an on-demand key rotation. If the key material in PENDING_ROTATION state is deleted or expires before initiating on-demand rotation, this operation will fail.
```
aws kms rotate-key-on-demand --key-id "${EXTERNAL_KEY1_ARN}"
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041"
}
```
AWS KMS uses a background worker to perform the rotation, so there’s a delay between initiating the on-demand key rotation and its completion. Use the GetKeyRotationStatus command to monitor the rotation status. Until the rotation is completed, the GetKeyRotationStatus response will include the OnDemandRotationStartDate field. When this field disappears, the on-demand key rotation is complete.
```
aws kms get-key-rotation-status --key-id "${EXTERNAL_KEY1_ARN}"
{
    "KeyRotationEnabled": false,
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "OnDemandRotationStartDate": "2025-05-08T14:12:23.096000-07:00"
}
```

Use ListKeyRotations when rotation completes. The second key material changes state from PENDING_ROTATION to CURRENT.

aws kms list-key-rotations --key-id "${EXTERNAL_KEY1_ARN}" --include-key-material ALL_KEY_MATERIAL
{
    "Rotations": [
        {
            "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
            "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241",
            "KeyMaterialDescription": "First key material",
            "ImportState": "IMPORTED",
            "KeyMaterialState": "NON_CURRENT",
            "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE"
        },
        {
            "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
            "KeyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e",
            "KeyMaterialDescription": "Second key material",
            "ImportState": "IMPORTED",
            "KeyMaterialState": "CURRENT",
            "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE",
            "RotationDate": "2025-05-08T14:12:30.890000-07:00",
            "RotationType": "ON_DEMAND"
        }
    ],
    "Truncated": false
}

Rotation changes the current key material, which is reflected in the CurrentKeymaterialId field in the DescribeKey response. It should now match ${KEY_MATERIAL2_ID}.

echo "${KEY_MATERIAL1_ID}"
04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241

echo "${KEY_MATERIAL2_ID}"
b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e

aws kms describe-key --key-id "${EXTERNAL_KEY1_ARN}"
{
    "KeyMetadata": {
        "AWSAccountId": "111122223333",
        "KeyId": "97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "Arn": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "CreationDate": "2025-05-08T13:55:04.605000-07:00",
        "Enabled": true,
        "Description": "",
        "KeyUsage": "ENCRYPT_DECRYPT",
        "KeyState": "Enabled",
        "Origin": "EXTERNAL",
        "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE",
        "KeyManager": "CUSTOMER",
        "CustomerMasterKeySpec": "SYMMETRIC_DEFAULT",
        "KeySpec": "SYMMETRIC_DEFAULT",
        "EncryptionAlgorithms": [
            "SYMMETRIC_DEFAULT"
        ],
        "MultiRegion": false,
        "CurrentKeyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e"
    }
}

Step 5: Use the second key material to create and decrypt ciphertext

Similar to Step 2, this step demonstrates how to verify that the key material ID of your imported key in cryptographic operations has been rotated.

Use the GenerateDataKey operation and capture the ciphertext. This operation returns a data key in both plaintext and ciphertext forms. Note that the KeyMaterialId returned in the response matches the identifier of the second key material.

echo "${KEY_MATERIAL2_ID}"
b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e

export KEY1_CIPHERTEXT_BLOB2=$(aws kms generate-data-key --key-id "${EXTERNAL_KEY1_ARN}" --number-of-bytes 32 | tee /dev/stderr | jq -r .CiphertextBlob)
{
    "CiphertextBlob": "AQIBAHi0dJw62Bc3nKmyJkln/nkfyz/pKmoUygZlfDsebq/rTgGjuxQDA9WWHvqF6ZipVLZZAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMdXVYuY1Sa/Jb/SvrAgEQgDtQUp2gClGPBOuxFO89oAZmmkDghEKM6rXgCeS5A/NCLyX7UPdcpsJG/cJAFdjPE9sCfWjOUXCv9JTWmQ==",
    "Plaintext": "acSB+a6W2TQ3dsb2c78yDaZLi9HcuHSubeQkFaAdl1Y=",
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "KeyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e",
    "KeyOrigin": "EXTERNAL"
}

Decrypt the ciphertext and compare it to the plaintext key. The KeyMaterialId returned in the response matches the identifier of the second key material. The plaintext in decrypt response matches the plaintext data key in the previous GenerateDataKey response.

echo "${KEY_MATERIAL2_ID}"
b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e

aws kms decrypt --ciphertext-blob "${KEY1_CIPHERTEXT_BLOB2}"
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "Plaintext": "acSB+a6W2TQ3dsb2c78yDaZLi9HcuHSubeQkFaAdl1Y=",
    "EncryptionAlgorithm": "SYMMETRIC_DEFAULT",
    "KeyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e",
    "KeyOrigin": "EXTERNAL"
}

AWS KMS automatically uses the correct key material based on the ciphertext. When you decrypt the ciphertext produced in Step 2, AWS KMS uses the first key material.

echo "${KEY_MATERIAL1_ID}"
04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241

aws kms decrypt --ciphertext-blob "${KEY1_CIPHERTEXT_BLOB1}"
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "Plaintext": "3OQAOySTM3/E7QHH3a+GGzEz3HKS30rUcvUep+uueas=",
    "EncryptionAlgorithm": "SYMMETRIC_DEFAULT",
    "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241",
    "KeyOrigin": "EXTERNAL"
}

Step 6: Delete key material, make the key unusable

With the launch of this feature, the DeleteImportedKeyMaterial operation takes an optional KeyMaterialId parameter. If no KeyMaterialId is specified, AWS KMS deletes the current key material. This maintains backwards compatibility with existing behavior.

Delete the first imported key material by specifying its identifier.

aws kms delete-imported-key-material --key-id "${EXTERNAL_KEY1_ARN}" --key-material-id "${KEY_MATERIAL1_ID}"
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241"
}

Deleting the key material causes the key state to change to PendingImport.

aws kms describe-key --key-id "${EXTERNAL_KEY1_ARN}"
{
    "KeyMetadata": {
        "AWSAccountId": "111122223333",
        "KeyId": "97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "Arn": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "CreationDate": "2025-05-08T13:55:04.605000-07:00",
        "Enabled": false,
        "Description": "",
        "KeyUsage": "ENCRYPT_DECRYPT",
        "KeyState": "PendingImport",
        "Origin": "EXTERNAL",
        "KeyManager": "CUSTOMER",
        "CustomerMasterKeySpec": "SYMMETRIC_DEFAULT",
        "KeySpec": "SYMMETRIC_DEFAULT",
        "EncryptionAlgorithms": [
            "SYMMETRIC_DEFAULT"
        ],
        "MultiRegion": false,
        "CurrentKeyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e"
    }
}

ListKeyRotations also reflects the first key material as PendingImport.

aws kms list-key-rotations --key-id "${EXTERNAL_KEY1_ARN}" --include-key-material ALL_KEY_MATERIAL
{
    "Rotations": [
        {
            "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
            "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241",
            "KeyMaterialDescription": "First key material",
            "ImportState": "PENDING_IMPORT",
            "KeyMaterialState": "NON_CURRENT"
        },
        {
            "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
            "KeyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e",
            "KeyMaterialDescription": "Second key material",
            "ImportState": "IMPORTED",
            "KeyMaterialState": "CURRENT",
            "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE",
            "RotationDate": "2025-05-08T14:12:30.890000-07:00",
            "RotationType": "ON_DEMAND"
        }
    ],
    "Truncated": false
}

Cryptographic operations fail with a KMSInvalidStateException when a key is in PendingImport state even though the key material required to decrypt the specific ciphertext blob is imported into AWS KMS.

aws kms decrypt --ciphertext-blob "${KEY1_CIPHERTEXT_BLOB2}"

An error occurred (KMSInvalidStateException) when calling the Decrypt operation: arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041 is pending import.

Step 7: Reimport key material to enable the key

You need to re-import all expired or deleted key materials associated with a key for the key to become usable again.

Re-import the missing key material. For this example, we reused the wrapped key and import token we already have. This is an optimization. Optionally, you can get new parameters for wrapping and importing the key material.

aws kms import-key-material --key-id "${EXTERNAL_KEY1_ARN}" --encrypted-key-material fileb://WrappedKeyMaterial1.bin --import-token fileb://ImportTokenForKeyMaterial1.bin --expiration-model KEY_MATERIAL_DOES_NOT_EXPIRE
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241"
}


aws kms list-key-rotations --key-id "${EXTERNAL_KEY1_ARN}" --include-key-material ALL_KEY_MATERIAL
{
    "Rotations": [
        {
            "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
            "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241",
            "KeyMaterialDescription": "First key material",
            "ImportState": "IMPORTED",
            "KeyMaterialState": "NON_CURRENT",
            "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE"
        },
        {
            "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
            "KeyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e",
            "KeyMaterialDescription": "Second key material",
            "ImportState": "IMPORTED",
            "KeyMaterialState": "CURRENT",
            "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE",
            "RotationDate": "2025-05-08T14:12:30.890000-07:00",
            "RotationType": "ON_DEMAND"
        }
    ],
    "Truncated": false
}

Use DescribeKey to validate the key is now enabled.

aws kms describe-key --key-id "${EXTERNAL_KEY1_ARN}"
{
    "KeyMetadata": {
        "AWSAccountId": "111122223333",
        "KeyId": "97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "Arn": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
        "CreationDate": "2025-05-08T13:55:04.605000-07:00",
        "Enabled": true,
        "Description": "",
        "KeyUsage": "ENCRYPT_DECRYPT",
        "KeyState": "Enabled",
        "Origin": "EXTERNAL",
        "ExpirationModel": "KEY_MATERIAL_DOES_NOT_EXPIRE",
        "KeyManager": "CUSTOMER",
        "CustomerMasterKeySpec": "SYMMETRIC_DEFAULT",
        "KeySpec": "SYMMETRIC_DEFAULT",
        "EncryptionAlgorithms": [
            "SYMMETRIC_DEFAULT"
        ],
        "MultiRegion": false,
        "CurrentKeyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e"
    }
}

You can use the following commands to validate that the key works:

aws kms decrypt --ciphertext-blob "${KEY1_CIPHERTEXT_BLOB1}"
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "Plaintext": "3OQAOySTM3/E7QHH3a+GGzEz3HKS30rUcvUep+uueas=",
    "EncryptionAlgorithm": "SYMMETRIC_DEFAULT",
    "KeyMaterialId": "04f5cf50ee8f057f974b9820ab8fa8837d28363a85a8f8ef06a46c93adc5b241",
    "KeyOrigin": "EXTERNAL"
}

aws kms decrypt --ciphertext-blob "${KEY1_CIPHERTEXT_BLOB2}"
{
    "KeyId": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041",
    "Plaintext": "acSB+a6W2TQ3dsb2c78yDaZLi9HcuHSubeQkFaAdl1Y=",
    "EncryptionAlgorithm": "SYMMETRIC_DEFAULT",
    "KeyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e",
    "KeyOrigin": "EXTERNAL"
}

CloudTrail logging

AWS CloudTrail now includes the key material ID in the additionalEventData field for operations using symmetric-encryption keys (both AWS_KMS and EXTERNAL origin). The following is a sample CloudTrail event for the AWS KMS decrypt operation:

{
    "eventVersion": "1.11",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "AROAEXAMPLE:alice",
        "arn": "arn:aws:sts::111122223333:assumed-role/key-user/alice",
        "accountId": "111122223333",
        "accessKeyId": "ACCESSKEY",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": "AROAEXAMPLE",
                "arn": "arn:aws:iam::111122223333:role/key-user",
                "accountId": "111122223333",
                "userName": "key-user"
            },
            "attributes": {
                "creationDate": "2025-05-09T18:12:33Z",
                "mfaAuthenticated": "false"
            }
        }
    },
    "eventTime": "2025-05-09T18:15:02Z",
    "eventSource": "kms.amazonaws.com",
    "eventName": "Decrypt",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "1.2.3.4",
    "userAgent": "aws-cli/2.9.19 Python/3.9.11 Darwin/24.4.0 exe/x86_64 prompt/off command/kms.decrypt",
    "requestParameters": {
        "encryptionAlgorithm": "SYMMETRIC_DEFAULT"
    },
    "responseElements": null,
    "additionalEventData": { "keyMaterialId": "b4749c3ad817379ca9b2264967fe791fcb3fe92a6a14ca06657c3b1e6eafeb4e" },
    "requestID": "68c6013e-24af-4a66-aa88-64cdb27d693d",
    "eventID": "1feca1a6-cbf0-491d-897f-31c52c54c783",
    "readOnly": true,
    "resources": [{
        "accountId": "111122223333",
        "type": "AWS::KMS::Key",
        "ARN": "arn:aws:kms:us-east-1:111122223333:key/97c8e55d-5b04-45a1-8a01-1febde0dd041"
    }],
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "111122223333",
    "eventCategory": "Management",
    "tlsDetails": {
        "tlsVersion": "TLSv1.3",
        "cipherSuite": "TLS_AES_256_GCM_SHA384",
        "clientProvidedHostHeader": "kms.us-east-1.amazonaws.com"
    }
}

Key expiry and key deletion capabilities unique to imported keys

Unlike standard KMS keys that you create within AWS KMS, imported keys offer two unique capabilities for enhanced controls over key material within AWS.

When importing key material into a KMS key, you can optionally set an expiration date and time, up to 365 days from the import date. If you don’t specify an expiration, the key material doesn’t expire. When the expiration time is reached, AWS KMS immediately deletes the key material and the KMS key becomes unusable. This is in contrast to the 7–30 day waiting period required for KMS keys whose key material is generated by AWS KMS. To re-enable the key, you must reimport the key material. With key rotation, you can continue to set expiration periods for new key material that you import.
Unique to KMS imported keys, you can also delete specific key material without deleting the entire KMS key. Deleting the key material of a KMS imported key is temporary and reversible. To restore the key, reimport its key material.

Key expiry and import key material deletion can be useful if you need to demonstrate immediate key suspension in the cloud or when you want to temporarily seed AWS KMS with key material that can be inserted and repeatedly removed from cloud access (hydration and re-hydration of keys for improved digital sovereignty).

Special considerations

AWS KMS is designed to keep imported key material highly available. But AWS KMS doesn’t maintain the durability of imported key material at the same level as key material that AWS KMS generates. You must retain a copy of the imported key material outside of AWS KMS in a system that you control. We recommend that you store an exportable copy of the imported key material in a key management system, such as an HSM. As a best practice, you should store a reference to the KMS key ARN and key material description alongside the exportable copy of the key material.

The deletion or expiration of any key material associated with a KMS key makes that key unusable. You must re-import all the key materials associated with a key before it can be used for cryptographic operations.

The following types of KMS keys with imported key material do not support on-demand key rotation, but you can continue to use manual rotation with these keys.

Asymmetric keys
HMAC keys
Multi-AWS Region keys

Features and benefits

This new capability includes several important features:

Maintain key identifiers: Rotate key material while keeping the same AWS KMS key ID and ARN.
Flexible rotation: Rotate keys on-demand to meet your security requirements or use an external key manager to set a rotation schedule that can drive new key rotations into AWS KMS.
Audit trail: Track key material usage through CloudTrail logs.
Metadata management: Add unique descriptions to each key material version.
Retains support for key expiry and import key deletion (features unique to KMS imported keys)

Pricing

For AWS KMS keys that rotate automatically or on-demand, each key incurs a base cost, and the first two rotations add $1 per month (prorated hourly) in additional charges. The pricing is capped after the second rotation, meaning subsequent rotations beyond the second one aren’t billed. This simplified pricing provides you with more predictable costs while maintaining the flexibility to rotate keys frequently to meet your compliance and security audit requirements.

Getting started

You can start using this feature today in all AWS Regions where AWS KMS is available. To learn more, visit the AWS KMS Developer Guide.

We’re excited to see how you use this new capability to enhance your key management practices. Leave a comment below or on the AWS re:Post community forum to let us know what you think.

ML-KEM post-quantum TLS now supported in AWS KMS, ACM, and Secrets Manager

2025-04-07 Alex Weibel

Post Syndicated from Alex Weibel original https://aws.amazon.com/blogs/security/ml-kem-post-quantum-tls-now-supported-in-aws-kms-acm-and-secrets-manager/

Amazon Web Services (AWS) is excited to announce that the latest hybrid post-quantum key agreement standards for TLS have been deployed to three AWS services. Today, AWS Key Management Service (AWS KMS), AWS Certificate Manager (ACM), and AWS Secrets Manager endpoints now support Module-Lattice-Based Key-Encapsulation Mechanism (ML-KEM) for hybrid post-quantum key agreement in non-FIPS endpoints in all AWS Regions in the aws partition. The AWS Secrets Manager Agent, built on AWS SDK for Rust now also has opt-in support for hybrid post-quantum key agreement. With this, customers can bring secrets into their applications with end-to-end post-quantum enabled TLS.

These three services were chosen because they are security-critical AWS services with the most urgent need for post-quantum confidentiality. These three AWS services have previously deployed support for CRYSTALS-Kyber, the predecessor of ML-KEM. Support for CRYSTALS-Kyber will continue through 2025, but will be removed across all AWS service endpoints in 2026 in favor of ML-KEM.

Our migration to post-quantum cryptography

AWS is committed to following our post-quantum cryptography migration plan. As part of this commitment, and part of the AWS post-quantum shared responsibility model, AWS plans to deploy support for ML-KEM to all AWS services with HTTPS endpoints over the coming years. AWS customers must update their TLS clients and SDKs to offer ML-KEM when connecting to AWS service HTTPS endpoints. This will protect against future harvest now, decrypt later threats posed by quantum computing advancements. Meanwhile, AWS service HTTPS endpoints will be responsible for selecting ML-KEM when offered by clients.

Our commitment to negotiate hybrid post-quantum key agreement algorithms is enabled by AWS Libcrypto (AWS-LC), our open-source FIPS-140-3-validated cryptographic library used throughout AWS, and s2n-tls, our open-source TLS implementation used across AWS service HTTPS endpoints. AWS-LC has been awarded multiple FIPS certificates from NIST (#4631, #4759, and #4816), and was the first open-source cryptographic module to include ML-KEM in a FIPS 140-3 validation.

The effect of hybrid post-quantum ML-KEM on TLS performance

Migrating from an Elliptic Curve Diffie-Hellman (ECDH)-only key agreement to an ECDH+ML-KEM hybrid key agreement necessarily requires that the TLS handshake send more data and perform more cryptographic operations. Switching from a classical to a hybrid post-quantum key agreement will transfer approximately 1600 additional bytes during the TLS handshake and will require approximately 80–150 microseconds more compute time to perform ML-KEM cryptographic operations. This is a one-time TLS connection startup cost and is amortized over the lifetime of the TLS connection across the HTTP requests sent over that connection.

AWS is working to provide a smooth migration to hybrid post-quantum key agreement for TLS. This work includes performing benchmarks on example workloads to help customers understand the impact of enabling hybrid post-quantum key agreement with ML-KEM.

Using the AWS SDK for Java v2, AWS has measured the number of AWS KMS GenerateDataKey requests per second that a single thread can issue serially between an Amazon Elastic Compute Cloud (Amazon EC2) C6in.metal client and the public AWS KMS endpoint. Both the client and server were in the us-west-2 Region. Classical TLS connections to AWS KMS negotiated the P256 elliptic curve for key agreement, and hybrid post-quantum TLS connections negotiated the X25519 elliptic curve with ML-KEM-768 for their hybrid key agreement. Your own performance characteristics might differ and will depend on your environment, including your instance type, your workload profiles, the amount of parallelism and number of threads used, and your network location and capacity. The HTTP request transaction rates were measured with TLS connection reuse both enabled and disabled.

Figure 1 shows the number of requests per second issued at different percentiles when TLS 1.3 connection reuse is disabled. It shows that in the worst-case scenario—when the cost of a TLS handshake is never amortized and every HTTP request must perform a full TLS handshake—enabling hybrid post-quantum TLS decreases the transactions per second (TPS) by about 2.3 percent on average, from 108.7 TPS to 106.2 TPS.

Figure 1: AWS KMS GenerateDataKey requests per second <em>without</em> TLS connection reuse” width=”1747″ height=”1221″ class=”size-full wp-image-37873″ style=”border: 1px solid #bebebe”></p>
<p id=

Figure 1: AWS KMS GenerateDataKey requests per second without TLS connection reuse

Figure 2 shows the number of requests per second issued at different percentiles when TLS connection reuse is enabled. Reusing TLS connections and amortizing the cost of a TLS handshake over many HTTP requests is the default setting in the AWS SDK for Java v2. We show that enabling hybrid post-quantum TLS when using default SDK settings leaves the TPS rate almost unchanged, with only a 0.05 percent decrease on average, from 216.1 TPS to 216.0 TPS.

Figure 2: AWS KMS GenerateDataKey requests per second <em>with</em> TLS connection reuse” width=”1747″ height=”1226″ class=”size-full wp-image-37874″ style=”border: 1px solid #bebebe”></p>
<p id=

Figure 2: AWS KMS GenerateDataKey requests per second with TLS connection reuse

Our results show that the performance impact of enabling hybrid post-quantum TLS is negligible when using typical configuration settings in your SDK. Our measurements show that enabling hybrid post-quantum TLS for a default-case example workload only lowered maximum TPS rate by 0.05 percent. Our results also show that overriding SDK defaults to force the worst-case scenario of performing a new TLS handshake for every request only decreased maximum TPS rate by 2.3 percent.

The following table shows the benchmark data that we measured. Each benchmark performed 500 one-second TPS measurements for varying TLS key agreement settings and TLS connection reuse settings. The measurements used v2.30.22 of the AWS SDK for Java v2. The TLS key agreement was switched between classical and hybrid post-quantum by toggling the postQuantumTlsEnabled() configuration. TLS connection reuse was toggled by injecting a Connection: close HTTP header into each HTTP request. This header forces the TLS connection to be shut down after each HTTP request and requires that a new TLS connection be created for each HTTP request.

TLS key agreement	TLS conn resuse	Total HTTP requests	Average (TPS)	p01 (TPS)	p10 (TPS)	p25 (TPS)	p50 (TPS)	p75 (TPS)	p90 (TPS)	p99 (TPS)
Classical (P256)	No	54,367	108.7	78	86	96	102	129	137	145
Hybrid post-quantum (X25519MLKEM768)	No	53,106	106.2	76	85	93	100	126	134	141
Classical (P256)	Yes	108,052	216.1	181	194	200	216	233	240	245
Hybrid post-quantum (X25519MLKEM768)	Yes	107,994	216	177	194	200	216	233	239	245

Removing support for draft post-quantum standards

AWS service endpoints with support for CRYSTALS-Kyber, the predecessor of ML-KEM, will continue to support CRYSTALS-Kyber through 2025. We will slowly phase out support for the pre-standard CRYSTALS-Kyber implementations after customers have moved to the ML-KEM standard. Customers using previous versions of the AWS SDK for Java with CRYSTALS-Kyber support should upgrade to the latest SDK versions that have ML-KEM support. No code changes are necessary for customers using a generally available release of the AWS SDK for Java v2 to upgrade from CRYSTALS-Kyber to ML-KEM.

Customers currently negotiating CRYSTALS-Kyber who do not upgrade their AWS Java SDK v2 clients by 2026 will see their clients gracefully fall back to a classical key agreement once CRYSTALS-Kyber is removed from AWS service HTTPS endpoints.

How to use hybrid post-quantum key agreement

If using the AWS SDK for Rust, you can enable the hybrid post-quantum key agreement by adding the rustls package to your crate and enabling the prefer-post-quantum feature flag. See the rustls documentation for more information.

If using the AWS SDK for Java 2.x, you can enable hybrid post-quantum key agreement by calling .postQuantumTlsEnabled(true) when building your AWS Common Runtime HTTP client.

Step 1: Add the AWS Common Runtime HTTP client to your Java dependencies.

Add the AWS Common Runtime HTTP client to your Maven dependencies. We recommend using the latest available version. Use version 2.30.22 or greater to enable the use of ML-KEM.

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>aws-crt-client</artifactId>
    <version>2.30.22<version>
</dependency>

Step 2: Enable post-quantum TLS in your Java SDK client configuration

When configuring your AWS service client, use the AwsCrtAsyncHttpClient configured with post-quantum TLS.

// Configure an AWS Common Runtime HTTP client with Post-Quantum TLS enabled
SdkAsyncHttpClient awsCrtHttpClient = AwsCrtAsyncHttpClient.builder()
          .postQuantumTlsEnabled(true)
          .build();

// Create an AWS service client that uses the AWS Common Runtime client
KmsAsyncClient kmsAsync = KmsAsyncClient.builder()
         .httpClient(awsCrtHttpClient)
         .build();

// Make a request over a TLS connection that uses post-quantum key agreement
ListKeysReponse keys = kmsAsync.listKeys().get();

See the KMS PQ TLS example application for an end-to-end example of a post-quantum TLS setup.

Things to try

Here are some ideas about how to use this post-quantum-enabled client:

Run load tests and benchmarks. The AwsCrtAsyncHttpClient is heavily optimized for performance and uses AWS Libcrypto on Linux-based environments. If you aren’t already using the AwsCrtAsyncHttpClient, try it today to see the performance benefits compared to the default SDK HTTP client. After using AwsCrtAsyncHttpClient, enable post-quantum TLS support. See if using AwsCrtAsyncHttpClient with post-quantum TLS is an overall performance gain to using the default SDK HTTP client without post-quantum TLS.
Try connecting from different network locations. Depending on the network path that your request takes, you might discover that intermediate hosts, proxies, or firewalls with deep packet inspection (DPI) block the request. If this is the case, you might need to work with your security team or IT administrators to update firewalls in your network to unblock these new TLS algorithms. We want to hear from you about how your infrastructure interacts with this new variant of TLS traffic.

Conclusion

Support for ML-KEM-based hybrid key agreement has been deployed to three security-critical AWS service endpoints. The performance impact of enabling hybrid post-quantum TLS is likely to be negligible when TLS connection reuse is enabled. Our measurements showed only a 0.05 percent decrease to maximum transactions per second when calling AWS KMS GenerateDataKey.

Starting with version 2.30.22, the AWS SDK for Java v2 now supports ML-KEM-based hybrid key agreement on Linux-based platforms when using the AWS Common Runtime HTTP client. Try enabling post quantum key agreement for TLS in your Java SDK client configuration today.

AWS plans to deploy support for ML-KEM-based hybrid post-quantum key agreement to every AWS service HTTPS endpoint over the coming years as part of our post-quantum cryptography migration plan. AWS customers will be responsible for updating their TLS clients and SDKs to help ensure that ML-KEM key agreement is offered when connecting to AWS service HTTPS endpoints. This will protect against future harvest now, decrypt later threats posed by quantum computing advancements.

For additional information, blog posts, and periodic updates on our post-quantum cryptography migration, keep watching the AWS Post-Quantum Cryptography page. To learn more about post-quantum cryptography with AWS, contact the post-quantum cryptography team.

Additional resources:

If you have feedback about this post, submit comments in the Comments section below.

AWS KMS CloudWatch metrics help you better track and understand how your KMS keys are being used

2025-03-17 Norman Li

Post Syndicated from Norman Li original https://aws.amazon.com/blogs/security/aws-kms-cloudwatch-metrics-help-you-better-track-and-understand-how-your-kms-keys-are-being-used/

AWS Key Management Service (AWS KMS) is pleased to launch key-level filtering for AWS KMS API usage in Amazon CloudWatch metrics, providing enhanced visibility to help customers improve their operational efficiency and aid in security and compliance risk management.

AWS KMS currently publishes account-level AWS KMS API usage metrics to Amazon CloudWatch, enabling you to monitor and manage your API usage. However, if you’re using numerous KMS keys, pinpointing the ones with the highest request rate quota usage or significant API costs becomes challenging. For example, if you have more than 10 active KMS keys in your account, prior to this launch you would have needed to build a custom CloudTrail and Amazon Athena based solution to locate which specific keys are driving the majority of API usage and costs. With the new CloudWatch metrics, which are available under the AWS/KMS namespace in CloudWatch, you can track, understand, and set alerts on detailed API usage at the individual KMS key level without building a costly customized solution.

This blog post explores several use cases to help you better take advantage of these newly introduced CloudWatch metrics to manage your AWS KMS API usage and costs. The use cases cover viewing and understanding your API usage at the key level, and creating CloudWatch alerts to detect unintentional runaway usage.

Overview of new CloudWatch metrics for KMS keys

With CloudWatch metrics for KMS keys, you can now do the following:

View the API usage for a specific KMS key, filtered by individual API operations (for example, Encrypt, Decrypt, or GenerateDataKey).
See the aggregated usage across cryptographic operations for a given KMS key.
Set up an alarm if a specific KMS key exceeds a specified threshold on a single API operation, or a set of API operations.

This streamlined approach allows you to quickly monitor, understand, and troubleshoot the API usage patterns of your KMS keys, without the overhead of the previous multi-step process. Let’s detail how these key-level API usage metrics can be used in two real-world examples.

Example 1: How to locate the KMS keys that consume the most API usage quota or contribute the most API charges

When you surpass your AWS KMS API request rate quotas, you can view your AWS KMS API utilization within the Service Quotas console. However, you might still find it cumbersome to identify the KMS keys that consume the largest amount of your request quota. When you receive the AWS KMS API charges that exceed your expectation, you can check the detailed billing usage in each AWS Region in Cost Explorer, but you cannot easily locate the KMS keys with the most API charges. This process becomes even more challenging when you manage a large number of KMS keys.

With the key-level API usage CloudWatch metrics, you can use the advanced metric query option to query CloudWatch Metrics Insights data with a user-friendly dialect of SQL to locate the KMS keys that consume the largest portion of the API usage quota or contribute the most API charges.

Walkthrough

To use Amazon CloudWatch Metrics Insights to identify the top 20 KMS keys that have the most cryptographic API usage up to the last 3 hours, complete the following steps:

Open the CloudWatch console.
In the navigation pane, choose Metrics, and then choose All metrics.
Choose the Multi source query tab.
For the data source, choose CloudWatch Metrics Insights.
You can enter the following example query in Editor view:

Note: In Builder view, the metric namespace, metric name, filter by, group by, order by, and limit options are shown. In Editor view, the same options as in Builder view are shown in query format.
```
	SELECT SUM(SuccessfulRequest)
	FROM SCHEMA("AWS/KMS", KeyArn, Operation)
	GROUP BY KeyArn
	ORDER BY MAX () DESC
	LIMIT 20
```
Choose Run in the Editor view or Graph query in the Builder view.

Example 2: How to set a new detailed alarm on unintentional runaway AWS KMS API usage

Running big data processing workflows that read Amazon Simple Storage Service (Amazon S3) files encrypted by KMS keys is a common scenario for analytics, business reporting, or machine learning projects. Typically, these workflows read a limited number of files from S3 on each invocation. However, misconfigured workflows could unintentionally read a large number of S3 files, which could result in exceeding your AWS KMS API request rate quotas or incurring undesirable charges due to spiky AWS KMS API usage. Historically, to address this issue, you would have had to build a customized alarm system by following these steps: 1) send AWS CloudTrail events generated by AWS KMS to Amazon CloudWatch Logs; 2) write queries in Amazon CloudWatch Logs Insights to track your API request usage; and 3) enable anomaly detection on the corresponding CloudWatch Log Insights math expression.

Now, with key-level API usage CloudWatch metrics, you can directly enable anomaly detection on these metrics to set up alarms for anomalous AWS KMS API usage patterns. This provides a more streamlined and efficient way to monitor and detect potential runaway workflows. By using these CloudWatch metrics and anomaly detection capabilities, you can proactively identify and address unintended increases in AWS KMS API usage, helping to avoid unexpected charges or service disruptions in your analytics, reporting, or machine learning pipelines.

Walkthrough

Consider a scenario where you have an analytics workflow that runs frequently, which uses the Decrypt AWS KMS API operation on a KMS key to decrypt and read data from S3. You would like to enable anomaly detection on the KMS key to trigger an alarm when the Decrypt call volume to the specific KMS key sees a discernible trend or pattern. To do so, complete the following steps:

Open the CloudWatch console.
In the navigation pane, choose Metrics, and then choose All metrics.
Choose KMS, and then choose KeyArn, Operation.
In the search bar, enter the Amazon Resource Name (ARN) of the key, and then choose Search. Select the CloudWatch metric you would like to enable anomaly detection for.
Navigate to Graphed metrics, and using the Statistic and Period drop-down lists, choose the statistic and period that you would like to monitor. Then you can enable anomaly detection by selecting the Pulse icon.

Figure 1: How to enable anomaly detection on a SuccessfulRequest metric
You can adjust the anomaly detection by setting the sensitivity to adjust the bandwidth, if needed.

Figure 2: Anomaly detection is enabled on the SuccessfulRequest metric. The gray band illustrates the expected range of values and the anomaly is in red

Conclusion

This blog post highlighted the newly introduced key-level filtering capability for the AWS KMS API usage in CloudWatch. We showed two real-world use cases to demonstrate how you can use the new CloudWatch metrics. These use cases include improving operational visibility, setting up proactive alarms on anomalies in KMS API usage patterns, and potentially tracking detailed key usage for compliance purposes.

If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread in the AWS Key Management Service re:Post.

The importance of encryption and how AWS can help

2025-02-12 Ken Beer

Post Syndicated from Ken Beer original https://aws.amazon.com/blogs/security/importance-of-encryption-and-how-aws-can-help/

February 12, 2025: This post was republished to include new services and features that have launched since the original publication date of June 11, 2020.

Encryption is a critical component of a defense-in-depth security strategy that uses multiple defensive mechanisms to protect workloads, data, and assets. As organizations look to innovate while building trust with customers, they need to meet critical compliance requirements and improve data security. Encryption, when used correctly, adds a layer of protection against unauthorized access that can help you strengthen data protection, adhere to regulations and standards, and enhance the security of communications.

How and why does encryption work?

Encryption works by using an algorithm with a key to convert data into unreadable data (ciphertext) that can only become readable again with the right key. For example, a simple phrase like “Hello World!” may look like “1c28df2b595b4e30b7b07500963dc7c” when encrypted. There are several different types of encryption algorithms, all using different types of keys. A strong encryption algorithm relies on mathematical properties to produce ciphertext that can’t be decrypted using any practically available amount of computing power without also having the necessary key. Therefore, protecting and managing the keys becomes a critical part of any encryption solution.

Encryption as part of your security strategy

An effective security strategy begins with stringent access control and continuous work to define the least privilege necessary for persons or systems accessing data. When using the AWS Cloud, you adopt the model of shared responsibility. You are responsible for managing your own access control policies. Encryption is a critical component of a defense-in-depth strategy because it can mitigate weaknesses in your primary access control mechanism. What if an access control mechanism fails and allows access to the raw data on disk or traveling along a network link? If the data is encrypted using a strong key, as long as the decryption key is not on the same system as your data, it is computationally infeasible for a bad actor to decrypt your data.

To show how infeasible this is, let’s consider the Advanced Encryption Standard (AES) with 256-bit keys (AES-256). It’s the strongest industry-adopted and government-approved algorithm for encrypting data. AES-256 is the technology we use to encrypt data in AWS, including Amazon Simple Storage Service (S3) server-side encryption. It would take at least a trillion years to break using current (and foreseeable future) computing technology. Current research suggests that even the future availability of quantum-based computing won’t sufficiently reduce the time it would take to break AES-256 encryption.

But what if you mistakenly create overly permissive access policies on your data? A well-designed encryption and key management system can also help prevent this from becoming an issue, because it separates access to the decryption key from access to your data.

Requirements for an encryption solution

To get the most from an encryption solution, you need to think about two things:

Protecting keys at rest: Are the systems using encryption keys secured so the keys can never be used outside the system? In addition, do these systems implement encryption algorithms correctly to produce strong ciphertexts that cannot be decrypted without access to the right keys?
Independent key management: Is the authorization to use encryption independent from how access to the underlying data is controlled?

There are third-party solutions that you can bring to AWS to help meet these requirements. However, these systems can be difficult and expensive to operate at scale. AWS offers a range of options to simplify encryption and key management.

Protecting keys at rest

When you use third-party key management solutions, it can be difficult to gauge the risk of your plaintext keys leaking and being used outside the solution. The keys have to be stored somewhere, and you can’t always know or audit all the ways those storage systems are secured from unauthorized access. The combination of technical complexity and the necessity of making the encryption usable without degrading performance or availability means that choosing and operating a key management solution can present difficult tradeoffs. The best practice to maximize key security is using a hardware security module (HSM). This is a specialized computing device that has several security controls built into it to help prevent encryption keys from leaving the device in a way that could allow an adversary to access and use those keys.

One such control in modern HSMs is tamper response, in which the device detects physical or logical attempts to access plaintext keys without authorization, and destroys the keys before the attack succeeds. Because you can’t install and operate your own hardware in AWS datacenters, AWS offers two services using HSMs with tamper response to protect customers’ keys: AWS Key Management Service (AWS KMS), which manages a fleet of HSMs on the customer’s behalf, and AWS CloudHSM, which gives customers the ability to manage their own HSMs. Each service can create keys on your behalf, or you can import keys from your on-premises systems to be used by each service.

The keys in AWS KMS or AWS CloudHSM can be used to encrypt data directly, or to protect other keys that are distributed to applications that directly encrypt data. The technique of encrypting encryption keys is called envelope encryption, and it enables encryption and decryption to happen on the computer where the plaintext customer data exists, rather than sending the data to the HSM each time. For very large data sets (e.g., a database), it’s not practical to move gigabytes of data between the data set and the HSM for every read/write operation. Instead, envelope encryption allows a data encryption key to be distributed to the application when it’s needed. The “master” keys in the HSM are used to encrypt a copy of the data key so the application can store the encrypted key alongside the data encrypted under that key. Once the application encrypts the data, the plaintext copy of data key can be deleted from its memory. The only way for the data to be decrypted is if the encrypted data key, which is only a few hundred bytes in size, is sent back to the HSM and decrypted.

The process of envelope encryption is used in AWS services in which data is encrypted on a customer’s behalf (which is known as server-side encryption) to minimize performance degradation. If you want to encrypt data in your own applications (client-side encryption), you’re encouraged to use envelope encryption with AWS KMS or AWS CloudHSM. Both services offer client libraries and SDKs to add encryption functionality to their application code and use the cryptographic functionality of each service. The AWS Encryption SDK is an example of a tool that can be used anywhere, not just in applications running in AWS. To make it easier for customers to encrypt data in databases like Amazon DynamoDB, we built the AWS Database Encryption SDK. The AWS Database Encryption SDK is a set of software libraries that enable you to use client-side encryption in your database design, including record-level encryption of database items. Today, the AWS Database Encryption SDK supports Amazon DynamoDB with attribute-level encryption.

Because implementing encryption algorithms and HSMs is critical to get right, all vendors of HSMs should have their products validated by a trusted third party. HSMs in both AWS KMS and AWS CloudHSM are validated under the National Institute of Standards and Technology’s FIPS 140 program, the standard for evaluating cryptographic modules. This validates the secure design and implementation of cryptographic modules, including functions related to ports and interfaces, authentication mechanisms, physical security and tamper response, operational environments, cryptographic key management, and electromagnetic interference/electromagnetic compatibility (EMI/EMC). Encryption using a FIPS 140 level 3 validated cryptographic module is often a requirement for other security-related compliance schemes like FedRamp and HIPAA-HITECH in the U.S., or the international payment card industry standard (PCI-DSS).

Independent key management

While AWS KMS and AWS CloudHSM can protect plaintext master keys on your behalf, you are still responsible for managing access controls to determine who can cause which encryption keys to be used under which conditions. One advantage of using AWS KMS is that the policy language you use to define access controls on keys is the same one you use to define access to all other AWS resources. Note that the language is the same, not the actual authorization controls. You need a mechanism for managing access to keys that is different from the one you use for managing access to your data. AWS KMS provides that mechanism by allowing you to assign one set of administrators who can only manage keys and a different set of administrators who can only manage access to the underlying encrypted data. Configuring your key management process in this way helps provide separation of duties you need to avoid accidentally escalating privilege to decrypt data to unauthorized users. For even further separation of control, AWS CloudHSM offers an independent policy mechanism to define access to keys.

In 2022, AWS KMS launched support for external key stores (XKS), a feature that allows you to store AWS KMS customer managed keys on an HSM that you operate on premises or at a location of your choice. At a high level, AWS KMS forwards requests for encryption and decryption to your HSM. Your key material never leaves your HSM. This can help you unblock use cases for a small portion of highly regulated workloads where encryption keys should be stored and used outside of an AWS data center. However, XKS forces a significant shift in the shared responsibility model—you now have responsibility for the durability, throughput, latency, and availability of your KMS key. If that key is lost or destroyed, you could permanently lose access to data, and if an XKS key becomes unavailable, all workloads in AWS that are dependent on that XKS key will be inaccessible.

Even with the ability to separate key management from data management, you can still verify that you have configured access to encryption keys correctly. AWS KMS is integrated with AWS CloudTrail so you can audit who used which keys, for which resources, and when. This provides granular vision into your encryption management processes, which is typically much more in-depth than on-premises audit mechanisms. Audit events from AWS CloudHSM can be sent to Amazon CloudWatch, the AWS service for monitoring and alarming third-party solutions you operate in AWS.

Encrypting data at rest and in transit

AWS services that handle customer data, encrypt data that is sent from one system to another—known as data in transit—provide options to encrypt data at rest. AWS services that offer encryption at rest using AWS KMS or AWS CloudHSM use AES-256. None of these services store plaintext encryption keys at rest—that’s a function that only AWS KMS and AWS CloudHSM may perform using their FIPS 140 level 3 validated HSMs. This architecture helps minimize the unauthorized use of keys.

When encrypting data in transit, AWS services use the Transport Layer Security (TLS) protocol to provide encryption between your application and the AWS service. Most commercial solutions use an open source project called OpenSSL for their TLS needs. OpenSSL has roughly 500,000 lines of code with at least 70,000 of those implementing TLS. The code base is large, complex, and difficult to audit. Moreover, when OpenSSL has bugs, the global developer community is challenged to not only fix and test the changes, but also to make sure that the resulting fixes themselves do not introduce new flaws.

AWS’s response to challenges with the TLS implementation in OpenSSL was to develop our own implementation of TLS, known as s2n, or signal to noise. We released s2n in June 2015, which we designed to be small and fast. The goal of s2n is to provide you with network encryption that is easier to understand and that is fully auditable. We released and licensed it under the Apache 2.0 license and hosted it on GitHub.

We also designed s2n to be analyzed using automated reasoning to test for safety and correctness using mathematical logic. Through this process, known as formal methods, we verify the correctness of the s2n code base every time we change the code. We also automated these mathematical proofs, which we regularly re-run to ensure the desired security properties are unchanged with new releases of the code. Automated mathematical proofs of correctness are an emerging trend in the security industry, and AWS uses this approach for a wide variety of our mission-critical software.

Similarly, in 2022, we released s2n-quic, an open-source Rust implementation of the QUIC protocol that was added to our set of AWS encryption open source libraries. QUIC is an encrypted transport protocol designed for performance and is the foundation of HTTP/3. It is specified in a set of IETF standards that were ratified in May 2021. Amazon CloudFront HTTP/3 support is built on top of s2n-quic, due to its emphasis on performance and efficiency. You can learn more about s2n-quic in this Security Blog post.

Implementing TLS requires using encryption keys and digital certificates that assert the ownership of those keys. AWS Certificate Manager and AWS Private Certificate Authority are two services that can simplify the issuance and rotation of digital certificates across your infrastructure that needs to offer TLS endpoints. Both services use a combination of AWS KMS and AWS CloudHSM to generate and/or protect the keys used in the digital certificates they issue.

Encrypting data in use

You might also have use cases for protecting data that is actively being used by federated learning models or other applications. Cryptographic computing—a set of technologies that allow computations to be performed on encrypted data, so that sensitive data is not exposed—is a methodology for protecting data in use.

Consider the example of an insurance company that works with other companies to develop machine learning models for insurance fraud detection. You might need to use sensitive data about your customers as training data for your models, but you don’t want to share your customer data in plaintext form with the other companies. Cryptographic computing gives organizations a way to train models collaboratively without exposing plaintext data about their customers to each other, or to a cloud provider like AWS. You can read more about cryptographic computing in this AWS Security Blog post.

Today, you can see cryptographic computing at work in AWS Clean Rooms, a service that helps companies and their partners more easily and securely analyze and collaborate on their collective datasets—all without sharing or copying one another’s underlying data. AWS Clean Rooms has a feature called Cryptographic Computing for AWS Clean Rooms (C3R) that cryptographically protects your data even while it is being processed by an AWS Clean Rooms collaboration.

The role of end-to-end encryption in secure communications

End-to-end encryption (E2EE) is a method of secure communication between two or more parties that combines encryption in transit and encryption at rest to protect data from unauthorized access, interception, or tampering. Decryption happens only on the parties you intend to communicate with, and no service providers in between. Every call, message, and file is encrypted with a unique private key and remains protected in transit. Unauthorized parties can’t access communication content, because they don’t have the private key required to decrypt the data.

AWS Wickr is an end-to-end encrypted messaging and collaboration service that protects one-to-one and group messaging, voice and video calling, file sharing, screen sharing, and location sharing with 256-bit encryption. With Wickr, each message gets a unique AES private encryption key and a unique Elliptic-curve Diffie–Hellman (ECDH) public key to negotiate the key exchange with recipients. Message content—including text, files, audio, or video—is encrypted on the sending device (your iPhone, for example) by using the message-specific AES key. This key is then exchanged by using the ECDH key exchange mechanism, so that only intended recipients can decrypt the message.

Quantum computing and post-quantum cryptography

Quantum computing is a field of technology that uses quantum mechanics to solve complex problems faster than on classical computers. Quantum computers are able to solve certain types of problems faster by taking advantage of quantum mechanical effects, such as superposition and quantum interference. For cryptography, this has implications that affect traditional encryption mechanisms such as asymmetric key encryption, which is often used for protecting data in transit (TLS) or creating hash-based signatures to verify the integrity and authenticity of a message or file. Quantum computers, if they are performant and stable enough, could theoretically compromise the security of asymmetric key algorithms like RSA, Elliptic Curve Cryptography (ECC), or Diffie-Hellman key agreement schemes. Based on current research, symmetric key algorithms like AES are not considered to be at risk from a quantum computer, because the key length of 256 bits is already sufficient to compensate for a decrease in cryptographic key strength posed by quantum algorithms.

AWS gives customers the option of evaluating post-quantum algorithms alongside traditional algorithms, using hybrid schemes that make use of both classic cryptography and newer post-quantum cryptographic (PQC) algorithms that are designed to be resistant to quantum computer threats. AWS has taken the first step in deploying PQC by implementing ML-KEM, a module lattice-based key encapsulation mechanism, within AWS-LC, our open source FIPS-140-3 validated cryptographic library. AWS-LC is the core cryptographic library used throughout AWS. Specifically, AWS-LC is used in s2n-tls, our open source TLS implementation used across AWS services with HTTPS-based endpoints.

We have also deployed post-quantum s2n-tls with AWS KMS, AWS Certificate Manager (ACM), and AWS Secrets Manager TLS endpoints—bringing the benefits of post-quantum cryptography to customers who enable hybrid post-quantum TLS in their AWS SDK to connect to those services. AWS Transfer Family also supports post-quantum, hybrid SFTP file transfers. You can read more about our efforts to migrate more AWS managed service endpoints to PQC over TLS in this AWS Security blog post. You can also find information about the work of Amazon and AWS in cryptographic research and improvements on the Amazon Science Blog.

Encrypt everything, everywhere

AWS provides customers the ability to encrypt everything, everywhere. Customers can encrypt data at rest, in transit, and in memory, with a few clicks in the AWS Management Console, or an AWS API call. Services like Amazon Simple Storage Service (Amazon S3) encrypt new objects by default, and also support the use of customer managed AWS KMS keys to give customers more control over their encryption keys. Importantly, AWS KMS uses techniques like envelope encryption and highly scalable key management infrastructure to enable AWS services like Amazon S3 or Amazon Elastic Block Store (Amazon EBS) to encrypt data with minimal performance impact to customer applications.

AWS is also consistently working to improve the performance and security of our customers’ data as it moves between networks or devices. As of June 2024, all AWS API endpoints support TLS 1.3 and require at least TLS 1.2 or higher. By using TLS 1.3, you can decrease your connection time by removing one network round trip for every connection request, and can benefit from some of the most modern and secure cryptographic cipher suites available today.

Customers who require memory encryption can use AWS Graviton, our custom-built family of processors based on ARM. AWS Graviton2, AWS Graviton3, and AWS Graviton3E support always-on memory encryption. The encryption keys are securely generated within the host system, do not leave the host system, and are destroyed when the host is rebooted or powered down. Memory encryption is also supported for other instance types; see the EC2 documentation for more details.

As part of our AWS Digital Sovereignty Pledge, we commit to continue to innovate and invest in additional controls for encryption features so that our customers can encrypt everything, everywhere with encryption keys managed inside or outside the AWS Cloud.

Summary

At AWS, security is our top priority. We are committed to helping you control how your data is used, who has access to it, and how it is protected. By building and supporting encryption tools that work both on and off the cloud, we help you secure your data and enable compliance across your environment. We put security at the center of everything we do to make sure that you can protect your data using best-of-breed security technology in a cost-effective way.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on the AWS KMS forum or the AWS CloudHSM forum, or contact AWS Support.

Amazon S3 Express One Zone now supports AWS KMS with customer managed keys

2024-09-18 Elizabeth Fuentes

Post Syndicated from Elizabeth Fuentes original https://aws.amazon.com/blogs/aws/amazon-s3-express-one-zone-now-supports-aws-kms-with-customer-managed-keys/

Amazon S3 Express One Zone, a high-performance, single-Availability Zone (AZ) S3 storage class, now supports server-side encryption with AWS Key Management Service (KMS) keys (SSE-KMS). S3 Express One Zone already encrypts all objects stored in S3 directory buckets with Amazon S3 managed keys (SSE-S3) by default. Starting today, you can use AWS KMS customer managed keys to encrypt data at rest, with no impact on performance. This new encryption capability gives you an additional option to meet compliance and regulatory requirements when using S3 Express One Zone, which is designed to deliver consistent single-digit millisecond data access for your most frequently accessed data and latency-sensitive applications.

S3 directory buckets allow you to specify only one customer managed key per bucket for SSE-KMS encryption. Once the customer managed key is added, you cannot edit it to use a new key. On the other hand, with S3 general purpose buckets, you can use multiple KMS keys either by changing the default encryption configuration of the bucket or during S3 PUT requests. When using SSE-KMS with S3 Express One Zone, S3 Bucket Keys are always enabled. S3 Bucket Keys are free and reduce the number of requests to AWS KMS by up to 99%, optimizing both performance and costs.

Using SSE-KMS with Amazon S3 Express One Zone
To show you this new capability in action, I first create an S3 directory bucket in the Amazon S3 console following the steps to create a S3 directory bucket and use apne1-az4 as the Availability Zone. In Base name, I enter s3express-kms and a suffix that includes the Availability Zone ID wich is automatically added to create the final name. Then, I select the checkbox to acknowledge that Data is stored in a single Availability Zone.

In the Default encryption section, I choose Server-side encryption with AWS Key Management Service keys (SSE-KMS). Under AWS KMS Key I can Choose from your AWS KMS keys, Enter AWS KMS key ARN, or Create a KMS key. For this example, I previously created an AWS KMS key, which I selected from the list, and then choose Create bucket.

Now, any new object I upload to this S3 directory bucket will be automatically encrypted using my AWS KMS key.

SSE-KMS with Amazon S3 Express One Zone in action
To use SSE-KMS with S3 Express One Zone via the AWS Command Line Interface (AWS CLI), you need an AWS Identity and Access Management (IAM) user or role with the following policy . This policy allows the CreateSession API operation, which is necessary to successfully upload and download encrypted files to and from your S3 directory bucket.

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Action": [
				"s3express:CreateSession"
			],
			"Resource": [
				"arn:aws:s3express:*:<account>:bucket/s3express-kms--apne1-az4--x-s3"
			]
		},
		{
			"Effect": "Allow",
			"Action": [
				"kms:Decrypt",
				"kms:GenerateDataKey"
			],
			"Resource": [
				"arn:aws:kms:*:<account>:key/<keyId>"
			]
		}
	]
}

With the PutObject command, I upload a new file named confidential-doc.txt to my S3 directory bucket.

aws s3api put-object --bucket s3express-kms--apne1-az4--x-s3 \
--key confidential-doc.txt \
--body confidential-doc.txt

As a success of the previous command I receive the following output:

{
    "ETag": "\"664469eeb92c4218bbdcf92ca559d03b\"",
    "ChecksumCRC32": "0duteA==",
    "ServerSideEncryption": "aws:kms",
    "SSEKMSKeyId": "arn:aws:kms:ap-northeast-1:<accountId>:key/<keyId>",
    "BucketKeyEnabled": true
}

Checking the object’s properties with HeadObject command, I see that it’s encrypted using SSE-KMS with the key that I created before:

aws s3api head-object --bucket s3express-kms--apne1-az4--x-s3 \
--key confidential-doc.txt

I get the following output:

 
{
    "AcceptRanges": "bytes",
    "LastModified": "2024-08-21T15:29:22+00:00",
    "ContentLength": 5,
    "ETag": "\"664469eeb92c4218bbdcf92ca559d03b\"",
    "ContentType": "binary/octet-stream",
    "ServerSideEncryption": "aws:kms",
    "Metadata": {},
    "SSEKMSKeyId": "arn:aws:kms:ap-northeast-1:<accountId>:key/<keyId>",
    "BucketKeyEnabled": true,
    "StorageClass": "EXPRESS_ONEZONE"
}

I download the encrypted object with GetObject:

aws s3api get-object --bucket s3express-kms--apne1-az4--x-s3 \
--key confidential-doc.txt output-confidential-doc.txt

As my session has the necessary permissions, the object is downloaded and decrypted automatically.

{
    "AcceptRanges": "bytes",
    "LastModified": "2024-08-21T15:29:22+00:00",
    "ContentLength": 5,
    "ETag": "\"664469eeb92c4218bbdcf92ca559d03b\"",
    "ContentType": "binary/octet-stream",
    "ServerSideEncryption": "aws:kms",
    "Metadata": {},
    "SSEKMSKeyId": "arn:aws:kms:ap-northeast-1:<accountId>:key/<keyId>",
    "BucketKeyEnabled": true,
    "StorageClass": "EXPRESS_ONEZONE"
}

For this second test, I use a different IAM user with a policy that is not granted the necessary KMS key permissions to download the object. This attempt fails with an AccessDenied error, demonstrating that the SSE-KMS encryption is functioning as intended.

An error occurred (AccessDenied) when calling the CreateSession operation: Access Denied

This demonstration shows how SSE-KMS works seamlessly with S3 Express One Zone, providing an additional layer of security while maintaining ease of use for authorized users.

Things to know
Getting started – You can enable SSE-KMS for S3 Express One Zone using the Amazon S3 console, AWS CLI, or AWS SDKs. Set the default encryption configuration of your S3 directory bucket to SSE-KMS and specify your AWS KMS key. Remember, you can only use one customer managed key per S3 directory bucket for its lifetime.

Regions – S3 Express One Zone support for SSE-KMS using customer managed keys is available in all AWS Regions where S3 Express One Zone is currently available.

Performance – Using SSE-KMS with S3 Express One Zone does not impact request latency. You’ll continue to experience the same single-digit millisecond data access.

Pricing – You pay AWS KMS charges to generate and retrieve data keys used for encryption and decryption. Visit the AWS KMS pricing page for more details. In addition, when using SSE-KMS with S3 Express One Zone, S3 Bucket Keys are enabled by default for all data plane operations except for CopyObject and UploadPartCopy, and can’t be disabled. This reduces the number of requests to AWS KMS by up to 99%, optimizing both performance and costs.

AWS CloudTrail integration – You can audit SSE-KMS actions on S3 Express One Zone objects using AWS CloudTrail. Learn more about that in my previous blog post.

– Eli.

Build a mobile driver’s license solution based on ISO/IEC 18013-5 using AWS Private CA and AWS KMS

2024-09-04 Ram Ramani

Post Syndicated from Ram Ramani original https://aws.amazon.com/blogs/security/build-a-mobile-drivers-license-solution-based-on-iso-iec-18013-5-using-aws-private-ca-and-aws-kms/

A mobile driver’s license (mDL) is a digital representation of a physical driver’s license that’s stored on a mobile device. An mDL is a significant improvement over physical credentials, which can be lost, stolen, counterfeited, damaged, or contain outdated information, and can expose unconsented personally identifiable information (PII). Organizations are working together to use mDLs across various situations, ranging from validating identity during airplane boarding to sharing information for age-restricted activities.

The trust in the mDL system is based on public-private key cryptography where mDLs are signed by issuing authorities using their private key and verified using the issuing authority’s public key. In this blog post, we show you how to build an mDL issuing authority in Amazon Web Services (AWS) using AWS Private Certificate Authority and AWS Key Management Service (AWS KMS) according to mDL specification ISO/IEC 18013-5:2021. These AWS services align with the cryptographic requirements placed on the issuing authorities by ISO/IEC 18013-5. While we have tailored this post to an mDL use case, the sign and verify mechanism using AWS Private CA and AWS KMS can be used for multiple kinds of digital identity verification.

Solution overview

AWS Private CA provides you with a highly available private certificate authority (CA) service without the initial investment and ongoing maintenance costs of operating your own private CA. CA administrators can use AWS Private CA to create a complete CA hierarchy, including online root and subordinate CAs, without needing external CAs. You can issue, rotate, and revoke certificates that are trusted within your organization using AWS Private CA.

AWS Private CA can issue certificates formatted as required by ISO/IEC 18013-5. You can build a certificate authority (CA) in AWS Private CA—referred to as the issuing authority certificate authority (IACA) in ISO/IEC 18013-5. We create an IACA self-signed root certificate and an mDL document signing certificate in AWS Private CA.

AWS KMS is a managed service that you can use to create and control the cryptographic keys that are used to protect your data. AWS KMS uses FIPS 140-2 Level 3 validated hardware security modules (HSMs) to protect AWS KMS keys, which is a requirement for building an issuing authority as described in ISO/IEC 18013-5. We create an asymmetric key pair in AWS KMS for signing and verification of the mDL document. We programmatically create a certificate signing request (CSR) that’s signed by the asymmetric key pair stored in AWS KMS. The CSR is sent to the AWS Private CA service for issuing the mDL document signing certificate that matches the certificate profile requirement specified for the document signing certificate in ISO/IEC 18013-5.

We sign an mDL document using the private key of the asymmetric key pair created in AWS KMS with a KeyUsage value of SIGN_VERIFY. The signed mDL document is delivered to a mobile device where it’s stored in a digital wallet and produced for verification by mDL readers. The mDL readers are configured with IACA certificates from various issuing authorities that allow them to verify the mDL documents signed by respective issuing authorities. An example of an issuing authority could be a state government agency that issues driver’s licenses.

Least privilege

The solution in this post uses AWS KMS and AWS Private CA services. Before you implement the process described in this post, ensure that the AWS Identity and Access Management (IAM) principal you choose follows the principle of least privilege and that permissions are scoped to the minimum required permissions required. See Security best practices in IAM to learn more.

Solution architecture

A sample solution architecture for building an mDL issuing authority in AWS is shown in Figure 1. The figure shows the step-by-step process starting from setting up a private CA and issuing an mDL document signing certificate to mDL issuance and verification. The infrastructure that’s built using this architecture includes a root certificate authority, which issues a document signer certificate. You can find the certificate requirements in section B.1 Certificate Profile of ISO/IEC 18013-5.

Figure 1: mDL issuing authority architecture and process flow in AWS

In this post, we use AWS Command Line Interface (AWS CLI) commands, but these can be replaced by AWS SDK API calls if needed. Along with the AWS CLI steps, a GitHub sample is provided that’s used to programmatically create and sign an mDL document signing CSR using AWS KMS.

See the AWS CLI commands documentation for AWS Private CA and AWS KMS for detailed information on the commands used in this solution.

Solution walkthrough

Use the following steps to create the infrastructure needed for mDL signing and verification.

Step 1: Create IACA CA in AWS Private CA

In this step, the root of trust IACA (issuing authority CA) will be created. The IACA root CA is the root of trust that will be used for verification of the mDL.

Create a local ca_config.txt file with the following content. The contents of this file are derived from the Certificate profiles section (Annex B) within ISO/IEC 18013-5. You can change the Country and CommonName values in the file as needed for your requirements.
```
{
  "KeyAlgorithm": "EC_prime256v1",
  "SigningAlgorithm": "SHA256WITHECDSA",
  "Subject": {
    "Country": "US",
    "CommonName": "mDL IACA Root"
  }
}
```
The IACA root certificate will be paired with a certificate revocation list (CRL). See Planning a certificate revocation list (CRL) for information about configuring CRLs. Create a local file called revocation_config.txt with the following information to configure a CRL. The values for CustomCname and S3BucketName are examples, update them with the values that you have created within your AWS account. Update ExpirationInDays to fit your requirements. We recommend configuring encryption on the Amazon Simple Storage Service (Amazon S3) bucket containing your CRLs.
```
{
  "CrlConfiguration": {
    "CustomCname": "example.com",
    "Enabled": true,
    "S3BucketName": "crlmdlbucket",
    "ExpirationInDays": 5000,  
  }
}
```
Invoke an AWS CLI command to create a private certificate authority. Replace the region parameter as needed. Update the file:// paths in the following command to the locations where you’ve stored the ca_config.txt and revocation_config.txt files.
```
aws acm-pca create-certificate-authority \ 
    --region us-west-1 \
    --certificate-authority-configuration file://ca_config.txt \
    --revocation-configuration file://revocation_config.txt \
    —-certificate-authority-type "ROOT"
```
The command should produce the following output. The output contains the Amazon Resource Name (ARN) of the created CA. You will need this ARN in subsequent steps.
```
{
    "CertificateAuthorityArn": "arn:aws:acm-pca:us-west-1:123412345678:certificate-authority/0116z123-dv7a-59b1-x7be-1231v7257113"
}
```

Step 2: Retrieve the CSR for IACA root certificate

You’ll create an IACA root certificate, which starts with retrieving a CSR. This step retrieves the CSR for the IACA root certificate. The certificate-authority-arn parameter carries the CA ARN that was generated in Step 1.

The following command will output a Privacy-Enhanced Mail (PEM) formatted CSR.

aws acm-pca get-certificate-authority-csr \
    --region us-west-1 \
    --output text \
    --certificate-authority-arn arn:aws:acm-pca:us-west-1:123412345678:certificate-authority/0116z123-dv7a-59b1-x7be-1231v7257113

The following is the format of the output CSR:

-----BEGIN CERTIFICATE REQUEST-----
..
-----END CERTIFICATE REQUEST-----

Store the output text in a file called IACA.csr.

Step 3: Generate root certificate

This step issues the IACA root certificate. Create a file named extensions.txt using the following contents, which are derived from the Certificate profiles section of ISO/IEC 18013-5.
The KeyUsage extension with KeyCertSign and CRLSign should be set to true. A custom extension for the CRL distribution point is set and the validity of the certificate should be set to 9 years or 3285 days (set in the next step). Because the IACA root certificate is only used to issued mDLs, a maximum validity period of 9 years is sufficient, as indicated in Table B.1 of ISO/IEC 18013-5. Additionally, a CRL distribution point extension must be present. In the following example, the CRL URL encoded in the CDP extension is http://example.com/crl/0116z123-dv7a-59b1-x7be-1231v72571136.crl, aligning with both the CA CRL configuration applied to the CA at creation and to the CA ID. For base-64 encoding of the CDP extension, you can refer to this java sample.
```
{
  "Extensions": {
    "KeyUsage": {
      "KeyCertSign": true,
      "CRLSign": true
    },
    "CustomExtensions": [
      {
        "ObjectIdentifier": "2.5.29.31",
        "Value": "MEgwRqBEoEKGQGh0dHA6Ly9leGFtcGxlLmNvbS9jcmwvMDExNnoxMjMtZHY3YS01OWIxLXg3YmUtMTIzMXY3MjU3MTEzNi5jcmw="
       }
    ]
  }
}
```

Issue the following command to AWS Private CA to create the certificate.

aws acm-pca issue-certificate \
    --region us-west-1 \
    --certificate-authority-arn arn:aws:acm-pca:us-west-1:123412345678:certificate-authority/0116z123-dv7a-59b1-x7be-1231v7257113 \
    --template-arn "arn:aws:acm-pca:::template/BlankRootCACertificate_PathLen0_APIPassthrough/V1" \
    --signing-algorithm "SHA256WITHECDSA" \
    --csr fileb://IACA.csr \
    --validity Value=3285,Type="DAYS" \
    --api-passthrough file://extensions.txt

The preceding command will produce the following output:

{
  "CertificateArn": "arn:aws:acm-pca:us-west-1:123412345678:certificate-authority/0116z123-dv7a-59b1-x7be-1231v7257113/certificate/34a1dab03117f0e89c54b1234fe13318"
}

Note that the IACA root CA created with AWS Private CA currently doesn’t have a CRL distribution point (CDP) extension by default. However, that is a mandatory extension according to the IACA root certificate profile in ISO/IEC 18013-5. To implement this, we use a custom extension passed in using API passthrough, which embeds the CDP extension. The distribution point specified in that extension must be based on the CA ID, which is 0116z123-dv7a-59b1-x7be-1231v7257113 derived from the CA ARN that was created in Step 1.

Step 4: Retrieve root certificate

This step retrieves the IACA root certificate in PEM format.

Use the following code to retrieve the IACA root certificate:

aws acm-pca get-certificate \
    --region us-west-1 \
    --certificate-authority-arn arn:aws:acm-pca:us-west-1:123412345678:certificate-authority/0116z123-dv7a-59b1-x7be-1231v7257113 \
    --certificate-arn arn:aws:acm-pca:us-west-1:123412345678:certificate-authority/0116z123-dv7a-59b1-x7be-1231v7257113/certificate/34a1dab03117f0e89c54b1234fe13318 \
    --output text

The command output will be a PEM formatted certificate similar to the following:
```
-----BEGIN CERTIFICATE-----
..
-----END CERTIFICATE-----
```
Store the output text in a file named IACA-Root-CA-Cert.pem.

Step 5: Import root certificate

Use the following code to import the root certificate into AWS Private CA and make the certificate authority active and ready to issue certificates.

aws acm-pca import-certificate-authority-certificate \
    --region us-west-1 \
    --certificate-authority-arn arn:aws:acm-pca:us-west-1:123412345678:certificate-authority/0116z123-dv7a-59b1-x7be-1231v7257113 \
    --certificate fileb://IACA-Root-CA-Cert.pem

You should see success after running the command.

Step 6: Create an asymmetric key in AWS KMS

In this step, create an asymmetric signing key in AWS KMS which will be used to sign the mDL document signing CSR.

Use the following command to create an asymmetric key:

aws kms create-key \
    --region us-west-1 \
    --key-spec ECC_NIST_P256 \
    --key-usage SIGN_VERIFY

The command should produce the following output:

{
  "KeyMetadata": {
    "AWSAccountId": "123412345678",
    "KeyId": "3ab87971-1fe2-45d9-955a-5dc7f65558zf",
    "Arn": "arn:aws:kms:us-west-1:123412345678:key/3ab87971-1fe2-45d8-955c-5dc7f65558ef",
    "CreationDate": "2024-05-18T19:53:27.318000+00:00",
    "Enabled": true,
    "Description": "",
    "KeyUsage": "SIGN_VERIFY",
    "KeyState": "Enabled",
    "Origin": "AWS_KMS",
    "KeyManager": "CUSTOMER",
    "CustomerMasterKeySpec": "ECC_NIST_P256",
    "KeySpec": "ECC_NIST_P256",
    "SigningAlgorithms": [
      "ECDSA_SHA_256"
    ],
    "MultiRegion": false
  }
}

Note the Arn value from the output. You will use it in Step 7 to configure the CSR creation utility for the mDL document signing certificate.

Step 7: Use the CSR creation utility to generate the document signing CSR

We published a sample utility in GitHub that creates a CSR signed by an AWS asymmetric key.

Clone the GitHub repository and then follow the instructions in the README file from the repository to configure and run it.

This program will output a PEM formatted CSR similar to the following:

-----BEGIN CERTIFICATE REQUEST-----
..
-----END CERTIFICATE REQUEST-----

Copy the output and store it in a file named document-signing-kms.csr. You will use the file in Step 8 to create the mDL document signing certificate based on this CSR.

Step 8: Generate an mDL document signing certificate

This step creates the document signing certificate from the CSR that’s signed using the AWS KMS asymmetric key.

Create a file named extensionSigner.txt with the following contents. The contents of this file are derived from the Certificate profiles section of ISO/IEC 18013-5. The JSON snippet that follows shows the extension structure containing the KeyUsage extension with DigitalSignature field set to true.
```
{
     "Extensions": {
         "KeyUsage": {
             "DigitalSignature": true
         },
         "ExtendedKeyUsage": [
             {
                 "ExtendedKeyUsageObjectIdentifier": "1.0.18013.5.1.2"
             }
         ]
     }
}
```

Use the following AWS CLI command to create the certificate.

aws acm-pca issue-certificate \
    --region us-west-1 \
    --certificate-authority-arn arn:aws:acm-pca:us-west-1:123412345678:certificate-authority/0116z123-dv7a-59b1-x7be-1231v7257113 \
    --template-arn "arn:aws:acm-pca:::template/BlankEndEntityCertificate_APIPassthrough/V1" \
    --signing-algorithm "SHA256WITHECDSA" \
    --csr fileb://document-signing-kms.csr \
    --validity Value=1825,Type="DAYS" \
    --api-passthrough file://extensionSigner.txt

Output:

{
    "CertificateArn": "arn:aws:acm-pca:us-west-1:123412345678:certificate-authority/0116z123-dv7a-59b1-x7be-1231v7257113/certificate/d462fcd3b9h3beb45c7c312241d42fba"
}

You will use the CertificateArn from the output in Step 9 to retrieve the mDL document signing certificate.

Step 9: Retrieve the mDL document signing certificate

This step retrieves the document signing certificate in PEM format from AWS Private CA.

Use the following command to retrieve the document signing certificate:

aws acm-pca get-certificate \
    --region us-west-1 \
    --certificate-authority-arn arn:aws:acm-pca:us-west-1:123412345678:certificate-authority/0116z123-dv7a-59b1-x7be-1231v7257113 \
    --certificate-arn arn:aws:acm-pca:us-west-1:123412345678:certificate-authority/0116z123-dv7a-59b1-x7be-1231v7257113/certificate/d462fcd3b9h3beb45c7c312241d42fba \
    --output text

Store the output text in document_signing_cert.pem.

You now have the mDL document signing certificate for packaging later with the Concise Binary Object Representation (CBOR) structure required by ISO/IEC 18013-5.

Step 10: mDL reader ingests issuing authority’s mDL signing certificate chain

An mDL reader can trust the mDL presented by a user after cryptographically verifying the mDL. This verification requires the reader to possess the mDL signing certificate chain of the issuing authority that issued the user the mDL. As required by the decentralized public key infrastructure (PKI) trust model specified in ISO/IEC 18013-5, the mDL reader will ingest the mDL signing certificate chain of the issuing authority.

Step 11: User makes an mDL signing request to the issuing authority

The user makes a request to the issuing authority to sign the mDL.

Step 12: Issuing authority issues signed mDL to the user

The issuing authority will authenticate the user’s identity and issue a signed mDL. The issuing authority provisions mDL data to the user’s device along with a CBOR encoded object known as a mobile security object (MSO). MSOs contain a digest algorithm, individual digests of mDL data elements, and a validity period. After this MSO has been generated and encoded as required by ISO/IEC 18013-5:2021 section 9.1.2.4, the MSO can be signed by the issuing authority. This signature can be generated in AWS KMS as shown in the following command. Generating the encoded MSO is out of scope for this post.

Use the following command to produce the SHA-256 digest of encoded MSO object using the sha256sum utility.
```
sha256sum < EncodedMSO > EncodedMSODigest
```

Sign the digest using the AWS KMS asymmetric key created in Step 6.

aws kms sign \
 --region us-west-1 \
 --key-id 3ab87971-1fe2-45d8-955c-5dc7f65558ef \
 --message fileb://EncodedMSODigest \
 --message-type DIGEST \
 --signing-algorithm ECDSA_SHA_256 \
 --output text \
 --query Signature | base64 --decode

This signature will be combined with the issuing authority certificate and the MSO to form a CBOR Object Signing and Encryption (COSE) signed message and will be presented with the mDL data elements to readers. Readers can validate this signature to confirm the integrity of the MSO.

Step 13: User presents their mDL to an mDL reader

The user presents their mDL to the mDL reader for identity verification, such as at an airport. This process is called mDL Initialization in ISO/IEC 18013-5:2021 section 6.3.2.2. The mDL is activated during this initialization step.

Step 14: An mDL reader requests mDL data from a user’s mobile device

The mDL reader issues an mDL retrieval request to the user’s mobile device. A key feature of mDLs is that they allow mDL holders to present a subset of their PII. An mDL reader will request specific attributes such as name and date of birth, requiring the mDL holder to consent to the release of this information. The mDL reader’s request contains the list of PII data element identifiers that it is requesting the mDL holder to share.

Step 15: User consents to share their mDL data

The user receives a prompt notifying them of mDL sharing request. This prompt shows the user the list of PII data elements that are being requested. The user consents to the request and the mDL data that includes the MSO is shared with the reader.

Step 16: Reader validates mDL integrity

The reader receives the mDL data and validates it for integrity. The inclusion of the MSO with the mDL data elements provides mDL readers with a mechanism for validating the integrity of the data they’ve received. The mDL reader can then hash and verify individual mDL data elements presented by the device. If all data elements match their corresponding entries in the MSO, the mDL device reader can attest that the data hasn’t been tampered with.

As an example, assume that the mDL contains the following data elements:

24(<<
  {
    "digestID": 0,
    "random": h'BBA394B98088CAE238D35979F7210E18DFAF70354524D86149CA20046E4321B1',
    "elementIdentifer": "given_name",
    "elementValue": "John"
  }
>>),
24(<<
  {
    "digestID": 1,
    "random": h'901F63FD880A15B30EDCEEFA857201C52FB9EAD1D39C15BB592829D16CB8A368',
    "elementIdentifer": "family_name",
    "elementValue": "Doe"
  }
>>)

And a Mobile Security Object containing the following data element digests:

24(<<
  {
    "version": "1.0",
    "digestAlgorithm": "SHA-256",
    "valueDigests":
    {
      "org.iso.18013.5.1":
      {
        0: h’D6AA81E454036313A9A681809151DDDBDF702289094F18286DDC591C41C6434E',
        1: h'4C3D83940CA8C5DE8060A23EB649C175E79B745B6A7D9939B4D16B3E46BB14D5'
      }
    }
  }
>>)

The MSO’s integrity would first confirm that the validity period of the MSO (not shown) has not expired. It can then verify the signature (not shown) with the issuing authority’s public key. After this has been established, both data elements need to be verified. The CBOR representation of each element (digestID, random, elementIdentifier, and elementValue) is encoded as bytes and then hashed using SHA-256. For example, the following should equal D6AA81E454036313A9A681809151DDDBDF702289094F18286DDC591C41C6434E.

SHA256(CBOR byte representation of 24(<<
    {
      "digestID": 0,
      "random": h'BBA394B98088CAE238D35979F7210E18DFAF70354524D86149CA20046E4321B1',
      "elementIdentifer": "given_name",
      "elementValue": "John"
    }
  >>))
)

Likewise, the following example should equal
4C3D83940CA8C5DE8060A23EB649C175E79B745B6A7D9939B4D16B3E46BB14D5.

SHA256(CBOR byte representation of 24(<<
    {
      "digestID": 1,
      "random": h'901F63FD880A15B30EDCEEFA857201C52FB9EAD1D39C15BB592829D16CB8A368',
      "elementIdentifer": "family_name",
      "elementValue": "Doe"
    }
  >>)))

If all data elements pass this hash verification check, then the presented mDL contents can be trusted by the mDL reader.

Summary

As you saw in this solution, mobile driver’s licenses (mDLs) provide increased security and flexible consent management to preserve privacy for individuals. The principles of cryptographic signing and verification aren’t new and both AWS KMS and AWS Private CA are well suited for supporting digital identity applications, whether it’s a driver’s license or some other kind of identification. To learn more about AWS KMS asymmetric keys and AWS Private CA, see Digital signing with the new asymmetric keys feature of AWS KMS and How to host and manage an entire private certificate infrastructure in AWS.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on the AWS Certificate Manager re:Post and AWS AWS Key Management Service re:Post, or contact AWS Support.

Announcing AWS KMS Elliptic Curve Diffie-Hellman (ECDH) support

2024-08-19 Patrick Palmer

Post Syndicated from Patrick Palmer original https://aws.amazon.com/blogs/security/announcing-aws-kms-elliptic-curve-diffie-hellman-ecdh-support/

When using cryptography to protect data, protocol designers often prefer symmetric keys and algorithms for their speed and efficiency. However, when data is exchanged across an untrusted network such as the internet, it becomes difficult to ensure that only the exchanging parties can know the same key. Asymmetric key pairs and algorithms help to solve this problem by allowing a public key to be shared over an untrusted network. And by using a key agreement scheme, two parties can use each other’s public key in combination with their own private key to each derive the same shared secret.

We’re excited to announce that AWS Key Management Service (AWS KMS) now supports Elliptic Curve Diffie-Hellman (ECDH) key agreement on elliptic curve (ECC) KMS keys. You can use the new DeriveSharedSecret API action to enable two parties to establish a secure communication channel by using a derived shared secret.

In this blog post we provide an overview of the new API action and explain how it can help you establish secure communications by exchanging only public keys to obtain a derived shared secret. We then show example commands to demonstrate how AWS KMS and OpenSSL can be used by two parties to derive a shared secret.

With this new DeriveSharedSecret API action, customers can take an external party’s public key and, in combination with a private key that resides within AWS KMS, derive a shared secret which can be used to derive a symmetric encryption key with a key derivation function (KDF). Customers can then use this symmetric encryption key to encrypt data locally within their application.

The same external party can combine their own related private key with the customer’s corresponding public key from AWS KMS to derive the same shared secret.

Now that both parties have the same shared secret, they can generate a symmetric encryption key that can be used to encrypt and decrypt the data they exchange.

DeriveSharedSecret offers a simple and secure way for customers to use their private key from within their application, enabling new asymmetric cryptography use cases for keys protected by AWS KMS, such as elliptic curve integrated encryption scheme (ECIES) or end-to-end encryption (E2EE) schemes.

AWS KMS DeriveSharedSecret overview

The AWS KMS API Reference documentation covers the DeriveSharedSecret API action in more detail than we include in this post. We broadly describe how to interact with the API action, using the following steps:

Create an elliptic curve (ECC) KMS key, selecting that the key be used for KEY_AGREEMENT and choosing one of the supported key specs. You will not be able to modify existing ECC keys to be used for key agreement.
Have another party create an elliptic curve key that matches the key spec you defined for your KMS key.
Retrieve the public key associated with your KMS key by using the existing GetPublicKey API action.
Exchange public keys through a trusted means of exchange with the other party. Note that DeriveSharedSecret expects a base64-encoded DER-formatted public key.
Use the other party’s public key as an input, along with your specified KEY_AGREEMENT key. The only key agreement algorithm supported by AWS KMS at launch is ECDH.
The other party should use the public key retrieved from AWS KMS and the private key associated with their generated ECC key pair to derive a shared secret.

The result of the preceding steps is that both parties have the same output without exchanging secret information. Only public keys were exchanged between the two parties. The output of DeriveSharedSecret is the raw shared secret. This shared secret is the multiplication of points on the elliptic curves and can result in many more bytes than are needed for an encryption key. We recommend that customers use a KDF, following the National Institute of Standards and Technology (NIST) SP800-56A Rev. 3 section 5.8 guidance, to derive encryption keys from this shared secret.

For the purposes of this post, we will demonstrate the steps by using the AWS CLI and OpenSSL command line. AWS has incorporated best practices for customers within the AWS Encryption SDK. You can find more details at AWS KMS ECDH keyrings.

Example use case

An example use case where you might wish to use ECDH key agreement is for end-to-end encryption. Although protocols exist that provide a secure framework for secure communications (for example, within AWS Wickr), we will highlight the simplified high-level steps behind some of these protocols. In our example use case, Alice and Bob are both part of a messaging network. This network is managed by a centralized service, and this service must not be able to access Alice or Bob’s unencrypted messages.

Figure 1: High-level architecture for the service described in the example use case

As shown in Figure 1, Alice and Bob each have an ECC key pair and participate in the secret derivation by using ECDH, through the following steps:

Alice registers her public key in the centralized key storage service. A detailed discussion of the key storage service is beyond the scope of this post.
Bob, an AWS KMS user, calls the AWS KMS GetPublicKey action to obtain the public key for the ECC KMS key pair.
Bob registers his public key in the same centralized key storage service.
Alice, who wants to exchange encrypted messages with Bob, retrieves Bob’s public key from the centralized key storage service.
Bob gets a notification that Alice wants to communicate with him, and he retrieves Alice’s public key from the centralized key storage service.
Using Bob’s public key and her private key, Alice derives a shared secret by using her cryptography provider.
Using Alice’s public key and his private key, Bob derives a shared secret by using DeriveSharedSecret.
Alice and Bob now have an identical shared secret. From this shared secret, she can create a symmetric encryption key by using a suitable KDF. The symmetric encryption key can be used to create ciphertext that can be sent to Bob.

Example use case walkthrough

You can use the following steps to create a KMS key for ECDH use and derive a shared secret by using AWS KMS. For our demonstration purposes, the user Alice (from our example use case) is using OpenSSL as the cryptography tool. We will show how the AWS KMS user Bob and OpenSSL user Alice can derive a shared secret by using each other’s public key.

General prerequisites

You must have the following prerequisites in place in order to implement the solution:

AWS CLI — The latest version is recommended. The example here uses aws-cli/2.15.40 and aws-cli/1.32.110.
OpenSSL — The example here uses OpenSSL 3.3.0.
Both parties (Alice and Bob, from our example use case) have an ECC key on the same curve. The steps in the next section, Key creation prerequisite, explain how these keys can be created.

Key creation prerequisite

Alice and Bob must use the same ECC curve during key creation. The DeriveSharedSecret API action supports curves ECC_NIST_P256, ECC_NIST_P384, and ECC_NIST_P521, which map to P-256, P-384, and P-521 respectively in OpenSSL. The curves that AWS KMS supports are the curves approved by the U.S. National Institute of Standards and Technology (NIST). Additionally, AWS KMS supports the SM2 key spec only in Amazon Web Services China Regions.

Bob creates an asymmetric KMS key for key agreement purposes

Bob creates a key pair in AWS KMS by using the CreateKey API action. In the following example, Bob creates an ECC key pair with ECC_NIST_P256 for the KeySpec parameter and KEY_AGREEMENT for the KeyUsage parameter.

aws kms create-key \
--key-spec ECC_NIST_P256 \
--key-usage KEY_AGREEMENT \
--description "Example ECDH key pair"

The response looks something like this:

{
    "KeyMetadata": {
        "AWSAccountId": "111122223333",
        "KeyId": "a1b2c3d4-5678-90ab-cdef-EXAMPLE11111",
        "Arn": "arn:aws:kms:us-east-1:111122223333:key/a1b2c3d4-5678-90ab-cdef-EXAMPLE11111",
        "CreationDate": "2024-06-25T13:06:24.888000-07:00",
        "Enabled": true,
        "Description": "Example ECDH key pair",
        "KeyUsage": "KEY_AGREEMENT",
        "KeyState": "Enabled",
        "Origin": "AWS_KMS",
        "KeyManager": "CUSTOMER",
        "CustomerMasterKeySpec": "ECC_NIST_P256",
        "KeySpec": "ECC_NIST_P256",
        "KeyAgreementAlgorithms": [
            "ECDH"
        ],
        "MultiRegion": false
    }
}

You can follow the Creating asymmetric KMS keys documentation to see how to use the AWS Management Console to create a KMS key pair with the same properties as shown here. This example creates a KMS key with a default KMS key policy. You should review and configure your key policy according to the principle of least privilege, as appropriate for your environment.

Note: When a KMS key is created, it will be logged by AWS CloudTrail, a service that monitors and records activity within your account. API calls to the AWS KMS service are logged in CloudTrail, which you can use to audit access to KMS keys.

To allow your KMS key to be identified by a human-readable string rather than by the KeyId value, you can create an alias for the KMS key (replace the target-key-id value of a1b2c3d4-5678-90ab-cdef-EXAMPLE11111 with your KeyId value). This makes it easier to use and manage your KMS keys.

Bob creates an alias for his KMS key by using the CLI with the following command:

aws kms create-alias \
    --alias-name alias/example-ecdh-key \
    --target-key-id a1b2c3d4-5678-90ab-cdef-EXAMPLE11111

Alice creates an ECC key for key agreement purposes by using OpenSSL

Using the ecparam and genkey option of OpenSSL, Alice creates a P-256 ECC key. The P-256 curve is represented by AWS KMS as ECC_NIST_P256.

Note: For ECDH to work, the curve of the OpenSSL ECC key must be same as the ECC KMS key created by the other party (Bob, in our example use case).

openssl ecparam -name P-256 \
        -genkey -out openssl_ecc_private_key.pem

Key exchange and secret derivation process

The following sections outline the steps that Alice and Bob will follow to share their public keys, retrieve one another’s public key, and then derive the same shared secret using AWS KMS and OpenSSL. The shared secrets derived by Alice and Bob respectively are then compared to show that they both derived the same shared secret.

Step 1: Alice generates and registers her OpenSSL public key with a central service

AWS KMS expects the public key in DER format. Therefore, in this example Alice creates a DER-format public key by using her ECC private key. Alice runs the following command to produce a DER-format file that contains her public key:

openssl ec -in openssl_ecc_private_key.pem \
        -pubout -outform DER \
        > openssl_ecc_public_key.bin.der

The file openssl_ecc_public_key.bin.der will have the public key in DER format, which Alice can store in the centralized key storage service (or send to anyone she would like to communicate with). Details about the centralized key storage service are beyond the scope of this post.

Step 2: Bob obtains the public key for his ECC KMS Key

To retrieve a copy of the public key for his ECC KMS key, Bob uses the GetPublicKey API action. Bob calls this API by using the AWS CLI command get-public-key, as follows:

aws kms get-public-key \
    --key-id alias/example-ecdh-key \
    --output text \
    --query PublicKey | base64 --decode > kms_ecdh_public_key.der

The returned PublicKey value is a DER-encoded X.509 public key. Because the AWS CLI is being used, the public key output is base64-encoded for readability purposes. This base64-encoded value is decoded by using the base64 command, and the decoded value is stored in the output file. The file kms_ecdh_public_key.der contains the DER-encoded public key.

Note: If you call this API by using one of the AWS SDKs, such as Boto3, then the returned PublicKey value is not base64-encoded.

In our example use case, Alice is using OpenSSL, which expects the public key in PEM format. Bob converts his DER-format public key into PEM format by using the following command:

openssl ec -pubin -inform DER -outform PEM \
        -in kms_ecdh_public_key.der \
        -out kms_ecdh_public_key.pem

The file kms_ecdh_public_key.pem contains the public key in PEM format.

Step 3: Bob registers his public key with the centralized key storage service

Bob saves his public key in PEM format, obtained in Step 2, in the centralized key storage service.

Step 4: Alice retrieves Bob’s public key to derive a shared secret

To perform ECDH key agreement, the two parties involved (Alice and Bob, in our example use case) need to exchange their public key with each other. Alice, who wants to send encrypted messages to Bob, retrieves Bob’s public key from the centralized key storage service.

Bob’s public key, kms_ecdh_public_key.pem, is already in PEM format as expected by OpenSSL.

Step 5: Bob retrieves Alice’s public key to derive a shared secret

To perform ECDH key agreement, the two parties involved, Alice and Bob, need to exchange their public key with each other. Bob gets a notification that Alice wants to communicate with him, and he retrieves Alice’s public key from the centralized key storage service.

Alice’s public key, openssl_ecc_public_key.bin.der, is already in DER format as expected by AWS KMS.

Step 6: Alice uses OpenSSL to derive the shared secret

Alice, using her private key and Bob’s public key, can derive the shared secret by using OpenSSL. Alice derives the shared secret by using the OpenSSL pkeyutl command with the derive option, as follows:

openssl pkeyutl -derive \
-inkey openssl_ecc_private_key.pem \
-peerkey kms_ecdh_public_key.pem > openssl.ss

The file openssl.ss will have the shared secret in binary format.

Step 7: Bob uses AWS KMS to derive the shared secret

Bob, using his private key (which remains securely within AWS KMS) and Alice’s public key, can derive the shared secret by using AWS KMS. The following example shows how Bob uses the DeriveSharedSecret API action with the AWS CLI command derive-shared-secret. At launch, the only supported key agreement algorithm is ECDH. Bob passes Alice’s public key for the PublicKey parameter.

aws kms derive-shared-secret \
--key-id alias/example-ecdh-key \
--public-key fileb://path/to/openssl_ecc_public_key.bin.der \
--key-agreement-algorithm ECDH \
--output text --query SharedSecret |base64 --decode > kms.ss

Because the AWS CLI is being used, the returned SharedSecret value is base64-encoded for readability purposes. Using the base64 --decode command, the decoded binary format is stored to the file.

Note: If you call this API by using one of the AWS SDKs, such as Boto3, then the returned SharedSecret value is not base64-encoded.

The file kms.ss will have the shared secret in binary format.

Step 8: Using the shared secret and a suitable KDF, Alice derives an encryption key to encrypt her communication to Bob

You can use the following command to compare the two files containing the derived shared secrets that were obtained in Steps 6 and 7 and verify that they are identical:

diff -qs openssl.ss kms.ss

Because these files are identical, we can see that the same secret was derived using both AWS KMS and OpenSSL.

Using the shared secret, Alice should then derive a symmetric encryption key by using a suitable KDF. She can use this symmetric encryption key to encrypt data and send the ciphertext to Bob.

This blog post does not cover the steps to derive that symmetric encryption key, because that can be a complex topic depending on your use case. However, we note that you should not use the raw shared secret as an encryption key because it is not uniform. In other words, the shared secret has a lot of entropy, but the byte string itself is not random.

NIST recommends that you use a KDF function over the raw shared secret (value Z as described in section 5.8 of NIST SP800-56A Rev. 3). The KDFs that are recommended are described in more detail in NIST SP800-56C Rev. 2. One such example is OpenSSL Single Step KDF (SSKDF) EVP_KDF-SS, but using this KDF involves choosing the other values, such as FixedInfo, carefully.

To help customers make the right choice for the resulting KDF to use on the shared secret, the AWS Encryption SDK now includes AWS KMS ECDH keyrings. The keyring is a construct within the AWS Encryption SDK that you implement within your code. The keyring handles the management of encryption keys while applying best practices to protect your data. You can use the keyring to reference your KMS keys for key agreement, and then call a function to encrypt data. Data will be encrypted by using a derived shared wrapping key following NIST recommendations, and the Encryption SDK applies key commitment to the ciphertext.

Summary

In this blog post, we highlighted how you can use the recently launched DeriveSharedSecret API action to securely derive a shared secret. You’ve seen how ECDH can be used between two parties without having to share secret information across untrusted networks. We explained how you can audit your AWS KMS key usage through AWS CloudTrail logs. We highlighted that you would need to use a KDF to generate a symmetric encryption key from the shared secret. We strongly recommend that you use the AWS Encryption SDK to encrypt your data, which helps make sure that the recommended NIST key derivation functions are used for generating symmetric encryption keys.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, contact AWS Support.

Strengthening data security in AWS Step Functions with a customer-managed AWS KMS key

2024-07-29 Chris Munns

Post Syndicated from Chris Munns original https://aws.amazon.com/blogs/compute/strengthening-data-security-in-aws-step-functions-with-a-customer-managed-aws-kms-key/

This post is written by Dhiraj Mahapatro, AWS Principal Specialist SA, Serverless.

AWS Step Functions provides enhanced security with a customer-managed AWS KMS key. This allows organizations to maintain complete control over the encryption keys used to protect their data in Step Functions, ensuring that only allowed principals (IAM role, user, or a group) have access to the sensitive information that is processed in a state machine. This post explores the details of this feature and the new console experience of executing Step Functions workflows when a customer-managed KMS key is used.

Step Functions is a serverless orchestration service that enables you to coordinate multiple AWS services, microservices, and third-party integrations into business-critical applications. Step Functions is widely used for orchestrating complex workflows, such as loan processing, fraud detection, risk management, and compliance processes. By breaking down these processes into a series of steps, Step Functions provides a clear overview and control of the entire workflow. This ensures that it executes each stage correctly and in the right order. One of the critical aspects of using Step Functions in regulated industries is the importance of security and data protection. Step Functions manages sensitive customer data, including PII and financial records, and require protection against unauthorized access and data breaches. Enabling a customer-managed KMS key further strengthens the data security in a state machine.

Using customer-managed AWS KMS keys

With this launch, Step Functions enable encryption of the state machine definition and execution details, including event history using customer-managed symmetric KMS keys. As part of this feature, you also have the option to encrypt Step Functions activities using customer-managed key.

This post uses a sample application to show the implementation details of this new feature. See user guide for a detailed explanation of this feature.

The sample application shows a basic stock trading example where the state machine buys or sells a stock if the price of the stock is above or below 50 and finally saves the transaction.

Example workflow

The Step Functions Cloudformation resource of the state machine has a new property EncryptionConfiguration as shown in the following:

StockTradingStateMachine:
  Type: AWS::StepFunctions::StateMachine
  Properties:
    StateMachineName: !FindInMap ['StateMachine', 'Name', 'Value']
    RoleArn: !GetAtt StockTradingStateMachineExecutionRole.Arn
    EncryptionConfiguration:
      KmsKeyId: !Ref StocksKmsKey
      KmsDataKeyReusePeriodSeconds: 100
      Type: CUSTOMER_MANAGED_KMS_KEY
    Definition: . . .

Within EncryptionConfiguration, you specify the KmsKeyId and the Type. This sample application uses a CUSTOMER_MANAGED_KMS_KEY key type. The Type is a required field and it will be AWS_OWNED_KEY if it is not a customer managed key. The state machine also allows to specify the KmsDataKeyReusePeriodSeconds property to a value between 60 and 900 seconds (default: 300), which signifies the maximum duration for which the state machine reuses the data keys. When the period expires, Step Functions will call GenerateDataKey API on AWS KMS. Therefore, besides kms:Decrypt, Step Functions needs access to kms:GenerateDataKey action.

The sample application also creates a customer-managed KMS key with a condition to force the stock trading state machine to only use the key.

Security controls

Within an AWS Organization setup, the best practices guidance is to have a dedicated security organizational unit responsible for managing and enforcing security standards, including ownership of KMS keys. The security account provides cross-account access for the key usage. You grant admin access only to the root of the security account, while external or member accounts can access it for various purposes like decryption, encryption, description, and data key generation. This can be done through an IAM Role, User, or Group in the member account. The standard approach for cross-account access involves combining KMS key policies in the security account and IAM policies to the identity that gives permission for the service in the member account.

Cross account access

For Step Functions, you can go a step further to restrict access to the caller’s role in the member account and provide a condition. The condition forces Step Functions service to only use the key. For example, with a security account (id: 1111111111) and a member account (id: 1234567890), the KMS key policy can use a kms:ViaService condition to restrict access to Step Functions state machines present in us-east-1 region only:

{
  "Sid": "Allow access to member account via Step Functions service",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::1234567890:role/MemberAccountRole"
  },
  "Action": ["kms:Decrypt", "kms:GenerateDataKey"],
  "Resource": "*",
  "Condition": {
    "StringEquals": {
      "kms:ViaService": "states.us-east-1.amazonaws.com"
    }
  }
}

Constantly updating the key policy for every new Step Functions workflow in member accounts is cumbersome. Therefore, a combination of KMS key policy and IAM roles grants fine-grained and least-privilege access to key actions. For organizations that do not have a security account or security organizational unit, the member account owns the KMS key, as shown below. The key policy must be more restrictive to the Step Functions execution role and the Step Functions ARN that will use the key.

Member account ownership

For example, a member account with an account id 1234567890 sets the Step Functions execution role sfn-execution-role as the Principal and restricts the key usage to a specific Step Functions ARN in the same account by using kms:EncryptionContext:aws:states:stateMachineArn condition as shown in the following:

{
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::1234567890:role/sfn-execution-role"
  },
  "Action": ["kms:Decrypt", "kms:GenerateDataKey"],
  "Resource": "*",
  "Condition": {
    "StringEquals": {
      "kms:EncryptionContext:aws:states:stateMachineArn": 
      "arn:aws:states:us-east-1:1234567890:stateMachine:MyStateMachine"
    }
  }
}

Testing

To setup the application in your AWS account, you need the following tools:

AWS CLI
AWS SAM CLI
Install Node.js 20, including the NPM package management tool

Clone the git repository. To build and deploy your application for the first time, run the following in your shell from the repository home directory:

sam build && sam deploy –guided

You can find the State Machine’s ARN in the output values displayed after deployment.

Once deployed, run the application using the AWS CLI. Run the following command after replacing the state machine ARN from the output of the deployment and the region where you have the state machine:

aws stepfunctions start-execution \
  --state-machine-arn <state-machine-arn> \
  --region <region>

You get a successful response in the CLI. You can also see a corresponding execution listed in the AWS Console as RUNNING:

Running workflow

However, opening the execution details will show an “Access Denied” error as expected:

Access denied error

You get the same error while visualizing the Step Functions definition or editing the state machine. The sample application restricts the decryption by the KMS key to only the Step Functions workflow’s execution role. Therefore, any other entity cannot decrypt the state machine’s workflow execution details and the state machine’s definition. This secures the exposure of information, including the payload passed to Step Functions or the payload passed in between state transitions to external entities. This new feature will securely allow personally identifiable information (PII), credit card information (PCI), and other similar sensitive information in Step Functions. Existing sensitive workloads are now unlocked for Step Functions, therefore easing, making them AWS cloud native.

You can integrate Amazon CloudWatch Logs with Step Functions for logging and monitoring capabilities. To send logs, you must provide access for log delivery to decrypt your logs. In your State Machine customer-managed key policy, you must grant kms:decrypt permission to the principal delivery.logs.amazonaws.com. Logging a workflow will not work without above grant. You encrypted data is sent to CloudWatch logs with the same or different customer managed KMS key. See CloudWatch logs documentation to learn how to set permissions on the KMS key for your log group.

Cleanup

To delete the sample application, use the latest version of the AWS SAM CLI and run:

sam delete

Conclusion

Customer-managed AWS KMS keys in Step Functions allows for access control sensitive data. KMS key policy and IAM identity policies determine who decrypts and access various aspects of the state machine, including the definition, execution details, and input/output payload transitions for each task. This is an essential feature for highly regulated industries like financial services. Apply these security guardrails using customer-managed AWS KMS keys at the organizational unit, business unit, or at the individual account level.

The sample application shows a way of using the customer managed KMS key in Step Functions resource in CloudFormation. The user guide provides additional details. Support for this feature is available in AWS CDK now while Terraform support will fast follow. Dive deeper into additional details from the Step Functions user guide.

For more serverless learning resources, visit Serverless Land.

Why per-tenant encryption matters

The cache stampede problem

How the stampede occurs

Two paths forward

Option A: Hierarchical keyring with DynamoDB (AWS-recommended)

Option B: Custom KMS client caching – Solving the stampede at the API layer

Choosing between the two options

Results

Conclusion and next steps

Quick comparison

Quick decision guide

Shared benefits

Security

Regulatory compliance

Standard algorithms

Performance

Operational support

When to choose AWS KMS

AWS integration

Operational simplicity

Cost considerations

Region coverage

When to choose AWS CloudHSM

Lift-and-shift on-premises workloads

Legacy cryptographic algorithms

Conclusion

Customer use case and challenge

Solution: MDAA and its value propositions

Use case of MDAA: Governed data architecture

Publish, discover, subscribe, and consume data products between business units: SageMaker Unified Studio

Use case of MDAA: Restricting access to cardholder data using Lake Formation

Use case of MDAA: Ensuring data integrity with AWS Glue Data Quality

Protecting metadata at rest: AWS Glue Data Catalog encryption

Auditing every data access event: CloudTrail integration

Identifying sensitive data automatically: Macie integration

Multi-account data mesh

Technical implementation

Conclusion

About the author

Determine when a key was last used

About the tracking period

Getting started

API reference

Use case 1: Cost optimization through unused key cleanup

Solution with GetKeyLastUsage API

Use case 2: Preventing accidental key deletion with policy controls

Solution with policy based controls

Important considerations

Conclusion

The security hygiene gap

What is the Security Health Improvement Program (SHIP)?

Why SHIP matters in the AI era

Address foundational security gaps proactively

Establish the security baseline AI workloads require

Build a mechanism for continuous security improvement

Getting started is straightforward

Take the next step together

The Catalyst solution

Key platform capabilities

Overall result

Conclusion

About the authors

Amazon S3 server-side encryption options with Amazon EMR

Challenges to prevent encryption key reuse due to file system caching

Solution overview

Method 1: Disable file system cache

Considerations

Method 2: Use separate Spark applications or sessions

Considerations

Method 3: Use S3 bucket default encryption

Considerations

Choosing the right approach

Conclusion

About the authors

Create a post-quantum PKI hierarchy

CA hierarchy creation

Code-signer creation

Sign code

Verify signed code

Conclusion