Post Syndicated from Nathan Pogue original https://aws.amazon.com/blogs/compute/building-zero-trust-generative-ai-applications-in-healthcare-with-aws-nitro-enclaves/

In healthcare, generative AI is transforming how medical professionals analyze data, summarize clinical notes, and generate insights to improve patient outcomes. From automating medical documentation to assisting in diagnostic reasoning, large language models (LLMs) have the potential to augment clinical workflows and accelerate research. However, these innovations also introduce significant privacy, security, and intellectual property challenges.

Healthcare data often contains Protected Health Information (PHI), which is governed by strict regulations and compliance frameworks. At the same time, organizations or researchers who have invested substantial time and compute resources into training medical LLMs must protect their proprietary model architectures, weights, and fine-tuned datasets. Traditional deployment models necessitate mutual trust between the model publisher and the healthcare data provider — trust that sensitive data won’t be leaked, and that the model itself won’t be copied, tampered with, or exfiltrated. The absence of a secure and verifiable trust model between model publishers and consumers remains one of the main barriers to scaling generative AI in regulated medical environments.

To address this concern, both parties need a secure environment to publish and consume models without exposing data or intellectual property. Amazon Web Services (AWS) Nitro Enclaves provide isolated, attested, and cryptographically verified compute environments that help protect sensitive workloads. Model owners can encrypt their LLMs with AWS Key Management Service (AWS KMS) and allow only verified Nitro Enclaves to decrypt and run them, making sure that the model can’t be accessed outside the Nitro Enclave. Healthcare organizations and consumers can use this to process sensitive data within their own AWS environment entirely within the Nitro Enclave, helping keep PHI private and contained. Hardware-based attestation provides proof that the Nitro Enclave is running trusted code, so that both sides can exchange information with confidence.

In this post, we demonstrate how to deploy a publicly available foundational model (FM) using Nitro Enclaves for isolated, more secure compute, AWS KMS for model encryption, Amazon Simple Storage Service (Amazon S3) for storing model artifacts and images, and Amazon Simple Queue Service (Amazon SQS) for securely delivering queries, enabling private, privacy-preserving inferences while helping protect both model intellectual property and sensitive patient data.

Solution overview

This solution outlines how to build a more secure end-to-end pipeline that enables zero trust medical LLM publication and inference with Nitro Enclaves. This post demonstrates a guide for setting up an Amazon Elastic Compute Cloud (Amazon EC2) instance with Nitro Enclaves enabled, downloading and encrypting a publicly available FM to an S3 bucket with an AWS KMS key, sending medical text and image-based queries to an SQS queue for processing, and storing results in an Amazon DynamoDB table.

This project is intended solely for educational and demonstration purposes and isn’t suitable for production or clinical use. Its outputs aren’t validated for clinical accuracy and must not be used for patient care or medical decision-making. Before any real-world deployment, make sure that you implement comprehensive security, privacy, and compliance safeguards. These include health data protection controls, secrets management, and regulatory validation. Furthermore, you must consult the appropriate clinical, legal, and security experts.

For demonstration purposes, this solution is deployed in a single AWS account. Ideally, in production, it would be deployed across separate AWS accounts: one for the model owner and one for the model consumer. The model owner can use cross-account AWS Identity and Access Management (IAM) permissions and encrypted model sharing through AWS KMS to securely provide access to their model without exposing the underlying weights or logic. At the same time, the consumer can run sensitive inferences within their own environment, maintaining strict data privacy and zero trust principles. In a real-world implementation, the model provider should also establish a robust entitlement and licensing framework to manage customer access, enabling fine-grained control over who can invoke the model, track usage, and support license revocation to immediately remove permissions from specific customers when necessary.

The following diagram shows the solution architecture:

The steps of the solution include:

1. Amazon EC2 setup: An EC2 instance is launched with Nitro Enclaves and Trusted Platform Module (TPM) enabled. For this project, a c7i.12xlarge instance with a 150 GB Amazon EBS volume is used to provide the necessary compute resources for running LLMs.
2. Public FM download: A publicly available FM is retrieved from Hugging Face and stored in an S3 bucket within the model consumer’s AWS account.
3. Model encryption: The model is encrypted using AWS KMS envelope encryption. Only a Nitro Enclave presenting a valid attestation document can request the decryption key from AWS KMS, which helps prevent unauthorized access to the model weights outside the Nitro Enclave.
4. Nitro Enclave setup: A Docker image containing the llama.cpp inference runtime is built and deployed inside the Nitro Enclave.
5. Model decryption and setup: When the Nitro Enclave launched, it requests decryption of the model artifacts using its attestation credentials. Then, the model can be securely decrypted inside the memory of the Nitro Enclave and loaded by the llama.cpp server. This means that the decrypted model weights aren’t visible outside of the Nitro Enclave boundary.
6. Medical query: Users can submit either text or image-based queries to the model. Queries are sent through vsock, a secure communication channel from the client application to the model server inside the Nitro Enclave. Image queries necessitate that users upload images to an S3 bucket. The upload event triggers an SQS queue, which signals the Amazon EC2 parent to fetch and send the image to the Nitro Enclave image for the medical LLM to process with its multimodal capabilities.
7. Message history: Each interaction, including the user’s prompt and the model’s response, is logged to a DynamoDB table. This provides a persistent conversation history that enables traceability and auditing while keeping PHI securely stored within the consumer’s account. If necessary, the DynamoDB table can be encrypted and sealed for another layer of security and privacy.

About Google MedGemma 4B

Google MedGemma is a family of medically-optimized LLMs built on Gemma 3, with 4B and 27B parameter variants supporting both text and multimodal versions for medical image inputs. The 4B model offers efficiency and strong performance for multimodal tasks such as report generation and medical Q&A, while the 27B models excel at more demanding scenarios, such as electronic health record interpretation and complex longitudinal data analysis.

MedGemma models are well-suited for automated radiology report generation, clinical triage and documentation, patient education, medical image pre-interpretation, and medical education systems. The 4B model is ideal for portable or resource-constrained deployments, whereas the 27B multimodal delivers maximal performance.

In this project, MedGemma 4B serves as a reference medical LLM, showing how domain-adapted fine-tuning can enhance a model’s ability to interpret, reason about, and respond to complex medical queries. It also provides a foundation for exploring the safe and effective use of LLMs in healthcare applications, while being securely deployed within a Nitro Enclave. However, you can choose to deploy your own medical FM if needed. This is a deeper overview on the 4B model.

Prerequisites

To implement the proposed solution, make sure that you have the following:

• The AWS Command Line Interface (AWS CLI) installed on your machine to create the EC2 instance.
• AWS permissions with access to EC2 c7i.12xlarge instances and Nitro Enclaves.
• Knowledge of Amazon S3, AWS KMS, Amazon SQS, AWS Lambda, and DynamoDB.
• Basic knowledge of Nitro Enclaves and healthcare data security.
• The GitHub repository cloned to your local machine.

Environment setup

The following sections outline how to set up your environment for this solution.

Create S3 buckets

In this solution, you create two S3 buckets: one for the model artifacts and one for the image inputs.

To create the S3 buckets

1. Sign in to the Amazon S3 console, choose Create bucket, and follow the prompts to create a new S3 bucket.
2. For the model artifact bucket, give it a unique name (for example AWSACCOUNTNUMBER-medgemma-model) in the same Region you use for the other project resources.
3. Repeat the same process for the image bucket (for example AWSACCOUNTNUMBER-medgemma-image-inputs).
4. Update the S3_BUCKET_NAME variable with your model bucket name in envelope_encrypt_model.sh and run.sh.

Create an SQS queue

When images are uploaded to the S3 image bucket, they are sent to an SQS queue for processing in sequential order by the model running in the Nitro Enclave.

To create an SQS queue

1. Sign in to the Amazon SQS console, choose Create queue, and follow the prompts to create a new SQS queue.
2. Choose Standard Queue, provide a name, leave the rest as default, and choose Create queue.
3. Replace the SQS_QUEUE_URL variable in image_processor.py and lambda_function.py (in the client and assets folder, respectively) with your URL.

Create a Lambda function

For image-based queries, MedGemma 4B expects images encoded in base64 format to be passed in the prompt. To convert the images to this format, a Lambda function is invoked using an Amazon S3 trigger when an image is uploaded to the bucket.

To create a Lambda function

1. Sign in to the Lambda console, choose Create function, and follow the prompts to create a new Lambda function from scratch.
2. Choose a name, choose a Python runtime (for example Python 3.13), and paste in the Lambda function code from the assets folder.
3. Next, update the Lambda function’s IAM role in Permissions under the Configuration tab with access to your S3 image bucket and the SQS queue that you created with inline policy permissions. Attach the following policies:
   1. Amazon S3 policy:

      {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "Statement1",
               "Effect": "Allow",
               "Action": [
                   "s3:*"
               ],
               "Resource": [
                   "arn:aws:s3:::<IMAGE_BUCKET_NAME>",
                   "arn:aws:s3:::<IMAGE_BUCKET_NAME>/*"
               ]
           }
       ]
      }

   2. Amazon SQS policy:

      {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "VisualEditor0",
               "Effect": "Allow",
               "Action": "sqs:ListQueues",
               "Resource": "*"
           },
           {
               "Sid": "VisualEditor1",
               "Effect": "Allow",
               "Action": "sqs:*",
              "Resource": "arn:aws:sqs:<REGION>:<ACCOUNT_NUMBER>:<QUEUE_NAME>"
           }
       ]
      }

4. Finally, within the Lambda Designer, add a trigger, choose Amazon S3, and choose your image bucket. You should see the following example when the trigger is enabled.

Create a DynamoDB table

When the queries have been processed by the model for inference, the prompts and responses are logged to a DynamoDB table for auditing and message history purposes.

To create a DynamoDB table

1. Sign in to the DynamoDB console, choose Create table, and follow the prompts to create a new DynamoDB table.
2. Give it a partition key named ID as a String type.
3. Replace the TABLE_NAME variable with the table name and REGION variable with your AWS Region in direct_query.py and image_processor.py files.

Create an AWS KMS key

An AWS KMS key is used to envelope-encrypt the model artifacts before they are uploaded to the S3 model bucket. During encryption, the AWS KMS key policy is configured with conditions that restrict decryption to only those Nitro Enclaves presenting a valid attestation document. This attestation includes platform configuration registers (PCR) hashes that represent the measured state of the Nitro Enclave, which covers the signed Nitro Enclave image, runtime, and configuration. When the Nitro Enclave is launched, it generates an attestation document signed by the Amazon EC2 Nitro hypervisor, proving that its PCR values match the expected trusted measurements defined in the AWS KMS key policy. The key is released only if these PCR hashes align and the attestation is verified by AWS KMS, allowing the Nitro Enclave to decrypt and load the model securely in memory.

To create an AWS KMS key

1. Sign in to the AWS KMS console, choose Create key, and follow the prompts to create a new AWS KMS key.
2. Choose Symmetric as the key type and Encrypt and decrypt for the key usage. Make the alias AppKmsKey. Leave the default settings and choose Finish.
3. Replace the REGION variable in vsock-proxy.yaml in the client folder with your AWS Region.

Create an EC2 instance

Now that the necessary resources are set up, you can proceed to launch the EC2 instance and create the Nitro Enclave image. For this solution, a c7i.12xlarge instance with a 150 GB EBS volume is provisioned.

To launch an EC2 instance with Nitro Enclaves enabled

1. Within the GitHub repository on your local machine, run ./create_ec2.sh to create the EC2 instance.

cd scripts 
chmod +x create_ec2.sh 
./create_ec2.sh

2. The script launches an EC2 instance called MedGemmaNitroEnclaveDemo. When the instance is running, you must create an IAM policy and add it to the Amazon EC2 IAM role with necessary permissions to the resources created previously.
3. Sign in to the IAM console and navigate to Policies, choose Create policy, choose JSON, and paste the following policy, making sure that you update the bucket, queue URL, AWS Region, account number, and table variables:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "s3modelbucket",
      "Effect": "Allow",
      "Action": [
        "s3:*"
      ],
      "Resource": [
        "arn:aws:s3:::<MODEL_BUCKET_NAME>",
        "arn:aws:s3:::<MODEL_BUCKET_NAME>/*"
      ]
    },
    {
      "Sid": "dynamodb",
      "Effect": "Allow",
      "Action": [
        "dynamodb:*"
      ],
      "Resource": [
        "arn:aws:dynamodb:<REGION>:<ACCOUNT_NUMBER>:table/<TABLE_NAME>"
      ]
    },
    {
      "Sid": "sqslist",
      "Effect": "Allow",
      "Action": "sqs:ListQueues",
      "Resource": "*"
    },
    {
      "Sid": "sqsqueue",
      "Effect": "Allow",
      "Action": "sqs:*",
      "Resource": "arn:aws:sqs:<REGION>:<ACCOUNT_NUMBER>:<QUEUE_NAME>"
    }
  ]
}

4. Give it a name (for example enclave-permissions) and choose Create policy.
5. Navigate to Roles, choose Create role, choose EC2 as the AWS service for the Trusted entity type, then choose your policy that you created under Add permissions.
6. Update your EC2 instance to use the role by going to the Security setting under the Actions dropdown, then modifying its IAM role.
7. You can upload the modified repository to your EC2 instance using SCP. Alternatively, you can transfer the repository through rsync

rsync -avz -e "ssh -i /path/to/directory/sample-for-secure-medical-llm-inference-with-nitro-enclaves/nitro-enclave-key.pem" --exclude='*.pem' /path/to/directory/sample-for-secure-medical-llm-inference-with-nitro-enclaves/ ec2-user@<PUBLIC_IP>:~ 
cd sample-for-secure-medical-llm-inference-with-nitro-enclaves

8. Make the scripts executable (in client, server and scripts).

chmod +x *.sh

Nitro Enclave setup

With Amazon EC2 loaded with the necessary scripts, you can begin building the Nitro Enclave image. During this process, the Docker container is converted into an Enclave Image File (EIF), which generates cryptographic measurements (PCR hashes) that uniquely identify the code and configuration of the enclave. These measurements are embedded into the AWS KMS key policy, creating a hardware-attested trust boundary that makes sure only this specific, unmodified Nitro Enclave can decrypt and access the model weights.

1. Run the complete setup script, which sets up the client on the EC2 parent instance and the server running within the Nitro Enclave. You can observe the different scripts in the client, server, and scripts folders.

sudo ./run_complete_setup.sh

2. The various scripts run to download the MedGemma 4B model, encrypt the model with the AWS KMS key, build a Docker image to run a llama.cpp server, start a Nitro Enclave, and decrypt and run the model. This process takes approximately 10 minutes.
3. When the Nitro Enclave is running, it runs in debug mode so that you can observe the various startup logs outputted. Wait until llama.cpp server logs are outputted that indicate the server is ready and listening.

Inference examples

When the model is decrypted and running on the llama.cpp server within the Nitro Enclave, you can begin to invoke the model with either image or text-based queries. Open a new terminal session in your EC2 instance. You can navigate to the client folder to run the scripts for queries.

Image-based queries

For inference on medical images, upload an image to your Amazon S3 image bucket. When it is uploaded, run python3 image_processor.py to pass the image from the SQS queue to the Nitro Enclave for processing. The following are examples of image inputs and model outputs.

Brain CT scan:

Case courtesy of Dr Henry Knipe, Radiopaedia.org, rID: 46289

Model response:Chest X-ray:

Model response:Text-based queries

For inference on text-queries, run python3 direct_query.py "<YOUR_MEDICAL_QUERY>" to invoke the model. The following are examples of text-based inputs and model outputs.

Basic usage:

python3 direct_query.py "What are the symptoms of pneumonia?"

Model response:Lab result interpretation:

python3 direct_query.py "Patient has elevated troponin levels (15.2 ng/mL), elevated CK-MB, and ST elevation in leads II, III, aVF. What does this suggest?"

Model response:

Cleaning up

To avoid incurring future charges, delete the resources used in this solution:

1. Stop and terminate the EC2 instance.
2. Empty and delete the S3 buckets.
3. Delete the DynamoDB table.
4. Delete the SQS queue.
5. Delete the AWS KMS Key.
6. Delete the Lambda function.

Conclusion

You can combine the isolation and attestation capabilities of AWS Nitro Enclaves, the encryption controls of AWS KMS, and the scalability of services such as Amazon S3, Amazon SQS, and Amazon DynamoDB to build a more secure, zero trust pipeline for deploying generative AI models in healthcare. Using Google MedGemma 4B as your reference medical LLM, you can enable privacy-preserving inference where both PHI and model intellectual property remain protected. For more information, consult the following resources:

• AWS Nitro Enclaves Developer Guide
• Amazon S3 Developer Guide
• Amazon DynamoDB Developer Guide
• AWS KMS Developer Guide
• Amazon SQS Developer Guide
• AWS Lambda Developer Guide