Post Syndicated from Vushesh Babu Adhikari original https://aws.amazon.com/blogs/architecture/how-tui-scales-threat-modeling-for-100-teams-with-amazon-bedrock/

When your security team has four engineers and your organization has over 100 digital product teams, threat modeling becomes a bottleneck fast. That was the challenge TUI faced. TUI operates across the travel and tourism industry with airlines, hotels, cruise ships, and local travel experiences. Our Platform & Engineering division needs security assessments to keep pace.

At TUI, each threat modeling session took 4–6 hours of manual work: reviewing architecture diagrams, mapping threats to internal standards, researching security issues, and documenting findings in Confluence. The team could complete only 3–4 sessions per month, leaving a constant backlog.

TUI needed threat modeling to become mandatory for every new service, not a luxury TUI reserved for the highest-priority projects. TUI built an AI-powered threat modeling system using Amazon Bedrock, collaborating with the AWS Generative AI Innovation Center during a 10-week proof of concept (PoC) to develop and validate the architecture. Amazon Bedrock is a fully managed service for building generative AI applications, and TUI used the Claude Sonnet 4.5 foundation model from Anthropic. TUI chose Amazon Bedrock for managed access to Claude Sonnet 4.5 without operating model infrastructure, its extended thinking support that the workflow uses for deeper analysis, and its integration with the AWS services already in TUI’s stack. TUI deployed the agentic workflow on Amazon Bedrock AgentCore runtime, part of Amazon Bedrock AgentCore, a platform to build, connect, and optimize agents at scale with any framework or model. AgentCore runtime hosts the agent in production without provisioning or scaling infrastructure. After the PoC, TUI took the system to production within 30 days.

In this post, we describe how TUI built an agentic AI workflow for threat modeling, the iteration process that shaped the architecture, and the results after deployment.

The challenge

TUI’s security engineering team followed a thorough but manual process: for each application, an engineer reviewed the architecture, identified threats using STRIDE methodology, mapped them to TUI Technology Standards, P&E-specific standards, and ISO 27001 controls, and documented findings in Confluence with Jira tickets.

With 100+ digital product teams and only 3–4 sessions possible per month, most teams waited weeks for their review. Security engineers also had varying levels of expertise across technology stacks, and there was no automated way to verify these standards were consistently applied to every assessment.

Solution overview

The solution extends AWS Threat Designer with TUI-specific capabilities through a serverless architecture deployed in the eu-central-1 (Frankfurt) AWS Region. Model availability varies by Region. For model availability by Region, refer to Supported models by AWS Region in Amazon Bedrock. Together, these services replace the manual review process with an automated workflow that applies TUI Technology Standards and P&E-specific standards consistently. The system uses the following key services:

• Amazon Bedrock for threat analysis and generation using Claude Sonnet 4.5 from Anthropic, with configurable extended thinking budgets.

• Amazon Bedrock AgentCore runtime for hosting the agentic workflow as a managed, serverless runtime. It runs the containerized LangGraph agent without provisioning or scaling infrastructure, supports long-running sessions that exceed typical function timeouts, and provides per-session isolation for concurrent threat models.

• AWS Lambda for serverless backend compute with LangGraph workflow orchestration.

• Amazon DynamoDB for workflow state management with resume capability.

• Amazon Simple Storage Service (Amazon S3) for architecture diagram storage, TUI knowledge base, and web search cache.

• AWS Amplify for the React front end with Amazon Cognito authentication.

Figure 1: TUI Threat Designer system architecture. Users authenticate through Amazon Cognito and interact with the React front end hosted on AWS Amplify. Requests flow through Amazon API Gateway to AWS Lambda, which orchestrates the LangGraph workflow through Amazon Bedrock AgentCore. The agent uses Amazon Bedrock (Claude Sonnet 4.5 from Anthropic) for threat analysis, with Amazon DynamoDB for state management and Amazon S3 for architecture diagram storage and caching.

The core is a LangGraph-based agentic workflow that processes architecture diagrams through several specialized nodes. Engineers upload one or more diagrams (for example, a high-level architecture alongside a detailed network topology), which the workflow converts to base64 to identify components and data flows. It then loads relevant security standards and generates threats with iterative refinement. The workflow runs gap analysis, optionally compares against a subject matter expert (SME) assessment, and assembles the final threat model with export options (PDF, JSON, or Confluence).

The following screenshot shows the submission wizard. Engineers provide a title, upload architecture diagrams, add optional context and assumptions, and select a reasoning depth based on architecture complexity. The entire setup takes approximately two minutes.

Figure 2: The submission wizard guides engineers through a six-step process. The iterations slider controls the reasoning budget, from None for quick scans to High for complex architectures with 20 or more components.

Once submitted, the system processes the architecture through each workflow node with real-time progress tracking. Engineers can monitor which phase is active and return to review results when processing completes.

Figure 3: Real-time progress tracking shows the current workflow phase. The Thinking indicator on the Threats node reflects the extended thinking capability of Claude analyzing the architecture against TUI security standards.

The iteration journey: from brute force to intelligent selection

TUI’s team didn’t arrive at the final design on day one. They iterated through several approaches, and each failure pointed toward the next improvement.

Attempt 1: Load everything into the prompt. The first approach loaded all 192 TUI security standards (338 KB) directly into the foundation model context. This failed immediately. Some individual standard documents run 20+ pages, and the combined content exceeded token limits before the foundation model could begin threat analysis.

Even with a subset of standards, the foundation model struggled to follow structured instructions. When asked to recommend only TUI-approved tools for mitigations, it would ignore the approved tool list and suggest generic alternatives. Too much contradictory information from different standards confused the model.

Attempt 2: Metadata-driven selection. The breakthrough came from treating the problem as a search and selection challenge rather than a prompt-loading exercise. Instead of loading all standards, the team pre-generated lightweight metadata for each document: keywords, component types, STRIDE categories, and a relevance score.

At runtime, the foundation model reviews this metadata (about 60 KB) and selects only the 16 most relevant standards for the specific architecture being analyzed. This reduced context from 338 KB to 21 KB, a 94% reduction, while maintaining coverage. The reduction is calculated from the metadata approach versus loading all standards directly.

The metadata approach fits the structure of TUI’s knowledge base: some documents are always mandatory (security guidelines, approved tools), while others apply only to certain architectures. The team also evaluated retrieval-augmented generation (RAG), but with a bounded corpus of 192 standards, some mandatory regardless of the query, metadata-driven selection proved more predictable and easier to debug. RAG may complement it as the knowledge base grows.

Agentic threat generation: a self-correcting workflow

The threat generation node is where the agentic behavior matters most. Rather than generating threats in a single pass, the system uses a Reason and Act pattern with a LangGraph subgraph. The agent has four tools at its disposal: adding threats, deleting threats, reading the current threat catalog, and running gap analysis.

The agent autonomously decides which tools to call and in what order. It generates an initial set of threats using STRIDE methodology, then reviews its own output against the loaded TUI standards. Each threat includes a title, description, affected component, and threat vector. It also assigns likelihood and impact scores using TUI’s risk matrix. Threats include recommended mitigations with TUI-approved tools and mapped ISO 27001 controls.

A tool budget system helps prevent the agent from generating indefinitely. After three threat-generation calls, the system forces a gap analysis. The agent must step back and assess what’s missing before generating more. This mimics how human security experts work: generate, review, identify gaps, fill them.

A validation node checks three conditions before the workflow can proceed: the threat catalog isn’t empty, all six STRIDE categories are covered, and gap analysis has been performed at least once. If a check fails, the agent receives specific feedback and loops back.

Based on TUI’s production data, the combination of iterative generation and gap analysis typically produces 40–60 threats per architecture, compared to the 30–40 that manual sessions identified. It also supports configurable extended thinking budgets, from 24K reasoning tokens for standard architectures up to 64K for complex systems.

Web search for current threat intelligence

Security threats evolve regularly. A threat model generated today needs to account for issues disclosed last week. The workflow integrates web search to fill gaps where TUI’s internal standards don’t cover specific technologies.

For every technology identified in the architecture (Keycloak, Kong, Redis, and so on), the system generates two search queries: one for known issues and one for hardening best practices. These queries target authoritative sources such as OWASP, NIST, and CWE databases. Before making an API call, it checks an Amazon S3-based cache with a 7-day time-to-live (TTL). Only technology component names are used for external searches. Architecture details and customer data are not included in queries.

The cache key uses a hash of the sorted component names, deliberately excluding foundation model-generated descriptions. Those descriptions vary slightly between runs even for the same diagram, which would reduce cache effectiveness.

Based on TUI’s monitoring data, the cache achieves a 70–80% hit rate because many TUI applications share core technologies. A cache hit returns results in under 100 milliseconds, compared to 2–3 seconds for a live search. TUI’s team evaluated multiple search providers during the PoC and designed the system with a pluggable provider architecture, allowing provider switching as capabilities evolve.

SME comparison: human-in-the-loop validation

TUI’s security engineers don’t just use the system. They validate it. The workflow includes an optional node where engineers upload their own manually created threat assessment as a PDF. The workflow extracts threats using Claude, normalizes risk ratings to a standard scale (LOW, MEDIUM, HIGH, CRITICAL), and matches them against the AI-generated threats using semantic similarity.

When the AI and the SME disagree on a threat’s severity, the system flags the discrepancy with both ratings and the SME’s rationale. For example, the AI might rate a credential stuffing event (automated login attempts using unintended disclosure of credential databases) as HIGH while the SME rates it MEDIUM. Over time, these comparisons calibrate the system’s risk scoring and surface areas where the AI consistently over- or under-estimates risk.

The following screenshot shows an SME Comparison Alert for a credential stuffing threat. The system matched the AI-generated threat with a similar SME threat, compared both ratings (AI: High, SME: HIGH), and provided the SME’s rationale alongside a recommendation.

Figure 4: An SME Comparison Alert for a credential stuffing threat. The system displays both the AI and SME ratings, the matched SME threat, the expert’s rationale, and a recommendation. In this case, ratings are consistent. When they disagree, the alert highlights the discrepancy for human review.

Path to production: 30 days from handover to live

After the PoC handover, TUI’s team took the system from prototype to production-ready deployment in 30 days. The following results reflect the system’s performance in production.

Results

*Based on TUI’s internal measurements comparing pre- and post-deployment performance.

The solution processes each architecture in the eu-central-1 Region, with data encrypted at rest using AWS Key Management Service (AWS KMS) and in transit with TLS. Architecture diagrams and threat models are processed within the AWS infrastructure, helping TUI meet its data residency and security requirements.

The following screenshot shows the threat catalog, a card view of completed threat models. Each card displays the original architecture diagram, an AI-generated description, completion status, and threat counts by severity (red for high, yellow for medium, green for low).

Figure 5: The threat catalog provides an overview of completed threat models. Each card shows the architecture diagram, an AI-generated summary, and threat counts grouped by severity.

Engineers can select a threat model to view a dashboard with four analytical views: threats by likelihood, threats by STRIDE category, threats by target asset, and threat sources.

Figure 6: The insights dashboard for a completed threat model. This example shows 40 identified threats (6 high, 24 medium, and 10 low) distributed across all six STRIDE categories and mapped to specific architecture components.

“In the first four weeks after launch, 64 users registered and 19 unique creators produced 34 threat models with Threat Designer. What previously took one to two weeks per assessment is now available on demand. As of October 2026, adoption has grown to 255 registered users, 86 threat models, and 43 active creators, demonstrating that teams are embracing self-service security as part of their development workflow.”

— Marvin Jordan, Head of Technology for Security Platform and Practices, TUI

Conclusion

TUI’s experience shows that agentic AI workflows can address a real operational challenge in enterprise security: making thorough threat modeling fast enough to be mandatory. The combination of metadata-driven context selection, self-correcting threat generation, and intelligent caching allowed a four-person security team to support over 100 digital product teams. TUI achieved consistent, standards-aligned assessments at scale.

Looking ahead, TUI plans to expand the system with automated Jira ticket creation for mitigation tracking and integration with CI/CD pipelines for pre-deployment security checks.

You can get started with Amazon Bedrock to build AI-powered security workflows and Amazon Bedrock AgentCore to deploy agentic applications in production. For hands-on guidance with your specific use case, connect with the AWS Generative AI Innovation Center. To explore the open source foundation that this solution extends, see the Threat Designer project on GitHub and the AWS blog post Accelerate threat modeling with generative AI.

We welcome your feedback and questions in the comments.

About the authors

Post Syndicated from Natasha Sah original https://aws.amazon.com/blogs/architecture/how-boat-delivers-personalized-fitness-coaching-with-amazon-bedrock/

This post is co-written with Sridhara Seshadri from KaHa Technologies Pvt Ltd, a wholly-owned subsidiary of Imagine Marketing Limited (IML).

Recognizing that users need more than raw fitness data, boAt launched the “AI Coach” feature. It uses a combination of user history, preferences, and generative AI to deliver personalized coaching plans in real time. Using AWS, boAt built a solution that delivers personalized AI coaching. The AI Coach feature has been in production for over six months, reliably serving millions of boAt smartwatch users across India. In this post, we explore how boAt built the AI Coach on Amazon Bedrock and AWS Lambda, and the design choices that made it scalable and cost-effective.

Imagine Marketing Limited primarily operates under the brand boAt, launched in 2015 in the consumer electronics industry, focused on mass-premium, lifestyle-oriented products for India’s young, tech-savvy consumers at accessible price points. Starting with audio, boAt has expanded into adjacent categories such as wearables and charging solutions, selling through online marketplaces, D2C, and offline retail. In FY 2025, boAt led India’s branded personal audio with strong value and volume shares, sold over 34 million units in India as reported in FounderPin, and ranked among the top global audio brands by volume. Its agile, digital-first model drives rapid innovation, wide reach, and faster time-to-market aligned with evolving consumer preferences.

Opportunity

With the emergence of advanced AI capabilities, boAt saw an opportunity to deliver a highly personalized fitness coaching experience for its smartwatch users. The core objective was to help users build and sustain fitness and wellness habits aligned with their lifestyles. These recommendations needed to be straightforward for users to follow while clearly communicating their value.

The three primary design principles were personalization, affordability, and scalability.

Solution architecture

The following shows a simplified high-level architecture of the AI Coach components deployed on AWS.

Figure 1: High-level architecture of the boAt AI Coach on AWS

The following sections provide a brief overview of the architecture and its key components.

1. Data source: The boAt smartwatch serves as the primary data source, capturing metrics such as steps, user activities (for example, running or yoga), and sleep data. All data is encrypted at rest and in transit. The boAt Crest companion app for iOS and Android also captures additional data points, such as the user’s preferred communication style and fitness goals for a predefined period.
2. Historical data: Historical user data, when available, plays a key role in personalizing plans. These data points are transmitted by the companion application to backend servers and stored in Amazon Relational Database Service (Amazon RDS) for MySQL and Amazon DocumentDB (with MongoDB compatibility). When the AI coach creates a plan, this historical data provides important context about what the user has done earlier.
3. AWS Lambda: This component plays several roles:
   • It acts as a translation layer, transforming input data into a format suitable for large language models (LLMs) and converting model outputs into a format the companion application can parse and interpret. This layer also abstracts the underlying AI coach implementation.
   • On the input side, the data sent through the API is in a legacy format optimized for database operations. This is mapped to a more verbose format so that LLMs can better interpret what each key means.
   • On the output side, the system handles missing keys in the generated plan by substituting safe defaults and verifying that the response format remains consistent with the companion application’s requirements.
   • For production deployments, consider placing the Amazon Elastic Compute Cloud (Amazon EC2) instance in a private subnet and exposing it through an Amazon API Gateway private integration using a VPC link.

The AI Coach feature has two main logical pieces:

1. Personalized plan generation.
2. Nudges to aid adherence to the plan.

We use the Meta Llama 3.3 70B Instruct model on Amazon Bedrock to implement the AI Coach feature. For model availability by Region, see Region availability at a glance.

AWS Lambda retrieves historical user data for a predefined period in accordance with applicable privacy laws from the boAt Crest Companion app (full historical data is stored in Amazon RDS and Amazon DocumentDB), supplies it as context to the LLM for personalized plan generation, and parses the model response for the client. The team uses Amazon Bedrock on-demand inference exclusively, with structured data retrieval and prompt construction handled within Lambda. As adoption grows, they plan to transition to Amazon Bedrock batch inference to optimize costs at scale.

The plan is personalized to each user based on the inputs provided by the user during onboarding to this feature and historical user data patterns. The architecture was designed in such a way that future changes in user inputs don’t need a change in the backend components.

Relevant nudges are sent to the user at different times of each day, based on their actual metrics compared to the recommended ones. The style and tone of the nudges are offered as a configuration for the end user. These range from informational and objective to pop-culture and sarcastic, with a wide variety of style packs to choose from.

After evaluating options, the team selected the AWS Lambda and Amazon Bedrock combination for three reasons:

1. Model-agnostic access: The unified API in Amazon Bedrock let the team evaluate multiple foundation models (including Meta Llama 3.3) through a single integration. This avoided the need to build separate pipelines for each LLM and accelerated model selection.
2. Built-in observability and cost governance: Amazon Bedrock publishes per-invocation metrics (token counts, latency, and error rates) directly to Amazon CloudWatch, and it supports application inference profiles with cost allocation tags to attribute usage by tenant, feature, or environment. Combined with the average Lambda execution-time metrics, this gave the team precise per-user unit economics from day one, without building custom metering infrastructure.
3. Serverless scalability with zero infrastructure overhead: The on-demand inference mode in Amazon Bedrock requires no provisioned compute. It scales automatically with usage and charges only per token processed. As AI Coach adoption grows, the team can extend throughput through cross-Region inference profiles that dynamically route requests across AWS Regions, without changes to their Lambda-based backend or impact to the end user experience.

Throughout development, the team applied responsible AI practices—including output validation, safe default substitution for missing response keys, and prompt constraints limiting recommendations to watch-supported activities—to deliver a trustworthy and safe user experience.

The following screenshots illustrate the AI Coach onboarding experience, where users set their fitness goals, preferred activities, workout frequency, sleep schedule, and stress levels to generate a fully personalized coaching plan.

Beyond the user-facing experience shown in the preceding screenshots, the development journey involved several technical and design challenges worth highlighting. The following are notable challenges the team encountered during development and real-world testing.

1. Personalizing plans effectively: During the initial testing phase, the generated plans were not personalized enough to meet our expectations. The data the companion app provides is in JSON format, made of key-value pairs. We found that the keys needed to be more descriptive for the LLM to interpret the data effectively. For example, weight became weight_in_kg, and target became step_goal. We spent considerable time verifying that each legacy key was expanded and made as descriptive as possible. This higher level of descriptiveness significantly improved plan personalization.
2. Smartwatch capability matching: Recommended activities in the plan, such as swimming, had to align with the user’s onboarding preferences, their overall goals, and the smartwatch’s capability to support those activities. We added a dedicated section to the prompt that lists watch-supported activities, which reduced the likelihood of unsupported recommendations. However, because of the non-deterministic nature of LLMs, prompt optimization alone did not eliminate all errors. We therefore added a parsing layer and suitable defaults to handle exception cases.

Conclusion

The boAt AI Coach demonstrates how generative AI can transform raw smartwatch data into personalized, actionable wellness guidance at scale. By combining Amazon Bedrock, AWS Lambda, and thoughtful prompt engineering, the team shipped a production-grade feature that serves millions of users without sacrificing personalization or reliability. To learn more about the boAt AI Coach and explore the technologies behind it, check out the following resources:

• Read about how the AI Coach in boAt smartwatches can improve your fitness
• Explore Amazon Bedrock for building generative AI applications
• Explore the boAt Chrome Eon smartwatch featuring AI Coach

About the authors

Post Syndicated from Deepti Tirumala original https://aws.amazon.com/blogs/architecture/how-mhk-built-a-hipaa-eligible-agentic-ai-solution-on-amazon-bedrock/

Healthcare organizations face an increasingly complex challenge: processing vast volumes of medical documents, including clinical records, claims, prior authorizations, appeals, and pharmacy data, while maintaining strict HIPAA compliance and security standards. Traditional approaches require dedicated engineering teams to build individual AI systems for each use case, each needing its own compliance infrastructure, audit trails, and security controls. Agentic frameworks that can scale across use cases can reduce lengthy development cycles and high operational overhead.

MHK, a Hearst Health company ranked #1 in payer care management solutions in the 2024 Best in KLAS: Software & Services Report, faced this exact challenge. Their medical management solution serves health plans across multiple workflows (medical, pharmacy, grievance, and appeals), each requiring intelligent document processing and decision support. Building separate AI systems for each workflow was unsustainable as demand grew.

To solve this, they developed the SmartProminence AI Orchestrator, a HIPAA-eligible agentic workflow framework built on AWS that reduced manual medical review effort by 90%. New AI features that previously took 3+ months to deploy now ship in 2 weeks.

In this post, we walk through how MHK architected this solution using Amazon Bedrock, Amazon Elastic Container Service (Amazon ECS), and event-driven patterns to create a reusable, multi-tenant orchestrator for healthcare AI.

MHK uses Amazon Bedrock exclusively for foundation model inference. They built their own orchestration, retrieval, and validation layers because healthcare workflows require domain-specific controls: DAG-based multi-step execution, clinical document retrieval tied to case context, and HIPAA-specific validation logic that goes beyond general-purpose guardrails. This approach keeps Bedrock focused on scalable model access while MHK retains full control over workflow behavior and compliance enforcement.

Background

MHK provides healthcare cost management and compliance solutions to health plans across the United States. Their medical management system supports the full lifecycle of care decisions, from the moment a provider submits a request for service through final resolution. This includes prior authorization, claims adjudication, appeals processing, pharmacy benefit verification, and medical director reviews.

Each workflow involves analyzing unstructured medical documents such as clinical notes, lab results, imaging reports, and multi-page faxed records, against structured policy criteria. Before MHK’s SmartProminence AI Orchestrator, case managers spent 5 to 10 minutes manually processing each incoming document, while medical directors spent longer reviewing complex cases that required policy adherence determinations.

MHK needed to automate this research while maintaining healthcare’s audit trail and compliance requirements, and to do so across all product modules without building separate AI infrastructure for each one.

Business challenge

As MHK evaluated how to bring AI capabilities across their entire product suite, three core challenges emerged.

• Fragmented AI infrastructure. Each AI-powered feature would require its own deployment pipeline, HIPAA compliance certification, security controls, and monitoring. Every new AI roadmap item meant a new cluster, a new compliance engagement, and a new operational burden. For a company serving multiple health plans across multiple modules, this approach could not scale.
• Lengthy development cycles. Deploying a new AI workflow through traditional engineering took 3 to 6 months, not including requirements gathering. The engineering team could not keep pace with the product roadmap.
• Manual effort at premium cost. Case managers, nurses, pharmacists, and medical directors spent hours per case manually searching through patient records. The cost was especially acute for medical directors (physicians) and pharmacists, whose hourly rates make even small-time savings translate into significant ROI.

Solution overview: SmartProminence AI Orchestrator

MHK built the SmartProminence AI Orchestrator, a multi-tenant, agentic workflow framework running entirely on AWS. Rather than building separate AI systems for each use case, MHK created a single orchestrator where various AI workflows can be deployed through configuration. Define your prompts, specify your input/output schemas, and register the agent. The solution handles everything else including HIPAA compliance, encryption, audit trails, scaling, and orchestration.

The solution is architected around a controller-agent pattern in which a Workflow Engine Controller resolves workflow dependencies and dispatches individual steps to LLM processing agents. The entire system is stateless, event-driven, and independently scalable.

The following diagram illustrates the high-level architecture of the solution.

Figure 1: MHK SmartProminence AI Orchestrator architecture on AWS

At the core of the architecture, the Agent Orchestration Core serves as the central nervous system. Built on Spring Boot and running on AWS Fargate, it exposes a REST API that handles job submission, workflow management, LLM proxying, and token management. Critically, it is the only component that directly accesses the database: controllers and agents interact exclusively through the orchestration core’s API, enforcing strict data access boundaries.

Architecture overview

This section examines the key architectural patterns the orchestrator uses to process diverse healthcare workflows at scale.

Controller-agent pattern with Amazon Bedrock

MHK selected Amazon Bedrock for its multi-model access through a single API, letting them choose the best model per workflow step without separate integrations. As a managed AWS service, Bedrock inherits existing AWS Identity and Access Management (IAM), Amazon Virtual Private Cloud (Amazon VPC), and encryption controls, avoiding a new trust boundary. Built-in content filtering and invocation logging satisfy healthcare auditability requirements, and its model-agnostic architecture lets MHK adopt newer models without rearchitecting the solution.

The orchestrator enforces a strict separation between workflow orchestration and LLM processing. The Workflow Engine Controller determines what needs to happen and in what order, while LLM processing agents execute individual steps. This separation lets agent processing scale independently from workflow logic, and it makes the workflow the single source of truth while agents operate only on specific, actionable steps.

When a job arrives, the controller loads the version-pinned workflow definition, resolves step dependencies into a DAG using Kahn’s algorithm, and pre-creates step executions in a WAITING state. It uses conditional Spring Expression Language (SpEL) expressions to decide which steps to run versus skip, then dispatches agents layer by layer. Steps at the same depth run in parallel, and the controller polls for completion before advancing to the next depth.

Each agent runs a standardized pipeline: input binding (resolving expressions to gather prior step results), optional vision processing for scanned documents, prompt assembly with enriched context, LLM invocation to Amazon Bedrock (Claude), and post-processing for field extraction, type coercion, and structured output.

For parallel workloads within a single step, agents use Java virtual threads for each execution. This lets them process multiple items concurrently, such as extracting data from each page of a multi-page document simultaneously.

Event-driven orchestration with Amazon SQS

Communication between the orchestration core, controllers, and agents flows through Amazon Simple Queue Service (Amazon SQS) queues. To trigger a workflow, the orchestration core places a ControllerTaskMessage on the Controller Invoke Queue. To dispatch an individual step, it places an AgentTaskMessage on the Agent Invoke Queue. Each message is secured with a capability token scoped to only that operation’s data.

This design delivers four properties. Stateless processing means available instances can pick up pending messages. Independent scaling lets agents scale horizontally through ECS Fargate. Fault isolation keeps a failed task from blocking parallel steps, and dead letter queues capture failures. Decoupled deployment ships new agent versions without system-wide restarts.

The only blocking call in the pipeline is the LLM invocation to Amazon Bedrock. Everything else is asynchronous and event-driven, so the system can process hundreds of concurrent jobs without resource contention.

DAG-based parallel execution

The workflow engine uses depth-based parallel execution to maximize throughput. Consider a medical policy review workflow: at Depth 0, agents simultaneously extract patient demographics and pull claims history. At Depth 1, once both are complete, a policy lookup agent identifies the relevant criteria. At Depth 2, an evidence-gathering agent searches through the patient’s clinical history for documentation that satisfies each policy criterion. The controller only advances to the next depth when steps at the current depth have completed.

Conditional expressions can dynamically skip steps based on upstream results. For example, if the initial classification step determines that a case does not involve prescription drugs, the pharmacy verification step at the next depth is automatically skipped, saving both time and token costs. This conditional logic is evaluated by the controller using Spring Expression Language (SpEL) against the structured outputs of completed steps.

Dynamic agent registry

When MHK needs a new agent type, whether for a new medical management module or a new kind of analysis, the process is configuration-driven rather than engineering-driven.

A developer defines the agent configuration (prompt templates, input/output schemas, and model selection), then registers it through the orchestration core’s API. Terraform automatically provisions the supporting infrastructure: SQS queues, IAM roles, and ECS task definitions. The agent immediately becomes available for workflow step assignments, with no new compliance certification needed, since it runs within the already certified orchestrator.

This transformed MHK’s development velocity. The engineering team focuses on prompt design and workflow logic rather than infrastructure scaffolding.

Conversational memory and case association

The orchestrator maintains context across multiple workflow executions for the same patient case. Each execution returns a job ID that the upstream system associates with the case record. Over a case’s lifetime there may be three or more executions (initial intake, policy review, and appeal processing), each producing structured outputs that stay available for later executions.

Once a 30-page clinical record has been analyzed, its structured output is available for future queries on that case without re-running ingestion. When a medical director reviews an appeal weeks later, the patient’s history is already organized and searchable.

Prior context remains in Amazon Simple Storage Service (Amazon S3), encrypted with the client’s dedicated AWS Key Management Service (AWS KMS) key.

Responsible AI controls

MHK enforces safe LLM outputs through application-layer validation built into each agent’s processing pipeline. Every agent post-processes model responses against expected output schemas, cross-references extracted data with source documents to detect hallucinations, and rejects responses that fail confidence thresholds. Domain-specific checks verify that outputs reference only the patient’s own clinical records and match policy-specific medical criteria. LLM inputs and outputs are logged with full audit trails, which supports compliance review and reproducibility for every AI-assisted decision.

AWS services used

The following table summarizes the AWS services that compose the SmartProminence AI Orchestrator and the role each plays in the architecture.

Security and compliance

Healthcare data demands the highest security standards, and MHK’s architecture implements defense-in-depth across every layer. The orchestrator processes protected health information (PHI) for multiple health plan clients simultaneously, making multi-tenant data isolation a foundational feature.

• Per-client encryption. Every client has their own AWS KMS key. Documents stored in Amazon S3 are double-encrypted: S3 server-side encryption plus client-specific KMS encryption. Even if a job were somehow misrouted (which the token system helps prevent), the receiving agent could not decrypt another client’s data because it would not have access to that client’s KMS key. The database layer adds row-level encryption on top of Amazon Relational Database Service (Amazon RDS) storage-level encryption, providing defense-in-depth for data at rest.
• Capability token model. A least-privilege token system limits what each component can access. A controller-scoped token can read workflow definitions and job data, dispatch agent tasks, and create step executions. An agent-scoped token can only read its step’s input, write its own result, call the LLM through the proxy, and upload artifacts. Tokens are generated per-dispatch through KMS, so even a compromised agent cannot reach data from other steps, workflows, or clients.
• Network isolation. The database subnets have no internet access. AWS service communication (Amazon S3, Amazon SQS, AWS KMS, AWS Secrets Manager, Amazon CloudWatch, Amazon Elastic Container Registry (Amazon ECR)) flows through VPC endpoints, meaning no data ever traverses the public internet. Connections use TLS 1.3 for encryption in transit.
• Compliance controls. LLM request and response bodies are not logged. Only token counts and content hashes are recorded. Workflow configurations are stored immutably in S3 for complete version history. Agents receive only the minimum context needed for their step, following the principle of data minimization.

Results and impact

The SmartProminence AI Orchestrator delivered measurable business outcomes across both MHK’s internal operations and their health plan clients.

90% reduction in manual review effort. For document intake workflows, processing time dropped from 5–10 minutes per document (manual) to under 1 minute (automated with human-in-the-loop verification). For complex medical director reviews, the system pre-gathers the relevant evidence and presents a structured summary, reducing the physician’s task from hours of document searching to a 30-second approval or denial decision.

85% faster AI feature deployment. New AI capabilities that previously required a full 3–6 month engineering release cycle now deploy in approximately 2 weeks. The engineering team defines workflow configuration and prompt logic without building custom infrastructure, compliance pipelines, or security controls for each feature.

Unified compliance posture. Instead of attesting each AI feature independently, MHK maintains a single orchestrator-level HIPAA and SOC 2 attestation that covers the agents. New agents inherit the orchestrator’s security controls automatically: per-client encryption, audit logging, token-based access, and data minimization.

Multi-tenant extensibility. Health plan clients can run AI workflows through the framework without building their own HIPAA-eligible infrastructure. Because the orchestrator is configuration-driven, MHK can onboard new use cases for existing clients or deploy entirely new health plan customers with minimal engineering effort.

Conclusion

The orchestrator’s controller-agent architecture provides a blueprint for organizations that need to scale AI capabilities across multiple use cases without multiplying their compliance burden. The key insight is that compliance infrastructure should be an orchestrator-level concern, not a per-feature concern, and that agentic orchestration patterns can be both powerful and auditable when designed with healthcare-grade security from the ground up.

Looking ahead, MHK is extending the orchestrator with conversational interfaces so case managers and medical directors can interactively query case data, using the same workflow memory and security infrastructure. The dynamic agent registry continues to grow as new medical management modules adopt AI-powered decision support. MHK is also exploring AWS Marketplace as a distribution channel to bring their HIPAA-eligible agentic framework to organizations beyond healthcare that require similar compliance thresholds.

Share your experience building HIPAA-eligible AI workflows in the comments or reach out if you’re exploring agentic architectures for regulated industries.

To learn more, get started with Amazon Bedrock and explore the Amazon Bedrock code samples to build your own agentic AI solutions on AWS.

About the authors