Post Syndicated from Anand Bilgaiyan original https://aws.amazon.com/blogs/architecture/build-adaptive-ai-interfaces-with-the-ag-ui-protocol-agent-swarms-and-nova-act-on-aws/

Your generative AI applications produce different results each time they run. One medical scan shows a single fracture, and another reveals twenty ambiguous regions that require expert review. Static interfaces can’t adapt to this variability.

In this post, we show you how to build interfaces that automatically adapt to your AI’s variable outputs. This approach reduces interface development time and removes the need for custom integration code by using the AG-UI protocol, the Strands Agents Software Development Kit (SDK), and Amazon Nova Act. You learn to build adaptive interfaces using the agent-to-UI (AG-UI) protocol for dynamic UI generation, the Strands Agents SDK Swarm pattern for multi-agent collaboration, and Amazon Nova Act for legacy system integration. After reading this post, you understand when to use adaptive interfaces and how to deploy them in your applications.

The problem with static interfaces

You design screens with fixed layouts, predetermined controls, and static data binding. This works well when your application’s output space is known and consistent. An ecommerce checkout page needs the same fields for every transaction. A dashboard displays the same metrics regardless of the data. Static interfaces excel at these predictable scenarios.

AI-driven applications introduce new requirements. Consider two scenarios when you review bone X-rays. In the first scenario, one image shows a single obvious fracture requiring minimal interface controls. In the second scenario, another image reveals three subtle regions where multiple AI agents must debate findings, track confidence progression, and reach consensus before presenting results. A static interface optimized for the first scenario lacks the controls needed for the second. An interface built for the second scenario presents unnecessary complexity in the first scenario with empty panels and unused controls.

This problem appears in multiple domains. Fraud detection systems encounter variable evidence chains. Legal document review surfaces unpredictable numbers of relevant clauses. Security event response reveals different threat patterns requiring different analysis tools. Domains where AI discovers things dynamically rather than classifying into predetermined categories face this architectural challenge.

The core challenge is that AI agents discover and reason about the world dynamically, while traditional interface design assumes static, predetermined outputs.

Business impact

This mismatch costs development teams significant time and creates poor user experiences. You spend weeks building interface variations to handle different scenarios, then maintain multiple code paths as your AI models evolve. Your users face either overwhelming complexity when AI finds simple results, or insufficient controls when AI discovers complex patterns requiring deeper analysis. The development cost compounds as you add new AI capabilities. Each new agent or model requires rethinking your entire interface architecture.

Solution overview

Three AWS technologies address this challenge, so you can build interfaces that adapt to what AI agents discover.

The AG-UI protocol offers standardized streaming for agent-to-user interface (UI) communication. Before AG-UI, connecting agents to interfaces required custom code for each framework. You built custom WebSocket formats, polling mechanisms, and bespoke integration code every time you switched agent frameworks or added new capabilities. AG-UI removes this work by providing a standard format of typed events that stream over Server-Sent Events (SSE). Agent frameworks emit AG-UI events, and frontends consume them, creating a universal contract so you can swap frameworks without rewriting integration code.

The Strands Agents SDK offers the Swarm pattern for peer-to-peer multi-agent collaboration. In swarm patterns, your agents operate as peers that share hypotheses and iteratively refine findings until reaching consensus. This debate process, visible to you in real time, builds trust and catches errors that single-agent systems miss. For medical imaging, the swarm pattern mirrors how radiologists consult specialists, with multiple expert perspectives converging on accurate diagnoses.

Amazon Nova Act offers browser-based automation using natural language commands, so your AI agents can interact with legacy systems through their web interfaces. Your agent navigates login screens, searches for related records, fills form fields, and captures confirmation numbers, while streaming actions back to the primary interface so you can observe the process.

These three technologies work together naturally: AG-UI adapts your interface to swarm findings, the swarm produces explainable multi-agent analysis, and Nova Act bridges the gap with legacy systems that lack API access.

Prerequisites

Before starting, verify you have:

Required AWS Resources:

• AWS account with Amazon Bedrock access in a supported region (us-east-1, us-west-2, or eu-west-1)
• Your Identity and Access Management (IAM) user or role needs these specific permissions:
  • bedrock:InvokeModel – For calling foundation models.
  • bedrock:CreateAgent and bedrock:CreateAgentActionGroup – For agent deployment.
  • lambda:CreateFunction and lambda:InvokeFunction – For serverless compute.
  • s3:PutObject and s3:GetObject – For file storage.
  • dynamodb:PutItem and dynamodb:GetItem – For state management.
  • secretsmanager:GetSecretValue – For credential retrieval.
  • logs:CreateLogGroup and logs:PutLogEvents – For Amazon CloudWatch logging.

Development Environment:

• Python 3.9+ with pip installed.
• Node.js 16+ and React 18+ installed.
• AWS Command Line Interface (CLI) configured with your credentials.

Technical Skills:

• Intermediate Python programming experience.
• Familiarity with event-driven architectures (your interface listens for messages from agents and updates in real time, similar to how chat applications work).
• Basic understanding of Representational State Transfer (REST) APIs and Server-Sent Events (SSE).

Data protection and HIPAA compliance

This solution processes Protected Health Information (PHI) including medical images, patient MRN, and clinical findings. Apply the following safeguards before deploying to any environment handling real patient data.

Encryption at rest — Configure SSE-KMS with a customer managed key on all Amazon S3 buckets storing medical images. Enable encryption with a customer managed AWS KMS key on all DynamoDB tables storing session state, conversation history, and analysis findings.

Encryption in transit — Enforce TLS 1.2+ on all connections. Attach a bucket policy denying all S3 actions when aws:SecureTransport is false. Do not override DynamoDB SDK endpoints to HTTP. Do not set ignore_https_errors=True on Nova Act workflows. If the legacy system uses self-signed certificates, add its CA to your runtime trust store.

S3 Block Public Access — Enable Block Public Access at the account level and on every bucket in this solution. Medical images must never be exposed through public bucket policies or ACLs.

HIPAA-eligible services and BAA — All AWS services in this architecture (Amazon Bedrock, Amazon S3, DynamoDB, Lambda, API Gateway, CloudFront, Cognito, Secrets Manager, CloudWatch) are HIPAA-eligible. Before processing PHI, execute a Business Associate Agreement (BAA) with AWS covering these services.

PHI minimization — Never write MRN, patient name, or clinical findings to plaintext logs. Enable CloudWatch Logs data protection policies to detect and mask PHI patterns automatically. Suppress Nova Act trajectory logging during steps that display patient data. Verify Cognito JWT on the SSE endpoint before emitting any PHI-bearing event.

Important: Code samples in this post are for educational purposes. Review all configurations against your organization’s HIPAA Security Rule implementation before production deployment.

Architecture

You implement an orchestrator pattern where your frontend interacts with a single entry point that internally coordinates specialized sub-agents. The solution follows this pattern: your React app sends requests to Amazon API Gateway, which triggers AWS Lambda functions that coordinate AI agents through Amazon Bedrock, then streams results back over Server-Sent Events. We use Amazon Bedrock AgentCore, a platform to build, connect, and optimize agents at scale with any framework or model.

Figure 1: Logical view of the adaptive interface, showing the AG-UI client, the backend orchestrator, the Strands agent swarm, and integrations with legacy systems and Amazon Bedrock

Figure 2: End-to-end AWS architecture showing the 16 components that deliver authentication, content delivery, agent orchestration, foundation model inference, and legacy system automation

Architecture components

The architecture consists of 16 integrated components working together:

① User authentication You authenticate through Amazon Cognito, which provides secure identity management and generates JWT tokens for accessing the radiology portal application.

② Content delivery Amazon CloudFront serves the React application and static assets from Amazon S3, providing global low-latency access and caching for optimal performance.

③ API Gateway Amazon API Gateway handles both REST API requests and Server-Sent Events (SSE) connections, providing the entry point for client-server communication.

④ Authorization of user API request with token validation

⑤ Backend processing AWS Lambda functions act as the AG-UI handler, managing authentication, invoking agents through Amazon Bedrock AgentCore Gateway, and formatting responses as SSE streams.

⑥ Image storage Amazon S3 stores medical images with SSE-KMS encryption using a customer managed key. S3 Block Public Access is enabled at the bucket level. Lambda generates short-lived, tightly scoped pre-signed URLs for secure direct uploads, pinned to the PUT method, scoped to a per-user key prefix, restricted to the application/dicom content type, and set to expire in 300 seconds. A bucket policy enforces maximum upload size through the s3:content-length-range condition and denies requests where aws:SecureTransport is false.

⑦ Session management Amazon DynamoDB maintains session state, conversation history, analysis findings, and agent registry data for stateful multi-turn interactions.

⑧ AgentCore Gateway Amazon Bedrock AgentCore Gateway serves as the orchestration layer, routing agent requests, managing sessions, coordinating multi-agent workflows, and load balancing across runtime instances.

⑨ AG-UI handler A specialized component within AgentCore that manages AG-UI protocol events, formatting agent responses as standardized events for dynamic UI rendering.

⑩ AgentCore runtime AgentCore Runtime provides the execution environment for agent instances, running the Strands Agent Swarm with isolated runtime instances for each agent type.

⑪ Agent swarm execution The Strands Agent Swarm consists of three specialized agents (Image Analysis, Clinical Reasoning, Reporting) that collaborate through iterative debate rounds to reach consensus.

⑫ Foundation model inference Amazon Bedrock provides access to the Claude Sonnet foundation model for reasoning, interpretation, and natural language generation across agents using a large language model (LLM).

⑬ Knowledge base retrieval Amazon OpenSearch Serverless stores medical knowledge base with vector embeddings, providing semantic search for relevant medical literature and clinical guidelines.

⑭ Legacy system automation Amazon Nova Act performs browser-based automation to submit validated findings to a legacy hospital’s Radiology Information System (RIS) and Electronic Medical Record (EMR) systems, with each action streamed back to your UI.

⑮ Credential management AWS Secrets Manager securely stores and rotates credentials for legacy system access, providing Nova Act with authentication details at runtime.

⑯ Observability and monitoring Amazon CloudWatch captures logs and metrics, with data protection policies enabled to detect and mask PHI. AWS Distro for OpenTelemetry (ADOT) provides distributed tracing with AWS X-Ray-compatible trace export. AWS Security Token Service (AWS STS) manages temporary security credentials.

Key architectural decisions

Your frontend sees one agent (radiology-assistant), not three, through the orchestrator pattern. This simplifies integration and encapsulates workflow complexity. The orchestrator internally coordinates the swarm based on analysis stage.

Rather than generating arbitrary HTML, agents select from themed, accessible components (ROICard, DebatePanel, ConfidenceMeter). This balances flexibility with design consistency and security through the predefined component library approach.

SSE provides real-time updates as agents work through the streaming protocol. You see agent contributions character by character, creating transparency into the reasoning process.

Your frontend exposes state (current findings, validation decisions) to agents through bidirectional state synchronization. Agents update state through actions. This synchronization supports human-in-the-loop workflows where agents pause for validation before proceeding.

Solution walkthrough

The following steps walk through the solution, from authentication and swarm configuration to the AG-UI endpoint, legacy system integration, and deployment.

Step 0: Configure authentication

Add JWT validation to your API endpoint to enforce authentication before processing requests containing PHI.

from fastapi import Depends, HTTPException, Request
import os

COGNITO_USER_POOL_ID = os.environ["COGNITO_USER_POOL_ID"]
COGNITO_APP_CLIENT_ID = os.environ["COGNITO_APP_CLIENT_ID"]

async def verify_token(request: Request):
    """Validate Cognito JWT from Authorization header."""
    token = request.headers.get("Authorization", "").replace("Bearer ", "")
    if not token:
        raise HTTPException(status_code=401, detail="Missing authorization")
    # Validate token against Cognito JWKS endpoint
    # See: https://docs.aws.amazon.com/cognito/latest/developerguide/amazon-cognito-user-pools-using-tokens-verifying-a-jwt.html
    return validate_jwt(token, COGNITO_USER_POOL_ID, COGNITO_APP_CLIENT_ID)

@app.post("/api/agui")
async def agui_endpoint(request: Request, user=Depends(verify_token)):
    ...

Full Amazon Cognito user pool setup (pool creation, hosted UI, and token exchange) is covered in the Amazon Cognito Developer Guide. This post focuses on the agent architecture layer.

Step 1: Install the Strands Agents SDK

Install required packages by running the following command:

pip install strands-agents ag-ui-strands fastapi uvicorn

This command installs the required packages for building your multi-agent swarm, including the Strands framework, AG-UI integration, and web server components.

Step 2: Configure your swarm agents

Create specialized agents for your swarm by adding the following code to your project:

from strands import Agent
from strands.models import BedrockModel
from typing import Dict, List, AsyncGenerator
import os

region = os.environ.get("AWS_REGION", "us-east-1")

class RadiologySwarm:
    """Coordinates multi-agent analysis with visible debate."""

    def __init__(
        self,
        consensus_threshold: float = 0.80,
        max_rounds: int = 5,
    ):
        self.consensus_threshold = consensus_threshold
        self.max_rounds = max_rounds
        self.agents = self._initialize_agents()
        self.hypotheses = []

    def _initialize_agents(self) -> List[Agent]:
        """Create specialized agents for swarm."""
        model = BedrockModel(
            model_id=os.environ.get("BEDROCK_MODEL_ID", "us.anthropic.claude-sonnet-4-20250514-v1:0")
        )
        return [
            Agent(
                name="image_analysis",
                system_prompt="Detect abnormal patterns in medical images...",
                model=model,
            ),
            Agent(
                name="clinical_reasoning",
                system_prompt="Validate findings with clinical context...",
                model=model,
            ),
            Agent(
                name="reporting",
                system_prompt="Structure findings in clinical format...",
                model=model,
            ),
        ]

    async def analyze_with_debate_stream(
        self,
        image_data: bytes,
        patient_context: Dict,
    ) -> AsyncGenerator[Dict, None]:
        """Stream swarm analysis events for real-time UI updates."""
        # Implementation continues in next steps...
        pass

Note: The agents use a foundation model accessed through Amazon Bedrock. The example defaults to Claude Sonnet 4, but you should select a currently active model from the Amazon Bedrock model lifecycle page for your AWS Region. Read the model ID from an environment variable so you can update it without code changes as newer models become available. Amazon Bedrock retires models on a published lifecycle schedule, so hardcoding a model ID risks failure when that model reaches end of life. Check the Amazon Bedrock model lifecycle page and supported Regions, and use an environment variable or AWS Systems Manager Parameter Store to manage the model ID externally.

This code creates three specialized agents (Image Analysis, Clinical Reasoning, Reporting) that work together as peers in a swarm pattern, sharing hypotheses and refining findings through iterative debate until reaching consensus.

Step 3: Implement consensus mechanism

Configure your swarm to continue debate rounds until agents reach consensus or exhaust maximum iterations:

from typing import Dict, List

class RadiologySwarm:
    """Consensus mechanism implementation."""

    def __init__(
        self,
        consensus_threshold: float = 0.80,
        max_rounds: int = 5,
    ):
        self.consensus_threshold = consensus_threshold
        self.max_rounds = max_rounds
        self.hypotheses = []

    async def analyze_with_debate_stream(
        self,
        image_data: bytes,
        patient_context: Dict,
    ):
        """Stream swarm analysis with consensus checking."""
        yield {"type": "swarm_start", "max_rounds": self.max_rounds}
        for round_num in range(1, self.max_rounds + 1):
            # Each agent contributes based on current hypotheses
            for agent in self.agents:
                try:
                    context = {
                        "image_data": image_data,
                        "patient_context": patient_context,
                        "round": round_num,
                    }
                    async for chunk in agent.stream_response(context):
                        yield {
                            "type": "agent_contribution_chunk",
                            "agent": agent.name,
                            "round": round_num,
                            "text": chunk,
                        }
                except Exception as e:
                    yield {
                        "type": "agent_error",
                        "agent": agent.name,
                        "round": round_num,
                        "error": str(e),
                    }
            # Check if consensus reached after all agents contribute
            if self._check_consensus():
                yield {"type": "consensus_reached", "round": round_num}
                break
        yield {"type": "swarm_complete", "findings": self.hypotheses}

    def _check_consensus(self) -> bool:
        """Verify hypotheses meet confidence threshold."""
        if not self.hypotheses:
            return False
        # Check if all hypotheses have confidence above threshold
        for hypothesis in self.hypotheses:
            if hypothesis.get("confidence", 0) < self.consensus_threshold:
                return False
        return True

Your swarm reaches consensus when proposed findings achieve confidence scores above your configured threshold (80% for this example). Confidence progresses as agents validate or debate each other.

In the first example of subtle fracture detection, Round 1 shows the Image Agent proposing a Region of Interest (ROI) with confidence above 80%. Round 2 shows the Clinical Agent validating with anatomical context, maintaining confidence above the threshold. Round 3 shows agents agreeing on clinical significance, reaching consensus when the three agents reach confidence scores above the configured threshold.

In the second example of false positive challenge, Round 1 shows the Image Agent proposing an ROI with confidence above the threshold. Round 2 shows the Clinical Agent challenging it as an artifact, with confidence dropping below 80%. Round 3 shows the Image Agent acknowledging the challenge, with confidence dropping further. Round 4 shows agents continuing debate without consensus. Round 5 shows maximum rounds reached, with the finding marked as disputed.

This iterative refinement, visible to you in real time, provides explainability that black-box AI systems can’t match.

Step 4: Set up the AG-UI protocol endpoint

Create a streaming endpoint by adding the following code:

from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
import json

app = FastAPI()

@app.post("/api/agui")
async def agui_endpoint(request: Request):
    """AG-UI streaming endpoint emitting typed events."""
    body = await request.json()

    async def event_stream():
        """Stream standardized AG-UI events."""
        swarm = RadiologySwarm()
        async for event in swarm.analyze_with_debate_stream(
            image_data=body["image_data"],
            patient_context=body["patient_context"],
        ):
            yield format_sse_event(event)

    return StreamingResponse(
        event_stream(),
        media_type="text/event-stream",
    )

def format_sse_event(event: Dict) -> str:
    """Format as Server-Sent Event."""
    return f"data: {json.dumps(event)}\n\n"

This code creates a streaming endpoint that emits standardized AG-UI events, so your frontend receives real-time updates as agents work without requiring custom integration code for each agent framework.

The AG-UI protocol defines event types that cover agent-to-UI communication needs. Instead of custom formats, agents emit structured events over SSE, and frontends subscribe to this stream and react to each event type. TEXT_MESSAGE_CONTENT streams agent reasoning or responses token by token for real-time visibility into agent thinking. STATE_DELTA provides incremental state updates for bidirectional synchronization between agent and interface. TOOL_CALL_START and TOOL_CALL_END show tool execution when agents invoke external functions or APIs. With UI_COMPONENT_SPEC, agents control which UI components appear and how they’re configured through interface element specifications.

Configure your frontend to subscribe to this stream and handle each event type:

// Frontend event handler
eventSource.onmessage = (event) => {
  const data = JSON.parse(event.data);
  switch (data.type) {
    case "text_message_content":
      appendAgentText(data.agent, data.text);
      break;
    case "state_delta":
      updateApplicationState(data.path, data.value);
      break;
    case "ui_component_spec":
      renderComponent(data.component, data.props);
      break;
  }
};

Step 5: Implement real-time agent debate visualization

Create interface components that make multi-agent collaboration visible to you by implementing the following visualization features:

A key benefit of the swarm pattern combined with AG-UI is making multi-agent collaboration visible. Rather than presenting you with final results from a black-box system, the interface shows agents debating findings in real time.

Your interface includes specialized components for visualizing agent collaboration. The agent contribution display shows each agent’s reasoning streaming character by character with a typing effect, indicating which agent is currently “thinking.” Color-coding distinguishes agents (blue for Image Analysis, purple for Clinical Reasoning, cyan for Reporting). The confidence timeline uses a line graph to show how confidence evolves across rounds. Increasing confidence (72% → 78% → 85%) indicates agents converging on consensus. Decreasing confidence (68% → 52% → 45%) shows successful challenge of a false positive. Evidence cards display each region of interest with a status badge (Challenged, Consensus, Disputed) based on the debate outcome, an AI-generated summary of key reasoning points, and an expandable section showing complete agent contributions for radiologists who want detailed analysis.

This visibility serves multiple purposes. For explainability, you see why agents reached conclusions, not only what they concluded. For error detection, visible debate helps you spot flawed reasoning. For confidence calibration, watching agents debate each other helps you assess reliability. For educational value, radiologists learn from agent reasoning, improving their own analysis.

Step 6: Integrate Amazon Nova Act for legacy systems

Implement browser automation to submit findings to legacy systems by adding the following code:

import json
import boto3
from typing import Dict

class LegacyRISSubmission:
    """Browser automation for legacy RIS submission."""

    def __init__(
        self, secret_name: str = "ris-credentials", region_name: str = "us-east-1"
    ):
        """Initialize with AWS Secrets Manager configuration."""
        self.secret_name = secret_name
        self.secretsmanager = boto3.client("secretsmanager", region_name=region_name)

    def _get_credentials(self) -> Dict:
        """Retrieve credentials from AWS Secrets Manager."""
        response = self.secretsmanager.get_secret_value(SecretId=self.secret_name)
        return json.loads(response["SecretString"])

    def submit_report(self, report_data: Dict, ris_url: str) -> Dict:
        """Submit validated findings to legacy RIS."""
        from nova_act import NovaAct
        from nova_act.types.workflow import Workflow
        creds = self._get_credentials()
        with Workflow(
            model_id="us.amazon.nova-act-v1:0",
            workflow_definition_name="radiology-submission"
        ) as workflow:
            with NovaAct(
                starting_page=f"{ris_url}/login",
                headless=True,
                workflow=workflow,
            ) as nova:
                # Authentication
                nova.act("Click on the username input field")
                nova.type_text(creds["username"], sensitive=True)
                nova.act("Click on the password input field")
                nova.type_text(creds["password"], sensitive=True)
                nova.act("Click Sign In button")
                # Patient lookup
                mrn = report_data["patient_mrn"]
                nova.act(f"Search for patient MRN '{mrn}'")
                nova.act("Click View Record button")
                # Report submission
                nova.act("Click New Report button")
                findings_text = report_data["findings_summary"]
                nova.act(f"Fill findings textarea with: {findings_text}")
                nova.act("Upload annotated image file")
                nova.act("Click Submit Report button")
                # Capture confirmation
                result = nova.act("Find and return the RPT- confirmation number")
        return {"status": "success", "confirmation": result, "mrn": mrn}

Security note: Nova Act captures prompts and screenshots as trajectory data. Never interpolate credentials or PHI into act() commands. Use type_text with sensitive=True to prevent credential capture in logs and trajectories. In production, suppress trajectory capture entirely for authentication steps, or route trajectory storage to a KMS-encrypted, access-controlled bucket subject to your BAA

Many enterprise systems, particularly in healthcare, lack modern APIs. Hospital RIS and EMR systems often run on decades-old technology stacks that can’t be modified without significant effort. Amazon Nova Act offers a pragmatic solution: browser-based automation using natural language commands.

This code implements browser automation that interacts with legacy systems through their existing web interfaces, retrieving credentials securely from AWS Secrets Manager and streaming each action back to your interface for transparency.

Each Nova Act command streams back to the radiology portal, so you can observe the submission process. Your interface displays a live action log showing authentication, navigation, form filling, and confirmation capture. This transparency helps you understand what the automation is doing and intervene if issues arise.

Browser automation requires careful credential management. The implementation stores credentials in AWS Secrets Manager, retrieves them at runtime, and never exposes them to the frontend. Audit logging captures Nova Act actions for compliance and troubleshooting. For production deployment, additional controls include IP allowlisting, session timeout enforcement, and multi-factor authentication where supported by legacy systems.

Step 7: Deploy to AWS

Configure DynamoDB tables with customer-managed KMS encryption:

import boto3

dynamodb = boto3.client("dynamodb")

dynamodb.create_table(
    TableName="radiology-sessions",
    KeySchema=[{"AttributeName": "session_id", "KeyType": "HASH"}],
    AttributeDefinitions=[{"AttributeName": "session_id", "AttributeType": "S"}],
    BillingMode="PAY_PER_REQUEST",
    SSESpecification={
        "Enabled": True,
        "SSEType": "KMS",
        "KMSMasterKeyId": "arn:aws:kms:us-east-1:ACCOUNT:key/YOUR-KEY-ID"
    },
)

Deploy your solution using Amazon Bedrock AgentCore, which offers managed runtime for agents with built-in identity, memory, and observability. AgentCore supports long-running tasks (up to 8 hours), asynchronous tool execution, and native CloudWatch integration, making it ideal for production deployments requiring minimal operational overhead.

Pattern selection guidance

Choosing the right architecture pattern depends on problem characteristics and requirements.

When to use dynamic agent-generated UIs

Swarm compared to other multi-agent patterns

Use swarm when multiple perspectives improve accuracy (peer review, consensus building), debate process offers value (explainability, error detection), no clear hierarchy exists (agents are peers, not supervisor/worker), and iterative refinement is beneficial (findings improve through rounds).

Use agents as tools when clear task decomposition exists (supervisor delegates to specialists), subtasks are independent (parallel execution possible), hierarchy is natural (manager coordinating experts), and no need for peer debate exists (each agent’s output stands alone).

Use sequential workflow when strict ordering is required (step B needs step A’s output), checkpoints are needed (validate before proceeding), process is well-defined (known stages, dependencies), and no benefit from parallel exploration exists (linear pipeline).

Use cases beyond medical imaging

This architectural pattern applies to domains with high output variability and multi-agent value. For fraud detection, you encounter variable evidence chains (1-20 suspicious transactions), multiple analyst perspectives (financial, behavioral, network analysis), and legacy banking systems requiring browser automation. For legal document review, you face unpredictable numbers of relevant clauses, expert opinions from different legal domains (contract law, regulatory compliance, risk assessment), and integration with legacy case management systems. For security event response, you deal with variable threat indicators, team collaboration (network analysis, malware analysis, threat intelligence), and legacy Security Information and Event Management (SIEM) systems without modern APIs.

Anti-patterns

Avoid this approach when you have simple classification (predetermined categories with fixed confidence scores), deterministic calculations (no uncertainty or debate needed), low-latency requirements (multi-round debate adds latency), cost-sensitive scenarios with predictable outputs (swarm pattern increases token usage), or minimal explainability needs (users trust results without seeing reasoning).

Clean up

To avoid incurring future charges, delete the resources you created while following this walkthrough. Delete them in the following order. The sequence removes the frontend and compute layers first, then the data stores, and finally the encryption keys and IAM roles, so no deletion fails because another resource still depends on it. Because this solution can process PHI, this teardown also removes any stored medical images, patient identifiers, and findings.

1. Amazon CloudFront — Disable the distribution, let it propagate, then delete it. This stops traffic and releases the S3 app bucket.
2. Amazon API Gateway — Delete the REST API and SSE endpoint.
3. Amazon Bedrock AgentCore — Delete the Gateway, then the Runtime instances (and their identity, memory, and session resources). This stops all agent activity against the downstream services.
4. AWS Lambda — Delete the AG-UI handler and any other functions.
5. Amazon Bedrock model access — Confirm nothing is still invoking the models. Revoke access to Claude Sonnet or Amazon Nova Act if enabled only for this walkthrough.
6. Amazon OpenSearch Serverless — Delete the knowledge base collection and its access, network, and encryption policies. OCUs bill hourly.
7. Amazon DynamoDB — Delete the radiology-sessions table and any other tables you created.
8. Amazon S3 — Empty (including all object versions) and delete the medical-image and app-hosting buckets.
9. AWS Secrets Manager — Delete the legacy RIS/EMR credential secrets. A 7 to 30 day recovery window applies unless you force deletion.
10. Amazon Cognito — Delete the user pool and app client.
11. AWS KMS — Only now, schedule deletion of the customer managed keys used for Amazon S3 and DynamoDB. Deleting them earlier can make encrypted data unrecoverable.
12. AWS IAM — Delete the roles and policies created for Lambda and AgentCore.
13. Amazon CloudWatch — Delete the log groups, dashboards, and alarms (kept until last for troubleshooting).

Finally, review the AWS Billing and Cost Management console to confirm no unexpected charges remain.

Conclusion

In this post, we showed you how to build adaptive AI interfaces using three AWS technologies. You learned to use the AG-UI protocol for dynamic UI generation, apply Strands swarm patterns for multi-agent collaboration, and integrate Amazon Nova Act for legacy systems. Through a complete radiology assistant implementation, you saw how these technologies work together to handle variable AI outputs while maintaining explainability and practical enterprise integration.

The AG-UI protocol creates interfaces that adapt to what your agents discover, not what you anticipated. Use it when output variability is high and you can’t predetermine interface requirements. The Strands swarm pattern creates explainability through visible multi-agent debate. Use when multiple perspectives improve accuracy and you benefit from seeing reasoning processes. Amazon Nova Act bridges the gap with legacy systems lacking APIs. Use when browser automation is the only viable integration path and action transparency matters.

These technologies work together naturally because they address different aspects of the same problem: building AI systems that are flexible, explainable, and practical for real-world enterprise environments.

Next steps

• Explore the AG-UI protocol on GitHub.
• Read the Strands Agents SDK documentation.
• Learn more about Amazon Bedrock AgentCore.

Read related AWS blogs:

• Multi-Agent Collaboration Patterns with Strands Agents and Amazon Nova.
• Strands Agents SDK: A Technical Deep Dive into Agent Architectures and Observability.
• Build a Drug Discovery Research Assistant using Strands Agents and Amazon Bedrock.

About the authors