Post Syndicated from Mazrim Mehrtens original https://aws.amazon.com/blogs/big-data/powering-agentic-ai-with-real-time-streaming-data-on-aws/
Two years ago, the conversation about streaming data and generative AI centered on a straightforward question: how do you feed real-time context into a large language model (LLM) so it can answer questions using fresh data? We explored that question in our 2024 blog post, “Exploring real-time streaming for generative AI applications,” which introduced patterns for connecting streaming pipelines to foundation models.
The landscape has shifted. Today’s generative AI systems don’t only answer questions. They observe, reason, and act. Agentic AI applications have moved from research prototype to production reality. Agentic AI-powered data pipelines now monitor streaming telemetry, detect anomalies, decide on remediation strategies, and execute actions without human intervention. They maintain memory across sessions, query live data sources on demand, and coordinate with other agents to solve complex problems.
This shift demands a fundamentally different relationship between streaming infrastructure and AI. It’s no longer enough to inject context into a prompt. You need architectures where streaming data continuously powers autonomous agent action and keeps a real-time lakehouse fresh for training and retrieval. That data also flows into multiple consumption patterns, such as generative business intelligence (BI) for humans, standardized protocols for agent queries, and proactive memory hydration for low-latency agent context.
This post introduces three architectural patterns that together form a unified streaming backbone for the agentic AI era:
- Streaming feature engineering → real-time inference → action: Continuous data flows build features, invoke AI models, and act in a single pipeline.
- Event-driven agent invocation: Streaming pipelines detect patterns across millions of events and trigger agentic workflows with full context already assembled.
- Real-time context synchronization: Change data capture (CDC) and streaming pipelines keep agents’ memory current, so agents can respond instantly rather than making expensive external calls.
The following sections explore each pattern in depth.
Pattern 1: Streaming feature engineering → real-time inference → action
You’re watching a live football match. As a striker receives the ball in the box, AI-generated commentary appears on screen: “This is Smith’s third touch in the penalty area in the last 3 minutes. His conversion rate from this zone is 34% this season.” That insight was computed from streaming event data, passed through a feature pipeline, and fed to a generative AI model. All of this happened within the time it takes the striker to turn and shoot.
This pattern combines two capabilities that are often treated separately: using real-time data to continuously improve AI models, and using real-time data to invoke those models for immediate action. The streaming pipeline does both: it builds the features that train the model and the features that drive inference.
Streaming events (user interactions, sensor readings, game events, and transaction records) flow into Amazon Managed Streaming for Apache Kafka (Amazon MSK) or Amazon Kinesis Data Streams. Amazon Managed Service for Apache Flink processes these events through windowed aggregations (tumbling windows, sliding windows, or session windows) to produce features: rolling averages, counts, ratios, behavioral sequences, or other derived signals relevant to your use case.
These features serve two paths simultaneously:
The inference path: At the end of each window (or on each event, depending on your latency requirements), features are passed to a generative AI or machine learning (ML) inference endpoint: Amazon Bedrock for generative output, or Amazon SageMaker for custom models. The model produces a result (commentary, a recommendation, a personalization decision, or a risk score) and the pipeline acts: posting content to a user, updating a recommendation feed, sending a notification, or writing to a downstream system.
The training path: The same streaming features are continuously written to a real-time data warehouse or lakehouse such as Apache Iceberg tables on Amazon S3 Tables, a capability of Amazon Simple Storage Service (Amazon S3), that keeps training datasets fresh. Amazon SageMaker lakehouse architecture provides unified access for training jobs and fine-tuning pipelines. As new data streams in, your models can be retrained or fine-tuned on data that’s minutes old rather than days old. This matters for domains where patterns shift quickly, such as fraud detection, personalization, and industry dynamics.
Amazon S3 Tables handles the Iceberg table management automatically, including compaction, snapshot management, and metadata optimization. Your team focuses on feature logic rather than storage operations. The AWS Glue Data Catalog makes these tables discoverable across training jobs, inference pipelines, and analytics consumers. Glue Data Catalog supports business context and semantic search. This context helps models discover and select the right data asset for any given task.
Scenarios
Real-time sports commentary: Streaming game events (passes, shots, player positions) flow through Apache Flink on Managed Service for Apache Flink, which computes rolling features (possession percentage, shot frequency by zone, player heat maps). These features feed a generative AI model through Amazon Bedrock that produces natural-language commentary and statistical insights in real time. Simultaneously, the features are written to S3 Tables to improve the model’s understanding of game patterns over time.
Streaming personalization: User clickstream data flows through Managed Service for Apache Flink, which computes behavioral features (session duration, category affinity scores, recency-weighted purchase history). These features invoke a personalization model that updates the user’s experience in real time by reranking product recommendations, adjusting content feeds, or triggering targeted offers. The same features feed the lakehouse to retrain the personalization model nightly.
Figure 1: Streaming feature engineering feeding a real-time inference path and a continuous training path
Pattern 2: Event-driven agent invocation
At 2:47 AM, a pressure sensor on a manufacturing line begins drifting. Within seconds, a streaming pipeline detects the anomaly, assembles full context (device history, maintenance schedule, correlated sensor readings), and invokes an agent that opens a maintenance work order, adjusts the device’s sampling rate, and notifies the on-call engineer. All of this happens before a human sees an alert.
Pattern 1 invokes inference on every window or event. It runs continuously. Pattern 2 adds to this approach: the streaming pipeline continuously analyzes data and invokes an agentic workflow when specific conditions are met or a pattern is detected. The pipeline is the sensor. The agent is the responder. Dynamic rules are the bridge between them.
The key distinction is that the events and triggers are dynamic. They’re defined by rules programmed into the streaming pipeline or traditional ML models for prediction or detection. The pipeline determines when and how the agent is triggered, making the system fluid and adaptive. You can update detection logic without redeploying the agent. You can add new anomaly patterns without changing the response logic.
Streaming telemetry flows into Amazon MSK or Amazon Kinesis Data Streams. Managed Service for Apache Flink runs continuous anomaly-detection logic, such as statistical models, windowed aggregations, threshold-based rules, or ML-based scoring. Critically, when Flink detects an anomaly, it doesn’t only publish a raw alert. It assembles a context package: the anomaly details, relevant historical data, correlated signals from other streams, and metadata the agent needs to act immediately.
This context package is published to a downstream topic and consumed by an Amazon Bedrock AgentCore agent. Because the pipeline has already assembled full context, the agent doesn’t waste time gathering information. It can reason and act immediately. AgentCore Runtime hosts the agent, AgentCore Observability provides tracing and logging, and AgentCore Memory maintains state across invocations (so the agent knows, for example, that this is the third anomaly from this device this week).
The benefit of this pattern over a polling-based or scheduled approach is twofold:
- Latency: The agent is invoked within seconds of the anomaly, not at the next polling interval.
- Context richness: The pipeline has already done the work of correlating signals and assembling context. A polling-based agent would need to make multiple queries to reconstruct what the pipeline already knows.
The rules that trigger invocation are a powerful abstraction. They can be simple thresholds (“temperature exceeds 95°C”), statistical (“value deviates more than 3σ from the rolling mean”), or ML-based (“anomaly score from an embedded model exceeds 0.85”). You can update these rules dynamically by adding new detection patterns, adjusting sensitivity, or routing different anomaly types to different agents.
Figure 2: Event-driven agent invocation triggered by anomaly detection in the streaming pipeline
Pattern 3: Real-time agent context
A customer messages their bank: “Was that $847 charge at the airport legitimate?” The agent responds in under two seconds with full context (the customer’s recent travel pattern, the merchant’s fraud-risk score, and the transaction details) because all of this was already loaded into the agent’s context layer through streaming CDC. A reactive agent without this synchronization would need to make five separate API calls across three systems, taking 8–12 seconds and risking timeout failures.
This pattern addresses a fundamental question: how proactive should your agent be about gathering context?
A proactive agent has the full context, continuously synchronized with the state of the world. When a user asks a question, the agent already has the relevant knowledge from context. It responds from memory rather than making expensive external calls. A reactive agent starts cold. It knows nothing until it queries for information, making multiple calls across security boundaries, handling authentication, and stitching together data from disparate sources. For latency-sensitive use cases, where a user sends a prompt and expects a fast response, this difference is critical.
Real-time context synchronization uses CDC and streaming pipelines to keep agent memory current. The agent’s knowledge graph becomes a synchronized replica of the distributed systems it needs to reason about.
No agent is purely proactive or purely reactive. The design decision is: what data should be pre-loaded, and what should be fetched on demand? This is a spectrum, and where you land depends on three factors:
- Latency sensitivity: If users expect fast, contextually relevant responses, pre-load the data the agent needs most frequently.
- Data volume: Synchronizing everything is impractical. An efficient, fast search that still produces accurate results matters more than exhaustive pre-loading. Be selective about what you push.
- Data freshness requirements: Some data changes every second (stock prices, session state). Other data changes rarely (customer preferences, account configuration). Load what changes frequently and matters immediately.
Streaming pipelines (Managed Flink reading from Amazon MSK, Kinesis Data Streams, or CDC streams from operational databases) continuously process events and write aggregated results to the agent’s knowledge graph, or the context layer. These stores can take multiple forms depending on your access patterns:
- AWS Context automatically maps relationships across your existing data into a knowledge graph and supports agentic search so AI agents can access governed data relationships, business rules, and domain knowledge at runtime. Data stewards manage the graph through an intuitive console, reviewing inferred relationships, promoting them to production, and attaching domain-specific knowledge like business definitions and usage rules.
- Amazon Bedrock AgentCore Memory for structured agent context that persists across sessions.
- Amazon DynamoDB for low-latency key-value lookups (customer profiles, account state).
- Amazon OpenSearch Serverless for semantic search over unstructured context (past conversations, documents).
- Amazon Neptune for relationship-rich data (knowledge graph).
- Amazon S3 Tables fully managed Apache Iceberg tables in Amazon S3, for interoperability between multiple query engines.
For data that isn’t pre-loaded, the agent falls back to on-demand retrieval. This applies when the data is too large, changes too rarely to justify streaming, or is needed only in edge cases. The Model Context Protocol (MCP) provides a standardized interface for this. MCP servers expose heterogeneous data sources through a uniform protocol. The agent queries MCP when it needs context that isn’t in its synchronized memory.
This same real-time context synchronization pattern serves different consumers:
AI agents access fresh context through a real-time knowledge graph or a context layer, and MCP servers (pull tier), as in the preceding sections.
Human analysts and executives access the same context layer, which can directly query Apache Iceberg tables on S3 Tables through its direct query mode. Amazon Quick chat provides natural-language access to real-time lakehouse data. No intermediate warehouse is required. This is the generative BI expression of the same underlying pattern: streaming data keeps the lakehouse current, and Amazon Quick gives humans conversational access to it.
Training and fine-tuning pipelines access the synchronized lakehouse through Amazon SageMaker Lakehouse, keeping models fresh (as described in Pattern 1).
The underlying principle is the same across consumers: streaming pipelines synchronize distributed data into accessible stores, and each consumer accesses those stores through the interface that fits their needs.
Figure 3: Real-time context synchronization serving agents, analysts, and training pipelines from shared stores
Bringing it together
The three patterns in this post form a unified architecture built on a single streaming backbone:
Pattern 1 uses streaming pipelines to build features that simultaneously drive real-time inference and keep training data fresh. Your models improve continuously while serving predictions in real time.
Pattern 2 uses streaming pipelines as intelligent sensors that detect anomalies and invoke agents with full context already assembled. This separates detection logic from response logic for maximum flexibility.
Pattern 3 uses streaming pipelines to synchronize distributed system state into the agent’s context layer, making agents more proactive and serving multiple consumers (agents, humans, and training jobs) from the same pre-loaded data.
The streaming infrastructure you build (Amazon MSK, Amazon Kinesis Data Streams, Amazon Managed Service for Apache Flink, and Amazon S3 Tables) serves all three patterns simultaneously. A Flink application can compute features for inference (Pattern 1), detect anomalies that trigger agents (Pattern 2), and synchronize state into agent memory (Pattern 3).
To get hands on with the patterns described in this post, refer to Agentic AI-Powered anomaly detection: Spotting anomalies in real-time.
You don’t need to implement all three patterns at once. Start with the one that addresses your most pressing need. But design your streaming infrastructure knowing it will serve multiple patterns. In the agentic AI era, every stream is a potential input to an agent, a model, and a human decision-maker.
We’re using the default Java truststore which uses the password 
