[$] Cindy Cohn on privacy battles old and new

Post Syndicated from jake original https://lwn.net/Articles/1061979/

Cindy Cohn is the executive director of the Electronic Frontier Foundation (EFF) and
she gave the Saturday morning keynote at SCALE 23x in Pasadena
about some of the work she and others have done to help protect online
rights, especially digital privacy. The talk recounted some of the history
of the court cases that the organization has brought over the years to try
to dial back privacy invasions. One underlying theme was the
role that attendees can play in protecting our rights, hearkening back to
earlier efforts by the technical community.

AI-powered event response for Amazon EKS

Post Syndicated from Aritra Nag original https://aws.amazon.com/blogs/architecture/ai-powered-event-response-for-amazon-eks/

Cloud environments with dozens of microservices are now easier to manage than ever, and modern DevOps teams are well-equipped to balance rapid deployments with operational stability — even as monitoring tools surface thousands of daily signals.

AWS DevOps Agent is a fully managed autonomous AI Agent that resolves and proactively prevents incidents, continuously improving reliability and performance of applications in AWS, multicloud, and hybrid environments. It brings Kubernetes-native intelligence to incident response. It understands how Pods relate to Deployments, which Services route traffic, what ConfigMaps provide configuration, and how these components interact across your environment. Rather than seeing isolated infrastructure issues, the agent comprehends the architectural relationships that matter most for fast, accurate root cause analysis. In this post, you’ll learn how AWS DevOps Agent integrates with your existing observability stack to provide intelligent, automated responses to system events.

Architecture Diagram of DevOps AgentFigure 1: This is an example of target architecture of how Amazon EKS workloads are deployed and how AWS DevOps agent can interact with the different managed services like Amazon CloudWatch

How AWS DevOps Agent discovers Kubernetes resources

Built on Amazon Bedrock, the agent can analyze complex operational scenarios and correlate data from multiple sources. AWS DevOps Agent combines natural language processing (NLP) of logs and error messages with root cause analysis, powered by machine learning (ML), to automatically identify issues across your infrastructure.

Telemetry-based discovery

The agent analyzes OpenTelemetry data to infer runtime relationships:

  • Service Mesh Analysis: Examines network traffic patterns between pods to identify service-to-service communication
  • Trace Correlation: Uses distributed traces to map request flows across microservices
  • Metric Attribution: Associates performance metrics with specific pods, containers, and nodes

Metadata enrichment

The agent enriches discovered resources with contextual information:

  • Labels and Annotations: Extracts application metadata, ownership information, and deployment details
  • Resource Specifications: Captures CPU/memory requests and limits, health check configurations, and environmental variables
  • Network Topology: Maps pod IPs, service cluster IPs, ingress rules, and network policies

Discovery process

When you start an investigation, the agent executes the following discovery workflow:

  1. Initial Scan: Queries the Kubernetes API for all resources in relevant namespaces
  2. Dependency Analysis: Builds a dependency graph showing how resources relate to each other
  3. Telemetry Correlation: Matches discovered resources with their corresponding metrics, logs, and traces
  4. Context Building: Aggregates resource state, recent events, and performance data into a unified view

Implementation details

Prerequisites

Before implementing this solution, verify that you have the following:

Development environment

As part of the setup and the applications that are mentioned in the following sections. We have this AWS samples repo with deployable scripts and setup instructions.

This section walks you through deploying and configuring the complete AWS DevOps Agent demo environment.

Step 1: Deploy AWS DevOps Agent infrastructure

Begin by deploying the AWS DevOps Agent using the AWS CDK. The infrastructure includes the Agent Space configuration, IAM roles and policies, and integration with your EKS cluster.

Screenshot of the AWS DevOps Agent web interface showing a form to create a new Agent Space named "OTEL-DevOpsAgent-Demo-v1". The form includes fields for agent space name and description, radio button options for configuring IAM roles for AWS resource access and web app access, auto-generated role names, and Cancel and Create action buttons.

Figure 2: This is screenshot of configuring the Agent Space in the AWS Console

Configure the Agent Space in the AWS Console by navigating to the AWS DevOps Agent service. You will then create a new Agent Space for your EKS cluster. Finally, you will set up data source integrations including Prometheus workspace endpoints, Amazon CloudWatch Log groups, and X-Ray service configuration.

Screenshot of the AWS DevOps Agent Capabilities configuration tab for the OTEL-DevOpsAgent-Demo-v1 agent space. The interface displays a Cloud capability section with a primary AWS account (ID: 123456789012) showing a Valid status, a secondary sources section with no entries, and options to add additional source accounts.Figure 3: This is screenshot of validating the connectivity to data sources and access to the AWS account

Validate the deployment by accessing the DevOps Agent web interface, verifying connectivity to data sources, and confirming that the agent can discover your EKS cluster resources.

Screenshot of the AWS DevOps Agent Incident Response Dashboard for the OTEL-DevOpsAgent-Demo-v1 project. The dashboard includes a text area to describe a new investigation, quick-start template buttons for common scenarios (Latest alarm, High CPU usage, Error rate spike), a bar chart showing daily investigation frequency from January 29 to February 4 with one investigation on February 4, and an empty investigations table filtered by Pending Start status.

Figure 4: This is screenshot of checking the incident response and if there is any ongoing investigation

Step 2: Set up port forwarding for applications

Configure port forwarding to enable the traffic generator to access your deployed applications. Set up port forwarding for all sample applications using “kubectl” commands. Each application runs on different ports to simulate a realistic microservices environment:

# Sample Metrics App (port 8000)
kubectl port-forward svc/sample-metrics-app 8000:8000 -n default &
# Python OTEL App (port 8080)
kubectl port-forward svc/otel-sample-app 8080:8000 -n default &
# Go OTEL App (port 8090)
kubectl port-forward svc/go-otel-sample-app 8090:8080 -n default &
# Java OTEL App (port 8081)
kubectl port-forward svc/java-otel-sample-app 8081:8080 -n default &

Step 3: Install and configure Traffic Generator

The traffic generator is a Python-based tool that creates realistic load patterns and error scenarios for testing AWS DevOps Agent capabilities. Install the required Python dependencies and make the traffic generator executable:

# Install required Python packages
pip install requests
# Make the script executable
chmod +x traffic-generator.py

The traffic generator supports multiple configuration options for creating different testing scenarios.

Step 4: Generate baseline traffic

Create baseline operational data by generating normal traffic patterns across all applications. This establishes normal operational patterns that AWS DevOps Agent can learn from. Generate steady traffic to establish baseline metrics:

# Generate normal baseline traffic
python traffic-generator.py --app all --duration 900 --rps 10 --error-rate 0.05

Terminal output showing the initialization of an EKS Platform Traffic Generator script targeting four OTEL applications — Sample Metrics App, Python OTEL App, Go OTEL App, and Java OTEL App — each configured for 900 seconds at 15 requests per second with a 10% error rate. All four services are confirmed available at their respective localhost ports.

This command generates traffic to all applications for 15 minutes at 10 requests per second with a 5% error rate, simulating normal operational conditions. Monitor the baseline traffic generation by checking application metrics and HPA scaling behavior:

# View HPA status (should show scaling based on metrics)
kubectl get hpa -A
# Check custom metrics availability
kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq .
# View specific metric values
kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/*/sample_app_requests_rate | jq .

System log output from a load testing session displaying real-time performance metrics including elapsed time, total requests, success and failure counts, success rate percentage, and requests per second (RPS). The log spans timestamps from 22:38:49 to 22:40:41, showing a consistent RPS of approximately 14.8–14.9 and alternating cycles of 0%, ~90%, and 100% success rates across approximately 9,000 to 10,650 total requests.

Step 5: Configure AWS DevOps Agent Investigation

Set up AWS DevOps Agent to monitor your EKS cluster and prepare for event investigation workflows.

Screenshot of the AWS DevOps Agent Incident Response interface showing the investigation timeline for the "Demo CPU Spike" incident created on 2026-02-04. The timeline displays a user request describing a 97.5% CPU spike on an EKS workload, followed by two assistant responses tracking investigation progress. The investigation was completed at 20:02:40 on the same day.Figure 5: This is screenshot of overall timeline of the ongoing investigation through AWS DevOps agent

Access the AWS DevOps Agent through the AWS Console:

  1. Navigate to AWS DevOps Agent in the AWS Console.
  2. Select your configured Agent Space.
  3. Select Operator access to open the DevOps Agent web application.
  4. Configure data source connections to verify proper integration.

Figure 6: This is screenshot of validating the access to the observability data from the cluster

Verify that the AWS DevOps Agent can access observability data from your cluster including metrics from Amazon Managed Prometheus, logs from Amazon CloudWatch Logs, traces from AWS X-Ray, and topology information from your EKS cluster. DevOps agent can also pull the service map information of your kubernetes resources.

Testing scenarios and use cases

This section demonstrates different testing scenarios that showcase AWS DevOps Agent capabilities in various operational situations.

Scenario 1: Normal load testing

This scenario establishes baseline operational patterns that AWS DevOps Agent can learn from and use for anomaly detection. Generate steady traffic to establish baseline metrics:

python traffic-generator.py --app all --duration 900 --rps 10 --error-rate 0.05

Command-line output report summarizing traffic generation test results for three applications. The Sample Metrics App achieved a 100% success rate across 13,365 requests, while both the Go OTEL App and Java OTEL App recorded 0% success rates with all 13,455 requests failing. The overall success rate was 90.49% at an average of 14.92 requests per second. A green checkmark confirms traffic generation completed for all applications, followed by next steps for monitoring with Prometheus, Grafana, and AWS DevOps Agent.What this test does: The command runs a 15-minute (900-second) steady traffic test across all applications at 10 requests per second, with a 5% simulated error rate. This low, consistent load represents typical production traffic and gives the agent enough signal to establish a reliable operational baseline.
Screenshot of the AWS DevOps Agent Root Cause analysis tab for the "Demo CPU Spike" incident. The investigation, completed on 2026-02-04 at 20:02:40, identifies one key finding: the HorizontalPodAutoscaler (HPA) for the go-otel-sample-app failed to retrieve the custom metric "go_app_requests_rate" from the custom metrics API 472 times between 16:19:44Z and 18:17:43Z, preventing automatic workload scaling during the incident.Figure 7: This is screenshot of investigation and root cause found in one of the cluster

What you should observe: During this scenario, the AWS DevOps Agent learns normal operational baselines. In the agent’s investigation dashboard, you will see the following being captured and recorded:

Typical request patterns and response times — The agent records average latency and throughput across all services, establishing what “healthy” looks like for your environment.
Normal error rates and distribution — With a 5% error rate, the agent learns the expected noise floor for errors, so it can distinguish genuine incidents from routine fluctuations.
Resource utilization patterns — CPU, memory, and network usage are tracked per pod and node, giving the agent a reference point for what normal resource consumption looks like under standard load.
Service dependency relationships — The agent maps how services communicate with each other, identifying upstream and downstream dependencies that will be critical for root cause analysis in future incidents.

After the test completes, you should see a stable metrics summary in the agent dashboard showing consistent throughput, low error variance, and steady resource utilization — confirming that a reliable baseline has been captured.

Expected outcomes:

During this scenario, the AWS DevOps Agent learns normal operational baselines. These baselines include typical request patterns and response times, normal error rates and distribution, resource utilization patterns, and service dependency relationships.

Scenario 2: Simulated production event

This scenario demonstrates the AWS DevOps Agent’s ability to investigate and analyze events with elevated error rates and performance degradation:

python traffic-generator.py --app java-otel --duration 600 --rps 30 --error-rate 0.25

Figure 8: This is screenshot of details of the Nodes and telemetry data which is relevant for the investigations

What this test does: The command targets the java-otel application specifically, running a 10-minute (600-second) high-load test at 30 requests per second — three times the baseline — with a 25% error rate. This simulates a degraded service experiencing both a traffic surge and a significant increase in failures.

Figure 9: This is screenshot of review and recommended next steps after the analysis of the root cause

What you should observe: Once the test begins, the AWS DevOps Agent detects the deviation from the established baseline and initiates an investigation. In the agent’s incident response view, you will see the following outcomes:

Affected application identification — The agent pinpoints java-otel-app as the impacted service, distinguishing it from other applications running normally in the cluster.
Error pattern analysis — The agent breaks down the 25% error rate into specific failure modes (for example, HTTP 500 errors, timeout spikes, or connection refusals), helping you understand not just that errors are occurring, but why and where.
Resource utilization correlation — The agent correlates CPU and memory spikes on the affected pods with the observed performance degradation, showing a clear relationship between resource exhaustion and increased error rates.
Root cause identification with confidence scoring — The agent presents a ranked list of potential root causes, each with a confidence score, so you can prioritize your investigation. For example, it may identify a memory leak or thread pool exhaustion as the most likely cause with high confidence.

Advanced analysis capabilities: Beyond the immediate incident, the agent performs deeper analysis that you can explore in the investigation timeline view:

Cross-service impact correlation — The agent identifies whether the degradation in java-otel-app has cascading effects on dependent services, showing you the full blast radius of the incident.
Timeline reconstruction — The agent reconstructs the sequence of events leading up to and during the incident, helping you understand how the situation evolved over time.
Dependency mapping — Upstream and downstream service dependencies are visualized, making it clear which services are affected directly and which are at risk.
Prioritized remediation recommendations — The agent provides actionable remediation steps ranked by business impact, so your team can address the most critical issues first. Recommendations may include scaling the affected deployment, adjusting resource limits, or rolling back a recent configuration change.

After the test completes, you should see a full incident report in the agent dashboard summarizing the root cause, affected components, timeline, and recommended next steps — giving your team everything needed to resolve the issue and prevent recurrence.

Expected outcomes:

The agent identifies which application is affected (java-otel-app), analyzes error patterns and rates for specific failure modes, correlates resource utilization with performance degradation, and provides potential root causes with confidence scoring.

Figure 10: This is screenshot of mitigation plan suggested by the AWS DevOps agent

Advanced analysis capabilities:

The agent performs analysis including cross-service impact correlation, timeline reconstruction of event progression, dependency mapping to identify upstream/downstream effects, and prioritized remediation recommendations based on business impact.

AWS DevOps Agent Investigation workflow

This section details how to use AWS DevOps Agent for event investigation and analysis.

Screenshot of the "Start an investigation" modal dialog in the AWS DevOps Agent Incident Response Dashboard. The dialog displays pre-filled investigation details describing an increase in application error rates, an investigation starting point noting an error spike for a workload tagged DemosFor: OTEL-DevOpsAgent-AiforOperations within the last 20 minutes, and an incident timestamp of 2026-02-04T21:30:00. Cancel and Start investigating buttons are shown at the bottom.Figure 11: This is screenshot of starting an investigation from the AWS Console inside AWS DevOps agent service

Starting an investigation

Access the AWS DevOps Agent web interface and initiate a new investigation:

  1. Investigation Trigger: Choose from predefined scenarios like “High CPU usage,” “Error rate spike,” or “Performance degradation”
  2. Time Range Selection: Select the time period when you generated traffic or observed issues.
  3. Scope Definition: Provide the AWS Account ID, AWS Region (us-east-1), and specific cluster or application context.
  4. Data Source Configuration: Make sure all observability data sources are properly connected.

Investigation Process

AWS DevOps Agent follows a systematic investigation methodology:

Data Collection Phase:

Screenshot of the AWS DevOps Agent Incident Response interface showing an active investigation for the "Demo Error Spike" incident created on 2026-02-04 at 22:57:20. The investigation timeline on the left shows sequential updates including a user request, assistant responses, and a planning phase. A chat assistant panel on the right displays a welcome message and an input field for asking questions about the investigation.Figure 12: This is screenshot of investigation timeline done by the AWS DevOps agent

This approach correlates metrics from Amazon Managed Prometheus workspace, and analyzes logs from Amazon CloudWatch Logs for error patterns and anomalies. It also reviews distributed traces from AWS X-Ray for service dependencies, and examines application topology and service relationships to provide comprehensive observability during the migration process.

Analysis phase:

Screenshot of the AWS DevOps Agent Incident Response interface during the "Demo Error Spike" investigation. The investigation timeline shows an Update step and a Fetching data step. An assistant response identifies relevant AWS resources in us-east-1 including the dev-eks-automode EKS cluster, and displays a CloudWatch describe_alarm_history API call targeting the incident window from 2026-02-04T21:00:00Z to 21:57:31Z. A chat assistant panel is visible on the right.Figure 13: This is screenshot of analysis done from the AWS Console inside AWS DevOps agent service

This approach identifies patterns and anomalies using MLalgorithms. It correlates events across multiple data sources for comprehensive understanding, applies statistical analysis to determine the significance of observed changes, and compares current behavior against established baselines for accurate detection and assessment of system behavior.

Root cause identification:

Figure 14: This is screenshot of root cause and investigation summary done from the AWS Console inside AWS DevOps agent service

This approach provides systematic root cause analysis with confidence scoring, identifies contributing factors and potential trigger events, maps event timeline with correlated evidence from multiple sources, and suggests most likely causes based on data correlation and pattern analysis to enable efficient troubleshooting and resolution.

Mitigation strategy:

Figure 15: This is screenshot of mitigation summary recommended by the AWS DevOps agent

This approach recommends immediate mitigation actions to resolve current issues. It also suggests long-term prevention strategies to avoid recurrence, provides runbook-style guidance for event response teams, and integrates with existing DevOps workflows and tools for seamless incident response and improvement.

Key features and benefits

Preventing future incidents

AWS DevOps Agent analyzes patterns across your incident investigations to deliver targeted recommendations that continuously improve your operational posture and prevent future incidents.

Screenshot of the AWS DevOps Agent Prevention tab showing a newly started weekly evaluation run for the OTEL-DevOpsAgent-Demo-v1 project. The evaluation has used 3 minutes of a 15-hour budget. The agent summary states no new recommendations were generated for the past week. A recommendation frequency breakdown shows zero items across all four categories: Code optimization, Observability, Infrastructure, and Governance.Figure 16: This is screenshot of prevention tab in the AWS Console for the AWS DevOps agent

DevOps Agent topology

AWS DevOps Agent Topology automatically discovers and maps your entire infrastructure into an interactive, living blueprint. It reveals not only what resources exist, but how they interconnect, depend on each other, and drive system behavior.

Screenshot of the AWS DevOps Agent Topology graph view for Agent Space 442bc2d6-616a-49dd-b13f-e8a43fab0450, displaying 1,806 total discovered resources. The graph is filtered to show Container resources and visualizes three interconnected sections: a left section with OTEL-DevOpsAgent-Demo-v1 ECS clusters and network components, a middle section with StackSets and multiple standalone resource groups, and a right section with additional standalone resources, connected by relationship lines.Figure 17: This is screenshot of topology of Amazon EKS cluster discovered by AWS DevOps agent

Clean up

To avoid ongoing charges, delete the resources that you created while following this walkthrough.

Remove AWS DevOps Agent resources:

  • Delete investigation data and Agent Space configuration through the AWS Console.
  • Remove IAM roles and policies created specifically for the DevOps Agent.
  • Delete any CloudFormation stacks created during deployment.

Conclusion

As organizations continue to embrace cloud-native architectures and DevOps practices, tools like AWS DevOps Agent will become essential for maintaining competitive advantage in an increasingly complex technological landscape.

Ready to add AI powered observability to your container infrastructure? Visit the AWS documentation to access implementation guides, or reach out to your AWS account team to discuss how this automated migration approach can accelerate your cloud modernization journey, while reducing operational overhead.

About the authors

Best practices for Amazon Redshift Lambda User-Defined Functions

Post Syndicated from Sergey Konoplev original https://aws.amazon.com/blogs/big-data/best-practices-for-amazon-redshift-lambda-user-defined-functions/

While working with Lambda User-Defined Functions (UDFs) in Amazon Redshift, knowing best practices may help you streamline the respective feature development and reduce common performance bottlenecks and unnecessary costs.

You wonder what programming language could improve your UDF performance, how else can you use batch processing benefits, what concurrency management considerations might be applicable in your case? In this post, we answer these and other questions by providing a consolidated view of practices to improve your Lambda UDF efficiency. We explain how to choose a programming language, use existing libraries effectively, minimize payload sizes, manage return data, and batch processing. We discuss scalability and concurrency considerations at both the account and per-function levels. Finally, we examine the benefits and nuances of using external services with your Lambda UDFs.

Background

Amazon Redshift is a fast, petabyte-scale cloud data warehouse service that makes it simple and cost-effective to analyze data using standard SQL and existing business intelligence tools.

AWS Lambda is a compute service that lets you run code without provisioning or managing servers, supporting a wide variety of programming languages, automatically scaling your applications.

Amazon Redshift Lambda UDFs allows you to run Lambda functions directly from SQL, which unlock such capabilities like external API integration, unified code deployment, better compute scalability, cost separation.

Prerequisites

  • AWS account setup requirements
  • Basic Lambda function creation knowledge
  • Amazon Redshift cluster access and UDF permissions.

Performance optimization best practices

The following diagram contains necessary visual references from the best practices description.

Use efficient programming languages

You can choose from Lambda’s wide variety of runtime environments and programming languages. This choice affects both the performance and billing. More performant code may help reduce the cost of Lambda compute and improve SQL query speed. Faster SQL queries could also help reduce costs for Redshift Serverless and potentially improve throughput for Provisioned clusters depending on your specific workload and configuration.

When choosing a programming language for your Lambda UDFs, benchmarks may help predict performance and cost implications. The famous Debian’s Benchmarks Game Team provides publicly available insights for different languages in their micro-benchmark results. For example, their Python vs Golang comparison shows up to 2 orders of magnitude run time improvement and twice memory consumption reduction if you could use Golang instead of Python. That may positively reflect on both Lambda UDF performance and Lambda costs for the respective scenarios.

Use existing libraries efficiently

For every language provided by Lambda, you can explore the whole collection of libraries to help you implement tasks better from the speed and resource consumption point of view. When transitioning to Lambda UDFs, review this aspect carefully.

For instance, if your Python function manipulates datasets, it might be worth considering using the Pandas library.

Avoid unnecessary data in payloads

Lambda limits request and response payload size to 6 MB for synchronous invocations. Considering that, Redshift is doing best effort to batch the values so that the number of batches (and hence the Lambda calls) would be minimal which reduces the communication overhead. So, the unnecessary data, like one added for future use but not immediately actionable, may reduce efficiency of this effort.

Keep in mind returning data size

Because, from the point of view of Redshift, each Lambda function is a closed system, it is impossible to know what size the returned data can possibly be before executing the function. In this case, if the returned payload is higher than the Lambda payload limit, Redshift will have to retry with the outbound batch of a lower size. That will continue until a fit return payload will be achieved. While it is the best effort, the process might bring a notable overhead.

In order to avoid this overhead, you might use the knowledge of your Lambda code, to directly set the maximum batch size on the Redshift side using the MAX_BATCH_SIZE clause in your Lambda UDF definition.

Use benefits of processing values in batches

Batched calls provide new optimization opportunities to your UDFs. Having a batch of many values passed to the function at once, allows to use various optimization techniques.

For example, memoization (result caching), when your function can avoid running the same logic on the same values, hence reducing the total execution time. The standard Python library functools provides convenient caching and Least Recently Used (LRU) caching decorators implementing exactly that.

Scalability and concurrency management

Increase the account-level concurrency

Redshift uses advanced congestion control to provide the best performance in a highly competitive environment. Lambda provides a default concurrency limit of 1,000 concurrent execution per AWS Region for an account. However, if the latter is not enough, you can always request the account level quota increase for Lambda concurrency, which might be as high as tens of thousands.

Note that even with a restricted concurrency space, our Lambda UDF implementation will do the best effort to minimize the congestion and equalize the chances for function calls across Redshift clusters in your account.

Restrict function concurrency with reserved concurrency

If you want to isolate some of the Lambda functions in a restricted concurrency scope, for example you have a data science team experimenting with embedding generation using Lambda UDFs and you don’t want them to affect your account’s Lambda concurrency much, you might want to set a reserved concurrency for their specific functions to operate with.

Learn more about reserved concurrency in Lambda.

Integration and external services

Call existing external services for optimal execution

In some cases, it might be worth considering using existing external services or components of your application instead of re-implementing the same tasks yourself in the Lambda code. For example, you can use Open Policy Agent (OPA) for policy checking, a managed service Protegrity to protect your sensitive data, there are also a variety of services providing hardware acceleration for computationally heavy tasks.

Note that some services have their own batching control with a limited batch size. For that we implemented a per-function batch row count setting MAX_BATCH_ROWS as a clause in the Lambda UDF definition.

To learn more on the external service interaction using Lambda UDFs refer the following links:

Conclusion

Lambda UDFs provide a way to extend your data warehouse capabilities. By implementing the best practices from this post, you may help optimize your Lambda UDFs for performance and cost efficiency.The key takeaways from this post are:

  • performance optimization, showing how to choose efficient programming languages and tools, minimize payload sizes, and leverage batch processing to reduce execution time and costs
  • scalability management, showing how to configure appropriate concurrency settings at both account and function levels to handle varying workloads effectively
  • integration efficiency, explaining how to benefit from external services to avoid reinventing functionality while maintaining optimal performance.

For more information, visit the Redshift documentation and explore the integration examples referenced in this post.

About the author

Sergey Konoplev

Sergey Konoplev

Sergey is a Senior Database Engineer on the Amazon Redshift team who is driving a range of initiatives from operations to observability to AI-tooling, including pushing the boundaries of Lambda UDF. Outside of work, Sergey catches waves in Pacific Ocean and enjoys reading aloud (and voice acting) for his daughter.

How Vanguard transformed analytics with Amazon Redshift multi-warehouse architecture

Post Syndicated from Alex Rabinovich original https://aws.amazon.com/blogs/big-data/how-vanguard-transformed-analytics-with-amazon-redshift-multi-warehouse-architecture/

This is a guest post by Alex Rabinovich, Anindya Dasgupta, and Vijesh Chandran from Vanguard, Financial Advisor Services division, in partnership with AWS.

Vanguard stands as one of the world’s leading investment companies, serving more than 50 million investors globally. The company offers an extensive selection of low-cost mutual funds and ETFs with over 450 funds/ETFs along with comprehensive investment advice and related financial services. With a workforce of approximately 20,000 crew members, Vanguard has built its reputation on providing low-cost, high-quality investment solutions that help investors achieve their long-term financial goals.

Within this massive organization, Vanguard’s Financial Advisor Services (FAS) division stands as one of the most prominent B2B operations in the financial services industry. Operating at an extraordinary scale, FAS oversees a broad range and diverse range of assets through the intermediary channel while supporting a vast network of advisory firms and financial advisors across the country. This division delivers a full suite of investment products, model portfolios, research capabilities, and technology-driven support services designed to help financial advisors serve their clients more effectively.

Business use cases and initial architecture

The scale and complexity of FAS operations generate enormous amounts of data that require sophisticated analytics capabilities to drive business insights, regulatory compliance, and operational efficiency. To address this, Vanguard launched the FAS 360 initiative. This initiative aims to empower Financial Advisor Services (FAS) with a centralized cloud data warehouse that integrates both internal and external data sources into a unified, intelligent system.

Key business use cases:

  1. Business operations – Enables sales goal setting, tracking, and compensation management to drive operational excellence. It delivers insights on product usage patterns across financial advisor clients.
  2. Data science – Powers customer segmentation models and call transcription analytics to drive strategic insights. It also supports marketing campaign preparation and customer insights for sales call preparation.
  3. Exploratory analytics – Enables ad-hoc leadership questions, what-if scenario analysis, and sales trend analysis for channel managers competitor comparative analysis.

By consolidating these use cases into a centralized system, FAS 360 enables consistent reporting and data-driven decision-making across Vanguard’s Financial Advisor Services division.

Centralized data warehouse FAS 360:

Vanguard’s first wave of modernization established FAS 360 as a centralized enterprise data warehouse, migrating from a fragmented “data swamp” of Parquet files on Amazon Simple Storage Service (Amazon S3) to a structured, unified system.

The following architecture diagram leverages Amazon S3 for raw data storage with Amazon Redshift serving as the core processing engine, providing integrated access for BI tools, analyst exploration, and data science workloads.

Here are the key benefits achieved with this architecture:

  • Single source of truth – Consolidated fragmented data sources into a unified system, minimizing multiple versions of truth and establishing consistent reporting practices across the organization
  • 10x faster query performance – Dramatically improved query response times compared to the previous solution, helping enhance analyst productivity and enabling more complex analytical workloads
  • Seamless data lake integration – Maintained connectivity with the broader data lake environment while providing structured warehouse capabilities
  • Enhanced business agility – Increased trust in metrics and unlocked new use cases that were previously untenable, directing the new migration efforts toward the FAS360 system

This centralized architecture successfully addressed the limitations of Vanguard’s previous approach, where data was scattered across individuals with limited governance, and established a foundation for their subsequent architectural evolution.

Significant growth and expanding use cases

Vanguard FAS experienced remarkable growth in their data analytics requirements over a two-year period, demonstrating the rapid evolution of modern data needs:

Initial State:

  • 20 AWS Glue ETL jobs processing daily data loads
  • Approximately 100 tables in their data warehouse
  • 20 Tableau dashboards serving business users
  • Around 60 analysts accessing the system

Two Years Later:

  • 20 TB in data volume in Amazon Redshift and another 150 TB in S3 data lake
  • 600+ AWS Glue ETL jobs (a 30x increase) handling complex data transformations
  • 300+ tables (3x growth) storing diverse business data
  • 250+ Amazon Redshift materialized views optimizing query performance
  • Over 500 Tableau dashboards (25x expansion) serving various business functions
  • 500,000+ user queries/months

This exponential growth reflected FAS’s increasing reliance on data-driven decision making across the business functions, from risk management and compliance to client service optimization and operational efficiency improvements.

Resource contention and performance bottlenecks

As Vanguard FAS’s data environment expanded, their initial architecture, a single Amazon Redshift provisioned cluster with 2 nodes (ra3.4xlarge), began experiencing severe performance challenges that threatened business operations:

ETL performance issues:

  • Frequent ETL SLA failures disrupting critical business processes
  • Tableau extract failures resulting in stale dashboard data
  • Resource conflicts between data ingestion and transformation workloads

End-user experience degradation:

  • Poor query performance during peak usage periods
  • Table and object locking issues preventing concurrent access
  • Frustrated analysts unable to perform deep data exploration
  • Limited ability to run long-running analytical queries

Operational challenges:

  • Resource contention between ETL workloads and interactive analytics
  • Inability to scale compute resources independently for different workload types
  • Single point of failure affecting the data operations
  • Difficulty in workload prioritization and resource allocation

These challenges were fundamentally limiting FAS’s ability to leverage their data assets effectively, impacting everything from daily operational reporting to strategic business analysis.

Solution overview

To address these critical challenges, Vanguard FAS implemented following multi-warehouse architecture that leverages the advanced data sharing capabilities of Amazon Redshift for workload isolation and independent scaling.

Producer – Amazon Redshift Provisioned Cluster

The central hub consists of the original Amazon Redshift provisioned cluster with RA3 nodes, optimized for consistent, predictable workloads:

  • Dedicated ETL processing: Handles data ingestion, transformation, and loading operations
  • Write workload optimization: Manages data writes and updates without interference
  • Cost optimization: Utilizes reserved instances for predictable, steady-state workloads
  • Data governance: Serves as the single source of truth for the enterprise data

Consumer – Amazon Redshift Serverless Workgroups

Multiple Amazon Redshift Serverless instances serve as specialized consumer endpoints which auto-scales compute resources based on demand:

  • Analyst Exploration: Dedicated environment for analyst data discovery and experimentation
  • BI Tools: Instance optimized specifically for Tableau dashboard and visualization workloads
  • Data Science: For complex and long running machine learning workloads in completely isolated environment

The solution leverages the native data sharing capabilities of Amazon Redshift to enable secure connectivity between the producer and consumers instances. Consumer clusters can access live data from the producer without data movement, providing real-time access to the most current information available. This zero-copy sharing approach alleviates the need for data duplication or complex synchronization processes, helping reduce both storage costs and operational complexity.

Results

The implementation of the multi-warehouse architecture delivered significant improvements across the key performance indicators:

Predictable Performance

Nightly ETL cycles now consistently complete before the 9 AM SLA, eliminating the previous SLA failures that disrupted business operations and ensuring fresh data is available for morning business activities. Dashboards and reports now reflect the most current data available, providing teams with up-to-date insights for decision-making.

Improved Analyst Productivity and Experience

The new architecture removed the restrictive 10-minute query timeout that previously prevented deep ad hoc exploratory queries. Analysts can now run complex analytical workloads exceeding 30 minutes in a fully isolated environment without impacting other users or ETL processes. This change, combined with significantly faster query response times, has led to higher analyst satisfaction and productivity across the team.

New Analytical Capabilities

The architecture introduced a dedicated “Data Lab” environment where analysts have write access to experiment with data using CREATE TABLE AS SELECT (CTAS) commands. Each workload type can now scale independently based on demand, with different consumer clusters optimized for specific use cases, enabling more sophisticated analytical approaches.

Operational Excellence

The separation of workloads enabled efficient utilization of compute resources across different patterns, leading to better cost control through appropriate sizing, serverless pay-as-you-go pricing, and reserved instance usage. The cleaner separation of concerns between ETL and analytics workloads has simplified overall management of the data platform.

Ongoing modernization: Evolution toward data mesh architecture

As Vanguard’s data environment matured and their success with the multi-warehouse architecture enabled broader adoption across the organization, they recognized an opportunity to evolve their architecture to match their organizational growth. The expanding portfolio of data products and increasing number of teams leveraging the system created new opportunities for innovation.

As Vanguard’s data environment grew, three key challenges emerged:

  1. Centralized ownership bottleneck – Single-team data ownership couldn’t scale with the growing number of data products
  2. Write workload contention – Resource contention persisted for write operations on shared endpoints
  3. Cross-domain dependencies – Data object interdependencies across business domains slowed data product development

Rationale for Data Mesh

Vanguard’s decision to adopt Data Mesh was driven by the need to:

  • Decentralize data ownership by establishing data domains with dedicated stewards
  • Remove write contention by isolating each domain’s data loads to separate endpoints
  • Enable autonomous development allowing stewards to own the complete data product lifecycle and governance
  • Leverage modern data lake capabilities using AWS Glue and Apache Iceberg format for data product curation

This evolution supports Vanguard’s ability to scale organizationally while building on the technical foundation and operational excellence achieved with their multi-warehouse architecture. Building on the success of their Amazon Redshift multi-warehouse implementation, Vanguard FAS is now exploring on the next phase of their data architecture evolution, implementing following data mesh approach.

This new data mesh architecture has several key components that work together to enable scalable, domain-oriented data management.

Domain-Oriented Data Ownership

Vanguard is establishing distinct data domains aligned with business functions and assigning dedicated data stewards to each domain for clear ownership and accountability. This strategy shifts from centralized data management to a decentralized model where data ownership and responsibility can be distributed across business domains, enabling teams closer to the data to make informed decisions about their domain-specific needs.

Distributed Data Architecture

The new architecture isolates domain-specific data loads to separate compute endpoints and creates independent data processing pipelines for each domain. This approach helps reduce cross-domain dependencies and conflicts that previously slowed development cycles, allowing teams to iterate and deploy changes without waiting for coordination across the entire organization.

Data Product Approach

Vanguard is curating data products on the data lake using Apache Iceberg format and leveraging AWS Glue for metrics computation and data lake integration. This approach treats data as products with defined SLAs and quality metrics, helping facilitate reliable, high-quality data delivery that downstream consumers can depend on with confidence.

Self-Service Analytics

The implementation enables domain teams to manage their complete data product lifecycle independently while maintaining enterprise governance standards. Vanguard provides comprehensive tools and systems for independent data management, allowing teams to innovate quickly without compromising data quality or security, ultimately accelerating time-to-insight across the organization.This evolution represents a natural progression from centralized data warehouse to multi-warehouse architecture, and finally to a fully distributed, domain-oriented data mesh that can scale with Vanguard’s continued growth.

Conclusion

Vanguard Financial Advisor Services’ journey demonstrates that scaling analytics is no longer about scaling a single warehouse bigger, but about architecting for workload isolation, independent scaling, and organizational growth.

By evolving from a single 2-node RA3 provisioned cluster to a multi-warehouse architecture using Amazon Redshift Serverless and Provisioned, Vanguard achieved measurable, production-grade outcomes:

  • 500,000+ monthly queries supported without ETL or dashboard contention
  • 100% ETL SLA adherence, with nightly pipelines completing before 9 AM
  • 25x growth in BI consumption (20 → 500+ Tableau dashboards) without performance degradation
  • 8x growth in analyst population (60 → 500+) enabled through workload isolation
  • 30x increase in ETL pipelines (20 → 600+) without re-architecting ingestion logic
  • Zero-copy Amazon Redshift data sharing across producer and consumer warehouses, minimizing data duplication and synchronization costs
  • Removal of 10-minute query limits, unlocking advanced exploratory and long-running analytics

Critically, these gains were not achieved by over-provisioning compute, but by right-sizing and specializing compute per workload, reserving capacity where demand was predictable (ETL) and using Amazon Redshift Serverless auto-scaling where demand was bursty (BI and ad-hoc analysis).

As Vanguard now progresses toward a domain-oriented data mesh, their experience reinforces a key lesson: Multi-warehouse architecture is a foundational enabler for organizational scale, data product ownership, and autonomous analytics.For organizations experiencing exciting growth in their data analytics requirements, Vanguard’s approach showcases the tremendous possibilities that await. With the right architecture and the help of AWS services, organizations can transform their data infrastructure to achieve remarkable improvements in performance, significant cost reductions, and unlock powerful new analytical capabilities that accelerate business value creation.

AWS encourages you to connect with your AWS Account Team to engage an AWS analytics specialist who can provide expert architectural guidance and tailored recommendations to help you achieve your data transformation goals.

© 2026 The Vanguard Group, Inc. and Amazon Web Services, Inc. All rights reserved. This material is provided for informational purposes only and is not intended to be investment advice or a recommendation to take any particular investment action.

About the authors

Alex Rabinovich

Alex Rabinovich

Alex is a Director of Data Engineering at Vanguard, aligned to Financial Advisory Services division. In this role, he leads large‑scale data engineering platforms and modernization initiatives, focusing on building reliable, scalable, and high‑performance data systems in the AWS cloud.

Anindya Dasgupta

Anindya Dasgupta

Anindya is a solutions architect in Vanguard’s Financial Advisor Services Technology division. He has over 25 years of experience building enterprise technology solutions to address complex business challenges. His work focuses on architecting and designing scalable, cloud‑native and data‑driven systems, with hands‑on contributions across application development, system integration, and proof‑of‑concept initiatives.

Vijesh Chandran

Vijesh Chandran

Vijesh is Head of Solution Design, overseeing the architecture and design of enterprise technology solutions that support critical business outcomes. His background spans data architecture on cloud‑native platforms, and data‑driven systems, with a strong focus on aligning technology design to business strategy. He plays a hands‑on role in guiding solution direction, integration patterns, and proof‑of‑concept initiatives.

Raks Khare

Raks Khare

Raks is a Senior Analytics Specialist Solutions Architect at AWS based out of Pennsylvania. He helps customers across varying industries and regions architect data analytics solutions at scale on the AWS platform. Outside of work, he likes exploring new travel and food destinations and spending quality time with his family.

Poulomi Dasgupta

Poulomi Dasgupta

Poulomi is a Senior Analytics Solutions Architect with AWS. She is passionate about helping customers build cloud-based analytics solutions to solve their business problems. Outside of work, she likes travelling and spending time with her family.

Scale fine-grained permissions across warehouses with Amazon Redshift and AWS IAM Identity Center

Post Syndicated from Raghu Kuppala original https://aws.amazon.com/blogs/big-data/scale-fine-grained-permissions-across-warehouses-with-amazon-redshift-and-aws-iam-identity-center/

Amazon Redshift is a fully managed, petabyte-scale cloud-based data warehouse that you can use to scale analytics workloads effortlessly. As organizations expand their analytics capabilities across multiple business units, they need streamlined approaches for defining and managing fine-grained permissions for each warehouse. Many organizations use external identity providers (IdPs) like Microsoft Entra ID, Okta, or Ping to manage workforce identities centrally and need streamlined data warehouse integration with consistent access controls. We address these challenges by introducing Amazon Redshift federated permissions with AWS IAM Identity Center integration so that you can define security policies once and automatically enforce them across the warehouses in your account.

Amazon Redshift federated permissions are now supported with IAM Identity Center across multiple AWS Regions, where you can use identities from supported identity provider (IdP) such as Microsoft Entra ID, Okta, Ping Identity, or OneLogin across supported AWS Regions with IAM Identity Center. This enables you to align with business requirements including resiliency and proximity to users. You can now extend IAM Identity Center from your primary AWS Region to additional Regions of your choice based on your data residency requirements. In that region, you can get horizontal multi-warehouse scalability by adding new warehouses using Amazon Redshift federated permissions across multiple warehouses. With Redshift federated permissions, you define data permissions once from any Redshift warehouse in that region and automatically enforce them across all warehouses in the account in that region.

This post provides a comprehensive technical walkthrough for implementing Amazon Redshift federated permissions with AWS IAM Identity Center to help achieve scalable data governance across multiple data warehouses. It demonstrates a practical architecture where an Enterprise Data Warehouse (EDW) serves as the producer data warehouse with centralized policy definitions, helping automatically enforce security policies to consuming Sales and Marketing data warehouses without manual reconfiguration. You will learn how to do the following:

  • Configure IAM Identity Center connections for both data sharing producers and consumers
  • Register Amazon Redshift serverless namespaces with AWS Glue Data Catalog
  • Set up trusted identity propagation (TIP)
  • Create and attach Dynamic data masking policies to help protect personally identifiable information (PII) like customer dates of birth
  • Implement row-level security policies to control data visibility based on user roles
  • Map IdP groups to Amazon Redshift database roles for seamless access management

Prerequisites

Before you begin, verify that you have the following:

  • An AWS account with admin role privileges
  • Assign data lake admin permissions to above admin role. For instructions, see Create a data lake administrator
  • Enable IAM Identity Center integration using the Lake Formation
  • Review the blog post to understand the setup process of AWS IAM Identity Center integration with Amazon Redshift Query Editor v2
  • IAM Identity Center enabled in your AWS account, with users and groups created as listed under Solution overview section of User access (figure 2)
  • As an Amazon Redshift superuser, grant CONNECT, CREATE TABLE, INSERT, SELECT, and sys:secadmin permissions to AWSIDC:awssso-admin database role
  • An IAM role for IAM Identity Center access:
    • Step 1:Create an IAM policy for Amazon Redshift access. To integrate Amazon Redshift with IAM Identity Center, create an IAM policy (for example, aws-idc-policy) in the account where your Amazon Redshift data warehouse exists: 
      {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "redshift:DescribeQev2IdcApplications",
        "redshift-serverless:ListNamespaces",
        "redshift-serverless:ListWorkgroups",
        "redshift-serverless:GetWorkgroup"
      ],
      "Resource": [
        "arn:aws:redshift-serverless:<AWS Region>:<AWS Account ID>:workgroup/*",
        "arn:aws:redshift-serverless:<AWS Region>:<AWS Account ID>:namespace/*"
      ]
    },
    {
      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": [
        "sso:DescribeApplication",
        "sso:DescribeInstance"
      ],
      "Resource": [
        "arn:aws:sso:::instance/<IAM Identity Center Instance ID>",
        "arn:aws:sso::<AWS Account ID>:application/<IAM Identity Center Instance ID>/*"
      ]
    }
  ]
}

    • Step 2: Create the IAM role. Create an IAM role (Amazon Redshift – Customizable) in the account where your Amazon Redshift data warehouse exists (for example, IAMIDCRedshiftRole).
    • Step 3: Attach IAM policies to the role. Attach the following two IAM policies to the previously mentioned role:
    • Step 4: Update the trust relationships. Update the trust relationships for this role with the following: 
      {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "redshift.amazonaws.com"
      },
      "Action": [
        "sts:AssumeRole",
        "sts:SetContext"
      ]
    }
  ]
}

      Note: AmazonRedshiftFederatedAuthorization is a managed policy that provides the necessary permissions for running queries with Amazon Redshift federated authorization.

  • Attach above IAMIDCRedshiftRole IAM role to all Redshift serverless endpoints

Solution overview

The following architecture diagram demonstrates federated permissions in a multi-warehouse environment, enabling scalable data governance across Amazon Redshift warehouses by automatically enforcing security policies.

Figure 1 : Sample architecture diagram

Figure 1: Sample architecture diagram

User access

Users can access data warehouses through Amazon Redshift Query Editor v2, third-party SQL editors (such as DBeaver and SQL Workbench), or custom client applications. The access methods help provide consistent security enforcement.

Figure 2: Solution overview flow

Figure 2: Solution overview flow

AWS IAM Identity Center integration

IAM Identity Center provides centralized authentication with single sign-on capabilities and automatically assigns role-based permissions based on organizational roles. This identity federation links corporate identities directly to AWS resources, making sure that authentication occurs at the identity layer before warehouse access.

Multi-warehouse architecture

This architecture uses three distinct data warehouses that serve different business functions while sharing centralized security policies.

Enterprise Data Warehouse (EDW)

The EDW serves as the central repository for enterprise data. In this architecture, customer and product data are stored in the Customer Profile Database (CPD), where administrators define two critical security policies:

  • Dynamic data masking (DDM) – Masks sensitive customer Date of Birth (DOB) fields for both Sales Analyst and Marketing Analyst roles, helping protect personally identifiable information (PII) while allowing analytical work
  • Row-level security (RLS) – Controls product visibility based on user roles. Sales Analysts view only launched products, while Marketing Analysts view both launched and planned products

The EDW registers with the AWS Glue Data Catalog, creating a unified metadata repository that makes data discoverable across the warehouses in the account. This registration establishes the foundation for federated permissions, enabling automatic policy propagation.

Sales data warehouse

When Sales Analysts query customer and product tables, the system automatically enforces policies defined in the EDW through federated permissions. The registered namespace from the EDW automatically mounts as an external database, alleviating the need to recreate or reattach policies. Customer DOB fields appear masked, and only launched products are visible without additional configurations.

Marketing data warehouse

The Marketing Data Warehouse automatically inherits and enforces EDW security policies. Customer DOB fields remain masked to help protect PII, but with RLS policies, Marketing Analysts can view both launched and planned products. This provides the broader visibility needed for marketing planning. This differentiated access control is automatically enforced based on user roles.

Walkthrough

In this walkthrough, you create two Amazon Redshift IAM Identity Center (IDC) connections:

  1. Data sharing producer identity center connection – Assigned to the edw-wg Amazon Redshift serverless workgroup
  2. Data sharing consumer identity center connection – Assigned to the cpd-sales-wg and cpd-marketing-wg Amazon Redshift serverless workgroups

Set up IDC connections for Amazon Redshift federated permissions

In this section, you configure the IAM Identity Center connections that enable federated authentication across your warehouses. You will create separate connections for the producer (policy-defining) warehouse and consumer warehouses.

Configure Amazon Redshift data sharing producer IDC connection

To create the producer IDC connection:

  1. Open the Amazon Redshift Serverless console.
  2. Choose IAM Identity Center connections by expanding the hamburger menu.
  3. Choose Create application.
  4. Verify that you see “Amazon Redshift connected to IAM Identity Center”, and then choose Next.
  5. Configure the connection properties:
    • For IAM Identity Center display name, enter a name.
    • For Managed application name, enter rs-multicluster-producer.
    • For Identity provider namespace, choose AWSIDC.
    • For IAM role for IAM Identity Center access, choose the TIP IAM role that you created.
    • For Query editor v2 application, choose Enable the query editor v2 application.
    • For IAM Identity Center application type, choose Configure Amazon Redshift federated permissions using AWS IAM Identity Center (Recommended).
    • Choose Next.
  6. For Configure client connections that use third-party IdPs, choose No.
  7. Choose Next.
  8. Verify that the configuration details match your inputs and then choose Create Application.
Figure 3: Data sharing producer IDC connection

Figure 3: Data sharing producer IDC connection

Configure data sharing consumer IDC connection

To create the consumer IDC connection:

  1. Open the Amazon Redshift Serverless console.
  2. Choose IAM Identity Center connections by expanding the hamburger menu.
  3. Choose Create application.
  4. Verify that you see “Amazon Redshift connected to IAM Identity Center”, and then choose Next.
  5. Configure the connection properties:
    • For IAM Identity Center display name, enter a name.
    • For Managed application name, enter rs-multicluster-consumer.
    • For Identity provider namespace, choose AWSIDC.
    • For IAM role for IAM Identity Center access, choose the TIP IAM role that you created.
    • For Query editor v2 application, you will see the notification “You already have a query editor v2 application.”
    • For IAM Identity Center application type, deselect Configure Amazon Redshift federated permissions using AWS IAM Identity Center (Recommended).
    • For Trusted identity propagation, choose AWS Lake Formation access grants and Amazon Redshift Connect.
    • Choose Next.
  6. For Configure client connections that use third-party IdPs, choose No.
  7. Choose Next.
  8. Verify that the configuration details match your inputs, and then choose Create Application.
  9. Add your required users or groups to the IDC application for Amazon Redshift data sharing consumers.
Figure 4: Data sharing consumer IDC connection

Figure 4: Data sharing consumer IDC connection

Configure Amazon Redshift data sharing producer IDC connection for Amazon Redshift serverless namespace

To register the edw-ns namespace with federated permissions:

  1. Open the Amazon Redshift Serverless Namespace console.
  2. Choose your Amazon Redshift Serverless namespace.
  3. Choose Actions, and then select Register with AWS Glue Data Catalog.
  4. Choose Register with Amazon Redshift federated permissions.
  5. Choose Amazon Redshift federated permissions using AWS IAM Identity Center.
  6. Choose Register.
Figure 5: Amazon Redshift data warehouse registration with Glue Data Catalog

Figure 5: Amazon Redshift data warehouse registration with Glue Data Catalog

Figure 6: Amazon Redshift data warehouse registration with Glue Data Catalog

Figure 6: Amazon Redshift data warehouse registration with Glue Data Catalog

Note: IAM Identity Center managed application ARN Data sharing producer IDC connection created would be used.

Configure Amazon Redshift data sharing consumer IDC connection for existing serverless namespace

For cpd-sales-wg and cpd-marketing-wg serverless workgroups, gather the following information from your registered IAM Identity Center connection:

  • IAM Identity Center display name
  • Identity provider namespace
  • IAM Identity Center managed application ARN
  • IAM role for IAM Identity Center access

Run the following SQL command as a database administrator to enable the integration:

CREATE IDENTITY PROVIDER "<IAM Identity Center display name>" TYPE AWSIDC
NAMESPACE '<Identity provider namespace>'
APPLICATION_ARN '<IAM Identity Center managed application ARN>'
IAM_ROLE '<IAM role for IAM Identity Center access>';

To modify an existing identity provider, use the ALTER IDENTITY PROVIDER command:

ALTER IDENTITY PROVIDER "<IAM Identity Center display name>"
NAMESPACE '<Identity provider namespace>';
ALTER IDENTITY PROVIDER "<IAM Identity Center display name>"
IAM_ROLE default | '<IAM role for IAM Identity Center access>';

Data preparation and access setup from producer

In this section, you create the customer and product tables, load sample data, create DDM and RLS policies, attach the policies to database roles and grant SELECT permissions to the roles.

Prepare data on EDW

Connect to the EDW data warehouse as an IDC Admin user and run the following SQL commands.

Create the product table:

CREATE TABLE product (
  product_id VARCHAR(16) NOT NULL,
  product_desc VARCHAR(200),
  current_price NUMERIC(7,2),
  wholesale_cost NUMERIC(7,2),
  category_desc VARCHAR(50),
  launch_status VARCHAR(50)
);

Insert sample product data:

INSERT INTO product 
VALUES 
  ('AAAAAAAAAFNPEAAA','At least concerned authors adopt just brown, federal',7.12,4.12,'Jewelry','launched'),
  ('AAAAAAAAOAAGDAAA','Complex services may not find totally changing accountants. Tiny, available ministers could not know always systems. Hot, male speakers discer',8.08,5.49,'Shoes','planned'),
  ('AAAAAAAAMJJMCAAA','Rows could prevent political, old duties. Just international stairs would regret police. Conditions discard always interesting, warm years. Present jobs shall take nearby relatively dreadful',8.18,5.31,'Jewelry','launched'),
  ('AAAAAAAAKLBLBAAA','Suddenly external sentences believe then by the assets. Simultaneously young feet could not probe separately shortly new men. Forms work again individuals. Images',17.96,7.9,'Shoes','launched'),
  ('AAAAAAAAMBKMCAAA','Clubs see finally materials. Significant objectives sell fairly left, civil power',3.18,3.84,'Books','launched'),
  ('AAAAAAAACPCAAAAA','Perhaps past preferences tell rather to a accounts. Very common feet can command never available final years; minutes expect recent, due employers. Altogether english shoes',9.84,0.19,'Electronics','planned'),
  ('AAAAAAAAFOIABAAA','More responsible characters go left factors. Championships shall stand twice new, important shows. Books could receive too able, national pounds. Central',3.55,2.2,'Books','launched'),
  ('AAAAAAAAKGBIAAAA','High, political changes shall not',9.55,5.25,'Electronics','launched');

Create the customer table:

CREATE TABLE customer (
  customer_id VARCHAR(16),
  first_name VARCHAR(20),
  last_name VARCHAR(30),
  date_of_birth VARCHAR(32),
  birth_country VARCHAR(20),
  email_address VARCHAR(50)
);

Insert sample customer data:

INSERT INTO customer
VALUES
  ('AAAAAAAALAMKHGBA','Regina','Coleman','1926-12-17','GAMBIA','[email protected]'),
  ('AAAAAAAAMCMKHGBA','John','Bell','1980-01-07','PAPUA NEW GUINEA','[email protected]'),
  ('AAAAAAAANNMKHGBA','Jacqueline','Pierre','1951-12-18','SAMOA','[email protected]'),
  ('AAAAAAAANFNKHGBA','Frank','Mackay','1992-03-19','HONG KONG','[email protected]'),
  ('AAAAAAAAOGNKHGBA','Anthony','Miller','1948-02-26','ALGERIA','[email protected]'),
  ('AAAAAAAACPOKHGBA','Bradley','Sawyer','1956-12-25','ZAMBIA','[email protected]'),
  ('AAAAAAAAOIPKHGBA','Robert','Carter','1951-01-01','UNITED STATES','[email protected]'),
  ('AAAAAAAALJPKHGBA','Ola','High','1980-11-19','SUDAN','[email protected]');

Create DDM and RLS policies

Create the masking policy for customer date of birth:

CREATE MASKING POLICY mask_cust_dob  
WITH (date_of_birth VARCHAR(32))  
USING (sha2(date_of_birth, 256)::TEXT);

Create RLS policies for product launch status:

CREATE RLS POLICY product_launch_status  
WITH (launch_status VARCHAR(50))   
USING (launch_status = 'launched');
  
CREATE RLS POLICY product_launch_status_all
WITH (launch_status VARCHAR(50))   
USING (launch_status IN ('launched','planned'));

Create Amazon Redshift DB roles for Sales and Marketing groups

Create the database roles:

CREATE ROLE "AWSIDC:awssso-sales";
CREATE ROLE "AWSIDC:awssso-marketing";

Attach masking policies

Attach the masking policy to both roles:

ATTACH MASKING POLICY mask_cust_dob  
ON dev.public.customer (date_of_birth)  
TO ROLE "AWSIDC:awssso-marketing";
ATTACH MASKING POLICY mask_cust_dob  
ON dev.public.customer (date_of_birth)  
TO ROLE "AWSIDC:awssso-sales";

Attach RLS policies and enable RLS on product table

Attach the RLS policies and enable row-level security:

ATTACH RLS POLICY product_launch_status  
ON dev.public.product  
TO ROLE "AWSIDC:awssso-sales"; 
ATTACH RLS POLICY product_launch_status_all  
ON dev.public.product  
TO ROLE "AWSIDC:awssso-marketing";
ALTER TABLE dev.public.product ROW LEVEL SECURITY ON;

Grant access to tables to roles

Grant SELECT permissions to both roles:

GRANT SELECT ON dev.public.customer TO ROLE "AWSIDC:awssso-sales";
GRANT SELECT ON dev.public.customer TO ROLE "AWSIDC:awssso-marketing";
GRANT SELECT ON dev.public.product TO ROLE "AWSIDC:awssso-sales"; 
GRANT SELECT ON dev.public.product TO ROLE "AWSIDC:awssso-marketing";

Connect to SALES data warehouse using IAM Identity Center

To connect as a Sales Analyst:

  1. Connect to cpd-sales-wg using the IAM Identity Center connection type as user sales-analyst, and then choose Continue.
  2. Choose sales-analyst, and then choose Next.
  3. Enter your password, and then choose Sign in.
  4. Enter your MFA code, and then choose Sign in.

You are now connected to Amazon Redshift Query Editor V2 with a successful connection to cpd-sales-wg as sales-analyst.

Figure 7: Connect to Sales data warehouse as IDC user

Figure 7: Connect to Sales data warehouse as IDC user

Query shared data as Sales Analyst

Query the customer table with dynamic data masking applied:

SELECT * FROM "dev@edw-ns"."public"."customer";

You can successfully access the customer table, but the sensitive information in the date_of_birth column is encrypted.

Figure 8: Result set of customer table

Figure 8: Result set of customer table

Query the product table with row-level security enabled:

SELECT * FROM "dev@edw-ns"."public"."product";

You can successfully access the product table, but only view data for products with a launch_status value of launched.

Figure 9: Result set of product table

Figure 9: Result set of product table

Note: To connect to the data sharing producer onboarded to Amazon Redshift federated permissions as an IDC user, a superuser is required to provide a CONNECT privilege to the IDC user trying to connect. For more information about how to grant the CONNECT privileges to the user, see Connect privileges in the Amazon Redshift Database Developer Guide.

Connect to Marketing data warehouse using IAM Identity Center

To connect as a Marketing Analyst:

  1. Connect to cpd-marketing-wg using the IAM Identity Center connection type as user marketing-analyst, and then choose Continue.
  2. Choose marketing-analyst, and then choose Next.
  3. Enter your password, and then choose Sign in.
  4. Enter your MFA code, and then choose Sign in.

You are now connected to Amazon Redshift Query Editor V2 with a successful connection to cpd-marketing-wg as marketing-analyst.

Figure 10: Connect to Marketing data warehouse as IDC user

Figure 10: Connect to Marketing data warehouse as IDC user

Query shared data as Marketing Analyst

Query the customer table with dynamic data masking applied:

SELECT * FROM "dev@edw-ns"."public"."customer";

You can successfully access the customer table, but the sensitive information in the date_of_birth column is encrypted.

Figure 11: Result set of customer table

Figure 11: Result set of customer table

Query the product table with row-level security enabled:

SELECT * FROM "dev@edw-ns"."public"."product";

You can successfully access the product table and view data for products with launch_status values of both launched and planned.

Figure 12: Result set of product table

Figure 12: Result set of product table

Additional resources

For more information about implementing federated permissions in your environment, see the following resources:

AWS Documentation

AWS Blogs

AWS Demo

Key benefits

  • Reduced administrative overhead – Centralized policy management removes manual replication
  • Consistent security enforcement – Policies apply uniformly across the warehouses and access methods
  • Seamless identity integration – Single sign-on with existing identity providers through trusted identity propagation and role-based access control

Conclusion

This post showed you how Amazon Redshift federated permissions with AWS IAM Identity Center integration helps streamline multi-warehouse data governance by centralizing security policy management. You define dynamic data masking and row-level security policies once in a central Enterprise Data Warehouse, and they automatically enforce across the connected data warehouses in the same account and Region.

About the authors

Raghu Kuppala

Raghu Kuppala

Raghu is an Analytics Specialist Solutions Architect experienced working in the databases, data warehousing, and analytics space. Outside of work, he enjoys trying different cuisines and spending time with his family and friends.

Satesh Sonti

Satesh Sonti

Satesh is a Principal Specialist Solutions Architect based out of Atlanta, specializing in building enterprise data platforms, data warehousing, and analytics solutions. He has over 20 years of experience in building data assets and leading complex data platform programs for banking and insurance clients across the globe.

Sandeep Adwankar

Sandeep Adwankar

Sandeep is a Senior Product Manager with Amazon SageMaker Lakehouse. Based in the California Bay Area, he works with customers around the globe to translate business and technical requirements into products that help customers improve how they manage, secure, and access data.

Sumukh Bapat

Sumukh Bapat

Sumukh is a Software Engineer at AWS. He works on improving customer experience for Amazon Redshift by solving complex problems in authentication, connectivity, and security. His work focuses on identity management, secure access, and distributed database systems.

Praveen Kumar Ramakrishnan

Praveen Kumar Ramakrishnan

Praveen is a Senior Software Engineer at AWS. He has nearly 20 years of experience spanning various domains including filesystems, storage virtualization and network security. At AWS, he focuses on enhancing the Redshift data security.

Ashish Ghodke

Ashish Ghodke

Ashish is a Software Engineer at Amazon Web Services, where he works on identity and access management systems for large-scale cloud services like Amazon Redshift. His work focuses on building secure authentication and single sign-on solutions for distributed systems. He is passionate about distributed systems, cloud security, and building reliable infrastructure at scale.

Samba 4.24.0 released

Post Syndicated from corbet original https://lwn.net/Articles/1063517/

Version 4.24.0 of the Samba SMB filesystem implementation has been
released. There are a number of significant changes, including audit
support for authentication information, remote password management, a
number of Kerberos improvements, asynchronous-I/O rate limiting, and more.

GNOME 50 released

Post Syndicated from jzb original https://lwn.net/Articles/1063466/

GNOME 50 has been
released. Notable changes in this release include enhancements to the
Orca screen-reader application, interface and performance improvements
for GNOME’s file manager (Files), a “massive set of stability and
performance updates” for its display-handling technologies, and
much more. See also the “What’s new
for developers” article that covers changes of interest to GNOME
and GNOME application developers.

Our First 2026 Heroes Cohort Is Here!

Post Syndicated from Taylor Jacobsen original https://aws.amazon.com/blogs/aws/our-first-2026-heroes-cohort-is-here/

We’re thrilled to celebrate three exceptional developer community leaders as AWS Heroes. These individuals represent the heart of what makes the AWS community so vibrant. In addition to sharing technical knowledge, they build connections, forge genuine human relationships, and create pathways for others to grow. From pioneering cloud culture in mountain villages to leading cybersecurity education across continents, these Heroes demonstrate that true leadership extends beyond technical expertise to the communities we build and the lives we impact.

Maurizio – Pignola, Italy

Community Hero Maurizio is a CTO and organizer of the AWS User Group Basilicata, recognized for his dedication to building tech ecosystems where they previously did not exist. For over a decade, he has pioneered cloud culture through a philosophy centered on genuine human connection and knowledge transfer. He founded an international tech conference in a small mountain village, creating a unique space where global experts and local talent meet, blending deep technical sessions on cloud architectures, DevOps, and web scaling with unconventional networking experiences. Beyond organizing events, Maurizio is a tireless mentor working across generations, which span from introducing children to coding to helping university students and professionals transition into cloud architecture. His impact is defined by a rare combination of technical leadership and inclusive community building that draws people from across Europe.

Ray Goh – Singapore

Artificial Intelligence Hero Ray Goh is a seasoned AWS machine learning and AI community leader based in Singapore and a long-standing contributor in various AWS community programs since 2018, from AWS ASEAN Cloud Warrior and AWS Dev/Cloud Alliance to being part of the pioneer batch of AWS Community Builders in 2020. He founded The Gen-C (a Generative AI Learning Community) in 2024, organizing regular public workshops at libraries across Singapore on topics ranging from LLM fine-tuning to AI agents on AWS. Ray has spoken at AWS re:Invent, AWS Summit ASEAN, AWS Community Day Hong Kong, and numerous user group meetups, and guest-authored for the AWS Machine Learning Blog. He spearheaded the world’s largest enterprise AWS DeepRacer program for DBS Bank in 2020, upskilling over 3,100 employees, and trained more than 1,300 ASEAN students in LLM techniques in 2025. His community work extends to skills-based CSR initiatives teaching AI and machine learning to women, children, and youths, with contributions featured on CNBC and Euromoney.

Sheyla Leacock – Panama City, Panama

Security Hero Sheyla Leacock is an IT security professional, mentor, technical author, and international speaker contributing to the global cloud and cybersecurity community. She has spoken at AWS Summit Mexico, AWS Summit LATAM in Peru, and led PeerTalk sessions at AWS re:Invent, while also leading the AWS User Group in Panama and regularly participating in AWS Community Days and regional meetups. Beyond AWS-focused events, she has delivered talks at more than 20 international conferences and publishes technical articles and educational content on AWS cloud computing and cybersecurity. She collaborates with universities as a guest lecturer, supporting the development of emerging technology and cybersecurity talent. Through community leadership, knowledge sharing, and education, she contributes to strengthening the AWS and cybersecurity ecosystem.

Learn More

Visit the AWS Heroes webpage if you’d like to learn more about the AWS Heroes program, or to connect with a Hero near you.

— Taylor

Amazon threat intelligence teams identify Interlock ransomware campaign targeting enterprise firewalls

Post Syndicated from CJ Moses original https://aws.amazon.com/blogs/security/amazon-threat-intelligence-teams-identify-interlock-ransomware-campaign-targeting-enterprise-firewalls/

Amazon threat intelligence has identified an active Interlock ransomware campaign exploiting CVE-2026-20131, a critical vulnerability in Cisco Secure Firewall Management Center (FMC) Software that could allow an unauthenticated, remote attacker to execute arbitrary Java code as root on an affected device, which was disclosed by Cisco on March 4, 2026.

After Cisco’s disclosure, Amazon threat intelligence began research into this vulnerability using Amazon MadPot’s global sensor network—a system of honeypot servers that attract and monitor cybercriminal activity. While looking for any current or past exploits of this vulnerability, our research found that Interlock was exploiting this vulnerability 36 days before its public disclosure, beginning January 26, 2026. This wasn’t just another vulnerability exploit, Interlock had a zero-day in their hands, giving them a week’s head start to compromise organizations before defenders even knew to look. Upon making this discovery, we shared our findings with Cisco to help support their investigation and protect customers.

A misconfigured infrastructure server—essentially, a poorly secured staging area used by the attackers—exposed Interlock’s complete operational toolkit. This rare mistake provided Amazon’s security teams with visibility into the ransomware group’s multi-stage attack chain, custom remote access trojans (backdoor programs that give attackers control of compromised systems), reconnaissance scripts (automated tools for mapping victim networks), and evasion techniques.

AWS infrastructure and customer workloads on AWS were not observed to be involved in this campaign. This advisory shares comprehensive technical analysis and indicators of compromise to help organizations identify potential compromise and defend against Interlock’s operations. Organizations running Cisco Secure Firewall Management Center should immediately apply Cisco’s security patches and review the indicators provided below.

Discovery and investigation timeline

Amazon threat intelligence identified threat activity potentially related to CVE-2026-20131 beginning January 26, 2026, predating the public disclosure. Observed activity involved HTTP requests to a specific path in the affected software. Request bodies contained Java code execution attempts and two embedded URLs: one used to deliver configuration data supporting the exploit, and another designed to confirm successful exploitation by causing a vulnerable target to perform an HTTP PUT request and upload a generated file. Multiple variations of these URLs were observed across different exploit attempts.

To advance the investigation and obtain additional threat intelligence, we performed the expected HTTP PUT request with the anticipated file content—essentially, we pretended to be a successfully compromised system. This successfully prompted Interlock to proceed to the next stage, issuing commands to fetch and execute a malicious ELF binary (a Linux executable file) from a remote server.

When analysts retrieved the binary, they discovered the same host (attacker-controlled server) is used for distributing Interlock’s entire operational toolkit. The exposed infrastructure organized artifacts into separate paths corresponding to individual targets, with the same paths used for both downloading tools to compromised hosts and uploading operational artifacts back to the staging server.

Attribution to Interlock ransomware

The ELF binary and associated artifacts are attributable to the Interlock ransomware family based on convergent technical and operational indicators. The embedded ransom note and TOR negotiation portal are consistent with Interlock’s established branding and infrastructure. The ransom note’s invocation of multiple data protection regulations reflects Interlock’s documented practice of citing regulatory exposure to pressure victims, essentially threatening organizations not just with data encryption, but with regulatory fines and compliance violations. The campaign-specific organization identifier embedded in the note aligns with Interlock’s per-victim tracking model.

Interlock has historically targeted specific sectors where operational disruption creates maximum pressure for payment. Education represents the largest share of their activity, followed by engineering, architecture, and construction firms, manufacturing and industrial organizations, healthcare providers, and government and public sector entities.

Temporal analysis performed on timestamps from observed threat activities, artifacts stored on the misconfigured infrastructure server, and metadata embedded within recovered threat artifacts indicates the actor most likely operates in UTC+3 with 75–80% confidence. Systematic analysis across all UTC offsets showed UTC+3 produced the best fit: first activity around 08:30, peak activity between 12:00 and 18:00, and a probable sleep window of 00:30–08:30.

Interlock ransomware negotiation portal where victims enter their organization ID and email address to receive an auth token to begin a negotiation chat session.

Figure 1: Interlock ransomware negotiation portal where victims enter their organization ID and email address to receive an auth token to begin a negotiation chat session.

Technical analysis: Interlock’s operational toolkit

Post-compromise reconnaissance script

Once Interlock gains initial access, they use a variety of priority tools to complete their attack. Amazon threat intelligence teams recovered a PowerShell script designed for systematic Windows environment enumeration (automated information gathering about the victim’s network). The script collects operating system and hardware details, running services, installed software, storage configuration, Hyper-V virtual machine inventory, user file listings across Desktop, Documents, and Downloads directories, browser artifacts from Chrome, Edge, Firefox, Internet Explorer, and 360 browser (including history, bookmarks, stored credentials, and extensions), active network connections correlated with responsible processes, ARP tables, iSCSI session data, and RDP authentication events from Windows event logs.

The script stages results to a centralized network share (\JK-DC2\Temp) using each system’s fully qualified hostname to create dedicated directories—essentially creating a folder for each compromised computer. Following collection, it compresses data into ZIP archives named after each hostname and removes original raw data. This structured per-host output format indicates the script operates across multiple machines within a network—a hallmark of ransomware intrusion chains that prepare for organization-wide encryption.

Custom remote access trojans

Remote access trojans (RATs) are malicious programs that give attackers persistent control over compromised systems, functioning like unauthorized remote desktop software.

JavaScript implant: Amazon threat intelligence recovered an obfuscated JavaScript remote access trojan that suppresses debugging output by overriding browser console methods (hiding its activity from basic detection tools). On execution, it profiles the infected host using PowerShell and Windows Management Instrumentation (WMI), collecting system identity, domain membership, username, OS version, and privilege context before transmitting this data during an encrypted initialization handshake.

Command-and-control communication occurs over persistent WebSocket connections with RC4-encrypted messages using per-message 16-byte random keys embedded in packet headers—essentially, each message uses a different encryption key, making interception more difficult. The implant cycles through multiple operator-controlled hostnames and IP addresses in randomized order with exponential backoff between reconnection attempts.

The implant provides interactive shell access, arbitrary command execution, bidirectional file transfer, and SOCKS5 proxy capability for tunneling TCP traffic (routing malicious traffic through other systems to hide its origin). Self-update and self-delete capabilities allow operators to replace or remove the implant without reinfection, supporting operational cleanup to hinder forensic investigation.

Java implant: A functionally equivalent client implemented in Java provides identical command-and-control capabilities. Built on GlassFish ecosystem libraries, it uses Grizzly for non-blocking I/O transport and Tyrus for WebSocket protocol communication. In simpler terms, Interlock built the same backdoor in two different programming languages, ensuring they maintain access even if defenders detect one version.

Infrastructure laundering script

Sophisticated threat actors don’t attack from their own infrastructure, they build disposable relay networks to hide their tracks. Amazon threat intelligence teams identified a Bash script that configures Linux servers as HTTP reverse proxies (intermediary servers that forward traffic to hide the attacker’s true location). The script performs system updates, installs fail2ban with SSH brute-force protection, and compiles HAProxy 3.1.2 from source. The HAProxy instance listens on port 80 and forwards all inbound HTTP traffic to a hardcoded target IP, with systemd ensuring persistence across reboots.

A notable component is a log erasure routine running as a cron job every five minutes. The routine truncates all *.log files under /var/log and suppresses shell history by unsetting the HISTFILE variable. This aggressive evidence destruction, wiping logs every five minutes, combined with the purpose-built HTTP forwarding proxy, indicates the script establishes disposable traffic-laundering relay nodes. These nodes obscure exploit traffic origin, relay command-and-control communications, or proxy data exfiltration, making it nearly impossible to trace attacks back to their source.

Memory-resident webshell

Amazon threat intelligence teams observed a Java class file delivered as an alternative to the ELF binary drop. When loaded by the Java Virtual Machine (JVM), its static initializer registers a ServletRequestListener with the server’s StandardContext, essentially installing a persistent memory-resident backdoor that intercepts HTTP requests without writing files to disk. This “fileless” approach evades traditional antivirus scanning that looks for malicious files.

The listener inspects incoming requests for specially crafted parameters containing encrypted command payloads. Payloads are decrypted using AES-128 with a key derived from the MD5 hash of the hardcoded seed “geckoformboundary99fec155ea301140cbe26faf55ed2f40″ (using the first 16 characters: 09b1a8422e8faed0). Decrypted payloads are treated as compiled Java bytecode, dynamically loaded into the JVM, and executed—a technique designed to evade file-based detection by running malicious code entirely in memory.

Connectivity verification tool

Amazon threat intelligence teams recovered Java class files implementing a basic TCP server listening on port 45588 (encoded as Unicode character 넔 to obscure the port number from static analysis). The server accepts connections, logs connecting IP addresses, sends a greeting message, and immediately closes connections. This operational profile is consistent with a lightweight network beacon—essentially a “phone home” tool used to verify successful code execution or confirm network port reachability following initial exploitation.

Legitimate tool abuse

Interlock deployed ConnectWise ScreenConnect, a legitimate commercial remote desktop tool, alongside custom implants. When ransomware operators deploy legitimate remote access tools alongside their custom malware, they’re buying insurance—if defenders find and remove one backdoor, they still have another way in. This indicates multiple redundant remote access mechanisms—a pattern consistent with ransomware operators seeking to maintain access even if individual footholds are removed. The tool’s legitimate network footprint helps blend with authorized remote administration traffic, making detection more challenging.

Amazon threat intelligence teams also recovered Volatility, an open-source memory forensics framework typically used by incident responders (the same tool defenders use to investigate attacks). While no artifacts indicated automated use, its presence alongside custom implants and reconnaissance scripts is consistent with advanced threat operations. Both ransomware groups and nation-state actors have been observed deploying Volatility during intrusions. The tool’s focus on parsing memory dumps provides access to sensitive data such as credentials stored in RAM, which can enable lateral movement (spreading through the network) and deeper environment compromise in support of ransom operations or espionage objectives.

Interlock also used Certify, an open source offensive security tool designed to exploit misconfigurations in Active Directory Certificate Services (AD CS). For ransomware operators, Certify provides a pathway to identify vulnerable certificate templates and enrollment permissions that allow requesting authentication-capable certificates. These certificates can be used to impersonate users, escalate privileges, or maintain persistent access. These capabilities directly support both initial compromise and long-term persistence objectives in ransomware operations.

Indicators of compromise (IoCs)

The following indicators support defensive measures by organizations that may be affected. Due to Interlock’s use of content variation techniques, most file hashes are not included as reliable indicators. The threat actor modified most artifacts like scripts and binaries downloaded to different targets. This resulted in different file hashes for functionally identical tools. The customization allowed each attack to evade signature-based detection that looks for exact file matches.

206.251.239[.]164

Exploit source IP

Active Jan 2026

199.217.98[.]153

Exploit source IP

Active Mar 2026

89.46.237[.]33

Exploit source IP

Active Mar 2026

Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:136.0) Gecko/20100101 Firefox/136.0

Exploit HTTP User-Agent

Observed Jan 2026 and Mar 2026

b885946e72ad51dca6c70abc2f773506

Exploit TLS JA3

Observed Jan 2026 and Mar 2026

f80d3d09f61892c5846c854dd84ac403

Exploit TLS JA3

Observed Mar 2026

t13i1811h1_85036bcba153_b26ce05bbdd6

Exploit TLS JA4

Observed Jan 2026 and Mar 2026

t13i4311h1_c7886603b240_b26ce05bbdd6

Exploit TLS JA4

Observed Mar 2026

144.172.94[.]59

C2 Fallback IP

Active Mar 2026

199.217.99[.]121

C2 Fallback IP

Active Mar 2026

188.245.41[.]78

C2 Fallback IP

Active Mar 2026

144.172.110[.]106

Backend C2 IP

Active Mar 2026

95.217.22[.]175

Backend C2 IP

Active Mar 2026

37.27.244[.]222

Staging host IP

Active Mar 2026

hxxp://ebhmkoohccl45qesdbvrjqtyro2hmhkmh6vkyfyjjzfllm3ix72aqaid[.]onion/chat.php

Ransom negotiation portal

Active Mar 2026

cherryberry[.]click

Exploit Support Domain

Active Jan 2026

ms-server-default[.]com

Exploit Support Domain

Active Mar 2026

initialize-configs[.]com

Exploit Support Domain

Active Mar 2026

ms-global.first-update-server[.]com

Exploit Support Domain

Active Mar 2026

ms-sql-auth[.]com

Exploit Support Domain

Active Mar 2026

kolonialeru[.]com

Exploit Support Domain

Active Mar 2026

sclair.it[.]com

Exploit Support Domain

Active Mar 2026

browser-updater[.]com

C2 domain

Active Mar 2026

browser-updater[.]live

C2 domain

Active Mar 2026

os-update-server[.]com

C2 domain

Active Mar 2026

os-update-server[.]org

C2 domain

Active Mar 2026

os-update-server[.]live

C2 domain

Active Mar 2026

os-update-server[.]top

C2 domain

Active Mar 2026

d1caa376cb45b6a1eb3a45c5633c5ef75f7466b8601ed72c8022a8b3f6c1f3be

Offensive security tool (Certify)

Observed Mar 2026

6c8efbcef3af80a574cb2aa2224c145bb2e37c2f3d3f091571708288ceb22d5f

Screen locker

Observed Mar 2026

Defensive recommendations

Organizations should take the following actions to protect against Interlock ransomware operations.

Immediate actions:

  • Apply Cisco’s security patches for Cisco Secure Firewall Management Center
  • Review logs for the indicators of compromise listed above
  • Conduct security assessments to identify potential compromise
  • Review ScreenConnect deployments for unauthorized installations

Detection opportunities:

  • Monitor for PowerShell scripts staging data to network shares with hostname-based directory structures
  • Detect Java ServletRequestListener registrations in web application contexts (unusual modifications to Java web applications)
  • Identify HAProxy installations with aggressive log deletion cron jobs (proxy servers that erase their own logs every five minutes)
  • Watch for TCP connections to unusual high-numbered ports (e.g., 45588)

Long-term measures:

  • Implement defense-in-depth strategies with multiple layers of security controls
  • Maintain continuous threat monitoring and hunting capabilities
  • Ensure comprehensive logging with secure, centralized log storage (stored separately from systems that could be compromised)
  • Regularly test incident response procedures for ransomware scenarios
  • Educate security teams on Interlock’s tactics, techniques, and procedures

The real story here isn’t just about one vulnerability or one ransomware group—it’s about the fundamental challenge zero-day exploits pose to every security model. When attackers exploit vulnerabilities before patches exist, even the most diligent patching programs can’t protect you in that critical window. This is precisely why defense in depth is essential—layered security controls provide protection when any single control fails or hasn’t yet been deployed. Rapid patching remains foundational in vulnerability management, but defense in depth helps organizations not to be defenseless during the window between exploit and patch.

Amazon Threat Intelligence teams continue to monitor Interlock ransomware operations and will provide updates as additional information becomes available. The intelligence gathered from this campaign is being integrated into AWS security services to protect customers proactively.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, contact AWS Support.

CJ Moses

CJ Moses

CJ Moses is the CISO of Amazon Integrated Security. In his role, CJ leads security engineering and operations across Amazon. His mission is to enable Amazon businesses by making the benefits of security the path of least resistance. CJ joined Amazon in December 2007, holding various roles including Consumer CISO, and most recently AWS CISO, before becoming CISO of Amazon Integrated Security September of 2023.

Prior to joining Amazon, CJ led the technical analysis of computer and network intrusion efforts at the Federal Bureau of Investigation’s Cyber Division. CJ also served as a Special Agent with the Air Force Office of Special Investigations (AFOSI). CJ led several computer intrusion investigations seen as foundational to the security industry today.

CJ holds degrees in Computer Science and Criminal Justice, and is an active SRO GT America GT2 race car driver.

Local-privilege escalation in snapd

Post Syndicated from jzb original https://lwn.net/Articles/1063453/

Qualys has discovered
a local-privilege escalation (LPE) vulnerability affecting Ubuntu
Desktop 24.04 and later:

This flaw (CVE-2026-3888) allows an unprivileged local attacker to
escalate privileges to full root access through the interaction of two
standard system components: snap-confine and systemd-tmpfiles.

More details are available in the security
advisory. Canonical has published updated packages as well as instructions
for verifying if a system is vulnerable and how to upgrade if so.

Fedora Asahi Remix 43 released

Post Syndicated from jzb original https://lwn.net/Articles/1063451/

Fedora Asahi Remix 43 is
now available:

This release incorporates all the exciting improvements brought by
Fedora
Linux 43. Notably, package management is significantly
upgraded with RPM 6.0 and the new
DNF5 backend for PackageKit for Plasma Discover and GNOME Software
ahead of Fedora Linux 44. It also continues to provide extensive
device support. This includes newly added support for the Mac Pro,
microphones in M2 Pro/Max MacBooks, and 120Hz refresh rate for
the built-in displays for MacBook Pro 14/16 models.

[$] BPF comes to io_uring at last

Post Syndicated from daroc original https://lwn.net/Articles/1062286/

The kernel’s asynchronous

io_uring interface maintains two shared ring buffers:
a submission queue for sending requests to the kernel, and a completion queue
containing the results of those requests. Even with shared memory removing much
of the overhead of communicating with user space, there is still some overhead
whenever the kernel must switch to user space to give it the opportunity to
process completion requests and
queue up any subsequent work items. A

patch set from Pavel Begunkov minimizes this overhead by letting
programmers extend the io_uring event loop with a BPF program that can enqueue
additional work in response to completion events. The patch set has
been in development for a long time, but has
finally been accepted.

Security updates for Wednesday

Post Syndicated from jzb original https://lwn.net/Articles/1063446/

Security updates have been issued by AlmaLinux (.NET 10.0, .NET 9.0, compat-openssl11, container-tools:rhel8, grub2, and libvpx), Debian (ansible, gst-plugins-base1.0, and nodejs), Fedora (chromium, forgejo, and systemd), Oracle (container-tools:rhel8, grub2, kernel, libpng, libvpx, nginx, opencryptoki, python3.12, and vim), Red Hat (firefox, python-wheel, python3.12-wheel, and thunderbird), SUSE (389-ds, chromium, clamav, container-suseconnect, curl, freerdp, gvfs, kea, kubernetes, ruby4.0-rubygem-minitar, ruby4.0-rubygem-multi_xml, ruby4.0-rubygem-nokogiri, ruby4.0-rubygem-puma, ruby4.0-rubygem-rack, ruby4.0-rubygem-rack-session, ruby4.0-rubygem-rails, ruby4.0-rubygem-rails-html-sanitizer, ruby4.0-rubygem-railties, ruby4.0-rubygem-rubyzip, vim, and xen), and Ubuntu (flask, libssh, linux-aws-5.15, linux-gcp-5.15, linux-gke, linux-hwe-5.15,
linux-intel-iotg-5.15, linux-lowlatency-hwe-5.15, linux-oracle-5.15, linux-gcp-6.17, linux-realtime, linux-realtime, linux-realtime, linux-realtime-6.8, snapd, and vim).

The Attack Cycle is Accelerating: Announcing the Rapid7 2026 Global Threat Landscape Report

Post Syndicated from Rapid7 Labs original https://www.rapid7.com/blog/post/tr-accelerating-attack-cycle-2026-global-threat-landscape-report

The predictive window has collapsed.

In 2025, high-impact vulnerabilities weren’t quietly accumulating risk. They were operationalized, and often within days.

Today, Rapid7 Labs released the 2026 Global Threat Landscape Report, an in-depth analysis of how attacker behavior is evolving across vulnerability exploitation, ransomware operations, identity abuse, and AI-driven tradecraft. The data shows a clear pattern: exposure is being identified and weaponized faster than most organizations are set up to defend.

From disclosure to exploitation in days, not weeks

In 2025, confirmed exploitation of newly disclosed CVSS 7–10 vulnerabilities increased 105% year over year, rising from 71 to 146. The median time from publication to inclusion in CISA’s Known Exploited Vulnerabilities list fell from 8.5 days to 5.0 days.

At the same time, the number of high-probability vulnerabilities that remained unexploited dropped sharply. The buffer that once allowed teams to triage and schedule remediation is shrinking to the point where some severe flaws were seen to have been exploited almost immediately.

The broader trend is unmistakable: vulnerability management programs built around reactive remediation cycles are struggling to keep pace with adversaries operating at machine speed.

Cybercrime as a structured market

Cybercrime in 2025 no longer resembles chaotic hacking. It resembles platform capitalism.

The report highlights how the underground economy now mirrors legitimate SaaS ecosystems. Initial Access Brokers obtain and validate network footholds. Ransomware operators focus on encryption and extortion. Infostealer operators sell subscription-style access to fresh credential logs.

This specialization lowers barriers to entry and increases scale creating a supply chain in which access is acquired, packaged, priced, and sold to anyone who wants it. 

Ransomware is a good example of this business maturity. It was present in 42% of Rapid7 MDR investigations in 2025 with leak posts increasing 46.4% year over year, and the number of active groups growing from 102 to 140. That kind of growth is anything but random or coincidental: it is an indication of systemic changes to the ransomware ecosystem indicating growing sophistication, specialization, and, ultimately, risk. 

Logging in, not breaking in

Authentication-based attacks remain incredibly common as the lack of consistency across organizations can lead to easy exploitation. Valid accounts without multi-factor authentication (MFA) were responsible for 43.9% of incidents over that year. Rather than forcing their way past defenses, attackers increasingly authenticate with stolen credentials, hijacked sessions, or abused tokens. This is where the increase in AI-driven attacks is particularly acute with the benefits generative AI can play in improving the maturity and sophistication of social engineering attacks. 

As enterprises extend trust across cloud platforms, SaaS ecosystems, APIs, and remote work environments, authentication systems have become the backbone of operational control. This represents a structural shift with the control layer of cyber risk moving away from network perimeters toward authentication flows.

Attacks are using reliable vectors, just at alarming speeds

One hallmark of the attack landscape in 2025 was the use of tried and true attack vectors rather than novel exploits and zero-day vulnerabilities. CVE disclosures continued to climb last year, but confirmed exploitation clustered around dependable weakness types like deserialization, authentication bypass, and memory corruption vulnerabilities.

Attackers are targeting flaws that enable pre-authentication access, repeatable execution, and rapid data theft. They are not, necessarily, chasing every vulnerability. Just the ones they deem reliable. This pattern reinforces a key theme of the report: exploitability and context matter more than raw volume.

AI as an accelerant

AI is serving as a force multiplier and an expanding attack surface at the same time. 

Generative AI is accelerating established attack methods by reducing the time, skill, and coordination previously required to execute them at scale. Rather than introducing entirely new categories of exploitation, threat actors are integrating AI into existing workflows to industrialize phishing, automate reconnaissance, and refine malicious scripts with greater speed and precision. 

AI-assisted phishing campaigns were more polished and tailored to specific industries or executive roles, reflecting a measurable improvement in personalization and believability. They accelerated open-source intelligence collection to create details from fragmented data. AI was used to troubleshoot malware development in near real time, effectively compressing the cycle between initial research and malware deployment. The result is not radical technical innovation, but efficiency, speed, and fewer missed opportunities. 

Meanwhile, AI platforms themselves are emerging as targets with model servers, orchestration frameworks, and token-based integrations, inheriting familiar weaknesses such as unsafe deserialization and weak authentication. As organizations operationalize AI quickly, governance gaps create new high-impact pathways to risk.

The geography of attacks

When it comes to targeted regions, no area of the globe represents a better convergence of exposure and financial opportunity than North America. Organizations on this continent accounted for 82.04% of observed incidents, with the United States representing roughly 70% of leak posts on ransomware leak sites. Manufacturing, business services, and retail were among the most targeted industries as these sectors often combine operational dependence, sensitive data, and financial leverage making them fat targets for attackers looking for reliability not only in their attack vectors, but in gains available from their chosen targets. 

Across criminal and state-aligned activity, attackers are converging on identity systems, edge infrastructure, collaboration platforms, and cloud control planes where trust, scale, and business continuity intersect.

What this means for security leaders

There is a sobering reality in this year’s data: the underlying weaknesses remain familiar. Weak credentials. Social engineering. Exposed services. Unpatched edge infrastructure.

What has changed is the speed.

Security programs can no longer rely on moving slightly faster than attackers. The model must shift toward reducing exposure before it is operationalized.

That means:

  • Continuous exposure visibility with contextual prioritization

  • Strong MFA enforcement and hardened identity controls

  • Protected and monitored edge infrastructure

  • Governance around AI systems and integrations

  • AI-enabled security workflows capable of matching attacker velocity

The organizations that maintain clear, continuous insight into their exposure – and reduce it before it is monetized – will be best positioned to manage risk in this accelerated cycle.

The question is no longer whether exposure exists.
It is whether you can reduce it before attackers capitalize on it.

Read the full Rapid7 2026 Threat Landscape Report to explore the data and strategic implications in detail.

The collective thoughts of the interwebz