Post Syndicated from Lijan Kuniyil original https://aws.amazon.com/blogs/architecture/how-mapfre-usa-modernized-fraud-claims-with-amazon-emr-serverless/

Insurance fraud remains a significant challenge for the insurance industry because fraudulent claims can increase loss costs, reduce trust, and consume investigation capacity that could otherwise be focused on serving customers. Traditional fraud detection approaches typically rely on rules-based controls, manual investigation triggers, historical claim patterns, and structured-data-only analysis. These approaches are useful for known fraud patterns, but they can struggle to detect sophisticated fraud rings or hidden relationships across claimants, policies, vehicles, providers, addresses, and prior suspicious activities.

Mapfre USA is the number one auto and home insurer in Massachusetts, serving customers in 11 states nationwide. Our coverage includes auto, home, motorcycle, watercraft, business insurance, and more. As part of Mapfre Group, we’re a worldwide leader serving over 31.1 million customers in more than 100 countries with a team of 31,000 employees. In collaboration with AWS and Neo4j, Mapfre USA modernized its fraud prevention capabilities by combining graph-based features with machine learning (ML) models deployed on AWS. This initiative, focused initially on Massachusetts Auto insurance and later expanded to Home (HO), has delivered significant business impact, exceeding $5 million Net Present Value (NPV), with realized savings already outperforming projections.

In this post, we share how Mapfre USA designed and implemented this solution, highlight the technical architecture running on AWS specifically on the Mapfre Data Platform called Atenea, and explore lessons learned that can apply to other industries facing complex fraud challenges.

Business challenge

Fraudulent claims aren’t always isolated events. They often involve hidden networks of policyholders, vehicles, providers, and prior suspicious activities. Detecting these complex relationships requires going beyond traditional structured data analysis.

Mapfre set out with a clear goal:

• Goal: Improve fraud detection accuracy and claims handling efficiency.
• Key KPI: Identify fraudulent claims missed by traditional methods.
• Approach: Develop several ML models that use both traditional structured data and graph-based features derived from claim relationships.
• Deployment: Integrate seamlessly with Guidewire Claims, so front-line adjusters automatically receive fraud alerts with explanations.

Each flagged claim exposure generates a Guidewire activity showing the top three model drivers, helping investigators understand why the claim was identified and act quickly.

Technical solution on AWS (Atenea Data Platform)

The fraud detection platform is built on a modern data architecture on AWS, designed to scale efficiently and provide long-term governance.

At its core, the solution uses Apache Iceberg tables stored on Amazon Simple Storage Service (Amazon S3), with metadata managed through the AWS Glue Data Catalog and access governed through AWS Lake Formation as part of the Atenea lakehouse governance model. The platform feature store is implemented through feature-store-managed Iceberg tables that manage model features, predictions, and Guidewire activities. The implementation is structured across three logical layers:

• Silver Layer – Iceberg tables that contain source data from each of the sources, used as the initial consumption point of the platform.
• Gold Layer – Iceberg tables storing intermediate data, such as unified Guidewire activity logs, Auto features, and Home features.
• Platinum Layer – Feature Store-managed Iceberg tables containing encoded features and model predictions, making them reusable across models and ensuring strong metadata governance.

Processing pipelines are executed on Amazon EMR Serverless, with orchestration managed by Apache Airflow operators running on Amazon Managed Workflows for Apache Airflow (Amazon MWAA). This provides elastic, cost-efficient compute for both batch processing and fast-time scoring, while keeping orchestration, monitoring, and recovery centralized.

For graph enrichment, the platform connects to Neo4j using a dedicated driver, enabling advanced network-based features like suspicious claim linkages, provider fraud ratios, and centrality metrics.

This architecture supports efficient, reliable, and transparent production execution through repeatable Airflow orchestration, environment-based continuous integration and delivery (CI/CD) promotion, centralized monitoring, failure notifications, retry mechanisms, dead-letter queue handling for Guidewire integration, and controlled secret management. At the same time, the layered lakehouse design keeps the platform flexible enough to evolve with new business needs and fraud detection use cases.

The data sources are policy, claims, vehicles, and notes (from AS400 and Guidewire), which include structured data and derived features capturing entity relationships (graph data).

The following list describes the architecture overview:

1. Data ingestion – Claim batch data uploaded to Amazon S3. Data gets standardized and materialized in Iceberg tables within the Silver layer.
2. Graph enrichment – Data processed to update Neo4j graph database hosted on AWS.
3. Model training and scoring – Batch scoring for several ML models.
4. Model orchestration – Unified orchestration for ingestion, training, and inference using Apache Airflow operators. CI/CD pipelines for promotion across environments.
5. Execution platform – Amazon EMR Serverless for cost-efficient Spark processing. Migration to Apache Iceberg plus AWS Glue Data Catalog for scalable metadata handling.
6. Integration with claims systems – Fraud predictions automatically create Guidewire activities, enriched with a description for investigators.
7. Secrets and security – AWS Secrets Manager securely stores credentials and tokens for Guidewire API integration, with environment-specific and Region-specific access controls.
8. Monitoring and reliability – Amazon CloudWatch and Amazon Simple Notification Service (Amazon SNS) provide visibility into pipeline health and notify teams on failures. Data quality checks are executed at key stages of the pipeline to validate data availability, schema consistency, completeness, and business-rule expectations before outputs are consumed by models or sent to Guidewire.

Guidewire integration with MLOps on AWS

One of the most important parts of Mapfre’s solution was closing the loop between ML predictions and the claims handling system. This required a resilient integration between the Atenea data platform on AWS and Guidewire Claims.

The following describes the integration flow:

1. When an ML use case finishes scoring, the results are written as JSON files into the S3 path: <bucket_name>/guidewire/.
2. An S3 event notification triggers an AWS Lambda function.
3. This Lambda function:
   • Reads the JSON file.
   • Calls the Guidewire Predictive Model API.
   • Because Guidewire doesn’t support batch requests, the Lambda function sends each JSON payload individually. This keeps the integration compatible with Guidewire and isolates failures at the individual activity level, but it increases the number of API calls and makes retry, throttling, DLQ handling, and monitoring controls important.
4. If successful, the API responds with HTTP 201 (activity created).
   • If not, the Lambda function retries up to two times.
   • Failed requests are sent to an Amazon Simple Queue Service (Amazon SQS) dead-letter queue (DLQ), and an Amazon SNS notification is published for monitoring.
5. Secrets are stored in AWS Secrets Manager and injected as Lambda environment variables, along with AWS Region-specific URLs for token retrieval and API endpoints.
6. The following JSON shows the example structure for Guidewire integration:

{
  "method": "createPredictiveActivity",
  "params": [
    {
      "claimNumber": "AUXXXXXXX",
      "exposureNumber": 1,
      "subject": "Fraud alert from ML model",
      "description": "Claim flagged as potential fraud based on graph + ML features",
      "shortSubject": "ML_Fraud_Flag",
      "priority": "high",
      "availableForClosedClaim": true,
      "autoCloseOnExposureClosure": false,
      "targetDays": 4,
      "escalationDays": 6
    }
  ]
}

Key benefits of this integration:

• Real-time actionability – Fraud predictions automatically create Guidewire activities for front-line adjusters.
• Resilience – Built-in retries, DLQ handling, and Amazon SNS alerts make sure failed events aren’t lost.
• Security – Secrets and tokens are managed using AWS Secrets Manager, with strict environment separation (dev, pre, pro).
• Scalability – Any new MLOps use case writes results into the S3 output path, automatically flowing into Guidewire.

This integration shows that fraud models don’t exist in isolation but actively augment daily claim workflows in production. It connects Atenea’s MLOps pipelines on AWS directly with business decisioning systems, which is critical to realizing the fraud savings impact.

Data quality and resilience

For robustness, data quality checks are applied on ingestion pipelines and graph features. Automated validation detects anomalies early, monitoring dashboards track KPIs and model performance, and standardized recovery and promotion processes operate across environments.

Visualization and investigative tools

Neo4j Bloom supports SIU workflows by visually exploring entity relationships, such as a provider linked across multiple suspicious claims, accelerating fraud ring identification.

Conclusion

The fraud detection model in auto claims has enhanced Mapfre USA’s ability to identify fraudulent activity, driving significant savings and improving overall claims efficiency.

During the pilot phase alone, savings exceeded projections, and in production the initiative has proven a Net Present Value (NPV) of more than $5M. These results confirm the business case and highlight the strength of combining structured data with graph-based features to uncover fraud networks that traditional approaches miss.

The results have been compelling:

• Accuracy gains – Detection improved by 50–135 percent compared to baseline methods.
• Substantial realized value – Both during the pilot and in production.
• Cross-functional success – The initiative brought together Claims, IT Data, Advanced Analytics, and Neo4j teams in an agile, collaborative model.

Beyond the financial outcomes, several lessons have emerged. First, cross-functional collaboration between groups like Claims, Data Engineering, Advanced Analytics, and technology partners like AWS and Neo4j was critical to success. Second, explainability proved essential. By presenting adjusters with the top model drivers directly in Guidewire, trust and adoption of the system increased substantially. Finally, building resilience into the architecture through monitoring, retries, and data quality processes helped the models operate reliably in production.

Looking ahead, the platform is well-positioned to expand beyond fraud detection. New use cases such as underwriting anomaly detection and customer entity resolution are already on the roadmap. With robust architecture built on AWS using Amazon EMR Serverless, Apache Iceberg on Amazon S3 supported by AWS Glue Data Catalog and Lake Formation, a custom-built Feature Store, and Neo4j, Mapfre now has a scalable foundation to continue driving innovation and business impact.

To learn more about Amazon EMR Serverless, see the Amazon EMR Serverless documentation.