Post Syndicated from Tim Kopacz original https://aws.amazon.com/blogs/big-data/how-swiss-life-germany-automated-data-governance-and-collaboration-with-amazon-sagemaker/

Data has become an indispensable strategic asset for the entire financial services industry, driving innovation and competitive advantage in an increasingly digital marketplace. At Swiss Life Germany, maximizing the value of this asset means empowering internal teams to derive actionable insights and deliver personalized financial solutions to diverse clientele. This led to the need to establish seamless data sharing workflows that enhance cross-departmental collaboration while maintaining strict security and compliance standards. To accomplish this, Swiss Life Germany decided to implement advanced data processing and governance capabilities using Amazon SageMaker.

Integrating SageMaker into a highly regulated enterprise environment required aligning the service’s agility with Swiss Life’s rigorous infrastructure as code (IaC) automation standards. This post demonstrates how Swiss Life Germany addressed these sophisticated deployment requirements by developing a custom Terraform pattern designed specifically for platform engineers and data architects.

Swiss Life Germany cloud journey

Swiss Life Germany is a leading provider of customized pension products and financial advice. Building on over 100 years of delivering insurance, retirement planning, and wealth management solutions, a key driver of the company’s recent evolution was the strategic transition from legacy on-premises data centers to a modern, cloud-centric architecture. After an extensive evaluation of various providers, Swiss Life Germany selected Amazon Web Services (AWS) as the strategic foundation to modernize their data operations. By using AWS, the organization was able to transition from capital-intensive data centers to a flexible pay-as-you-go model, significantly reducing the operational costs.

Following their comprehensive AWS cloud migration over the last two years—combining 30% re-platforming with 70% lift-and-shift strategies—Swiss Life Germany modernized infrastructure management through IaC. The company introduced the governance concept of an IT System. An IT System is a fundamental unit of management that defines a software component regardless of its origin. Whether a component is purchased from a vendor, self-developed or consumed as software as a service (SaaS), it’s integrated into this single governance structure. This ensures that off-the-shelf products and custom-coded applications are held to the same high standards of visibility and accountability. Every IT system is required to maintain specific attributes that allow for seamless oversight such as unique identifiers, assigned ownership and the associated AWS resources logically grouped under the IT System they support.

Where traditional approaches would store and expose this information in configuration management database (CMDB)-like systems to store static snapshots of asset data, Swiss Life adopted a more dynamic model. By using GraphQL API as a unified meta-model, the company queries application data directly from its primary source systems. This approach eliminates the delays common in batch-processed databases, ensuring maximum freshness. The API serves as a single entry point for infrastructure data, documentation, organizational metadata, and even inter-application dependencies. The transparency and automation gained through this everything-as-code and API-first approach provided a blueprint for the Swiss Life Data Platform: complete transparency, reproducibility, and end-to-end automation.

This robust technical foundation served as a catalyst and prerequisite for Swiss Life’s broader strategic goals and governed framework.

Defining the vision for a unified data solution

With the architectural foundations in place, the next challenge was to establish efficient data flows from production systems through data engineering teams to end users across various business divisions, with hundreds of specific use cases demanding attention.

For instance, Swiss Life’s customer portal specialists had to validate the effectiveness of campaign management and push notification systems in real-time, requiring secure and immediate access to interaction data.

Security requirements added another layer of complexity, because Swiss Life’s solution needed to incorporate robust compliance standards including two-factor authentication, session-based access controls, and granular row and column-level security protections.

To align with the overarching Swiss Life Germany cloud strategy, the company aimed to build a modern data solution atop their existing AWS data and analytics services. AWS introduced SageMaker to Swiss Life Germany following its announcement at AWS re:Invent 2024. A proof-of-concept quickly validated that this was the right tool to advance Swiss Life’s data journey. By deploying a fully automated framework, Swiss Life Germany sought to create a secure, compliant framework with SageMaker democratizing data access for authorized users, ultimately enabling faster business insights and more responsive customer experiences across the entire data environment.

Having met the infrastructure requirements, let’s look at what SageMaker looks like for end users and how data platform administrators can control access and resources at a granular level.

Users and their types of projects

A typical end user experience within Amazon SageMaker Unified Studio starts with creating a project. A project is a logical boundary within a domain where the data teams can collaborate and work on a business use case. Administrators would provision the blueprints and project profile templates for the data teams, as shown in the following figure.

However, at Swiss Life, they have extended the data platform administrator’s role to also create projects so they can maintain regulatory compliance and remove initial onboarding hurdles. The end user experience in SageMaker Unified Studio is simplified with data teams selecting their respective projects to work on a business initiative, as shown in the following figure.

To implement this solution effectively, Swiss Life identified different user groups:

• A solution team developing an IT System that can act as producer or consumer of data assets.
• A data scientist doing advanced data processing. They will most likely consume a lot of data assets and might produce some high aggregated data assets. The data processing software is also categorized as an IT System.
• Business users who have some SQL skills and want to process data to get insights for their daily business.
• A platform team administering the data platform. They provide core services to all users to make participation as straightforward as possible.
• A data officer who wants to have a single point of interpretation for data.

Given this diverse set of user groups, the resulting data platform had to support a federated data organization with a centralized governance, decentralized data stores and data-processing organized at the IT System level. This architecture means the SageMaker management account—which orchestrates the data domain—contains no actual data, instead, data and compute resources reside in the individual IT System AWS accounts. Swiss Life’s implementation distinguishes between two fundamental project types:

• IT System projects (for technical users)
• Team projects (for non-technical users)

Swiss Life decided to align team projects with specific organizational units and operate them without staging environments, providing dedicated workspaces for departmental data initiatives. In contrast, IT System projects are associated with specific solutions such as customer portal or CRM systems. These follow a structured staging methodology, with each solution team managing dedicated DEV, TEST, and PROD environments to maintain proper development lifecycles and quality control.

This federated architecture is designed to handle the immense scale and diversity of Swiss Life’s data landscape. Swiss Life’s data platform would then aim to provide unified access to over 180 database servers with over 1,800 databases and 18 thousand tables across all stages (DEV, TEST and PROD).

In this post, we focus on the IT System projects.

How Swiss Life built the automation framework

Because Terraform is the preferred IaC tool across Swiss Life Germany, the team faced an interesting architectural challenge: while the existing infrastructure framework incorporates numerous AWS services that are readily supported by Terraform, SageMaker required a custom integration approach to align with Swiss Life’s advanced automation patterns.

Rather than adopting a manual ClickOps approach to infrastructure management, Swiss Life developed an innovative solution to keep the entire infrastructure—including SageMaker—within their Terraform automation, preserving key benefits like state management. The team accomplished this by using Terraform’s AWS Lambda invoke function resource with a create, read, update, delete (CRUD) lifecycle scope. By using this approach, the organization could maintain a single source of truth for infrastructure, while accommodating specific requirements of SageMaker. This component is called the Management Lambda and it serves as a bridge between Terraform’s declarative configuration and SageMaker, so that Swiss Life can provision, modify, and decommission Amazon SageMaker resources through established Terraform workflows.

The following is the snippet of a new domain creation using Terraform and Management Lambda:

resource"aws_lambda_invocation" "domain" {
  function_name = "management-lambda-function-name"
  lifecycle_scope = "CRUD"
  input = jsonencode({
  resource = "domain"
  domain_name = "SwissLife"
  domain_execution_role = "arn:aws:iam::012345678912:role/sus_domain_execution_role"
  domain_service_role = "arn:aws:iam::012345678912:role/sus_service_role"
  })
}

Using this approach, Swiss Life successfully automated every aspect of deploying a complete SageMaker domain installation within the Swiss Life cloud data platform. The automation encompasses the entire domain creation process, using the SageMaker domain unit feature as an organizational framework for diverse project portfolio.

Deployment architecture

Let’s dive deeper into the individual steps of the automation process itself. As said, all resources within SageMaker are controlled by the Terraform-invoked Management Lambda whereas other resources are directly managed by Terraform itself. The Management Lambda and SageMaker resources such as domains, metadata fields and others live in the central SageMaker account. Users of the data platform have their own AWS accounts. To start with, AWS Lake Formation had to be enabled across all AWS accounts, which could then act as consumer or provider to the platform. Using the established AWS Landing Zones mechanism, this was done by a single deployment to the management account. This early step also verified the management role being present in all accounts and assumable by the Management Lambda.

The following steps are used to set up Swiss Life’s data platform from scratch, as shown in the following diagram:

1. The Management Lambda is deployed to Swiss Life’s designated SageMaker account. This Lambda function uses the described CRUD pattern for all subsequent SageMaker-specific operations.
2. The domain provisioning begins by creating the service and domain execution roles, after which the Management Lambda creates the domain and uses these roles. During this step, administrative users and their associated permissions are also configured.
3. Upon successful domain creation, the Lambda function returns the domain identifier as output. This identifier is then used to let all AWS accounts of the company join this domain. These can now act as providers or consumers on the platform, resulting in a frictionless onboarding of teams.
4. Because Swiss Life decided to stage data products in a single domain, the DEV, TEST, and PROD domain units are then created, establishing the hierarchical structure under which IT System projects are subsequently created in the next implementation phase.

All projects and teams with the necessary prerequisites set up are then created automatically. This is done by using the enterprise GraphQL API mentioned to retrieve all IT products, their teams and roles. With that, each team already has their ready-to-use project in place upon singing into the platform. In detail this process looks like the following:

Continuing with the earlier example: the customer portal team needs to share their data with others in the organization and is using their dedicated project for this purpose. The process is shown in the following figure.

1. The deployment initiates with a cross-account role assumption by the Management Lambda to activate the blueprint configuration in the team’s AWS account. A standardized creation process was built to help facilitate all accounts are configured identically, maintaining consistency across the environment.
2. Next, a project profile specifically tailored for the customer portal project is created. This profile establishes the foundational settings and permissions framework that will govern the project’s operations.
3. With the profile in place, the actual project within this previously established project profile can now be provisioned, instantiating the working environment, where data sharing and collaboration will occur. This results in an identical amount of project profiles and projects in the SageMaker Unified Studio domain.
4. Finally, an automated membership management process is triggered. The system again queries Swiss Life’s Enterprise GraphQL API to identify all members of the solution team and automatically adds them as project members with appropriate permissions. This process executes daily, to help ensure that project access permissions remain current and accurately reflect team composition changes.

In the third and final deployment step, the user experience is enhanced by making the data platform immediately usable for teams in production. When teams and their members first access the domain URL, they find a project environment already populated with all necessary assets, so they can begin working without delay. This is accomplished through the following steps, shown in the following figure:

1. An automated discovery process is triggered that identifies all Amazon Simple Storage Service (Amazon S3) buckets and AWS Glue assets associated with the specific customer portal IT System. This inventory is created by using the AWS Resource Tagging API with specific filters targeting these asset types, so that all relevant resources for exactly that IT System are captured.
2. When identified, all discovered S3 buckets are registered as data lake locations within the platform. For each location, they create an AWS Identity and Access Management (IAM) role with precise access permissions, adhering to the least privilege security model.
3. Then grantable permissions are granted to the SageMaker project role for these assets, establishing a permission delegation framework that allows project members to manage access within their project scope—managing cross project access—while maintaining overall governance.
4. Finally, the AWS Glue databases are added as data sources within the project. These data sources are configured with daily synchronization schedules to automatically load new metadata into SageMaker, helping to ensure that catalog information remains current without manual intervention.

What a team needs to start with all of this

The overarching goal throughout this implementation has been to simplify the adoption process for the internal data teams. To ensure the data teams could immediately use the powerful capabilities of SageMaker without needing to manage its underlying architecture, Swiss Life Germany streamlined the experience by pre-packing the entire onboarding process into a high-level Terraform module. Teams can then use the module to deploy a complete, production-ready environment with minimal configuration, accelerating their path from setup to insight.

The following is an example of the code used by the module.

module "membership" {
	source = "<git-source>"
	it_system_labels = ["kundenportal"]
	domain_name = "SwissLife"
	vpc_id = "vpc_id"
	subnet_ids = ["subnet_a", "subnet_b", "subnet_c"]
}

To initiate this, the data teams define their basic parameters such as network configuration or their IT-System identifier as outlined previously and submit a pull request in the central Git repository. After the Swiss Life data platform team reviews and approves the request, the automated processes run in the background, preparing the complete environment. This automated approach has reduced deployment time for new environments from several weeks of manual coordination to under 20 minutes.

Rather than requiring users to understand the intricate deployment steps and managing the infrastructure, the automated deployment process empowers business units, like the customer portal team, to focus on deriving insights. At the same time, the Swiss Life Germany data platform team also maintains precise control over resource allocations, access rights and cost management.

Future enhancements

Looking ahead, Swiss Life plans to elevate its automation to a higher level of business abstraction. The next major enhancement focuses on removing the requirement for teams to request specific technical assets. Instead, the vision is to implement an intuitive interface where teams can specify the business terms or data domains they require. The system will automatically identify and provision the correct underlying technical assets associated with those business definitions.

This semantic layer will create a more natural interaction model, so that business users can think and work in familiar concepts rather than technical constructs. For example, rather than requesting access to specific S3 buckets or AWS Glue databases, a marketing analyst might indicate they need customer interaction data or campaign response metrics. An automated system will then map these business terms to the appropriate technical resources, provision access, and configure the environment accordingly.

By elevating automation to this business terminology level, Swiss Life aims to further reduce friction in the data access process while maintaining its robust security and governance framework. This evolution represents Swiss Life Germany’s commitment to continuously improving how data serves the business, making sophisticated data capabilities increasingly accessible to all parts of the organization.

Conclusion

Through the comprehensive automation of Amazon SageMaker, Swiss Life Germany has transformed their usage of data from a complex technical challenge into a streamlined business enabler. By using AWS services and their innovative Terraform-Lambda integration approach, Swiss Life created a secure, compliant data platform that maintains governance while democratizing access across the full organization. The automated deployment process helps ensure consistency across environments while dramatically reducing the technical knowledge required for teams to begin using advanced data capabilities. Business units, such as the customer portal team, can now focus on deriving insights rather than managing infrastructure, accelerating data-driven decision making throughout the company. This implementation represents a significant milestone in Swiss Life Germany’s cloud journey, demonstrating how thoughtful automation can simultaneously enhance security, improve operational efficiency, and accelerate business outcomes.

As of today, 5 organizational unit teams and 15 IT System teams were onboarded to the platform. To speed things up, Swiss Life has decided to onboard all 180 database clusters and consume data using SageMaker over the coming months. This expansion is designed to enable teams to use the data platform and enhance the efficiency of data discovery and data sharing processes across the organization.

About the authors