All posts by Antoine Boucherie

How Generali Malaysia optimizes operations with Amazon EKS

Post Syndicated from Antoine Boucherie original https://aws.amazon.com/blogs/architecture/how-generali-malaysia-optimizes-operations-with-amazon-eks/

This post is co-authored with Ivan Amemoutou, DevOps and Cloud Lead at Generali Malaysia (“Generali”).

The insurance industry’s shift to cloud computing has accelerated the development and expansion of digital services. To support this transformation, insurers are modernizing their technology stack with solutions that enhance scalability, portability, and operational efficiency. This digital evolution is driven by growing customer expectations for seamless insurance services across all touchpoints. Generali faced this industry-wide challenge head-on, needing both to migrate their legacy applications to the cloud and meet increasing demands for new digital services. To address these needs, they embraced a modern approach by implementing containerized microservices architecture, significantly improving their operational capabilities and service delivery.

Generali started its migration to AWS in 2019. They selected Amazon Elastic Kubernetes Service (Amazon EKS) as the target container service for their modernized applications for its capabilities as an enterprise-grade container management solution and its seamless integration with other AWS services. Previous experience of the Generali DevOps and Cloud team was also a strong factor in selecting Amazon EKS. Although the selection of the target platform was straightforward, the main challenge Generali was facing was to enable the scale of adoption while maintaining a lean operational base.

Today, digital applications and several core insurance solutions are hosted on their EKS clusters, making it an important piece of infrastructure for the company. In this post, we look at how Generali is using Amazon EKS Auto Mode and its integration with other AWS services to enhance performance while reducing operational overhead, optimizing costs, and enhancing security.

Solution overview

Generali strives to implement Amazon EKS best practices and actively align their implementation with the AWS Well-Architected Framework. To that end, they follow the six pillars of Operational Excellence, Security, Reliability, Performance Efficiency, Cost Optimization, and Sustainability to build a robust and scalable platform. By applying Well-Architected principles to their EKS environment, Generali benefits from improved system resilience through automated operations and monitoring, enhanced security through AWS Identity and Access Management (IAM) integration and network policies, optimized costs through right-sizing and automatic scaling, and sustainable practices that minimize their environmental impact while maintaining high performance and reliability.

The following diagram illustrates the architecture of their EKS cluster and some of its integration points with different AWS services.AWS security and monitoring architecture diagram showing integration between Inspection VPC and EKS VPC with multiple AWS services for container workload protection and observability.

This solution offers the following benefits:

  • Simplified management of multiple containerized applications
  • Automated node provisioning and scaling
  • Enhanced security integration
  • Optimized resource utilization and simplified cost management
  • Granular multi-tenant observability

In the following sections, we discuss the integration with AWS services in more detail and how these components align with the AWS Well-Architected Framework.

Operational Excellence, Reliability, and Performance Efficiency with Amazon EKS Auto Mode

Generali faced challenges managing their expanding portfolio of containerized applications. The growth of their containerized services introduced operational inefficiencies and complexities: multiple applications from multiple tenants created operational overhead from manual orchestration and scaling to infrastructure maintenance, making it difficult to optimize costs while enforcing security and compliance across diverse application stacks. These challenges led to over-provisioning of resources and inconsistent security postures across different containerized environments.

To address these pain points, Generali has been adopting Amazon EKS Auto Mode, which automates their cluster infrastructure management, provides production-ready environments with minimal operational overhead, dynamically scales resources based on application demands, and implements consistent security practices with automated upgrades, so their teams can focus on application development rather than infrastructure complexity.

EKS Auto Mode manages the underlying nodes, load balancers, and storage configuration automatically. EKS Auto Mode takes care of scaling the cluster depending on the need of the workloads, while optimizing cost across a set of Amazon Elastic Compute Cloud (Amazon EC2) instances types selected by Generali in the node pools configuration.

With EKS Auto Mode’s expanded Shared Responsibility Model, compared to non-Auto Mode clusters, it also takes care of the patching of the underlying operating system (Bottlerocket), the different Amazon EKS add-ons installed by default, and the upgrade of the cluster, so Generali DevOps and Cloud team can focus on supporting their application teams.

While starting up EKS Auto Mode, the Generali DevOps and Cloud team had to adjust their operations to allow for those new features. For example, EKS Auto Mode releases a new version of its AMI, which automatically upgrades nodes on a regular basis, usually every week. To do so, nodes are terminated to be replaced with upgraded ones. The team had to create disruption control configurations to prevent those disruptions from impacting workloads. For example, they specified a maintenance window during off-peak hours for those upgrades. They also specified Pod Disruption Budgets and Node Disruptions Budgets to make sure critical applications would not see all the pods of a micro-service being terminated at the same time. The team can then focus on monitoring the current services and making sure they stay compliant with upcoming Amazon EKS upgrades, an activity that usually takes a fair amount of time every quarter, which is now automated with EKS Auto Mode.

Finally, the Generali DevOps and Cloud team also follow several principles to maintain reliability of their applications: they only allow stateless micro-services, they treat the underlying pods as immutable, they use Helm chart as a standardize deployment mechanism, and they use Horizontal Pod Autoscaler (HPA) to scale services based on traffic.

Security using Amazon GuardDuty, Amazon Inspector, Amazon Network Firewall, and AWS Secrets Manager

Generali implemented Amazon GuardDuty Extended Threat Detection for their EKS clusters to automatically correlate security signals across Amazon EKS audit logs, runtime behaviors, malware execution, and AWS API activity to identify sophisticated multistage attacks that traditional monitoring approaches often miss. By enabling both Amazon GuardDuty Amazon EKS protection and runtime monitoring, Generali gained comprehensive visibility into complex attack patterns such as container exploitation, privilege escalation, and unauthorized movement within their Kubernetes environment, with detailed timelines mapped to MITRE ATT&CK tactics and techniques. The benefits Generali realizes include reduced investigation time through consolidated security insights, rapid assessment of which containerized infrastructure components require immediate attention, and the ability to prioritize remediation efforts on the most critical affected resources while minimizing the potential blast radius of Amazon EKS targeted attacks.

Generali also uses the new Amazon Inspector capability to map Amazon ECR images to running containers, helping their security teams prioritize vulnerabilities based on containers currently running in their environment rather than just identifying vulnerabilities in repository images. The enhanced service provides Generali with visibility into which container images are actively running across their EKS environments, including cluster Amazon Resource Names (ARNs), the number of EKS pods where images are deployed, and last in-use dates for each vulnerability finding. The key benefits Generali realizes include the ability to prioritize remediation efforts based on actual container usage patterns rather than repository events alone, and comprehensive vulnerability management across container images.

Generali set up AWS Network Firewall to filter outbound HTTPS traffic from applications hosted on their EKS cluster by restricting outbound connections to only a set of hostnames provided by Server Name Indication (SNI) in the allow list, deploying their EKS cluster in private subnets with Network Firewall endpoints in public subnets and NAT gateways in protected subnets. The benefits Generali realizes include enhanced security through egress filtering that monitors and restricts outbound network traffic based on certificate hostnames rather than changing IP addresses, the ability to collect and analyze hostnames accessed by applications through Amazon CloudWatch alert logs for traffic pattern analysis, and improved compliance with security requirements by making sure applications can only access approved external services.

Getting secrets into pods can be done either through environment variables or as mounted volumes. Hard-coding them directly into the deployment template is not recommended, and it is better to store them in AWS Secret Manager and retrieve them dynamically. As a best practice and to reduce operational complexity, Generali choses to only host stateless containers in their cluster, alleviating the need for storage volume. To that end, the best option is to retrieve secrets dynamically and add them as environment variables to the pod. To do so, they implemented the External Secrets Operator on their EKS cluster to use Secrets Manager for centralized secret management, which reads the necessary secrets and automatically stores them as Kubernetes secrets without requiring application code changes or daemonsets. The benefits Generali realizes include improved security, management, and auditability of secret usage through centralized secret management outside their Kubernetes clusters and automatic secret synchronization on a recurring basis to capture credential rotations.

Cost Optimization using tags and Savings Plans

Although EKS Auto Mode already offers some cost optimization features, it’s important for Generali to keep track of resource consumption per business project. To that end, Generali uses AWS Billing split cost allocation data for Amazon EKS to analyze and allocate costs using the AWS Billing Console, gaining insights into Kubernetes costs alongside other AWS spend. The feature allows for split along cost allocation tags for some Kubernetes attributes. These tags include aws:eks:cluster-name, aws:eks:deployment, aws:eks:namespace, and aws:eks:node, so the company can map Amazon EKS consumption against lines of business and applications.

Generali also takes advantage of the following:

Operational Excellence and observability using custom dashboards in Amazon Managed Grafana

Hosting multiple projects from multiple business unit means that different application owners need their own custom analytics dashboards. To provide per-project granularity, Generali uses the integration between CloudWatch and Amazon Managed Grafana to create observability dashboards per EKS namespace. By connecting CloudWatch as a data source in Amazon Managed Grafana, they can visualize Amazon EKS metrics, logs, and traces through Grafana’s powerful visualization capabilities without managing the underlying Grafana infrastructure. Through this integration, Generali can create unified views of cluster health, node performance, pod resource utilization, and application performance indicators, while using Grafana’s advanced alerting and templating features for dynamic dashboard creation.

Lessons learned

Generali’s adoption of EKS Auto Mode, combined with integrated AWS security services and comprehensive observability tools, has transformed their container operations from a complex, manually managed environment to an automated, secure, and efficient platform. The integration with services like GuardDuty, Amazon CloudWatch Container Insights, and Amazon Managed Grafana has created a cohesive ecosystem that maximizes operational efficiency while minimizing management overhead. This transformation has helped the Generali DevOps and Cloud team shift its focus from infrastructure maintenance to strategic application support, resulting in improved security posture, cost optimization, and overall platform reliability.Generali realized the following key benefits:

  • Significant reduction in operational overhead with EKS Auto Mode
  • Enhanced security with automated threat detection and response
  • Reduction in infrastructure costs through optimization
  • Improved mean-time-to-resolution
  • Accelerated application deployment cycles

Conclusion

Amazon EKS Auto Mode has proven to be a transformative service for Generali, helping them build a modern, secure, and efficient container environment that aligns with AWS Well-Architected best practices. With EKS Auto Mode and its integration with AWS services like GuardDuty, Amazon Inspector, and CloudWatch, Generali created a robust foundation that not only enhances their security posture and operational efficiency but also optimizes costs. The Generali DevOps and Cloud team is now able to focus on applications teams’ support with expansion plans to host AI models and upcoming agentic applications.As organizations continue their cloud-based journey, Generali’s experience demonstrates how AWS’s comprehensive container services can help enterprises focus on innovation and business value while maintaining operational excellence, security, and cost-efficiency at scale.

If you’re interested in learning more about Amazon EKS, refer to Amazon EKS Best Practices Guide.

About Generali Malaysia

Generali Malaysia is one of the largest general insurers and an emerging life insurer in the country, dedicated to delivering best in class general and life insurance protection solutions for individuals, families, and businesses. As part of the Generali Group, a global insurance leader with over 190 years of heritage, Generali Malaysia carries forward a deep legacy of protection, service excellence, and innovation.

Today, the company is supported by more than 1,600 employees, over 9,000 agents and partners, and an extensive network of branches nationwide. Guided by its ambition to be a trusted Lifetime Partner, Generali Malaysia is committed to its purpose of empowering lives and dreams. The company continues to drive excellence by leveraging AI, data, and customer centric solutions, while embedding sustainability at the heart of its business.

About the authors

Pilot light with reserved capacity: How to optimize DR cost using On-Demand Capacity Reservations

Post Syndicated from Antoine Boucherie original https://aws.amazon.com/blogs/architecture/pilot-light-with-reserved-capacity-how-to-optimize-dr-cost-using-on-demand-capacity-reservations/

For digital enterprises to remain competitive, resilience is essential for maintaining reliability and building customer trust. End users expect applications to be available 24 hours a day, leading companies to develop increasingly sophisticated methods to provide continuous operation of critical services. Some companies, such as financial services companies, have to meet regulatory requirements such as Digital Operational Resilience Act (DORA) and are expected to manage the risk of outsourcing critical applications. They must design for high availability and plan for potential impairments. By proactively planning for potential disruptions, they’re not just mitigating risks – they’re building trust and delivering unparalleled value to their customers.

When assessing your own applications, you should define a set of objectives, perform a business impact analysis and a risk assessment. This way, you can estimate the impact to the business if the application isn’t available. This results in categorization of the applications and influence their design according to the AWS resilience lifecycle framework. Each application is given a specific Recovery Point Objective (RPO) and Recovery Time Objective (RTO), depending on its criticality for the business.

Not all applications fall in the most critical category. You allocate resources according to the results of the assessment and make trade-offs when designing applications. For example, you’ll have a more stringent RTO and RPO for—and be willing to spend more time and money on—a critical application than on a less critical application. The challenge becomes how to minimize the risk of breaching a specific RTO while optimizing for resources, such as cost and operational complexity.

At AWS, we provide guidance through the Well-Architected Framework and specifically within the Reliability pillar. Disruption can happen at several levels, and we recommend that you explore and prepare for four types of disruptions in the AWS Resilience Hub: application, infrastructure, Availability Zone, and AWS Region.

We recommend that you use managed services and make sure that all production workloads are designed to take advantage of multiple Availability Zones in AWS Regions. If your application also needs to be protected against the unlikely risk of Regional impairment, you should consider a multi-Region disaster recovery (DR) strategy.

You can select from several DR strategies: backup and restore, pilot light, warm standby, and multi-site active-active:

  • Backup and restore – This strategy might not provide the necessary RPO or RTO required for a highly critical application.
  • Multi-site active-active – This strategy increases significantly the cost and operational complexity of your application.
  • Pilot light – This strategy allows for a RPO or RTO in the tens of minutes by having the data asynchronously copied to the secondary Region and ready to be accessed. However, unlike a warm standby, the application servers aren’t deployed and aren’t ready to serve traffic. The pilot light strategy allows for a lower cost but brings a risk that you might not be able to provision the compute capacity you need when you want to fail over to the secondary Region, especially if you require a specific instance type.

In this post, we explore an intermediate strategy between the pilot light and the warm standby strategies: pilot light with reserved capacity. You can use this strategy to reserve compute capacity in a secondary Region while also limiting cost.

The following diagram illustrates where the pilot light with reserved capacity solution lies in the spectrum of disaster recovery strategies.

spectrum of disaster recovery strategies

Reserving capacity, on demand

On-Demand Capacity Reservations were launched in 2018. They make it possible to reserve capacity in the Availability Zone of your choice without a long-term contract. You have the flexibility to create, modify, or cancel reservations at your discretion. It’s especially well-suited if your application is dependent on a specific instance type or size.

Optimizing the cost of On-Demand Capacity Reservations with a Savings Plan

On-Demand Capacity Reservations is a reservation mechanism and doesn’t require a commitment. However, you can optimize your spending by combining the capacity reservation with an AWS Savings Plan. By using Savings Plans, you can achieve up to a 72% discount, a very significant cost reduction for DR instances that have to stay available all year long.

Optimizing the cost of On-Demand Capacity Reservations by sharing Capacity Reservations

To further optimize the cost, you can use your reserved capacity in another account when you don’t need it for DR.

Here’s an example in which we share On-Demand Capacity Reservations with our development and test account:

We have a three-tier application running in production in a primary AWS Region. This application is composed of a load balancer forwarding traffic to a fleet of application servers running on Amazon Elastic Compute Cloud (Amazon EC2) instances, backed by an Amazon Relational Database Service (Amazon RDS) database. All services used by this application are configured to use multiple Availability Zones in this primary Region.

We use the pilot light strategy, so the application data is being replicated to the disaster recovery environment in a secondary Region using Amazon RDS cross-Region read replicas. However, the load balancer and EC2 services aren’t running in DR to limit cost and operational complexity. Following best practices, each environment is running in a different AWS account.

The following diagram illustrates the pilot light strategy setup for our example.

pilot light strategy setup for our example

To reserve capacity in case of failover to the secondary Region, we create an On-Demand Capacity Reservation in the DR account, according to our baseline compute capacity. Because we don’t need this capacity until we fail over the application from the primary to the secondary Region, we share those On-Demand Capacity Reservations with a development and test account hosting our nonproduction environment in the secondary Region. On-Demand Capacity Reservations are Availability Zone specific (and hence Region specific) and can be shared with either AWS accounts or AWS Organizations using AWS Resource Access Manager (AWS RAM).

Best practices are to share those On-Demand Capacity Reservations with a nonproduction organizational unit (OU) within an organization or to directly share with the account(s) hosting the testing environments (for example, user acceptance testing or preproduction). Those environments are usually very similar to the production account in baseline sizing, in order to perform load and performance testing. This is an important point: you want to be able to retrieve those On-Demand Capacity Reservations when needed without impacting other critical applications.

The following diagram illustrates the Capacity Reservations sharing with the development and test account.

Capacity Reservation sharing with development and test account

If an impairment affects our production environment in the primary Region, we can trigger failover to the secondary Region. To reclaim capacity, we need to terminate the EC2 instances running in our development and test account. Capacity becomes available nearly immediately after these instances are successfully terminated. Separately, we can also stop the sharing of On-Demand Capacity Reservations to make sure that the development and test account can’t consume that capacity again. Know that merely unsharing your reservation without terminating development and test instances might not result in complete or immediate capacity retrieval. This is because when you unshare an On-Demand Capacity Reservation, the instances in the consumer account continue to run, and capacity is only returned to the owner account if additional capacity is available in the Amazon EC2 service on-demand pool.

The following diagram illustrates the failover to the DR environment in a secondary Region.

cancellation of the capacity reservations sharing

Steps

Here is a possible approach to take advantage of On-Demand Capacity Reservations to reduce the application’s total infrastructure cost:

  1. Calculate the baseline compute capacity necessary for the DR environment in the secondary Region in the event of failover, including the compute that might already be running in this secondary Region for data stores (for example, a Kafka broker running on Amazon EC2). How much vCPU and RAM is required or what are the exact EC2 instances necessary to host the whole application in case of failover of the production from the primary to the secondary Region.
  2. Create an On-Demand Capacity Reservation for the exact EC2 instances that the application need as a baseline in the DR account. Capacity Reservation Fleet is also a possible choice to reserve capacity across multiple instance types, which is often the case for Amazon Elastic Kubernetes Service (Amazon EKS) or Amazon Elastic Container Service (Amazon ECS) clusters, for example. Creating a Capacity Reservation Fleet will create multiple Capacity Reservations that can be shared independently. It’s also recommended to apply for Savings Plans on those On-Demand Capacity Reservations to save up to 72%.
  3. Share those On-Demand Capacity Reservations from the DR account to one or several accounts, depending on your need. In our example, we share the On-Demand Capacity Reservations with the development and test account, effectively allowing the development and test environment to use compute capacity that has already been reserved.
  4. In case of impairment in the primary Region, terminate the development and test instances first and then stop the On-Demand Capacity Reservation sharing The DR account will recover those reservations. If you want to keep the development and test instances, you will be charged at the on-demand rate.
  5. Redeploy in an automated manner the application in the DR account on new EC2 instances behind a load balancer.

Benefits

By purchasing On-Demand Capacity Reservations in the DR account, you make sure that you always have Amazon EC2 capacity access when required and for as long as you need it. By sharing those On-Demand Capacity Reservations with another AWS account or organization, you can share the cost of the application’s compute capacity with other environments, reducing your application’s total cost of ownership. The additional cost of the DR instances can even reach zero, if your instances are completely consumed by nonproduction environments such as development and testing.

DR savings over On-Demand
Compute Savings Plan – 1 year, no upfront Around 27% (for example, for m7i instance)
Compute Savings Plan – 3 years, all upfront Up to 66%
Instance Savings Plan – 3 years, all upfront Up to 72%
Reservation shared and consumed 100% by development and test environment Up to 100%

Limits

Although you can reserve DR capacity at a minimal cost using the pilot light with reserved capacity solution, there are some limits to keep in mind.

Firstly, we advise looking at this solution only if the Recovery Time Objective of the application, in case of Regional disruption, is in hours because you need to take into account the time needed to:

  • Detect the impairment in the primary Region.
  • Trigger the failover procedure.
  • Terminate the used instances to retrieve capacity (estimated time in minutes)
  • Stop the On-Demand Capacity Reservations sharing and automatically retrieve them in the DR account (estimated time in minutes).
  • Launch the compute infrastructure with the necessary application software in the DR account. You need to make sure that it matches the On-Demand Capacity Reservations according to the criteria used (open or targeted)

If your application requires a lower RTO, we recommend exploring the warm standby strategy.

Secondly, this strategy can only be used for application servers running EC2 instances and ECS or EKS clusters on EC2 because On-Demand Capacity Reservations aren’t available for managed services such as AWS Fargate or AWS Lambda. For those managed services, we recommend having them up and running like in a warm standby strategy, with a minimum baseline capacity that you’re comfortable with.

Thirdly, it requires some nonproduction development and test usage in the selected secondary Region to use the shared On-Demand Capacity Reservation.

Finally, it’s important to consider that this solution brings some complexity and extra operational work. You should plan well ahead, automate the operational tasks where possible, but most importantly, regularly test that the failover of the application works according to plan. We encourage you to perform your own game days to support your operational resilience.

Deciding whether this strategy is a good fit for your application will ultimately be a decision based on your business and regulatory requirements.

Conclusion

In this post, we explained how to reserve capacity in a secondary Region using On-Demand Capacity Reservations. We highlighted how cost can be optimized using Savings Plans and by sharing reserved capacity with noncritical workloads. We saw how we can recover that capacity for the DR environment, in the event of a disaster, to allow the application to continue to serve end users. We looked at the benefits and limits of the pilot light with reserved capacity solution and the necessary steps to put it in place.

About the Authors