Tag Archives: AWS Server Migration Service

Field Notes: Choosing a Rehost Migration Tool – CloudEndure or AWS SMS

Post Syndicated from Ebrahim (EB) Khiyami original https://aws.amazon.com/blogs/architecture/field-notes-choosing-a-rehost-migration-tool-cloudendure-or-aws-sms/

Field Notes provides hands-on technical guidance from AWS Solutions Architects, consultants, and technical account managers, based on their experiences in the field solving real-world business problems for customers.

For customers that choose a rehost migration strategy (also known as lift-and-shift) for their large-scale workloads, the recommendation is to select a tool that will orchestrate, automate and schedule the migration with minimum downtime. CloudEndure Migration and AWS Server Migration Service (AWS SMS) are two services provided by AWS that both automate rehost migration, but customers often struggle determining which service is a best fit for their migration use case.

CloudEndure Migration is a block-level replication tool that simplifies the process of migrating applications from physical, virtual, and cloud-based servers to AWS.

CloudEndure Migration

Figure 1 – CloudEndure Migration

AWS Server Migration Service is an agentless migration service to migrate on-premises virtual machines to AWS using virtual appliance.

AWS Server Migration Service

Figure 2 – AWS Server Migration Service

In this post, I provide some considerations and patterns where it’s recommended based on your migration requirements to choose one tool over the other.

What type of infrastructure are you migrating?

First, consider your source infrastructure:

CloudEndure Migration supports any source infrastructure as long as it runs on x86 operating systems supported by Amazon Elastic Cloud Compute (EC2). This includes physical servers, P2V (virtual servers converted from physical), VMware, Hyper-V, and other cloud providers like Azure, GCP, IBM, or Oracle (see  Supported Operating Systems). Be aware of some limits that are specific to some the operating systems. For example, dynamic type Windows disks that have GPT partitions are not supported. For more information, see How do I determine if my disks are GPT or dynamic?

AWS SMS supports migrations of virtual machines from VMware, Hyper-V or Microsoft Azure. AWS SMS doesn’t currently support physical infrastructure or other public cloud provider except Microsoft Azure. For a full list of supported operating systems, see Operating Systems Supported by AWS SMS.

If your source environment includes bare metal servers, and you can install agents (more on agents in the next section), the recommendation is to use CloudEndure Migration. You can use a combination of CloudEndure Migration and AWS SMS in an environment that has a mix of physical and virtual servers, but this approach adds unnecessary complexity to your operation– you must maintain both as opposed to having a unified solution that works for all sources.

Do you require agent or agentless solution?

CloudEndure Migration requires that you install CloudEndure Agent individually on each server or virtual machine with root/admin access. CloudEndure Agent is a lightweight software (utilizes approximately 5% CPU and 250MB of RAM) and non-disruptive. CloudEndure Agent performs an initial block-level read for the content of the volumes attached to the server and replicates it to the replication server. After the initial sync is complete, the agent then captures only the writes and synchronizes any block-level modification to the replication server. For handshake, monitoring, and management purposes, the agent requires egress access on TCP port 443 with the CloudEndure User Console. For communicating with the replication server in your account in AWS, it requires egress access on TCP port 1500. For list of networking requirements, see Networking and Ports.

AWS SMS doesn’t require an agent on each VM. Instead, it uses the Server Migration Connector (SMS Connector), which is a pre-configured FreeBSD virtual machine that you install on your on-premises virtualization environment, on the hypervisor level, before the migration can start. The format of the SMS Connector varies based on your source environment (see Install the Server Migration Connector for types). The SMS Connector is responsible for taking snapshots, sending them to AWS SMS, and converting them to an Amazon Machine Image (AMI).

In some situations you can’t meet some of the agent requirements. For example, you may not be able to have admin access on source machines to install the agent, or maybe you just prefer an agentless solution that requires less administration tasks on the operating system level. In such cases, assuming you have the necessary access to the hypervisor (or Azure Portal), AWS SMS is a better fit.

How frequently do you require source environment to be replicated?

With CloudEndure Migration, once the agent is installed and activated, it begins initial replication, reading all of the data on source machines at the block level and replicating it to a staging environment in the customer’s individual account in their preferred target region on AWS. The initial replication can take anywhere from several minutes to several days, depending on the amount of data to be replicated and the bandwidth available between the source and target infrastructure. After the initial replication is complete, the source machines are continuously monitored to ensure constant synchronization, up to the last second. Any changes to source machines are continuously and asynchronously replicated into the “staging area” in the target infrastructure. The replication at this point enters a Continuous Data Replication (CDR) state.

The block-level replication can be performed on any type of disk as long as it’s presented to CloudEndure Migration as a block device. This includes physical disks, disks hosted on Storage Area Network (SAN), VMDK, VHD and iSCSI.

The continuous, asynchronous, block-level replication maintains the data on AWS up-to-date with sub-second latency, and that enables you to achieve a cutover time between 5-20 minutes, depending on the operating systems and network latency. When the replication reaches the CDR state, you can launch a target machine in Test mode. Once you complete all your testing and are ready to fully transition your machine to the target cloud, you can perform the Cutover mode action. This process is fully automated. The following image shows the steps CloudEndure Migration performs during the conversion. For more details, see Performing a Migration Cutover.

 

CloudEndure Target Machine Launch

CloudEndure Target Machine Launch

Figure 3 – CloudEndure Target Machine Launch

The block-level replication and CDR make CloudEndure Migration a better fit for use cases where you require near real-time replication (i.e., the data in the destination environment is replicated sub-seconds behind the source environment.)

AWS SMS replication is done by taking snapshots of source machines. After you configure the SMS Connector in the virtualized environment and you create the replication job, the replication job starts and performs the following steps:

  1. Take a snapshot for the source VM
  2. Export VM to VMDK/VHD file for vCenter/Hyper-V respectively
  3. Upload VMDK /VHD to the S3 bucket
  4. Clean the snapshot
  5. Convert VMDK/VHD file to EBS Snapshot using VM import/export
  6. Delete the VMDK/VHD file in the S3 bucket
  7. Create an AMI

These steps are fully automated and are repeated for each recurrence of the replication job. Since AWS SMS uses snapshot-based replication, the subsequent snapshot is taken incrementally and, unlike CloudEndure Migration, there is no need for a staging area to receive the replication data.

Currently, the minimum time between reach replication in SMS is 1-hour, and the maximum is 24-hours. Using this setup, you still can build the migrated environment in minutes, but the migrated environment is at least 1-hour behind the source environment. If your migration requirements tolerate the 1-hour window, then AWS SMS can be a good fit, otherwise you should use CloudEndure Migration.

Do you require migration to be managed entirely from AWS Console?

Currently, CloudEndure Migration is a SaaS offering that you administer in the CloudEndure Console outside of the AWS Management Console, so you need two separate accounts, an AWS account and a CloudEndure account.

CloudEndure Console

Figure 4 – CloudEndure Console

AWS SMS is a managed AWS service, so you operate it entirely from inside your AWS account without having to use an external portal or additional accounts.

If you have such a requirement to operate centrally from within AWS Console, currently, AWS SMS is the right choice.

What level of customization do you require after launching target environment?

In many cases, customers run discovery tools (like TSO Logic) to gather information about on-premises data centers. This information includes server utilizations, dependency mapping, or Total Cost of Ownership (TCO) estimate. Most discovery tools provide rightsizing for on-premises servers and recommend instance types and configurations when migrating these servers to Amazon EC2 on AWS. Both CloudEndure Migration and AWS SMS include these capabilities in different ways.

CloudEndure Migration uses Blueprint, which is a set of instructions on how to launch a target machine for a selected source machine. The Blueprint settings serve as the base settings for the creation of the Target machine. Using Blueprint, you can customize EC2 properties like: machine type, launch type, subnet, security group, private or public IP, disks type (standard, SSD or provisioned SSD) and others. CloudEndure Migration can also do the rightsizing by launching target machines that best match the current hardware (RAM, CPU, Cores) on source machine.

 

CloudEndure Blueprint Configurations

Figure 5 – CloudEndure Blueprint Configurations

In some cases, you may want to import the recommendations from a discovery tool without having to customize the blueprint manually. This is also supported by CloudEndure Migration using the mass blueprint setter script which takes the inventory and rightsizing configurations in csv format and imports them into the blueprint configurations using CloudEndure API. The configuration is then used to launch target machines. For more details, see CloudEndure API.

AWS SMS supports similar functionality when choosing Application Migration. Whereas server migration is accomplished by replicating a single server as an Amazon Machine Image (AMI), application migration replicates all of the servers in an application as AMIs and generates an AWS CloudFormation template to launch them in a coordinated fashion.

The following steps launch and migrate an application:

  1. Create a new application (i.e. SharePoint)
  2. Add servers to the application (i.e. database and application servers)
  3. Configure replication settings (i.e. license type)
  4. Configure launch settings: this defines how the application will be launched on EC2 instances. Some of the configurations you can specify: order of launch (launch database server first and then app server), instance type, VPC, subnet, security group, whether the application is publicly accessible or not and, optionally, you could add post launch scripts.

AWS SMS Target Instance Configurations

Figure 6 – AWS SMS Target Instance Configurations

From there, you start the replication and launch the application as per the configuration above. AWS SMS generates a CloudFormation template for the application settings that you can download, edit, and use for later launches. See also Migrating a SharePoint application using the AWS Server Migration Service on the AWS Compute Blog.

If you have a requirement to import your discovery and right-sizing data and use them to launch your target machines, then CloudEndure Migration is the right tool. If you have a requirement for generating a reusable CloudFormation template of migrated environment, then AWS SMS is the right tool.

Cost

You can use both CloudEndure Migration and AWS SMS at no charge. You are only charged for the resources each tool creates in your account to complete the migration.

Additional charges for each tool are as follows:

CloudEndure Migration uses a staging area to receive replication data. The staging area contains low-cost resources to keep your cost minimum. By default, CloudEndure Migration uses the t3.small EC2 instance type for replication server and EBS magnetic for its attached volumes as long as the volume is less than 500 GB in size. There is a one-to-one mapping between each EBS volume attached to the replication server, and a disk attached to a server in your source environment. You should also consider that depending on the number of disks you replicate from your source, you may need to have more than one replication server. The typical ratio of volumes to replication server is 15:1.

CloudEndure Migration also incurs charges for storing the snapshot it takes for each volume. There are 5-7 internal snapshots needed to replicate each disk. Frequency and exact number depend on various factors, such as change rate on the Source machine and network stability. Snapshots that are no longer needed, for example, after a source server has been migrated, are deleted automatically.

The AWS SMS is similar in incurring charges for storing snapshots it takes for every replication. The number of snapshots depends on frequency of the replication job (from 1 to 24 hours). To prevent additional charges, delete snapshot copies you no longer need. AWS SMS also replicates server volumes from your on-premises environment to Amazon S3 temporarily and purges them from Amazon S3 right after creating Amazon EBS snapshots, incurring a transient charge for S3.

Encryption

CloudEndure Migration supports encryption at rest and in transit. For encryption in transit, CloudEndure Migration uses Advanced Encryption Standard (AES-256) to encrypt replication traffic between agents in the source environment and the replication server. The traffic is passed using TCP port 1500. When the CloudEndure agent is installed on each source server, CloudEndure Migration generates a separate encryption key and passes it along to the designated replication using Transport Layer Security (TLS). The key is kept there in memory (confirm) for decryption.

For encryption at rest, CloudEndure Migration gives you the option to store your volumes encrypted in the staging area. You do that by checking Enable volume encryption in the project’s replication settings. From there, you choose either the default encryption key, or a custom KMS CMK key. Note that CloudEndure Migration doesn’t support OS-based disk encryption features such as BitLocker. These should be disabled before using any CloudEndure services.

CloudEndure Volume Encryption

Figure 7- CloudEndure Volume Encryption

AWS SMS creates an Amazon S3 bucket in the Region on your behalf, with server-side encryption enabled and a bucket policy to delete any items in the bucket after seven days. When you configure a replication job you have the option to enable AMI encryption. If you do so, AWS SMS encrypts the generated AMIs. Your default CMK is used unless you specify a non-default CMK. Replicated server volumes are encrypted in transit by Transport Layer Security TLS 1.2.

Disaster Recovery (DR)

In many cases, I work with customers who start their experience on AWS by moving or building a disaster recovery site on AWS. Sometime later, they realize the benefits of AWS and they decide to migrate their entire workload.

Although AWS SMS and CloudEndure Migration products are not designed for disaster recovery purposes, in this scenario, it is easier for customers to use CloudEndure Disaster Recovery for the DR project, and then leverage their familiarity with the product by using CloudEndure Migration for the migration portion. CloudEndure Disaster Recovery also supports region-to-region disaster recovery within AWS. To learn more, see CloudEndure Disaster Recovery.

Conclusion

Both CloudEndure Migration and AWS SMS are good options for a lift-and-shift migration. In this post, I reviewed a few use cases where one service is better fit over the other. For example, CloudEndure Migration is a better option when you migrate physical servers to AWS, when you want to deploy a block-level replication solution, or when you have near real-time replication requirements. In contrast, AWS SMS is a better option when you want to leverage an agentless solution, when you can tolerate one hour between replication jobs, or when you want a solution that’s managed from one place in the AWS Management Console.

You can get started here with CloudEndure Migration and AWS SMS. If you have comments about this blog post, please submit them in the comments section. I look forward to hearing from you.

Migrating a SharePoint application using the AWS Server Migration Service

Post Syndicated from Chris Munns original https://aws.amazon.com/blogs/compute/migrating-a-sharepoint-application-using-the-aws-server-migration-service/

This post contributed by Ashwini Rudra, Solutions Architect; Rajesh Rathod, Sr. Product Manager; Vivek Chawda, Senior Software Engineer, EC2 Enterprise

Many AWS customers are migrating on-premises SharePoint workloads to AWS for greater reliability, faster performance, and lower costs. While planning the migration, customers are looking for tools and methodologies that reduce the time to migrate, application downtime, and performance disruption. They use continuous replication to optimize cost and effort required to migrate applications reliably.

To accelerate these migrations, AWS provides a comprehensive set of tools. In this blog post, we explore how to use AWS Server Migration Service (AWS SMS) for migrating a SharePoint application from on-premises to AWS.

Overview of solution

AWS Server Migration Service is an agentless service. It makes it easier and faster to migrate thousands of workloads from on-premises or Microsoft Azure to AWS. In this article, we discuss one of the approaches and steps to migrate SharePoint farm using AWS SMS.

AWS SMS also supports migrating a group of servers organized as an application. This can simplify migration of applications with complex dependencies that must be migrated together. This service provides a customized replication schedule designed to simplify migration at scale. This also tracks the progress of each migration using AWS Migration Hub.

For SharePoint migrations, it facilitates migration failovers quickly. After the initial sync, the migration uses an incremental change capture approach to synchronize changes made to the on-premises SharePoint servers. This method also reduced required network bandwidth for the migration.

SharePoint Migration Architecture

SharePoint Migration Architecture

Here is how this service and solution works:

Walkthrough

A basic SharePoint deployment is a 3-tier architecture comprising of web frontend servers, application servers, and backend SQL database servers. It also includes authentication services servers with Active Directory domain controllers.

To migrate this application, you must deploy a Server Migration Connector, which is a preconfigured virtual machine. This connector creates a server catalog. Based on selected server and configuration, it takes snapshots of virtual machines and stores them in S3 buckets.

In the background, AWS VM import/export service converts these snapshots into Amazon Machine Images (AMIs). Using these AMIs, you can configure launch settings where you define an order of application launch. You can also select instance types and set user-defined PowerShell scripts.

At the end, you define the cloud network topology: a VPC, subnets, and security groups. With these launch settings, SMS creates an AWS CloudFormation template, which can launch the SharePoint application in the target AWS account.

To migrate a SharePoint using SMS:

  1. Install and register the SMS connector.
  2. Create an application from SMS server catalog.
  3. Configure replication settings for your SharePoint farm.
  4. Configure launch settings.
  5. Start the replication.
  6. Launch the SharePoint application in AWS.

Install and register the Server Migration Connector

The Server Migration Connector is a VM that you install in your on-premises virtualization environment. The supported platforms are VMware vSphere, Microsoft Hyper-V/SCVMM, and Microsoft Azure. Follow the links below to install in your environment:

For example, VMWare is a common scenario. First, you must set up Server Migration Connector and import server catalog:

  1. Login to AWS Server Migration Service Console, and click to Connectors.
  2. Download .ova file and deploy it to your VMWare environment using vSphere client from AWS Server Migration Setup page.
  3. Install OVA file on your VMWare environment. This is your AWS SMS Connector.
  4. Use Connector Host IP address access Connector page and start five step registration process. Refer AWS documentation, Install the Server Migration Connector on VMWare.
  5. Follow the five-step process of registration. Here, you set up the password and network configuration between the connector virtual machine and AWS accounts.
    Setup network info

    Setup network info

    vCenter credentials

    vCenter credentials

    At the end, provide your vCenter host name and credentials.

    In setup, provide all details of the VMWare and AWS environment to the connector. After establishing the connection, the connector setup looks like this in the AWS Management Console:

    SMS Connectors

    SMS Connectors

Create an application from SMS server catalog

After configuring the connector and selecting “Import Server Catalog”, you are able to view the server catalog in AWS Server Migration Service console. To migrate your SharePoint application, select the application server, SQL Server, and Active Directory server from the server catalog.

Depending on the application architecture, you can group these servers to apply server-specific configuration settings and select appropriate instance types. Here are the steps:

  1. Navigate to the application feature of SMS and create a “new application” for the SharePoint farm. Provide the application name, description, and IAM role.

    Create new application - Application settings

    Create new application – Application settings

  2. Select servers to migrate from the catalog.
  3. Create different groups for these servers in your console. For this, select servers and choose Add servers to group. This helps in defining different instance types and run user-defined PowerShell scripts for all servers in a group during application launch. You may create different groups for the application, web frontend, database, and Active Directory servers. In the below example, there are two groups – one for application servers, and the other for database servers. This process assumes that you have the authentication services servers already in place and operational in AWS. For more information on SharePoint authentication services, Active Directory and Domain Services, Refer Active Directory Domain Services on AWS Deployment Scenario and Architecture.

    Create new application - Add servers to groups

    Create new application – Add servers to groups

  4. Add tags, per your organization tagging strategy or policy.
  5. Review your application and click “Next” when it is ready for replication settings configurations.

Configure replication settings

  1. Define “replication job type”, “when to start replication job”, and “automatic AMI deletion” based on your requirements. Choose Next.
  2. On the Configure server-specific settings page, in the License type column, select the license type for AMIs created from the replication job. Windows Servers can use either an AWS-provided license or Bring Your Own License (BYOL). Check Microsoft Licensing to review the licensing options. You can also choose Auto to allow AWS SMS to detect the source-system operating system (OS) and apply the appropriate license to the migrated virtual machine. Choose Next.

    Server-specific settings

    Server-specific settings

  3. Review your application replication setting and choose Configure Launch Settings.

Configure launch settings

An important aspect of migration is how this application should launch on EC2. This is configured on this page of SMS:

  1. On the Configure launch settings page, for the IAM CloudFormation role, provide an IAM role for launch settings. Refer AWS Documentation on IAM Roles for AWS SMS.

    Configure launch settings

    Configure launch settings

  2. Under Specify launch order, configure a launch order for your groups. For this SharePoint application, you may prefer Active Directory first, followed by the SQL database, and then the application servers.
  3. Under Configure launch settings for the application, edit the server settings individually:
    Target instance details

    Target instance details

    • Logical ID: AWS CloudFormation resource ID. This is the logical ID of the CloudFormation template that AWS SMS generates for the application. A value is created automatically when you use the console, but you must supply it manually when using the API, CLI, or SDKs. For more information, see Resources in the AWS CloudFormation User Guide.
    • Instance type: specifies the EC2 instance type on which to launch the server.
    • Key pair: specifies the SSH key pair for access to the server.
    • Configuration script: a script to run configuration commands at the startup of EC2 instances launched as part of an application. This is important for your SharePoint migration, as you can provide registry settings and configuration settings using PowerShell script for your SharePoint servers and SQL database servers.

    For example, the PowerShell script below retrieves the IP address of the current SQL Server and replaces the old SQL Server IP in the SharePoint configuration database and connection strings. You can automate many SharePoint configuration tasks using PowerShell scripts.

      Start-Transcript -Path "C:\UserData.log" -Append
      $oldIP = <<Your old IP goes here>>
      $newIP = ([System.Net.Dns]::GetHostAddresses("sp-ip-sql-server.aws.local")).IPAddressToString
      $registryPath = "HKLM:\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\16.0\Secure\ConfigDB"
      $Name = "dsn"
      $regValue = (Get-ItemProperty -Path $registryPath -Name $Name).dsn
      $updatedRegValue = $regValue.Replace($oldIP, $newIP)
      New-ItemProperty -Path $registryPath -Name $Name -Value $updatedRegValue -PropertyType String -Force | Out-Null
  4. Configure the target network: VPC, subnets and security groups. Refer to the SharePoint on AWS documentation (Reference Deployment) for more guidance. The network topology varies based on platform requirements. Here is a reference architecture for SharePoint on AWS:

    Architecture topology

    Architecture topology

Start application replication

To start replication, select Start Replication under the Actions menu on the Applications page. The replication time depends on the amount of data replicated and available network bandwidth. On the application details page, you can observe the status of the replication in the Replication status field. If the replication fails, the status message field shows the reason.

Start replication

Start replication

Launch SharePoint in AWS

  1. On the Application page, choose Actions, Launch application. A replication job must complete before you perform this action.
  2. In the Launch application window, choose Launch. On the application details page, you can observe the status of the launch in the Launch status field. If the launch fails, you are able to find the reason in the status message field. You can also generate a CloudFormation template and download this template to use in different AWS accounts.

    Launch application

    Launch application

Test your migration

When the SharePoint application is launched, you can connect to Amazon EC2 instances based servers via Remote Desktop Protocol (RDP). You can access the application based on your Internet Information Services (IIS) Server settings runs on SharePoint Web Front End (WFE) application server (on Amazon EC2).  It is also recommended to investigate optimizations using the AWS Compute Optimizer. For this blog, we have not verified the migration steps with SharePoint 2007 and 2010.

Conclusion

AWS Server Migration Service simplifies SharePoint application migration. Using AWS SMS, you can easily migrate a SharePoint farm and reduce your migration timeline using the launch setting and launch order features.

To learn more, watch Application migration Using AWS Server Migration Service (SMS) or view a demo on the AWS Online Tech Talks Channel. If you have feedback, let us know in the comments section below.

Learn about AWS Services & Solutions – September AWS Online Tech Talks

Post Syndicated from Jenny Hang original https://aws.amazon.com/blogs/aws/learn-about-aws-services-solutions-september-aws-online-tech-talks/

Learn about AWS Services & Solutions – September AWS Online Tech Talks

AWS Tech Talks

Join us this September to learn about AWS services and solutions. The AWS Online Tech Talks are live, online presentations that cover a broad range of topics at varying technical levels. These tech talks, led by AWS solutions architects and engineers, feature technical deep dives, live demonstrations, customer examples, and Q&A with AWS experts. Register Now!

Note – All sessions are free and in Pacific Time.

Tech talks this month:

 

Compute:

September 23, 2019 | 11:00 AM – 12:00 PM PTBuild Your Hybrid Cloud Architecture with AWS – Learn about the extensive range of services AWS offers to help you build a hybrid cloud architecture best suited for your use case.

September 26, 2019 | 1:00 PM – 2:00 PM PTSelf-Hosted WordPress: It’s Easier Than You Think – Learn how you can easily build a fault-tolerant WordPress site using Amazon Lightsail.

October 3, 2019 | 11:00 AM – 12:00 PM PTLower Costs by Right Sizing Your Instance with Amazon EC2 T3 General Purpose Burstable Instances – Get an overview of T3 instances, understand what workloads are ideal for them, and understand how the T3 credit system works so that you can lower your EC2 instance costs today.

 

Containers:

September 26, 2019 | 11:00 AM – 12:00 PM PTDevelop a Web App Using Amazon ECS and AWS Cloud Development Kit (CDK) – Learn how to build your first app using CDK and AWS container services.

 

Data Lakes & Analytics:

September 26, 2019 | 9:00 AM – 10:00 AM PTBest Practices for Provisioning Amazon MSK Clusters and Using Popular Apache Kafka-Compatible Tooling – Learn best practices on running Apache Kafka production workloads at a lower cost on Amazon MSK.

 

Databases:

September 25, 2019 | 1:00 PM – 2:00 PM PTWhat’s New in Amazon DocumentDB (with MongoDB compatibility) – Learn what’s new in Amazon DocumentDB, a fully managed MongoDB compatible database service designed from the ground up to be fast, scalable, and highly available.

October 3, 2019 | 9:00 AM – 10:00 AM PTBest Practices for Enterprise-Class Security, High-Availability, and Scalability with Amazon ElastiCache – Learn about new enterprise-friendly Amazon ElastiCache enhancements like customer managed key and online scaling up or down to make your critical workloads more secure, scalable and available.

 

DevOps:

October 1, 2019 | 9:00 AM – 10:00 AM PT – CI/CD for Containers: A Way Forward for Your DevOps Pipeline – Learn how to build CI/CD pipelines using AWS services to get the most out of the agility afforded by containers.

 

Enterprise & Hybrid:

September 24, 2019 | 1:00 PM – 2:30 PM PT Virtual Workshop: How to Monitor and Manage Your AWS Costs – Learn how to visualize and manage your AWS cost and usage in this virtual hands-on workshop.

October 2, 2019 | 1:00 PM – 2:00 PM PT – Accelerate Cloud Adoption and Reduce Operational Risk with AWS Managed Services – Learn how AMS accelerates your migration to AWS, reduces your operating costs, improves security and compliance, and enables you to focus on your differentiating business priorities.

 

IoT:

September 25, 2019 | 9:00 AM – 10:00 AM PTComplex Monitoring for Industrial with AWS IoT Data Services – Learn how to solve your complex event monitoring challenges with AWS IoT Data Services.

 

Machine Learning:

September 23, 2019 | 9:00 AM – 10:00 AM PTTraining Machine Learning Models Faster – Learn how to train machine learning models quickly and with a single click using Amazon SageMaker.

September 30, 2019 | 11:00 AM – 12:00 PM PTUsing Containers for Deep Learning Workflows – Learn how containers can help address challenges in deploying deep learning environments.

October 3, 2019 | 1:00 PM – 2:30 PM PTVirtual Workshop: Getting Hands-On with Machine Learning and Ready to Race in the AWS DeepRacer League – Join DeClercq Wentzel, Senior Product Manager for AWS DeepRacer, for a presentation on the basics of machine learning and how to build a reinforcement learning model that you can use to join the AWS DeepRacer League.

 

AWS Marketplace:

September 30, 2019 | 9:00 AM – 10:00 AM PTAdvancing Software Procurement in a Containerized World – Learn how to deploy applications faster with third-party container products.

 

Migration:

September 24, 2019 | 11:00 AM – 12:00 PM PTApplication Migrations Using AWS Server Migration Service (SMS) – Learn how to use AWS Server Migration Service (SMS) for automating application migration and scheduling continuous replication, from your on-premises data centers or Microsoft Azure to AWS.

 

Networking & Content Delivery:

September 25, 2019 | 11:00 AM – 12:00 PM PTBuilding Highly Available and Performant Applications using AWS Global Accelerator – Learn how to build highly available and performant architectures for your applications with AWS Global Accelerator, now with source IP preservation.

September 30, 2019 | 1:00 PM – 2:00 PM PTAWS Office Hours: Amazon CloudFront – Just getting started with Amazon CloudFront and [email protected]? Get answers directly from our experts during AWS Office Hours.

 

Robotics:

October 1, 2019 | 11:00 AM – 12:00 PM PTRobots and STEM: AWS RoboMaker and AWS Educate Unite! – Come join members of the AWS RoboMaker and AWS Educate teams as we provide an overview of our education initiatives and walk you through the newly launched RoboMaker Badge.

 

Security, Identity & Compliance:

October 1, 2019 | 1:00 PM – 2:00 PM PTDeep Dive on Running Active Directory on AWS – Learn how to deploy Active Directory on AWS and start migrating your windows workloads.

 

Serverless:

October 2, 2019 | 9:00 AM – 10:00 AM PTDeep Dive on Amazon EventBridge – Learn how to optimize event-driven applications, and use rules and policies to route, transform, and control access to these events that react to data from SaaS apps.

 

Storage:

September 24, 2019 | 9:00 AM – 10:00 AM PTOptimize Your Amazon S3 Data Lake with S3 Storage Classes and Management Tools – Learn how to use the Amazon S3 Storage Classes and management tools to better manage your data lake at scale and to optimize storage costs and resources.

October 2, 2019 | 11:00 AM – 12:00 PM PTThe Great Migration to Cloud Storage: Choosing the Right Storage Solution for Your Workload – Learn more about AWS storage services and identify which service is the right fit for your business.

 

 

Learn about hourly-replication in Server Migration Service and the ability to migrate large data volumes

Post Syndicated from Martin Yip original https://aws.amazon.com/blogs/compute/learn-about-hourly-replication-in-server-migration-service-and-the-ability-to-migrate-large-data-volumes/

This post courtesy of Shane Baldacchino, AWS Solutions Architect

AWS Server Migration Service (AWS SMS) is an agentless service that makes it easier and faster for you to migrate thousands of on-premises workloads to AWS. AWS SMS allows you to automate, schedule, and track incremental replications of live server volumes, making it easier for you to coordinate large-scale server migrations.

In my previous blog posts, we introduced how you can use AWS Server Migration Service (AWS SMS) to migrate a popular commercial off the shelf software, WordPress into AWS.

For details and a walkthrough on how setup the AWS Server Migration Service, please see the following blog posts for Hyper-V and VMware hypervisors which will guide you through the high level process.

In this article we are going to step it up a few notches and look past common the migration of off-the-shelf software and provide you a pattern on how you can use AWS SMS and some of the recently launched features to migrate a more complicated environment, especially compression and resiliency for replication jobs and the support for data volumes greater than 4TB.

This post covers a migration of a complex internally developed eCommerce system comprising of a polyglot architecture. It is made up a Windows Microsoft IIS presentation tier, Tomcat application tier, and Microsoft SQL Server database tier. All workloads run on-premises as virtual machines in a VMware vCenter 5.5 and ESX 5.5 environment.

This theoretical customer environment has various business and infrastructure requirements.

Application downtime: During any migration activities, the application cannot be offline for more than 2 hours
Licensing: The customer has renewed their Microsoft SQL Server license for an additional 3 years and holds License Mobility with Software Assurance option for Microsoft SQL Server and therefore wants to take advantage of AWS BYOL licensing for Microsoft SQL server and Microsoft Windows Server.
Large data volumes: The Microsoft SQL Server database engine (.mdf, .ldf and .ndf files) consumes 11TB of storage

Walkthrough

Key elements of this migration process are identical to the process outlined in my previous blog posts and for more information on this process, please see the following blog posts Hyper-V and VMware hypervisors, but a high level you will need to.

• Establish your AWS environment.
• Download the SMS Connector from the AWS Management Console.
• Configure AWS SMS and Hypervisor permissions.
• Install and configure the SMS Connector appliance.
• Import your virtual machine inventory and create a replication jobs
• Launch your Amazon EC2 instances and associated NACL’s, Security Groups and AWS Elastic Load Balancers
• Change your DNS records to resolve the custom application to an AWS Elastic Load Balancer.

Before you start, ensure that your source systems OS and vCenter version are supported by AWS. For more information, see the Server Migration Service FAQ.

Planning the Migration

Once you have downloaded and configured the AWS SMS connector with your given Hypervisor you can get started in creating replication jobs.

The artifacts derived from our replication jobs with AWS SMS will be AMI’s (Amazon Machine Images) and as such we do not need to replicate each server individually and that is because we have a three-tier architecture that has commonality between servers with multiple Application and Web servers performing the same function, and as such we can leverage a common AMI and create three replication jobs.

1. Microsoft SQL Server – Database Tier
2. Ubuntu Server – Application Tier
3. IIS Web server – Webserver Tier

Performing the Replication

After validating that the SMS Connector is in a “HEALTHY” state, import your server catalog from your Hypervisor to AWS SMS. This process can take up to a minute.

Select the three servers (Microsoft SQL Server, Ubuntu Server, IIS Web server) to migrate and choose Create replication job. AWS SMS now supports creating replications jobs with frequencies as short as 1 hour, and as such to ensure our business RTO (Recovery Time Objective) of 2 hours is met we will create our replication jobs with a frequency of 1 hour. This will minimize the risk of any delta updates during the cutover windows not completing.

Given the businesses existing licensing investment in Microsoft SQL Server, they will leverage these the BYOL (Bring Your Own License) offering when creating the Microsoft SQL Server replication job.

The AWS SMS console guides you through the process. The time that the initial replication task takes to complete is dependent on available bandwidth and the size of your virtual machines.

After the initial seed replication, network bandwidth requirement is minimized as AWS SMS replicates only incremental changes occurring on the VM.

The progress updates from AWS SMS are automatically sent to AWS Migration Hub so that you can track tasks in progress.

AWS Migration Hub provides a single location to track the progress of application migrations across multiple AWS and partner solutions. In this post, we are using AWS SMS as a mechanism to migrate the virtual machines (VMs) and track them via AWS Migration Hub.

Migration Hub and AWS SMS are both free. You pay only for the cost of the individual migration tools that you use, and any resources being consumed on AWS

The dashboard reflects any status changes that occur in the linked services. You can see from the following image that two servers are complete whilst another is in progress.

Using Migration Hub, you can view the migration progress of all applications. This allows you to quickly get progress updates across all of your migrations, easily identify and troubleshoot any issues, and reduce the overall time and effort spent on your migration projects.

After validating that the SMS Connector

Testing Your Replicated Instances

Thirty hours after creating the replication jobs, notification was received via AWS SNS (Simple Notification Service) that all 3 replication jobs have completed. During the 30-hour replication window the customers ISP experienced downtime and sporadic flapping of the link, but this was negated by the network auto-recovery feature of SMS. It recovered and resumed replication without any intervention.

With the replication tasks being complete. The artifact created by AWS SMS is a custom AMI that you can use to deploy an EC2 instance. Follow the usual process to launch your EC2 instance, noting that you may need to replace any host-based firewalls with security groups and NACLs and any hardware based load balancers with Elastic Load Balancing to achieve fault tolerance, scalability, performance and security.

As this environment is a 3-tier architecture with commonality been tiers (Application and Presentation Tier) we can create during the EC2 Launch process an ASG (Auto Scaling Group) to ensure that deployed capacity matches user demand. The ASG will be based on the custom AMI’s generated by the replication jobs.

When you create an EC2 instance, ensure that you pick the most suitable EC2 instance type and size to match your performance and cost requirements.

While your new EC2 instances are a replica of your on-premises VM, you should always validate that applications are functioning. How you do this differs on an application-by-application basis. You can use a combination of approaches, such as editing a local host file and testing your application, SSH, RDP and Telnet.

For our Windows Presentation and database tier, I can RDP in to my systems and validate IIS 8.0 and other services are functioning correctly.

For our Ubuntu Application tier, we can SSH in to perform validation.

Post validation of each individual server we can now continue to test the application end to end. This is because our systems have been instantiated inside a VPC with no route back to our on-premises environment which allows us to test functionality without the risk of communication back to our production application.

After validation of systems it is now time to cut over, plan your runbook accordingly to ensure you either eliminate or minimize application disruption.

Cutting Over

As the replication window specified in AWS SMS replication jobs was 1 hour, there were hourly AMI’s created that provide delta updates since the initial seed replication was performed. The customer verified the stack by executing the previously created runbook using the latest AMIs, and verified the application behaved as expected.

After another round of testing, the customer decided to plan the cutover on the coming Saturday at midnight, by announcing a two-hour scheduled maintenance window. During the cutover window, the customer took the application offline, shutdown Microsoft SQL Server instance and performed an on-demand sync of all systems.

This generate a new versioned AMI that contained all on-premise data. The customer then executed the runbook on the new AMI’s. For the application and presentation tier these AMI’s were used in the ASG configuration. After application validation Amazon Route 53 was updated to resolve the application CNAME to the Application Load Balancer CNAME used to load balance traffic to the fleet of IIS servers.

Based on the TTL (Time To Live) of your Amazon Route 53 DNS zone file, end users slowly resolve the application to AWS, in this case within 300 seconds. Once this TTL period had elapsed the customer brought their application back online and exited their maintenance window, with time to spare.

After modifying the Amazon Route 53 Zone Apex, the physical topology now looks as follows with traffic being routed to AWS.

After validation of a successful migration the customer deleted their AWS Server Migration Service replication jobs and began planning to decommission their on-premises resources.

Summary

This is an example pattern on migrate a complex custom polyglot environment in to AWS using AWS migration services, specifically leveraging many of the new features of the AWS SMS service.

Many architectures can be extended to use many of the inherent benefits of AWS, with little effort. For example this article illustrated how AWS Migration Services can be used to migrate complex environments in to AWS and then use native AWS services such as Amazon CloudWatch metrics to drive Auto Scaling policies to ensure deployed capacity matches user demand whilst technologies such as Application Load Balancers can be used to achieve fault tolerance and scalability

Think big and get building!