AWS Cloud Migration

Introduction

The objective of this article is to share perspective on some real-world AWS cloud migrations detailing out the as-is-state architecture, key requirements, AWS architecture and challenges. These are based on learnings from actual implementations of large number of applications.

Key Requirement and As-Is-State Architecture

Key Requirements:

The key requirement for most enterprise File Share are as follows:

·      Large Storage Requirements in 100’s of TB of data.

·      High Availability

·      Auto Scale free Storage

·      Low Latency/High Performance

·      Security – File Screening for Malware etc., Encryption at Rest and In-transit

·      Backup – Daily and Monthly and retention for 5 to 7 years

·      Authentication using Microsoft AD

On-Prem Architecture:

Most use cases, on prem architecture uses 2 windows servers in a High Availability setup with File Share mounted on user’s system authenticated using on-prem MS AD.

AWS Solution Design

The five pillars of well architected framework were used for designing the solution in AWS.

Key Considerations for FileShare Design 

Based on all the above consideration, here is the design created for the file share migration.

Key Challenges and Solution Options:

1.    Proactive File Block to prevent Malware/Ransomware Attacks

One of the key requirements was to be able to proactively block certain extension types being added to the File Share and revoke user access if any extension types were found.This was managed via File Screens on the on-prem Windows File Share but once FSx is mounted on the Client, it ceases to show as a local drive thus this capability could not be used.

Solution:

A custom solution with FSx CloudWatch File Access Audit Logs, Lambda and NACL was developed as highlighted in the above design

2.    AWS Backup vs Migration to Glacier

The data that was required to be backed up for the FileShare was about 100 TB with requirement to back it up for 7 years. The organization standard was to use AWS Backup.

Solution

An alternative solution where the data was full backed up on the Glacier using the DataSync which would run on the last day of the month through EventBridge.This enabled 2 to 3 times cost saving over a period of 7 years.

3.    Data Migration Tool – DataSync

DataSync was used for File Transfer from on-prem to AWS by installing the DataSync agent and configuring as per below AWS recommended configuration.

The key challenge here was that each DataSync Agent uses 4 IP’s and each folder that needs to be transferred in a File Share will need 4 IP’s (cannot do it at root share level). So, if there are 20 folders in a file share, you may need to block 80 IP’s incase you want to migrate all the folders together else they can be done sequentially. The challenge with doing it sequentially is that as the contents of the folder change till the time cutover is done, you need to create incremental tasks to update all the folders. So, reserving such large number of IP’s depending on the number of folders and size of File Share can be a challenge.

Please feel free to share insights on the architecture.

Watch the video on following link discussing the architecture : https://www.youtube.com/watch?v=hlViEFtAyHk&t=413s