AI Sciences

Introduction to Project

Project Architecture

Creating RDS MySql instance

Creating S3 Bucket

Creating DMS Source Endpoint

Creating DMS Destination Endpoint

Creating DMS Instance

MySql WorkBench

Connecting with RDS and Dumping Data

Quering RDS

DMS Full Load

DMS Replication Ongoing

Stoping Instances

Glue Job (Full Load)

Glue Job (Change Capture)

Glue Job (CDC)

Creating Lambda Function and Adding Trigger

Checking Trigger

Getting S3 file name in Lambda

Creating Glue Job

Adding Invoke for Glue Job

Testing Invoke

Writing Glue Shell Job

Full Load Pipeline

Change Data Capture Pipeline