JD:

9+ years of experience in developing Data Lakes with data ingestion from disparate sources, including relational databases, flat files, APIs, and streaming data.

Proficiency in programming languages such as Python, PySpark, and Spark, preferably Python.

Architect and implement robust ETL pipelines using AWS Glue, defining data extraction methods, transformation logic, and data loading procedures across different sources.

Expertise in core AWS services, including AWS IAM, VPC, EC2, S3, RDS, Lambda, CloudWatch, CloudFormation, and CloudTrail.

Experience in designing and developing event-driven data pipelines in AWS Cloud using Glue.

Ability to design, orchestrate, and schedule jobs using Apache Airflow.

Experience in developing event-driven distributed systems in the cloud using serverless architecture.

Strong experience in developing CI/CD pipelines using GitHub Actions, Jenkins.

9+ years of experience with AWS Cloud Data Ingestion Patterns and Practices, using S3 as the storage backbone.

9+ years of experience using Infrastructure as Code (IaC) tools like Terraform.

Experience in migrating data from on-premises sources to AWS Cloud using AWS Database Migration Service (DMS).