Data Platform Engineer

Data Platform Engineer

Project Role : Data Platform Engineer
Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models.
Must have skills : Python (Programming Language)
Good to have skills : PostgreSQL
Minimum 3 year(s) of experience is required
Educational Qualification : 10 years full time education

Role Overview
The Data Platform Engineer is responsible for designing, building, and evolving a cloud-native data platform. The role focuses on data ingestion, transformation, orchestration, metadata management, and data quality across AWS and Snowflake environments.

Core Responsibilities

Design and maintain data ingestion pipelines from internal and external sources into AWS S3 (Raw & Standard zones)
Develop and operate Airflow DAGs on EKS for ingestion, transformation, alerts, and data quality checks
Build and maintain dbt models for data transformations in Snowflake
Manage Snowflake integrations, including S3 external stages and environment separation (Prod / Non-Prod)
Own and implement data quality frameworks, orchestration metadata, and checks (stored in RDS)
Integrate with AWS Glue / Data Catalog for metadata, lineage, and data discovery (Atlan)
Ensure platform reliability, scalability, security, and cost efficiency
Collaborate with analytics and downstream consumers (BI tools such as Cognos, Tableau)


Required Skills (Must Have)

Strong experience with AWS cloud data platforms
Hands-on experience with Airflow, dbt, and Snowflake
Proficiency in SQL and Python
Experience with S3-based data lakes (raw / curated patterns)
Knowledge of metadata management, data catalogs, and data governance
Experience with CI/CD pipelines (GitHub, Azure DevOps, Octopus)
Linux and scripting skills (Bash, Python, PowerShell)


Technical Skills – Must Have

AWS Services: IAM, S3, SNS, SQS, API Gateway, Lambda, DynamoDB, EKS
Airflow
SQL & Python
CI/CD systems
Containers / Kubernetes fundamentals


Preferred / Nice to Have

Experience running data workloads on EKS / Kubernetes
PySpark and AWS Glue Catalog experience
Knowledge of IAM cross-account access and secure data sharing
Experience with data quality frameworks (custom or open-source)
Exposure to hybrid / on-prem integrations (file drops, legacy ETL)
Experience with hybrid architectures (on-prem + cloud)
Familiarity with dbt and Snowflake optimizations