AWS Data Pipeline Engineer

The Role

Are you ready to dive headfirst into the captivating world of data engineering at Kyndryl? As a Data Engineer, you'll be the visionary behind our data platforms, crafting them into powerful tools for decision-makers. Your role? Ensuring a treasure trove of pristine, harmonized data is at everyone's fingertips.

As an AWS Data Engineer at Kyndryl, you will be responsible for designing, building, and maintaining scalable, secure, and high-performing data pipelines using AWS cloud-native services. This role requires extensive hands-on experience with both real-time and batch data processing, expertise in cloud-based ETL/ELT architectures, and a commitment to delivering clean, reliable, and well-modeled datasets.

Key Responsibilities:

• Design and develop scalable, secure, and fault-tolerant data pipelines utilizing AWS services such as Glue, Lambda, Kinesis, S3, EMR, Step Functions, and Athena.
• Create and maintain ETL/ELT workflows to support both structured and unstructured data ingestion from various sources, including RDBMS, APIs, SFTP, and Streaming.
• Optimize data pipelines for performance, scalability, and cost-efficiency.
• Develop and manage data models, data lakes, and data warehouses on AWS platforms (e.g., Redshift, Lake Formation).
• Collaborate with DevOps teams to implement CI/CD and infrastructure as code (IaC) for data pipelines using CloudFormation or Terraform.
• Ensure data quality, validation, lineage, and governance through tools such as AWS Glue Data Catalog and AWS Lake Formation.
• Work in concert with data scientists, analysts, and application teams to deliver data-driven solutions.
• Monitor, troubleshoot, and resolve issues in production pipelines.
• Stay abreast of AWS advancements and recommend improvements where applicable.

Who You Are

You’re good at what you do and possess the required experience to prove it. However, equally as important – you have a growth mindset; keen to drive your own personal and professional development. You are customer-focused – someone who prioritizes customer success in their work. And finally, you’re open and borderless – naturally inclusive in how you work with others.

Required Skills and Experience

• Bachelor’s or master’s degree in computer science, Engineering, or a related field
• Over 8 years of experience in data engineering
• More than 3 years of experience with the AWS data ecosystem
• Strong experience with Pyspark, SQL, and Python
• Proficiency in AWS services: Glue, S3, Redshift, EMR, Lambda, Kinesis, CloudWatch, Athena, Step Functions
• Familiarity with data modelling concepts, dimensional models, and data lake architectures
• Experience with CI/CD, GitHub Actions, CloudFormation/Terraform
• Understanding of data governance, privacy, and security best practices
• Strong problem-solving and communication skills

Preferred Skills and Experience

• Experience working as a Data Engineer and/or in cloud modernization.
• Experience with AWS Lake Formation and Data Catalog for metadata management.
• Knowledge of Databricks, Snowflake, or BigQuery for data analytics.
• AWS Certified Data Engineer or AWS Certified Solutions Architect is a plus.
• Strong problem-solving and analytical thinking.
• Excellent communication and collaboration abilities.
• Ability to work independently and in agile teams.
• A proactive approach to identifying and addressing challenges in data workflows.