Data Engineer - Associate Director

Data Engineer - Associate Director

Job Requirements

Mandatory Skills

• 12+ years of experience in data engineering, with at least 3 years leading large technical teams.

• Industry vendor certifications are desired (e.g. AWS, Azure, GCP, CNCF/Kubernetes or Databricks certifications); although not essential if you have demonstrable ability.

• Strong hands‑on expertise with Python, PySpark, and Databricks (including Lakehouse & Databricks Workflows).

• Experience designing and deploying data solutions on Azure, AWS, or GCP.

• Strong understanding of data modelling techniques, data lifecycle management, and enterprise data principles.

• Expertise in designing, developing, and managing scalable, end-to-end data pipelines (ADF, Airflow or dbt,).

• Proficient in Big Data Platforms (Hadoop, Databricks, Hive, Kafka, Apache Iceberg or Microsoft Fabric), Data Warehouses (Teradata, Snowflake, BigQuery etc.) and lakehouses (Delta Lake, Apache Hudi)

• Proficient in programming languages such as SQL, Python and Pyspark with strong skills in writing scalable, readable and maintainable code using object-oriented programming concept.

• Implement DevOps practices, including Git workflows and CI/CD pipelines (Azure DevOps, Jenkins, GitHub Actions) to enhance automation and streamline deployments.

• Experience in project management frameworks such as Waterfall or Agile.

• Solid experience with SQL and handling large datasets.

• Familiarity with data governance frameworks (e.g., data quality controls, cataloging, lineage, roles & policies).

• Excellent communication skills—able to translate complex data concepts to both technical and non‑technical stakeholders.

Primary Roles and Responsibilities 

• Lead and mentor a technical team (8–15 engineers) in designing and delivering data‑engineering solutions across cloud platforms (Azure/AWS/GCP).

• Architect, develop, and optimize ETL/ELT pipelines using Python, PySpark, Databricks, and distributed processing frameworks.

• Design and maintain scalable data architectures, including data lakes, lakehouses, warehouses, Delta tables, and semantic layers.

• Establish best practices in data modelling (dimensional, wide‑table, data vault), data quality, metadata management, and data governance.

• Collaborate with business owners, architects, product managers, and analytics teams to ensure end‑to‑end project delivery.

• Define standards for data operations, monitoring, lineage, and CI/CD for data workloads.

• Drive cloud platform adoption and modernization, ensuring solutions meet performance, cost‑efficiency, and compliance requirements.

• Promote innovation by evaluating new data engineering tools, frameworks, and practices.

• Mentor junior team members, providing guidelines to ensure high-quality deliverables.

• Communicate complex technical solutions to senior management and diverse stakeholders effectively.

• Contribute to sales activities through data and platform architecture expertise

• Stay updated on industry trends and contribute to internal initiatives, R&D, and business development projects.

Preferred Skills

• Experience with Azure Data Factory, AWS Glue, or GCP Dataflow.

• Experience with Microsoft Fabric solutions

• Knowledge of Lakehouse architecture, Delta Live Tables, and Medallion design patterns.

• Familiarity with CI/CD, DevOps, containerization (Docker/Kubernetes).

• Exposure to data management tools (Purview, Collibra, Informatica, BigID, etc.).

• Experience with different data execution paradigms, including low latency/streaming, batch, and micro-batch processing (Apache Kafka, Databricks Streaming processing, Apache StreamSets).

• Knowledge of data management frameworks (data governance, data quality, data security, data dictionary, metadata management) with tools like Databricks Unity Catalog, Apache Atlas, Informatica etc.

• Familiarity with containerization and orchestration tools like Docker and Kubernetes.

• Understanding of API gateway and service mesh architectures (e.g., Istio).

• Familiarity working with Linux-based operating systems.

• Familiarity with working with REST APIs.

• Experience in pre‑sales, solutioning, or client‑facing delivery leadership.

• Fluency in English (verbal and written).

Other Information

Number of interview rounds

3

Mode of interview

Virtual

Job location

Bangalore

Clean room policy (specific to business)

NA

Culture

• Corporate Social Responsibility programs

• Maternity and paternity leave

• Opportunities to network and connect

• Discounts on products and services

Note: Benefits/Perks listed above may vary depending on the nature of your employment with KPMG and the country where you work.

1. © 2021 KPMG Global Services Private Limited, a company incorporated under the laws of India and a member firm of the KPMG global organization of independent member firms affiliated with KPMG International Limited (“KPMG International”), a private English company limited by guarantee. All rights reserved. The KPMG name and logo are trademarks used under license by the independent member firms of the KPMG global organization.
2. The term “KGS” refers to the KGS Platform of Indian delivery entities, which consist of KPMG Global Services Private Limited (“KGSPL”), KPMG Global Services Management Private Limited (“KGSMPL”), KPMG Global Delivery Center Private Limited (“GDCPL”) and KPMG Resource Centre Private Limited (“KRCPL”), unless the specific private limited entity is specifically noted.
3. Use in these materials of the term “Our”, or “Us” means KGSPL, KGSMPL, GDCPL and/or KRCPL as the case may be and all four such legal entities are referred to collectively as “KGS”.
4. Note that Audit services for the clients of the US and Canadian firms are delivered from the GDCPL and for UK (and other firms except US and Canada) Audit services are delivered from KRCPL.

KPMG is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances.

 

 

 

 

 

                          

• Lead and mentor a technical team (8–15 engineers) in designing and delivering data‑engineering solutions across cloud platforms (Azure/AWS/GCP).

• Architect, develop, and optimize ETL/ELT pipelines using Python, PySpark, Databricks, and distributed processing frameworks.

• Design and maintain scalable data architectures, including data lakes, lakehouses, warehouses, Delta tables, and semantic layers.

• Establish best practices in data modelling (dimensional, wide‑table, data vault), data quality, metadata management, and data governance.

• Collaborate with business owners, architects, product managers, and analytics teams to ensure end‑to‑end project delivery.

• Define standards for data operations, monitoring, lineage, and CI/CD for data workloads.

• Drive cloud platform adoption and modernization, ensuring solutions meet performance, cost‑efficiency, and compliance requirements.

• Promote innovation by evaluating new data engineering tools, frameworks, and practices.

• Mentor junior team members, providing guidelines to ensure high-quality deliverables.

• Communicate complex technical solutions to senior management and diverse stakeholders effectively.

• Contribute to sales activities through data and platform architecture expertise

• Stay updated on industry trends and contribute to internal initiatives, R&D, and business development projects.

• 12+ years of experience in data engineering, with at least 3 years leading large technical teams.

• Industry vendor certifications are desired (e.g. AWS, Azure, GCP, CNCF/Kubernetes or Databricks certifications); although not essential if you have demonstrable ability.

• Strong hands‑on expertise with Python, PySpark, and Databricks (including Lakehouse & Databricks Workflows).

• Experience designing and deploying data solutions on Azure, AWS, or GCP.

• Strong understanding of data modelling techniques, data lifecycle management, and enterprise data principles.

• Expertise in designing, developing, and managing scalable, end-to-end data pipelines (ADF, Airflow or dbt,).

• Proficient in Big Data Platforms (Hadoop, Databricks, Hive, Kafka, Apache Iceberg or Microsoft Fabric), Data Warehouses (Teradata, Snowflake, BigQuery etc.) and lakehouses (Delta Lake, Apache Hudi)

• Proficient in programming languages such as SQL, Python and Pyspark with strong skills in writing scalable, readable and maintainable code using object-oriented programming concept.

• Implement DevOps practices, including Git workflows and CI/CD pipelines (Azure DevOps, Jenkins, GitHub Actions) to enhance automation and streamline deployments.

• Experience in project management frameworks such as Waterfall or Agile.

• Solid experience with SQL and handling large datasets.

• Familiarity with data governance frameworks (e.g., data quality controls, cataloging, lineage, roles & policies).

• Excellent communication skills—able to translate complex data concepts to both technical and non‑technical stakeholders.

Experience Level

Mid Level