Capco Technologies Pvt Ltd

Data Engineer - PySpark and Databricks Specialist

Capco Technologies Pvt Ltd
Pune
Not disclosed
Work from OfficeWork from Office
Full TimeFull Time
Min. 5 yearsMin. 5 years

Job Description

Data Engineer (Databrick + Pyspark)

 

Job Title: Data Engineer (PySpark / Databricks)

Experience: 5–9 Years Location: Pune (Hybrid – Capco Office)

Job Summary

We are looking for a skilled Data Engineer with strong expertise in PySpark, Databricks, and modern data engineering practices. The ideal candidate will have hands-on experience in building scalable data pipelines, working with large datasets, and leveraging cloud-based data platforms.

Key Responsibilities Design, develop, and maintain scalable ETL/ELT data pipelines Work extensively with PySpark and Apache Spark for large-scale data processing Build and manage workflows using Apache Airflow Develop and optimize data solutions on Databricks (Jobs, Delta Lake) Work with cloud-based data lakes (S3 or equivalent) Write efficient and complex SQL queries for data transformation and analysis Run and manage Spark workloads on EMR Serverless or other managed Spark platforms Ensure data quality, reliability, and performance optimization of pipelines Must Have Skills Strong hands-on experience with PySpark and Apache Spark internals Experience with Databricks (Jobs, Delta Lake) Proficiency in Apache Airflow for workflow orchestration Solid experience building ETL/ELT pipelines at scale Strong SQL skills and experience with Data Warehouse (DWH) systems Experience running Spark workloads on EMR Serverless or managed Spark platforms Hands-on experience with cloud data lakes (S3 or equivalent) Good to Have Skills Experience with Delta Lake / Apache Iceberg Exposure to streaming frameworks (Spark Structured Streaming, Kafka) Familiarity with CI/CD pipelines for data engineering workflows Knowledge of data governance, cataloging, and lineage tools

Experience Level

Senior Level

Job role

Work location
Work locationIndia; India - Pune
Department
DepartmentData Science & Analytics
Role / Category
Role / CategoryDBA / Data warehousing
Employment type
Employment typeFull Time
Shift
ShiftDay Shift

Job requirements

Experience
ExperienceMin. 5 years

About company

Name
NameCapco Technologies Pvt Ltd
Job posted by Capco Technologies Pvt Ltd

Similar jobs you can apply for

Manufacturing / Production
Avirat Enterprise

Quality Control Engineer

Avirat Enterprise
Narhe, Pune
₹15,000 - ₹30,000*
Work from Office
Full Time
Min. 3 years
Basic English

SAP ABAP

THE NaukriWala
Pune
₹50,000 - ₹1,20,000
Work from Office
Full Time
Min. 2 years
Good (Intermediate / Advanced) English
S R Engineering

Quality Control Engineer

S R Engineering
Ambegaon Budruk, Pune
₹12,000 - ₹25,000
Work from Office
Full Time
Night Shift
Min. 1 year
No English Required
Future Agents

AI Automation & Training Specialist

Future Agents
Hinjewadi, Pune
₹50,000 - ₹1,00,000*
Work from Office
Full Time
Min. 1 year
Good (Intermediate / Advanced) English
Iotas Solutions Private Limited

Embedded Software Developer

Iotas Solutions Private Limited
Hinjewadi, Pune
₹15,000 - ₹35,000
Work from Office
Full Time
Min. 2 years
Basic English
Speed Express

Software Developer

Speed Express
Somwar Peth, Pune
₹20,000 - ₹25,000
Work from Office
Full Time
Min. 1 year
Good (Intermediate / Advanced) English