Data Engineer III - Databricks, Pyspark, Python, AWS
JP Morgan Services India Pvt Ltd
Work from Office
Full Time
Min. 3 yearsJob Description
Data Engineer III - Databricks, Pyspark, Python, AWS
As a Data Engineer III - Databricks, Pyspark, Python, AWS at JPMorgan Chase within the Commercial & Investment Bank, you'll serve as a seasoned member of an agile team to design and deliver trusted data collection, storage, access, and analytics solutions in a secure, stable, and scalable way. You are responsible for developing, testing, and maintaining critical data pipelines and architectures across multiple technical areas within various business functions in support of the firm’s business objectives.
Job responsibilities
- Design, develop, and maintain big data pipelines (batch and streaming) using PySpark/Spark and Databricks.
- Lead/own data modelling and solution design for data products, including defining target-state architecture, data flows, and transformation patterns.
- Build scalable ingestion and transformation workflows for high-volume datasets, ensuring reliability, quality, and performance.
- Develop and optimize complex SQL transformations, reconciliation queries, and analytical datasets; perform query tuning for large-scale workloads.
- Apply strong data warehousing concepts (dimensional modeling, SCDs, partitioning strategies, etc.) to build well-structured, analytics-ready data layers and Leverage common AWS services, with strong emphasis on S3 and AWS data processing capabilities, to support scalable storage and processing.
- Perform advanced debugging and troubleshooting across distributed Spark workloads (data skew, shuffle tuning, memory/compute optimization).
- Implement engineering best practices: modular design, efficient coding, code reviews, and CI-friendly development approaches.
- Use GitHub/Bitbucket and standard version control workflows to manage codebase, peer reviews, and releases.
- Partner with cross-functional stakeholders to convert requirements into robust big data solutions.
- Uses enterprise-authorized AI capabilities within the work environment to accelerate data pipeline/design analysis and documentation, validating outputs and handling data according to sensitivity and security requirements.
Applies reuse-first, AI-assisted practices to strengthen SDLC-quality routines for data pipelines (e.g., test generation and control validation), ensuring traceability/auditability and alignment to resiliency and security expectations.
- Formal training or certification on data engineering concepts and 3+ years applied experience
- Experience in data engineering / big data engineering, with strong hands-on delivery and Expert-level SQL skills (must be extremely strong): complex joins, window functions, CTEs, optimization, and analytical problem solving at scale.
- Strong hands-on coding experience with Python in production environments; demonstrated ability to write efficient, maintainable code.
- Deep expertise in Apache Spark (in depth) and PySpark, including performance tuning and distributed processing fundamentals.
- Strong experience with Databricks for large-scale data processing and pipeline development.
- Strong understanding of data warehousing concepts and best practices; proven capability in data modelling and solution design for scalable, maintainable data platforms/products.
- Experience implementing both batch and streaming data processing solutions.
- Familiarity with AWS S3 and common AWS services used in data platforms and processing
- Excellent debugging, troubleshooting, problem-solving skills and experience with GitHub, Bitbucket, and version control best practices.
- Demonstrated experience using enterprise-authorized AI capabilities within the work environment to support data engineering workflows with strong validation habits and awareness of data sensitivity.
Ability to review and validate AI-assisted outputs (e.g., query suggestions, test ideas, or model change summaries) before use, escalating when uncertain and following data handling requirements.
- Good to have: infrastructure provisioning in AWS using Infrastructure as Code (IaC) (e.g., Terraform, AWS CloudFormation).
Job role
Job requirements
About company
Similar jobs you can apply for
Software / Web Developer
Software Devel
Apnatime Tech Private LimitedWork from OfficeFull TimeAny experience
Good (Intermediate / Advanced) EnglishQuality Engineer
Kateel Engineering Industry Private LimitedWork from OfficeFull TimeMin. 2 yearsGood (Intermediate / Advanced) English
Engineering Supervisor
Zcientia Labs Private Limited
Field JobFull TimeMin. 2 yearsGood (Intermediate / Advanced) EnglishQuality Executive
Sri Sai H R Consultancy ServicesWork from OfficeFull TimeAny experienceBasic EnglishData Analyst
Flipkart Internet Private LimitedWork from OfficeFull TimeMin. 1 yearBasic EnglishQuality Assurance Executive
Accord212 Human Resource Solutions Private LimitedWork from OfficeFull Time
Night ShiftMin. 3 yearsGood (Intermediate / Advanced) EnglishYou can expect a minimum salary of 0 INR. The salary offered will depend on your skills, experience and performance in the interview.
The candidate should have completed the required education and people who have 3 to 31 years are eligible to apply for this job. You can apply for more jobs in Bengaluru/Bangalore to get hired quickly.
The candidate should have sound communication skills and sound communication skills for this job.
Both Male and Female candidates can apply for this job.
No, it's not a work from home job and can't be done online. You can explore and apply for other work from home jobs in Bengaluru/Bangalore at apna.
No work-related deposit needs to be made during your employment with the company.
Go to the apna app and apply for this job. Click on the apply button and call HR directly to schedule your interview.
The last date to apply for this job is . For more details, download apna app and find Full Time jobs in Bengaluru/Bangalore . Through apna, you can find jobs in 64 cities across India. Join NOW!