Big Data Developer (PySpark Specialist)

Data Developer

Job Description:

You will join the Airbus (DCS) organization in charge of the BigData Developments & Support. The mission of the PSL is to deliver innovative IT solutions that support digital transformation and efficient operations for customer services. More precisely the PSL is dedicated to support Customer Services activities related to the Airframe Services product and services portfolio.

We are looking for a performance-driven PySpark Developer to join the Airbus (DCS) team. In this role, you will go beyond basic data processing to design highly optimized, scalable server-side solutions in Python and PySpark. You will be responsible for building the data backbone of Airframe Services, ensuring that our "Skywise" platform delivers rapid insights to maximize aircraft availability and streamline customer contracts.

Key Responsibilities

• Performance Tuning: Optimize Spark jobs by managing partitioning, caching strategies, broadcast joins, and addressing data skew to ensure efficient resource utilization.

• Data Governance: Implement and manage data assets using Unity Catalog to ensure fine-grained access control and data lineage.

• CI/CD : Build and maintain automated deployment pipelines using Git and CI/CD frameworks to ensure seamless code promotion across environments.

• Scalable Engineering: Independently prototype and develop high-quality data objects, KPIs, and dashboards on the Skywise platform.

• Agile Collaboration: Participate in an Agile development environment, providing weekly updates on algorithms and KPIs while challenging business requirements for technical feasibility.

Technical Skill Set

Core Requirements (Must-Have)

• Advanced PySpark: 2–4 years of experience with a heavy focus on performance tuning (e.g., analyzing Spark UI, optimizing DAGs, and memory management).

• Python Mastery: Strong server-side programming skills for complex data processing.

• Data Architecture: Hands-on experience with Unity Catalog for metadata management and data governance.

• Version Control & Automation: Proficiency in Git (branching, merging, PR workflows) and integrated CI/CD pipelines.

• SQL & Relational Databases: Expert-level SQL for complex querying and data validation.

Preferred Qualifications (Good to Have)

• Cloud Platforms: Knowledge of AWS services (S3, EC2, Glue, or EMR) is a significant plus.

• Tech Skills: Java

• Skywise/Palantir: Experience with the Palantir ecosystem (Code Repository, Pipeline Builder, Contour, Workshop, Slate) and Ontology definitions.

• Methodology: Familiarity with Agile/Scrum modes of delivery and tools like Jira or VersionOne.

• Visualization: Exposure to Qlik or similar dashboarding tools.

Soft Skills & Qualifications

• Education: Graduate in Information Technology, Computer Science Engineering or equivalent.

• Analytical Thinking: Ability to debug complex data transformation pipelines and solve performance bottlenecks.

• Communication: English (Negotiation level): ability to collaborate with Project Leaders, POs, and multi-disciplinary support teams.

• Adaptability: Comfortable working in a fast-paced environment with weekly deliverables.

Deliverables

• Optimized Spark Algorithms and Data Pipelines.

• Production-ready KPIs and Dashboards (Qlik/Skywise).

• Clean, version-controlled code via Git.

• Documented data lineage and governance structures in Unity Catalog.

This job requires an awareness of any potential compliance risks and a commitment to act with integrity, as the foundation for the Company’s success, reputation and sustainable growth.

Company:

Airbus India Private Limited

Employment Type:

Permanent

-------

Experience Level:

Professional

Job Family:

Digital

By submitting your CV or application you are consenting to Airbus using and storing information about you for monitoring purposes relating to your application or future employment. This information will only be used by Airbus.
Airbus is committed to achieving workforce diversity and creating an inclusive working environment. We welcome all applications irrespective of social and cultural background, age, gender, disability, sexual orientation or religious belief.

Airbus is, and always has been, committed to equal opportunities for all. As such, we will never ask for any type of monetary exchange in the frame of a recruitment process. Any impersonation of Airbus to do so should be reported to emsom@airbus.com.

At Airbus, we support you to work, connect and collaborate more easily and flexibly. Wherever possible, we foster flexible working arrangements to stimulate innovative thinking.