Senior Data Engineering Expert

Engineering Expert - AI & Data Products, Runtime Platform

We help the world run better

At SAP, we enable you to bring out your best. Our company culture is focused on collaboration and a shared passion to help the world run better. How? We focus every day on building the foundation for tomorrow and creating a workplace that embraces differences, values flexibility, and is aligned to our purpose-driven and future-focused work. We offer a highly collaborative, caring team environment with a strong focus on learning and development, recognition for your individual contributions, and a variety of benefit options for you to choose from.

What you'll build

 

As an Engineering Expert, you will be instrumental in the expansion of our Foundation Services team in Bangalore. Your profound expertise in distributed data processing, Spark optimization, data transformation pipelines, data engineering architecture principles, AI Agent development, and LLM-driven engineering will drive scalable data solutions and intelligent automation, reinforcing SAP Data & Analytics’ pivotal role in SAP’s Data & AI strategy. This is a product expert role requiring deep hands-on expertise across the modern cloud-native stack.

• Lead the design and execution of optimized data processing solutions at scale.
• Define and enforce data engineering architecture principles including data lineage, schema evolution, idempotency, fault tolerance, data partitioning strategies, and separation of concerns across processing layers.
• Spearhead Spark optimization strategies using PySpark to maximize performance and scalability in data processing.
• Architect and refine pluggable data transformation pipelines for efficient processing of large datasets.
• Establish architectural standards for data lakehouse patterns, batch vs. streaming paradigms, data contracts, and metadata-driven pipeline design.
• Design and develop AI Agents, MCP (Model Context Protocol) Servers, and MCP Tools to enable intelligent, autonomous workflows.
• Craft and optimize advanced prompts using prompt engineering techniques (chain-of-thought, few-shot, Zero-shot prompting) to drive reliable and high-quality LLM outputs in product workflows.
• Contribute to LLM fine-tuning and training efforts — including data curation, training dataset preparation, RLHF (Reinforcement Learning from Human Feedback), evaluation benchmarking, and model alignment for domain-specific use cases.
• Build and maintain microservices architectures that support modular, scalable product capabilities.
• Implement GitOps practices using Argo CD and Helm to advance CI/CD pipelines and operational efficiency on Kubernetes.
• Apply advanced SQL skills to refine data transformations across vast datasets.
• Integrate AI & ML technologies into high-performance data solutions, staying informed on emerging trends in agentic AI and LLM-based tooling.
• Utilize SAP HANA Spark to drive innovation in data engineering processes.
• Drive architectural decisions around data quality, observability, scalability, and reliability across the platform.
• Mentor junior engineers, fostering a culture of continuous improvement and technical excellence.
• Collaborate effectively with global stakeholders to achieve successful project outcomes.

 

What you bring

 

• Extensive experience in data engineering, distributed data processing, and expertise in SAP HANA or similar databases.
• Deep understanding of data engineering architecture principles — including data pipeline architecture, ELT/ETL patterns, Delta tables and their optimization (Z-ordering, compaction, vacuum, partitioning, liquid clustering), schema-on-read vs. schema-on-write, and pipeline orchestration best practices.
• Proficiency in Python (PySpark) is essential. 
• Hands-on experience in AI Agent development, MCP Server development, and MCP Tool development.
• Strong prompt engineering skills — experience designing, testing, and iterating on prompts for LLMs (GPT-4, Claude, Gemini, or similar) across diverse use cases including code generation, data transformation, and agentic reasoning.
• Strong expertise in microservices architecture and development.
• Deep knowledge of Kubernetes, Helm, and Argo CD for container orchestration and GitOps-driven deployments.
• Advanced understanding of Spark optimization and scalable data processing techniques.
• Proven experience in architecting data transformation pipelines with a focus on reusability, extensibility, and maintainability.
• Strong understanding of AI & ML technologies, LLMs, and industry trends in agentic systems.
• Knowledge of data governance, data quality frameworks, and observability in distributed data systems.
• Familiarity with LLM frameworks and tooling such as LangChain, LlamaIndex, LangGraph
• Effective communication skills within a global, multi-cultural environment.
• Proven track record of leadership in data processing, platform initiatives, and product engineering.

 

Where you belong

 

SAP is the market leader in enterprise application software, helping companies of all sizes and industries run at their best. As part of the Data & Analytics Organization, the Foundation Services team is pivotal to SAP’s Data & AI strategy, delivering next-generation data experiences that power intelligence across the enterprise. Located in Bangalore, India, our team drives cutting-edge engineering efforts in a collaborative, inclusive and high-impact environment, enabling innovation and integration across SAP’s data platforms.

#LI-DC3

 

 ​Bring out your best

SAP innovations help more than four hundred thousand customers worldwide work together more efficiently and use business insight more effectively. Originally known for leadership in enterprise resource planning (ERP) software, SAP has evolved to become a market leader in end-to-end business application software and related services for database, analytics, intelligent technologies, and experience management. As a cloud company with two hundred million users and more than one hundred thousand employees worldwide, we are purpose-driven and future-focused, with a highly collaborative team ethic and commitment to personal development. Whether connecting global industries, people, or platforms, we help ensure every challenge gets the solution it deserves. At SAP, you can bring out your best.

We win with inclusion

SAP’s culture of inclusion, focus on health and well-being, and flexible working models help ensure that everyone – regardless of background – feels included and can run at their best. At SAP, we believe we are made stronger by the unique capabilities and qualities that each person brings to our company, and we invest in our employees to inspire confidence and help everyone realize their full potential. We ultimately believe in unleashing all talent and creating a better and more equitable world.
SAP is proud to be an equal opportunity workplace and is an affirmative action employer. We are committed to the values of Equal Employment Opportunity and provide accessibility accommodations to applicants with physical and/or mental disabilities. If you are interested in applying for employment with SAP and are in need of accommodation or special assistance to navigate our website or to complete your application, please send an e-mail with your request to Recruiting Operations Team: Careers@sap.com
For SAP employees: Only permanent roles are eligible for the SAP Employee Referral Program, according to the eligibility rules set in the SAP Referral Policy. Specific conditions may apply for roles in Vocational Training.

EOE AA M/F/Vet/Disability:

Qualified applicants will receive consideration for employment without regard to their age, race, religion, national origin, ethnicity, age, gender (including pregnancy, childbirth, et al), sexual orientation, gender identity or expression, protected veteran status, or disability.
Successful candidates might be required to undergo a background verification with an external vendor.

Requisition ID: 426957  | Work Area: Software-Design and Development  | Expected Travel: 0 - 10%  | Career Status: Professional  | Employment Type: Regular Full Time   | Additional Locations: #LI-Hybrid.

Experience Level

Senior Level