Principal Data Engineer

Principal Engineer - Data Engineering

Company:

Marsh Corporate

Description:

We are seeking a talented individual to join our Tech team at MMC Corporate.This role will be based in Gurgaon/ Noida/Pune/Mumbai. This is a hybrid role that has a requirement of working at least three days a week in the office.

Principal Engineer   – Data Engineering

What can you expect?

As a Data Engineer, you will be responsible for designing and implementing scalable data pipelines and AI Based solution using Databricks. You will handle end-to-end ETL/ELT processes, manage large datasets, and work with tools like Python, PySpark, and AWS S3 to ensure data is transformed and optimized for analytical use.

You’ll work on cutting-edge cloud and hybrid data projects, transforming raw data into meaningful insights and AI Analytics. You’ll be hands-on from day one, collaborating closely with architects and business stakeholders.

What is in it for you?

• Hybrid way of working
• Diversify your experience and learn new skills
• Opportunity to work with stakeholders globally to learn and grow

We will count on you to:

• Develop and maintain data pipelines using Databricks and the Medallion Architecture (Bronze, Silver, Gold layers).
• Design AI Based Solution using Databricks Genie and E2E integration.
• Knowledge of exposing/consuming Databricks features via API using cloud-native tools or other application.
• Write data transformation scripts using Python and PySpark.
• Store and manage real time data in AWS S3 and integrate with other cloud-based services.
• Use SQL to query, clean, and manipulate large datasets.
• Collaborate with cross-functional teams to ensure data is accessible for business intelligence and analytics.
• Monitor and troubleshoot data pipelines for performance and reliability.
• Document data processes and follow best practices for scalability and maintainability.
• Ingest and process structured and unstructured data across batch and streaming sources.

What you need to have: 

• Experience with Databricks components like :  Pipeline, scheduled / event based job , Genie , Unity Catalog and Datawarehouse.
• Proficiency in Python, PySpark, and SQL for data processing and transformation using AWS S3 data.
• Experience in Data Governance , data access security , and information of configuring Job compute for different Jobs in Databricks.
• Familiarity with version control using Git.
• Understanding of Databricks API and its integration with different Tools and application.
• Bulk data and real time data streaming understanding.
• Experience with Delta Lake and other Databricks technologies. Knowledge of additional AWS services (e.g., Athena, Glue, Lambda, S3, DMS ).

Why join our team:

• We help you be your best through professional development opportunities, interesting work and supportive leaders.
• We foster a vibrant and inclusive culture where you can work with talented colleagues to create new solutions and have impact for colleagues, clients and communities.
• Our scale enables us to provide a range of career opportunities, as well as benefits and rewards to enhance your well-being.

Marsh (NYSE: MRSH) is a global leader in risk, reinsurance and capital, people and investments, and management consulting, advising clients in 130 countries. With annual revenue of over $27 billion and more than 95,000 colleagues, Marsh helps build the confidence to thrive through the power of perspective. For more information, visit corporate.marsh.com, or follow us on LinkedIn and X.Marsh is committed to embracing a diverse, inclusive and flexible work environment. We aim to attract and retain the best people and embrace diversity of age, background, caste, disability, ethnic origin, family duties, gender orientation or expression, gender reassignment, marital status, nationality, parental status, personal or social status, political affiliation, race, religion and beliefs, sex/gender, sexual orientation or expression, skin color, or any other characteristic protected by applicable law.Marsh is committed to hybrid work, which includes the flexibility of working remotely and the collaboration, connections and professional development benefits of working together in the office. All Marsh colleagues are expected to be in their local office or working onsite with clients at least three days per week. Office-based teams will identify at least one “anchor day” per week on which their full team will be together in person.

Experience Level

Senior Level