Senior Data Engineer

Custom Software Engineer

Project Role : Custom Software Engineer
Project Role Description : Design, build and configure applications to meet business process and application requirements.
Must have skills : Microsoft Azure Data Services, Microsoft Azure Databricks
Good to have skills : NA
Minimum 3 year(s) of experience is required
Educational Qualification : 15 years full time education

Summary: We’re seeking a visionary Senior Data Engineer to design and build on our Azure-based data platform. This role requires deep expertise in Databricks, Medallion Architecture, and Delta Lake, combined with strong orchestration skills using Azure Data Factory (ADF). You’ll also leverage data virtualization technologies including Denodo and Snowflake to enable unified, efficient access to data across the enterprise. Experience with Veeva Vault for life sciences data integration adds significant value. You will lead a team building scalable, secure, and high-performance data solutions that power analytics and AI initiatives. Roles & Responsibilities: • Design and implement scalable data pipelines using Databricks and Azure Data Factory • Design and maintain Medallion Architecture (Bronze, Silver, Gold layers) for structured and semi-structured data • Develop and optimize Delta Lake tables for ACID-compliant, performant data storage and querying • Lead integration and virtualization of diverse data sources leveraging Denodo and Snowflake platforms • Collaborate with teams to ingest, harmonize, and manage life sciences data from Veeva Vault and related systems • Drive data lake and warehouse development on Azure Synapse and Data Lake Storage Gen2 • Ensure robust data governance, lineage, and security across the platform • Mentor junior engineers and promote engineering best practices • Drive CI/CD automation and infrastructure-as-code for data workflows Professional & Technical Skills: • 5+ years of experience in data engineering, including 3+ years in databricks. • Deep hands-on experience with Databricks (PySpark, Delta Lake, etc.) • Strong understanding of Medallion Architecture and practical implementation • Expertise with Delta Tables: schema evolution, time travel, compaction, and performance tuning • Proficiency in Azure Data Factory for orchestration and automation • Solid knowledge of Azure services: Data Lake Storage Gen2, Synapse Analytics, Azure Functions • Advanced Python and SQL skills for data processing, scripting, and modeling • Exposure to data virtualization technologies such as Denodo and Snowflake for unified data access • Experience working with Veeva Vault data integration and lifecycle • Familiarity with Git, CI/CD pipelines, and DevOps practices Additional Information: - The candidate should have minimum 3 years of experience in Microsoft Azure Data Services. - This position is based at our Bengaluru office. - A 15 years full time education is required.