Software Engineer / Senior Software Engineer - Azure Data Engineering

ROP - SE2/SSE - Azure Data

Overview

Microsoft is a company where passionate innovators come to collaborate, envision what can be and take their careers further. This is a world of more possibilities, more innovation, more openness, and the sky is the limit thinking in a cloud-enabled world. 

Microsoft's Azure Data engineering team is leading the transformation of analytics in the world of data with products like databases, data integration, big data analytics, messaging & real-time analytics, and business intelligence. The products in our portfolio include Microsoft Fabric, Azure SQL DB, Azure Cosmos DB, Azure PostgreSQL, Azure Data Factory, Azure Synapse Analytics, Azure Service Bus, Azure Event Grid, and Power BI. Our mission is to build the data platform for the age of AI, powering a new class of data-first applications and driving a data culture. 

Our team is part of Azure Data, building reliable, large-scale distributed systems for data engineering. We develop Fabric Materialized Lake Views, which automatically refresh analytical data in Microsoft Fabric so customers can query up-to-date results without manual pipelines. We also maintain Azure HDInsight in production, the PaaS platform running open-source Hadoop, Spark, HBase, and Kafka. You will get hands-on experience with both. 

Software engineer 2 : You will design and ship features for our Fabric-based SaaS platform with minimal supervision. You will write high-quality code, debug complex distributed systems, and work closely with Spark and Spark-based extension frameworks. You will own features through the full lifecycle: requirements, design, implementation, deployment, and production support. You will also maintain HDInsight components in production. This role will deepen your expertise in distributed systems, cloud platform engineering, and large-scale data infrastructure. 

Senior Software engineer : You will architect and build core infrastructure for our Fabric-based SaaS platform. You will drive technical decisions, write high-quality code, and own features end-to-end from requirements through production support. You will work closely with Spark and Spark-based extension frameworks, use AI tools for development, and mentor other engineers. You will also maintain HDInsight components in production. This role will sharpen your ability to make architectural trade-offs at scale, give you ownership of features used by customers worldwide, and develop your technical leadership.  

Microsoft's mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. 

Successful candidates will have strong experience in one or more of these areas: designing and coding cloud-scale SaaS products; building data lakes or data engineering tools; working with big data technologies such as Spark, Kafka, Hadoop, or HBase; enterprise software development in C#, Scala, Java, or Python; and working with the Azure stack (Storage, Compute, Networking, Fabric, Purview, Synapse, AKS, DevOps, Data Factory, Power BI). 

 

Responsibilities

• Write extensible, maintainable code in C#, Java, Scala, or Python for Fabric Materialized Lake View services and HDInsight components. Use AI tools and coding best practices across the development lifecycle. Produce code with few defects and clear test coverage. 

• Design data refresh, scheduling, and query optimisation features with minimal supervision. Evaluate design options, document trade-offs, and begin owning or co-owning the architecture of Fabric data engineering components. 

• Review code from teammates for correctness, test coverage, security risks, and adherence to team standards. Coach junior engineers through code reviews and provide feedback that raises code quality across the team. 

• Debug complex issues in distributed systems running on Azure, Linux, and Windows. Use debugging tools, logs, and telemetry to verify assumptions before issues reach production. Conduct incident retrospectives for Fabric and HDInsight services; identify root causes, implement repairs, and prevent recurrence. 

• Run live site operations on a rotational, on-call basis. Diagnose and fix both simple and complex issues across Fabric and HDInsight workloads. Improve troubleshooting guides, tests, and telemetry to make on-call better for the team. 

• Integrate logging and instrumentation to gather telemetry on system health, performance, reliability, and security. Classify metrics, build dashboards, and create alerts that surface problems across the Materialized Lake View pipeline. 

• Work with product managers, technical leads, and partners across geographies to define customer requirements for Materialized Lake View features. Incorporate customer feedback into designs. Understand and advocate for the security and privacy needs of customers. 

Qualifications

Required Qualifications 

• Bachelor's Degree in Computer Science or related technical field AND 3+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python; OR equivalent experience. 

Preferred Qualifications 

• Master's + 4 years technical engineering experience; OR Bachelor's + 5 years technical engineering experience; OR equivalent experience. 

• Experience with the Azure stack including Storage, Compute, Networking, Fabric, Purview, Synapse, AKS, DevOps, Data Factory, or Power BI. 

• Experience with big data technologies such as Spark, Kafka, Hadoop, or HBase. 

• Experience building data lake or data engineering products, tools, or pipelines. 

• Familiarity with container-based architectures (Docker, Kubernetes). 

• Ability to debug complex distributed systems on Linux and/or Windows platforms.

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.