Principal Data Platform Engineer

Principal Data Platform Engineer

Overview

Are you passionate about data platforms and tools? Are you an open-minded, structured thinker who is passionate about building systems at scale. Do you understand data platforms, know how to build them and want to help our users to make data actionable? If so, this role with our team at Atlassian is for you.
We are looking for a Principal Software Engineer to join our Data Platform Team to build a world class data platform that powers crucial business decisions throughout the organization.
You will partner with analytics teams, data engineers and data scientists across various initiatives and work with them to understand the gaps and bring your findings back to the team to build these capabilities.

Working at Atlassian

Atlassians can choose where they work – whether in an office, from home, or a combination of the two. That way, Atlassians have more control over supporting their family, personal goals, and other priorities. We can hire people in any country where we have a legal entity.

Responsibilities

We build analytical data platforms for Atlassian that serve both Engineering and Data Engineering to help them generate valuable insights from data.

As a Principal Engineer on this team you will:

• Own the long term architecture and technical direction for core platform capabilities and be accountable for their reliability, cost efficiency and adoption

• Design and build foundational platform components that other engineering teams build on top of, across areas like ingestion, transformation, orchestration, compute and consumption

• Architect robust multi-tenant platform standards by establishing clear cross-layer contracts, tenant isolation, and SLOs to enable autonomous producers and consumers, while driving compute and storage efficiency across workloads

• Lead cross team technical initiatives and architecture reviews, influence platform strategy with senior stakeholders and act as a technical mentor and multiplier for Senior engineers

• Stay hands on. You will be in the code on the hardest problems, not just reviewing designs

Qualifications

Required

• 12+ years of software or data engineering experience, with at least 6 years building data platform infrastructure

• You have built a data platform or a significant component of one and operated it in production. For example an ingestion framework, a transformation framework, a compute or query layer, an orchestration layer, a metadata or lineage service or a data quality framework

• You can describe the scale that component ran at in concrete numbers. Data volume, ingest throughput, number of pipelines or jobs, query concurrency and latency, number of onboarded teams or tenants and infrastructure cost.

• Deep understanding of big data and distributed systems challenges including partitioning, shuffle behaviour, skew, idempotency and backfills

• Experience designing both batch and streaming architectures and clear judgement on when each is the right answer

• Expert level programming skills in language such as Java, Kotlin or Python and built solutions on public cloud offerings such as Amazon Web Services

• Experience with big data processing and storage technologies such as Spark, DBT and with open table formats such as Iceberg or Delta Lake

• A track record of owning ambiguous cross team problems end to end and of influencing without authority

• Degree in Computer Science, EE or a related STEM discipline or equivalent practical experience

Nice to have

• Experience with high performance analytical or MPP databases such as StarRocks, Druid, ClickHouse or Pinot

• Experience with change data capture and event driven ingestion at high volume

• Experience building a semantic layer or metrics store and governing shared metric definitions

• Experience with the data foundations of ML platforms, such as point-in-time correct training data, online and offline feature parity or backfilling features at scale

• Experience with data governance, access control, lineage and compliance, including regulated environments such as FedRAMP

• Experience running platform components on Kubernetes

• Experience with multi region data architectures, including replication, failover and consistency tradeoffs

Benefits & Perks

Atlassian offers a wide range of perks and benefits designed to support you, your family and to help you engage with your local community. Our offerings include health and wellbeing resources, paid volunteer days, and so much more. To learn more, visit go.atlassian.com/perksandbenefits.

About Atlassian

At Atlassian, we're motivated by a common goal: to unleash the potential of every team. Our software products help teams all over the planet and our solutions are designed for all types of work. Team collaboration through our tools makes what may be impossible alone, possible together.

We believe that the unique contributions of all Atlassians create our success. To ensure that our products and culture continue to incorporate everyone's perspectives and experience, we never discriminate based on race, religion, national origin, gender identity or expression, sexual orientation, age, or marital, veteran, or disability status. All your information will be kept confidential according to EEO guidelines.

To provide you the best experience, we can support with accommodations or adjustments at any stage of the recruitment process. Simply inform our Recruitment team during your conversation with them.

To learn more about our culture and hiring process, visit go.atlassian.com/crh.

In line with local law, identity verification (which may include use of biometric data) is a condition of employment with Atlassian for employment fraud purposes.

Experience Level

Senior Level