Senior Software Engineer - Confluent Kafka Team

At IBM Software, we transform client challenges into solutions. Building the world’s leading AI-powered, cloud-native products that shape the future of business and society. Our legacy of innovation creates endless opportunities for IBMers to learn, grow, and make an impact on a global scale. Working in Software means joining a team fueled by curiosity and collaboration. You’ll work with diverse technologies, partners, and industries to design, develop, and deliver solutions that power digital transformation. With a culture that values innovation, growth, and continuous learning, IBM Software places you at the heart of IBM’s product and technology landscape. Here, you’ll have the tools and opportunities to advance your career while creating software that changes the world.

About the Role:

We are looking for a Senior Software Engineer to join the Tableflow team. In this role, you will build the critical components that turn Confluent Cloud data streams into open table formats. You will bridge the gap between operational streaming (Kafka) and analytical data lakes (Apache Iceberg, Delta Lake) by building the control plane and data plane components that materialize topics into tables in real-time.

About the Team:

We are a team of passionate engineers who love solving complex distributed systems and infrastructure problems. The Tableflow team creates the "intelligent connective tissue" between Confluent’s data streaming platform and the modern data lakehouse. The team’s work focuses on two core pillars:

• Control Plane: Building the robust orchestration layer that manages the entire lifecycle of tables across AWS, Azure, and GCP.

• Open Table Formats and Ecosystem: Building the translation logic to publish Kafka data as Tables (Apache Iceberg and Delta Lake) and also integrates with the Lakehouse ecosystem (AWS Glue, Unity Catalog).

What You Will Do:

• Design and implement data plane components that publish data into open-table formats and integrate with the ecosystem.

• Develop control plane services and APIs that orchestrate table lifecycles and resource management.

• Contribute to relevant open-Source projects such as Apache Iceberg.

• Own code quality, test coverage, documentation, and maintainability for your components.

• Drive technical designs, leading discussions to curate and moderate decisions with stakeholders.

• Collaborate cross-functionally with Product, Design, and other engineering teams to deliver complex features.

• Lead operational excellence by resolving high-severity incidents, building observability tools, and designing automated workflows to ensure 99.99% availability.

• Be an active team player by leading meetings, taking initiative, and participating in on-call rotations.

• A degree in Computer Science, or equivalent experience.

• 3+ years of hands-on software engineering experience.

• Hands-on experience building services on public clouds (AWS, Azure, or GCP).

• Self starter who is able to quickly ramp up on the important bits of our overall product.

• Good understanding of distributed systems design and development.

• Experience running production services, debugging complex integration issues, and driving post-mortems.

• Ability to move fast and iterate with a sense of urgency while delivering results without surprises.

• A growth mindset and the ability to collaborate effectively with team members and take on customer problems.

• Experience using and/or working on data infrastructure such as SQL databases, Apache Kafka, Apache Flink or Apache Spark.

• Hands-on experience with Open-Table Formats (Apache Iceberg, or Delta Lake) and Data Catalogs (Unity Catalog or AWS Glue).

• Interest in evangelism: Giving talks at tech conferences, writing blog posts evangelizing your work

In a world where technology never stands still, we understand that, dedication to our clients success, innovation that matters, and trust and personal responsibility in all our relationships, lives in what we do as IBMers as we strive to be the catalyst that makes the world work better.

Being an IBMer means you’ll be able to learn and develop yourself and your career, you’ll be encouraged to be courageous and experiment everyday, all whilst having continuous trust and support in an environment where everyone can thrive whatever their personal or professional background.

Our IBMers are growth minded, always staying curious, open to feedback and learning new information and skills to constantly transform themselves and our company. They are trusted to provide on-going feedback to help other IBMers grow, as well as collaborate with colleagues keeping in mind a team focused approach to include different perspectives to drive exceptional outcomes for our customers. The courage our IBMers have to make critical decisions everyday is essential to IBM becoming the catalyst for progress, always embracing challenges with resources they have to hand, a can-do attitude and always striving for an outcome focused approach within everything that they do.

Are you ready to be an IBMer?

IBM’s greatest invention is the IBMer. We believe that through the application of intelligence, reason and science, we can improve business, society and the human condition, bringing the power of an open hybrid cloud and AI strategy to life for our clients and partners around the world.

Restlessly reinventing since 1911, we are not only one of the largest corporate organizations in the world, we’re also one of the biggest technology and consulting employers, with many of the Fortune 500 companies relying on the IBM Cloud to run their business. 

At IBM, we pride ourselves on being an early adopter of artificial intelligence, quantum computing and blockchain. Now it’s time for you to join us on our journey to being a responsible technology innovator and a force for good in the world.

IBM is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, genetics, pregnancy, disability, neurodivergence, age, or other characteristics protected by the applicable law. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.

When applying to jobs of your interest, we recommend that you do so for those that match your experience and expertise. Our recruiters advise that you apply to not more than 3 roles in a year for the best candidate experience. For additional information about location requirements, please discuss with the recruiter following submission of your application.