Software Engineer - PySpark and Data Engineering

Software Engineer, PySpark

Join us as a Software Engineer, PySpark

• This is an opportunity for a driven Software Engineer to take on an exciting new career challenge
• Day-to-day, you'll be engineering and maintaining innovative, customer centric, high performance, secure and robust solutions
• It’s a chance to hone your existing technical skills and advance your career while building a wide network of stakeholders
• We're offering this role at associate level

What you'll do

In your new role, you’ll be working within a feature team to engineer software, scripts and tools, as well as liaising with other engineers, architects and business analysts across the platform.

You’ll also be:

• Producing complex and critical software rapidly and of high quality which adds value to the business
• Working in permanent teams who are responsible for the full life cycle, from initial development, through enhancement and maintenance to replacement or decommissioning
• Collaborating to optimise our software engineering capability
• Designing, producing, testing and implementing our working software solutions
• Working across the life cycle, from requirements analysis and design, through coding to testing, deployment and operations

The skills you'll need

To take on this role, you’ll need a background in software engineering, software design, and architecture, and an understanding of how your area of expertise supports our customers. You’ll need at least six years of experience in PySpark, data engineering with strong expertise in ETL design, data quality testing, cleansing, monitoring, sourcing and exploration.

You'll need experience working with AWS services such as Glue, Lambda, EMR or Spark and S3. You'll also need programming proficiency in Python, Spark, and SQL and demonstrate working knowledge of Large Language Models or LLMs and AI-assisted development tools like GitHub Copilot, GitLab Duo along with practical prompt engineering skills.

You’ll also need:

• Experience of working with development and testing tools, bug tracking tools and wikis
• Experience in multiple programming languages or low code toolsets
• Experience of DevOps and Agile methodology and associated toolsets
• A background in solving highly complex, analytical and numerical problems
• Experience of implementing programming best practice, especially around scalability, automation, virtualisation, optimisation, availability and performance

Hours

45

Job Posting Closing Date:

21/05/2026

Experience Level

Mid Level