Technical Architect

Technical Architect

Senior Databricks Engineer – SAS Modernisation & Migration

Role Summary

Looking for a strong Databricks Engineer who understands how SAS platforms work and can modernise SAS workloads into Databricks. The role covers discovery, migration, validation, optimisation and production deployment of SAS workloads onto Databricks. The individual should be capable of understanding SAS business logic and rebuilding it efficiently using Databricks, PySpark and modern data engineering practices.

Core Skills (Must Have)

Databricks (Primary Skill)

• Azure Databricks / Databricks Lakehouse
• PySpark
• Spark SQL
• Python
• Delta Lake
• Unity Catalog
• Databricks Workflows
• Medallion Architecture (Bronze/Silver/Gold)
• Performance tuning and optimisation
• CI/CD and Git-based development
• Production support and troubleshooting

SAS (Strong Working Knowledge)

• Base SAS
• SAS DATA Step
• PROC SQL
• SAS Macros
• SAS Enterprise Guide
• SAS Grid / SAS Viya
• Batch scheduling and job dependencies
• SAS datasets, libraries and file processing
• Understanding of SAS business logic and data lineage

Key Responsibilities

SAS Discovery & Assessment

• Analyse SAS estate and identify dependencies, business logic and migration complexity.
• Inventory SAS jobs, datasets, macros and interfaces.
• Classify workloads for retire, refactor, re-engineer or migrate.

SAS to Databricks Migration

• Convert SAS code to PySpark/Spark SQL.
• Re-engineer legacy SAS processing into scalable Databricks solutions.
• Develop Delta Lake pipelines and Databricks workflows.
• Support automated conversion and manual remediation where required.

Validation & Reconciliation

• Compare SAS and Databricks outputs.
• Build automated reconciliation frameworks.
• Perform record count, aggregation and business logic validation.
• Support UAT and business sign-off.

Architecture & Engineering

• Design Databricks Lakehouse solutions.
• Implement governance using Unity Catalog.
• Optimise performance, scalability and cost.
• Support Dev, Test and Production deployments.

Experience Required

• 8+ years in Data Engineering / Analytics.
• 4+ years hands-on Databricks experience.
• 3+ years working with SAS environments.
• Previous experience in SAS to Databricks, SAS to PySpark or similar data-platform modernisation programs.
• Experience in large enterprise environments involving migration, reconciliation and production cutovers.