Lead Software Engineer - Production Support & SRE

Lead Software Engineer - Production Support & SRE

Job Responsibilities 

• Serve as the technical production owner for critical Recon services and client-facing platforms, driving reliability, availability, scalability, operational efficiency, and client experience through SRE principles.
• Lead technical response and restoration efforts during major production incidents, accelerating recovery and coordinating across engineering, infrastructure, cyber, and support teams.
• Partner with development teams to identify recurring failure patterns, drive root cause remediation, reduce technical debt, and prevent recurrence.
• Define and improve SLOs, SLIs, error budgets, reliability metrics, monitoring, telemetry, dashboards, alerting, and operational analytics.
• Define and deploy monitoring, metrics, and logging systems on AWS and implement/Enhance infrastructure automation via IaaC using Terraform
• Demonstrates site reliability principles and practices every day and champions the adoption of site reliability throughout the team
• Works toward becoming an expert on the applications and platforms in your remit while understanding their interdependencies and limitations. Maintain strong production governance, including access controls, operational procedures, audit readiness, runbooks, documentation, MEPC execution, and operational review processes.
• Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.
• Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.

• Evolves and debug critical components of applications and platforms. Provides comprehensive and ongoing guidance, tools, and solutions to support the firms’ growth

   

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 5+ years applied experience
• Extensive experience supporting large-scale, mission-critical production environments in high-volume transaction or payment platforms.
• Experience in applying Production Engineering or Site Reliability Engineering principles to improve reliability and operational performance with strong background in Site Reliability Engineering principles, Production Engineering, Application Support, Platform Engineering, or Technology Operations.
• Proven experience leading complex production incidents, root cause analysis, problem management, service restoration, and recurrence elimination.
• Experience with production observability technologies such as Grafana, Splunk, Dynatrace, or comparable platforms.
• Experience developing automation using scripting, system integrations, engineering tooling, CI/CD pipelines, or release automation.
• Experience with automation tools like Terraform or AWS CloudFormation
• Demonstrated experience leading effective use of approved AI-assisted software development tools (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security.
• Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching engineers on safe, compliant adoption within delivery practices
• Proficient with DevOps practices and CI/CD pipelines with excellent problem-solving and troubleshooting skills

• Practical cloud native experience and ability to tackle design and functionality problems independently with little to no oversigh

   

Preferred qualifications, capabilities, and skills 

• Hands-on experience with AWS and enterprise observability platforms including Grafana, Splunk, Dynatrace.
• Experience with GitHub Copilot, AI-assisted engineering, and prompt engineering for operational analysis, automation, troubleshooting, or reporting.