Senior Quality Auditor - Machine Learning Data Operations

Senior Quality Auditor, Artificial General Intelligence

The Senior Quality Auditor serves as a specialist who becomes the organization's deepest expert on 1-2 complex machine learning data workflows while maintaining hands-on audit execution. This role within the Machine Learning Data Operations job family leads Quality Audit Design (QuAD) teams for high-complexity workflows, handles the most ambiguous audit scenarios, and generates workflow-specific quality analysis that drives customer insights and workflow optimization.

Key job responsibilities
Workflow Expertise Development
- Become the organization's deepest expert on 1-2 assigned complex workflows through intensive pilot participation and continuous learning
- Participate in workflow pilots as an annotator to build foundational knowledge before production begins
- Develop comprehensive understanding of workflow conventions, edge cases, and quality requirements
Team Leadership
- Lead QuAD teams of 3-5 Quality Auditors for Template 2 (High-Engagement Support) workflows
- Provide day-to-day leadership and make real-time quality decisions during iteration phases
- Handle escalations from team members and serve as the go-to expert for workflow-specific questions
- Mentor and develop Quality Auditor team members through coaching, feedback, and knowledge sharing
Audit Execution
- Execute audits on the most ambiguous and complex scenarios, reducing escalations to Quality Specialists
- Maintain 60-70% hands-on audit execution to preserve expertise required for real-time decision-making
- Validate team audit decisions and ensure consistency across QuAD team members
- Support Template 1 workflows by auditing high-complexity scenarios requiring specialized judgment
Analysis & Insights
- Generate workflow-specific deep dives and thematic analysis providing insights beyond conventions adherence scores
- Identify error patterns, convention evolution, and areas of ambiguity within assigned workflows
- Provide actionable recommendations for workflow optimization and quality improvement
- Partner with DSLs and customers on workflow-specific quality insights
Strategic Collaboration
- Provide input to audit strategy design led by Quality Specialist I
- Collaborate with Quality Specialists on cross-workflow patterns and tactical improvements
- Participate in workflow design discussions with customers and stakeholders
- Support new-tool onboarding for operationalized and bootstrapped interfaces
- Track record of generating actionable insights from quality data.

Basic Qualifications

- Bachelor's degree or equivalent
- 2-4 years of experience in quality assurance, data operations, or a related field
- Experience working with machine learning data workflows or similar technical domains
- Hands-on experience in leadership and mentoring
- Strategic thinking about workflow optimization and process improvement
- Effective stakeholder communication skills with the ability to influence decision-makers and customers
- Domain expertise in STEM, language, or technical domains highly valued.

Preferred Qualifications

- Familiarity with audit methodologies and quality frameworks
- Experience participating in pilot phases and workflow development
- Experience with explainable machine learning and artificial intelligence methodologies and tools
- Experience in leadership roles, overseeing teams conducting data analysis, process improvements, creative problem solving, and cross-functional collaboration
- Experience using data and metrics to drive actionable insights at scale
- Experience that includes strong analytical skills, attention to detail, and effective communication abilities
- Experience handling administrative details independently, while exercising good judgment in keeping team members adequately informed

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Experience Level

Senior Level