Senior Quality Auditor - Machine Learning Data Operations

Senior Quality Auditor, Artificial General Intelligence

The Senior Quality Auditor serves as a specialist who becomes the organization's deepest expert on 1-2 complex machine learning data workflows while maintaining hands-on audit execution. This role within the Machine Learning Data Operations job family leads Quality Audit Design (QuAD) teams for high-complexity workflows, handles the most ambiguous audit scenarios, and generates workflow-specific quality analysis that drives customer insights and workflow optimization.

Key job responsibilities
Workflow Expertise Development
- Become the organization's deepest expert on 1-2 assigned complex workflows through intensive pilot participation and continuous learning
- Participate in workflow pilots as an annotator to build foundational knowledge before production begins
- Develop comprehensive understanding of workflow conventions, edge cases, and quality requirements
Team Leadership
- Lead QuAD teams of 3-5 Quality Auditors for Template 2 (High-Engagement Support) workflows
- Provide day-to-day leadership and make real-time quality decisions during iteration phases
- Handle escalations from team members and serve as the go-to expert for workflow-specific questions
- Mentor and develop Quality Auditor team members through coaching, feedback, and knowledge sharing
Audit Execution
- Execute audits on the most ambiguous and complex scenarios, reducing escalations to Quality Specialists
- Maintain 60-70% hands-on audit execution to preserve expertise required for real-time decision-making
- Validate team audit decisions and ensure consistency across QuAD team members
- Support Template 1 workflows by auditing high-complexity scenarios requiring specialized judgment
Analysis & Insights
- Generate workflow-specific deep dives and thematic analysis providing insights beyond conventions adherence scores
- Identify error patterns, convention evolution, and areas of ambiguity within assigned workflows
- Provide actionable recommendations for workflow optimization and quality improvement
- Partner with DSLs and customers on workflow-specific quality insights
Strategic Collaboration
- Provide input to audit strategy design led by Quality Specialist I
- Collaborate with Quality Specialists on cross-workflow patterns and tactical improvements
- Participate in workflow design discussions with customers and stakeholders
- Support new-tool onboarding for operationalized and bootstrapped interfaces
- Track record of generating actionable insights from quality data.

Basic Qualifications

- Bachelor's degree or equivalent
- 2-4 years of experience in quality assurance, data operations, or a related field
- Experience working with machine learning data workflows or similar technical domains
- Hands-on experience in leadership and mentoring
- Strategic thinking about workflow optimization and process improvement
- Effective stakeholder communication skills with the ability to influence decision-makers and customers
- Domain expertise in STEM, language, or technical domains highly valued.

Preferred Qualifications

- Experience with explainable machine learning and artificial intelligence methodologies and tools
- Experience in leadership roles, overseeing teams conducting data analysis, process improvements, creative problem solving, and cross-functional collaboration
- Experience using data and metrics to drive actionable insights at scale
- Experience that includes strong analytical skills, attention to detail, and effective communication abilities
- Experience handling administrative details independently, while exercising good judgment in keeping team members adequately informed
- Familiarity with audit methodologies and quality frameworks
- Experience participating in pilot phases and workflow development

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Experience Level

Senior Level