AI Engineer - Model Evaluation and Quality Assurance

AI Engineer, Evaluation & Quality - 11319

Coupa makes margins multiply through its community-generated AI and industry-leading total spend management platform for businesses large and small. Coupa AI is informed by trillions of dollars of direct and indirect spend data across a global network of 10M+ buyers and suppliers. We empower you with the ability to predict, prescribe, and automate smarter, more profitable business decisions to improve operating margins.
Why join Coupa?
🔹 Pioneering Technology: At Coupa, we're at the forefront of innovation, leveraging the latest technology to empower our customers with greater efficiency and visibility in their spend.🔹 Collaborative Culture: We value collaboration and teamwork, and our culture is driven by transparency, openness, and a shared commitment to excellence.🔹 Global Impact: Join a company where your work has a global, measurable impact on our clients, the business, and each other. 
Learn more on Life at Coupa blog and hear from our employees about their experiences working at Coupa. 
The Impact of a AI Engineer, Evaluation & Quality at Coupa:    Coupa's AI platform serves thousands of enterprise customers, and model quality directly impacts business outcomes. The AI Engineer, Evaluation & Quality will build the evaluation infrastructure that ensures our AI models meet rigorous accuracy standards. You will design benchmarks, implement automated evaluation pipelines, and build dashboards that track model performance across iterations.

What You’ll Do

• Build and maintain automated evaluation pipelines for AI model quality.
• Implement task-specific benchmarks and test suites.
• Design quality dashboards tracking accuracy, regression, and safety metrics.
• Implement automated regression testing for every model iteration.
• Build comparison frameworks for side-by-side evaluation of model variants.
• Analyze evaluation results to identify failure modes and report to the ML team.
• Maintain evaluation datasets: versioning, quality validation, coverage analysis.
• Support A/B testing infrastructure for production model validation.

What You Will Bring to Coupa

• 3+ years of software engineering or quality engineering experience.
• Proficiency in Python with strong testing and automation skills.
• Experience with statistical analysis and data visualization.
• Understanding of ML model evaluation concepts (precision, recall, F1, human eval).
• Experience building automated test frameworks and CI/CD pipelines.
• Familiarity with dashboarding tools.
• Strong analytical and problem-solving skills.
• BS in Computer Science, Statistics, or equivalent experience.

Coupa complies with relevant laws and regulations regarding equal opportunity and offers a welcoming and inclusive work environment. Decisions related to hiring, compensation, training, or evaluating performance are made fairly, and we provide equal employment opportunities to all qualified candidates and employees. 
Please be advised that inquiries or resumes from recruiters will not be accepted.
By submitting your application, you acknowledge that you have read Coupa’s Privacy Policy and understand that Coupa receives/collects your application, including your personal data, for the purposes of managing Coupa's ongoing recruitment and placement activities, including for employment purposes in the event of a successful application and for notification of future job opportunities if you did not succeed the first time. You will find more details about how your application is processed, the purposes of processing, and how long we retain your application in our Privacy Policy.

Experience Level

Senior Level