AI Tester

Associate Engineer - AI Tester

Company Description

About Sutherland

Artificial Intelligence. Automation.Cloud engineering. Advanced analytics.For business leaders, these are key factors of success. For us, they’re our core expertise.

We work with iconic brands worldwide. We bring them a unique value proposition through market-leading technology and business process excellence.

We’ve created over 200 unique inventions under several patents across AI and other critical technologies. Leveraging our advanced products and platforms, we drive digital transformation, optimize critical business operations, reinvent experiences, and pioneer new solutions, all provided through a seamless “as a service” model.

For each company, we provide new keys for their businesses, the people they work with, and the customers they serve. We tailor proven and rapid formulas, to fit their unique DNA.We bring together human expertise and artificial intelligence to develop digital chemistry. This unlocks new possibilities, transformative outcomes and enduring relationships.
Sutherland

Unlocking digital performance. Delivering measurable results.

Job Description

We are hiring a quality-focused AI Tester to design and execute validation frameworks for AI models and systems. Reporting to the AI Manager, this role ensures that AI solutions meet accuracy, performance, reliability, and ethical compliance benchmarks before production deployment.

Key Responsibilities:

• Develop and execute test plans and cases for AI/ML models, GenAI solutions, and agent-based systems.
• Validate model outputs for accuracy, precision, recall, latency, and interpretability across diverse datasets.
• Conduct bias testing, data drift analysis, and adversarial robustness validation.
• Collaborate with AI developers, data scientists, and business analysts to define acceptance criteria.
• Automate testing pipelines and integrate with CI/CD environments where applicable.
• Maintain traceability matrices, defect logs, and model validation documentation.

Qualifications

Required Qualifications:

• 3–5 years of experience in software testing or QA, with at least 1–2 years in AI/ML or data-centric testing.
• Strong understanding of AI model evaluation metrics, data sampling, and validation techniques.
• Experience using testing frameworks (e.g., PyTest, Great Expectations) and visualization tools.
• Familiarity with Python, JSON, and REST APIs.
• Bachelor’s degree in Computer Science, Statistics, or a related field.

Preferred Skills:

• Experience testing LLMs, GenAI prompts, or RAG workflows.
• Exposure to responsible AI frameworks and model audit practices.
• ISTQB or other QA certifications are a plus.

Experience Level

Mid Level