Senior Quality Assurance Engineer

Specialist - Research Engineering

36433 36434 36484 3 Headcount

36433 Role 1 Enterprise QA Lead

   Position Title Enterprise QA Lead Senior QA Architect

   Experience 10 years in Software Quality Assurance

   About the Role

   Design and implement enterprise test architecture across functional regression performance  

    and nonfunctional testing Own quality strategy CICD test orchestration and release       

   quality gates

   Required Skills

   Core QA

     10 years enterprise QA with test architecture experience

     Expertise in functional regression smoke UAT testing

     Automated testing frameworks Selenium pytest TestNG Cucumber

     API testing Postman Rest Assured Swagger

   CICD DevOps

     Azure DevOps andor GitHub Actions test orchestration

     Quality gate implementation with auto passfail scoring

     Embedding automated tests in CICD pipelines

   Performance Engineering

     Handson load testing tools JMeter k6 Locust Gatling

     Performance workload modeling peak stress soak testing

     Resiliency and chaos testing concepts

   Data Governance

     SQL proficiency for data validation

     Familiarity with data lineagegovernance tools Purview Unity Catalog

     Test data management and synthetic data generation

   Leadership

     Led QA teams 5 members

     Stakeholder communication and quality reporting

     Regulated industry experience auditcompliance controls

   NicetoHave

     Basic understanding of AIML systems

     Cloud platforms Azure AWS

     Observability tools Splunk Datadog Grafana

   Key Deliverables

   1 Enterprise test architecture functional regression performance

   2 Unified test taxonomy and coverage model

   3 Automated test harness in CICD pipelines

   4 Performance workload models and loadtesting framework

   5 Quality gates in release pipelines with scoring

   6 Golden dashboards pass rate latency error rate coverage

36434 AIML QA Specialist

   Position Title AIML Evaluation Engineer LLM QA Specialist

   Experience 5 years in QA with 2 years in AIML systems

   About the Role

   Build and own LLM evaluation framework and AI agent testing suite Define metrics build       

   evaluation pipelines ensure AI model quality across hallucination grounding consistency    

   and correctness

   Required Skills

   AIML Evaluation

     Handson LLM evaluation experience GPT Claude Llama

     Evaluation metrics BLEU ROUGE perplexity F1 exact match

     Hallucination rate grounding rate factual accuracy measurement

     Toxicity and safety evaluation frameworks

   Technical

     Strong Python programming

     ML frameworks HuggingFace LangChain LlamaIndex

     Prompt engineering and testing methodologies

     API testing for AI endpoints

   AI Agent Testing

     Multistep AI agent workflow understanding

     Tool chaining function calling agent orchestration testing

     Context injection memory recovery path validation

   Data MLOps

     Dataset versioning and synthetic test data generation

     Model registry and promotion pipeline concepts

     AB testing and model comparison frameworks

   NicetoHave

     Evaluation frameworks DeepEval RAGAS TruLens promptfoo

     RAG evaluation experience

     Chaosresiliency testing for AI systems

     Regulated industry background

   Key Deliverables

   1 AI model evaluation suite hallucination grounding toxicity correctness

   2 AI agent workflow evaluation suite tool chaining recovery paths

   3 PoC validation framework for new AI capabilities

   4 LLM metrics integrated into quality dashboards

   5 Evaluation pipelines in CICD for model release gates

36484

Senior QA Engineer Playwright Python AI Data Platform Testing

Role Summary

EYai Simulation Hub is seeking a Senior QA Engineer with strong experience owning test strategy test plans and endtoend E2E validation and who is handson with test automation using Playwright and Python This role focuses on assuring quality across modern web platforms and AIdatacentric systems including synthetic data generators Python services APIs and workflow orchestration

You will design the E2E quality approach build reliable automation suites integrate tests into CICD and help teams adopt a shiftleft quality mindset with strong traceability and reporting

Key Responsibilities

1 Test Strategy Plans End