Clinical Evaluation Operations Associate

Clinical Evals Ops Associate, AHS

We are looking for a detail-oriented and operationally strong Associate to join our Healthcare AI Evaluation team. This role sits at the intersection of clinical operations and healthcare technology quality assurance, supporting the management of a team of clinician evaluators while also contributing to the evaluation of AI agent outputs across non-clinical parameters. The ideal candidate brings a blend of healthcare domain familiarity, strong organizational skills, and a structured analytical mindset to help ensure our team’s work meets the highest standards of safety, quality, and user experience. They should be willing to roll up their sleeves to with rigor and consistency to ensure high degree of effectiveness and efficiency.

Key job responsibilities
Clinician Management & Operations

• Coordinate day-to-day operations of the clinician evaluators, including scheduling, task assignment, and workload tracking
• Serve as the primary point of contact for clinician evaluators — managing onboarding, training, and ongoing communication
• Maintain evaluator rosters, track participation, performance and availability, and ensure adequate coverage for evaluation cycles
• Manage documentation of evaluation protocols, guidelines, and standard operating procedures (SOPs)

Non-Clinical AI Evaluation

• Independently evaluate AI agent outputs across non-clinical quality dimensions by applying structured rubrics and evaluation frameworks consistently across assigned cases, for example: a) Task Completion: Assessing whether the AI agent adequately addresses the user's stated need, b) Tone & Communication: Evaluating appropriateness, empathy, and clarity of language.
• Contribute to the development and refinement of evaluation rubrics and guidelines as the program evolves by documenting findings

Program Support

• Assist in preparing evaluation reports, summaries, and dashboards for internal stakeholders
• Support tooling and workflow improvements to increase evaluation efficiency and quality
• Participate in cross-functional reviews and contribute to continuous improvement initiatives

Basic Qualifications

- Bachelor's degree
- Speak, write, and read fluently in English
- • 1–3 years of experience in healthcare operations, team coordination, quality assurance, or a related role
- • Familiarity with healthcare terminology, clinical workflows, or patient communication standards
- • Proficiency with data handling, productivity and program management tools (Excel)

Preferred Qualifications

- Knowledge of Microsoft Office products and applications
- Speak, write, and read fluently in Hindi
- Work a flexible schedule/shift/work area, including weekends, nights, and/or holidays

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Experience Level

Mid Level