AI Evaluation Analyst

Role Overview

Key Responsibilities

• Assess AI-generated content for accuracy, relevance, safety, and consistency.
•  Apply established evaluation frameworks, scoring rubrics, and quality benchmarks.
•  Perform both qualitative and quantitative analysis of model outputs and performance.
•  Identify edge cases, recurring failure patterns, and areas that require improvement.
•  Document findings clearly and provide practical, actionable recommendations.
•  Work closely with data scientists, engineers, and product teams to refine models and workflows.
•  Support testing cycles, experiments, and benchmarking efforts.
•  Handle sensitive or confidential data with care and attention to detail.

Skills and Qualifications

• Strong analytical and critical thinking abilities.
•  Ability to interpret guidelines and consistently apply structured evaluation criteria.
•  Excellent written communication and documentation skills.
•  Comfortable working with datasets, spreadsheets, and reporting tools.
•  Basic understanding of AI concepts such as machine learning, NLP, or large language models is a plus.
•  Highly detail-oriented with the ability to manage repetitive review tasks accurately.
•  Capable of working independently while meeting quality standards and turnaround times.

Preferred Background

• Experience in quality assurance, data analysis, research, or content review roles.
•  Exposure to AI products, prompt evaluation, model testing, or data annotation workflows.
•  Comfort working in fast-paced, experimental, and rapidly evolving environments.