caferemote
Clera

Expert Talent Pool, Evals and Benchmarks (Associate)

Remote Worldwide Full-time $30 – $60 / hour Posted 2 days ago
Apply now
Work from
Worldwide
Contract
Full-time
Salary
$30 – $60 / hour
Posted
20 November 2026
Apply by
19 January 2027

You will help measure model capabilities by creating rigorous questions, rubrics, and comparative assessments, contributing to better understanding of AI performance.

About the role

Join a team that designs and runs human-led evaluations of AI models.

Responsibilities

  • Write questions and tasks that test specific model capabilities.
  • Develop rubrics that define strong, partial, and weak answers.
  • Create reference answers and edge-case scenarios to challenge models.
  • Grade model outputs against rubrics and explain each score clearly.
  • Analyze incorrect answers, group error patterns, and document findings.
  • Compare multiple models using the same prompts.
  • Validate benchmarks and flag questions that may no longer provide reliable measurements.
  • Support small studies of how people evaluate or interact with model outputs.

Requirements

  • Zero to five years of professional experience; prior AI-data work is not required.
  • Experience or training in QA, test engineering, assessment design, psychometrics, UX research, data analysis, ML evaluation, research support, or a subject area involving rigorous question writing.
  • Experience writing test cases, exam questions, rubrics, or evaluation tasks, and reviewing work consistently against written criteria.
  • Ability to use spreadsheets or Python to organize and analyze results, and to identify and document error causes.
  • Experience with evaluation tools such as Inspect, lm-eval-harness, HELM, or promptfoo, model comparisons, or user and rating studies is useful.
  • Clear, professional written communication.

Salary

Compensation is $30 to $60 per hour, depending on project, field, and experience.

Compensation & Benefits

Payment is for hours worked.

Location

Fully remote. Projects typically run from one week to several months.

Discussion

No comments yet. Ask a question or share a tip about this role.

Apply now

More jobs from Clera

View company →

Product Engineer (Consumer, Design Heavy)

Clera
Remote United States Full-time $200K – $350K / year

About the Role Own the full lifecycle of consumer products, from spotting user needs and shaping product direction to building and shipping polished…

4h ago Software Development

Freelance Engineer

Clera
Remote Worldwide Full-time

About the Role Work independently on software projects for a logistics technology company developing AI tools for freight operations. This role helps deliver…

4h ago Software Development

Expert Talent Pool, Agents and Reinforcement Learning (Associate, 0-5 Years)

Clera
Remote Worldwide Full-time $35 – $70 / hour

About the Role Join a paid talent pool supporting AI projects focused on agents and reinforcement learning. You will help build training environments and…

4h ago Data

Expert Talent Pool, Evals and Benchmarks

Clera
Remote Worldwide Full-time $75 – $175 / hour

About the Role Join a network of senior evaluation experts supporting AI research projects. When projects match your experience, you may lead benchmark design,…

5h ago Data

Similar remote jobs

Browse all →

Oracle PL/SQL Developer

Bright Vision Technologies
Remote United States Full-time $120K – $180K / year

Oracle PL/SQL Developer – Remote Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the…

4h ago Data

Senior Data Scientist (R14369)

Oportun
Remote India Full-time

Who We Are Oportun (Nasdaq: OPRT) is a mission-driven financial services company that puts its members' financial goals within reach. With intelligent borrowing, savings, and budgeting…

4h ago Data

AI for Data & Analytics Coach

Leland
Remote United States Contract

Leland is building the home for ambition in the age of AI. We connect people with the experts, programs, and communities they need to land the job, get into the school, or upskill for a…

4h ago Data

Azure Data Engineer ( EU REMOTE)

IRIUM Portugal
Remote Albania, Andorra, Austria, Belarus, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, Cyprus, Czechia, Denmark, Estonia, Faroe Islands, Finland, France, Germany, Gibraltar, Greece, Guernsey, Holy See (Vatican City State), Hungary, Iceland, Ireland, Full-time €2K – €3K / month

At IRIUM, we want you to always pursue your dreams! Here, get ready to achieve your goals while enjoying the journey. We are currently looking for an Azure Data Engineer to join our team!…

4h ago Data

Applied Scientist (LLM)

Squad
Remote Ukraine Contract

Team Summary Our distributed team is looking for an experienced Applied Scientist with a strong background in Large Language models to develop high-performance Generative AI features across…

4h ago Data

Engineering Manager, Data Feeds

Chainlink Labs
Remote Canada, United States Full-time $129K – $304K / year

About Chainlink Labs Chainlink Labs is one of the primary contributing developers of Chainlink, the standard for onchain finance enabling high-quality data, secure cross-chain…

4h ago Data

Get remote jobs the moment they are posted

Choose your categories and skills. We push matching jobs to your device. No email, ever.