caferemote
mercor

AI Rubric Reviewer - Fully Remote | Up to $75/hr

Remote United States Contract $40 – $75 / hour Posted 3 days ago
Apply now
Work from
United States
Contract
Contract
Salary
$40 – $75 / hour
Posted
18 November 2026
Apply by
17 January 2027

Position: AI Evaluation Rubric ReviewerType:ContractCompensation:$40–$75/hourLocation:Remote

About the role

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Requirements

  • Substantial experience writing or reviewing rubrics for LLM evaluations.
  • Ability to read long agent transcripts and identify errors.
  • Precise writing skills and a commitment to data verification.

Role Responsibilities

  • Review AI-drafted tasks and evaluate model attempts for fairness and accuracy.
  • Assess if the grading rubric aligns with the prompt and correct any unfair criteria.
  • Flag tasks that cannot be saved and provide concise feedback on each evaluation.
  • Analyze long agent transcripts to identify errors and improve AI model performance.
  • Ensure rubric structures are correct per guidelines and enhance training data quality.

Resources & Support

  • For details about the interview process and platform information, please check:
  • For any help or support, reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

How to apply

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Discussion

No comments yet. Ask a question or share a tip about this role.

Apply now

More jobs from mercor

View company →

Accounting Expert - CPA Certified

mercor
Remote India Contract $130 – $160 / hour

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include…

7h ago Management and Finance

Audio Engineer - Fully Remote | Up to $10/task

mercor
Remote United Kingdom Contract

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include…

8h ago Software Development

Audio Engineer - Fully Remote | Upto $16/hr

mercor
Remote United Kingdom Contract $16 / hour

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include…

9h ago Software Development

AI Software Engineer - Task Curator

mercor
Remote United States Contract $60 – $90 / hour

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include…

9h ago Data

Similar remote jobs

Browse all →

REO Closing Coordinator

MORTGAGE CONNECT
Remote United States Full-time $24 – $28 / hour

Overview As the REO Closing Coordinator, you are responsible for the coordination of all aspects involved with finalizing the loan closing process. You will communicate with the client,…

just now All Other Remote

Genetic Counselor, Laboratory Services- Full Time Days

Northwestern Memorial Healthcare
Remote United States Full-time $43 – $60 / hour

The Genetic Counselor, Laboratory Services reflects the mission, vision, and values of Northwestern Medicine, adheres to the organization’s Code of Ethics and Corporate Compliance…

just now All Other Remote

Counterintelligence Technology & Program SME — AI Evaluation (Contract) | Remote

Xperteez Technology
Remote United States Contract $90 – $140 / hour

Job Description Role Title: Counterintelligence Technology & Program SME — AI Evaluation (Contract) Role Type: Contractor Location: Remote We are engaging Counterintelligence Technology &…

just now All Other Remote

Educator (Casual)

Playgroup NSW
Remote Australia Temporary AUD 49 / hour

Reputable not for profit - Making real community impact every day – Opportunity to work and engage across your region. Incredibly supportive team! Do you believe every family deserves a…

1h ago All Other Remote

Monitor Technician Virtual Sitter-LGH FT/ 7p-7a

Centra Health
Remote United States Full-time

The Virtual Sitter is responsible for the visual and auditory camera monitoring of appropriate patients’ and reporting changes or unsafe behaviors to the medical staff. Incumbents should…

1h ago All Other Remote

Transportation and Capacity Analyst I/II/III/IV

EQT Corporation
Remote United States Full-time

At EQT, you won't just do a job, you'll help shape our future. Through Transparent Leadership, you'll work alongside leaders who value collaboration and ensure every voice is heard. Through…

1h ago All Other Remote

Get remote jobs the moment they are posted

Choose your categories and skills. We push matching jobs to your device. No email, ever.