- Work from
- Worldwide
- Contract
- Full-time
- Experience
- Senior
- Salary
- $50 – $120 / hour
- Posted
- 16 November 2026
- Apply by
- 15 January 2027
Join a remote expert network as a senior rater and reviewer, evaluating challenging AI model outputs and helping calibrate other graders. Your work supports the development of reliable, high-quality AI systems.
Responsibilities
- Rank and rate expert-level model answers in your domain using detailed rubrics and written rationales.
- Evaluate model reasoning step by step in specialized fields such as mathematics, code, law, medicine, or finance.
- Fact-check detailed claims against primary sources and verify accuracy.
- Assess answers against content policies, style guides, and edge cases.
- Review other raters' work, calibrate standards, and develop or refine grading guidelines.
- Assess search relevance and share feedback in live sessions.
Requirements
- At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or deep expertise in a relevant field.
- Native or near-native written English proficiency.
- Experience grading or reviewing expert-level answers against rubrics, policies, or style guides.
- Experience leading, reviewing, or mentoring other raters or reviewers.
- An advanced degree, professional license, or deep working expertise in the domain you evaluate.
- Hands-on experience with AI data annotation, preference ranking, rubric grading, process labeling, or search-quality rating is a plus.
Salary
Compensation is $50 to $120 per hour, depending on project, field, and expertise. Compensation Range: $104K - $249.6K
Compensation & Benefits
Payment is for hours worked.
Location
Fully remote. Projects typically last from one week to several months.
M
A
N
P
C
Discussion
No comments yet. Ask a question or share a tip about this role.