caferemote
mercor

AI Workflow Evaluator - Fully Remote | Upto $190/hr

Remote United States Contract $50 – $190 / hour Posted 3h ago
Apply now
Work from
United States
Contract
Contract
Salary
$50 – $190 / hour
Posted
18 November 2026
Apply by
17 January 2027

Position: MCP & Plug In Connectors ExpertType:ContractCompensation:$50–$190/hourLocation:RemoteCommitment:20+ hours/week

About the role

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Requirements

  • US-based only.
  • Strong MCP experience and plugin/connector usage.
  • Experience using LLM plugins/connectors like Google Drive, Expedia, and Notion.
  • Heavy personal usage of LLM products.
  • Active LLM account with 6+ months of history.
  • Willingness to sign a data-share consent form via DocuSign.
  • Experience using AI for multi-step planning, research, and decision-making.
  • Strong written judgment and attention to detail.
  • Ability to evaluate AI outputs and write against rubrics.

Nice to have

  • Extensive rubric experience with 100+ hours on prior projects.

Compensation & Legal

  • Hourly contractor.
  • Experts added to the project will begin in a trial period.

Role Responsibilities

  • Evaluate AI systems on complex personal workflows, including personal health, travel, and activity planning.
  • Design realistic prompts for complex personal-life tasks using Google Drive, Expedia, Notion, and other plugins/connectors.
  • Execute tasks while recording your screen and using personal plugins/connectors.
  • Write clear explanations of AI successes and failures to improve AI model performance.
  • Judge AI outputs for practicality, personalization, and reasoning.
  • Identify where models miss context, overreach, or fail to use tools correctly.

Resources & Support

  • For details about the interview process and platform information, please check:
  • For any help or support, reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

How to apply

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Discussion

No comments yet. Ask a question or share a tip about this role.

Apply now

More jobs from mercor

View company →

Architectural Designer - Fully Remote | Up to $60/hr

mercor
Remote India Contract $50 – $60 / hour

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include…

2h ago Software Development

Account Management Lead - Fully Remote | Up to $380/task

mercor
Remote United States Contract

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include…

2h ago Customer Support

AI Vulnerability Expert - Fully Remote | Upto $90/hr

mercor
Remote United States Contract $60 – $90 / hour

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include…

2h ago Data

AI/ML Engineer - Fully Remote | Upto $85/hr

mercor
Remote Germany Contract $85 / hour

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include…

3h ago Data

Similar remote jobs

Browse all →

Senior Project Manager, QA/QC

Cash Construction Company Inc.
Remote United States Full-time $150K – $175K / year

Overview As one of the nation's leading electric transmission line construction companies, MasTec Power Delivery companies engineer and construct efficient and reliable electrical…

2h ago Quality Assurance

QA Automation Test Engineer

Aretum
Remote United States Full-time $80K – $120K / year

Public Trust Eligibility Required About Aretum Aretum is a mission-driven organization committed to delivering innovative, technology-enabled solutions to our customers across defense,…

3h ago Quality Assurance

QA Manager, EPIC Loan Systems

Aquila Software Group
Remote Canada Full-time CAD 130K – CAD 150K / year

We are excited to speak to you about this opportunity. If you are interested in joining our growing team, please apply to this link directly. Who we are At Aquila, we invest in…

4h ago Quality Assurance

Andromeda Shimu Deep Research Reviewer (EN-US)

Welo Global
Remote United States Contract $28 / hour

About the Role We're looking for a QA Deep Research Reviewer to evaluate long-form, AI-generated research outputs against detailed rubrics and guidelines. You'll read documents that include…

5h ago Quality Assurance

Sr. QE Test Coordinator (ATL, LOU, or Remote)

Papa Johns
Remote United States Full-time

What’s Unique About You Is What Makes Us Better! Diversity is our strength and competitive advantage. Bring your flavor to the Papa John's team today! Job Summary The Senior Test…

5h ago Quality Assurance

SDET (Salesforce)

Alcumus
Remote Canada Full-time

As a Salesforce SDET, you will play a critical role in ensuring the quality and reliability of our Salesforce solutions. What that means day-to-day: Test Planning: Collaborate with the…

6h ago Quality Assurance

Get remote jobs the moment they are posted

Choose your categories and skills. We push matching jobs to your device. No email, ever.