- Work from
- Worldwide
- Contract
- Full-time
- Salary
- $80 – $200 / hour
- Posted
- 20 November 2026
- Apply by
- 19 January 2027
You will design training environments, build evaluation systems, and assess agent behavior as projects become available.
About the role
Join a network of experienced technical contributors working on paid AI human-data projects focused on agents and reinforcement learning.
Responsibilities
- Design reinforcement learning environments, including Python and Docker setups, state spaces, tool integrations, and scoring mechanisms.
- Build verifiers and reward functions that reliably evaluate agent performance and resist gaming.
- Translate real-world business workflows in areas such as ERP, CRM, finance, and operations into measurable scenarios.
- Integrate tools and REST APIs, and build test harnesses using frameworks such as pytest.
- Lead agent demonstration and computer-use programs, setting quality standards for trace collection.
- Grade and repair agent trajectories, and mentor contributors on evaluation practices.
Requirements
- At least 5 years of experience in software engineering, QA automation, machine learning, reinforcement learning, business systems, or operations paired with hands-on agent or RL data work.
- Strong Python skills and experience building tools or services that others rely on.
- Experience designing environments, verifiers, reward functions, or test harnesses to evaluate software or agent behavior, using tools such as pytest, Playwright, Selenium, or Gymnasium-style environments.
- Hands-on experience with agent or RL data, such as demonstrations, trajectory grading, trace collection, or computer-use demos.
- Experience integrating REST APIs and using Docker.
- Experience leading or reviewing work on data, evaluation, or demonstration programs is valued, as is expertise modeling workflows in business systems such as ERP or CRM platforms.
- A degree in computer science, engineering, or a related field is relevant.
Eligibility
- Visa sponsorship information was not provided.
Salary
Compensation is $80 to $200 per hour, depending on project, field, and experience level.
Compensation & Benefits
Pay is based on hours worked.
Location
Remote. Projects typically last from one week to several months.
N
C
L
W
D
T
Discussion
No comments yet. Ask a question or share a tip about this role.