- Work from
- United States
- Contract
- Contract
- Salary
- $50 – $90 / hour
- Posted
- 14 November 2026
- Apply by
- 13 January 2027
Role Type: ContractorLocation: Global (Fully Remote)Time Commitment: 15+ Hours per WeekSchedule: Flexible
About the role
We are seeking experienced Software Engineers with prior professional AI training or model evaluation experience to contribute to a project focused on improving advanced AI systems.
In this role, you will apply your software engineering expertise to create challenging technical tasks, evaluate AI-generated code, identify technical failures, and establish objective standards for correct solutions. This opportunity is intended for professionals with substantial prior experience in AI training, benchmarking, or model evaluation.
Responsibilities
- Create realistic software engineering tasks that challenge advanced AI models.
- Develop detailed task instructions, reproducible environments, reference solutions, tests, and evaluation criteria.
- Evaluate AI-generated code and technical deliverables for correctness, robustness, and completeness.
- Identify issues related to implementation, debugging, dependencies, performance, testing, and reproducibility.
- Assess technical solutions against established software engineering standards.
- Provide clear feedback to support improvements in AI model performance and reliability.
Requirements
- Prior paid professional experience in training, evaluating, benchmarking, or improving AI systems.
- Strong practical proficiency in Python.
- Professional experience with at least one additional programming language, such as Java, JavaScript, TypeScript, Rust, C++, C#, or Go.
- Strong understanding of software design, debugging, testing, version control, and dependency management.
- Ability to develop, review, and validate complex software engineering solutions.
Nice to have
- Experience creating AI training tasks, reference solutions, tests, graders, or benchmark problems.
- Experience reviewing technical submissions from other AI trainers.
- Codeforces rating of approximately 1900 or higher, or strong results in another programming competition.
- Experience with Linux, Docker, CI/CD, command-line tools, or sandboxed environments.
- Prior experience with long-horizon or agentic coding evaluations.
- Availability to begin immediately.
Public Proof of Technical Expertise
Strong applicants should provide publicly verifiable evidence of their technical skills, such as:
- Substantial GitHub or GitLab repositories.
- Meaningful open-source contributions.
- Maintainer or core-contributor responsibilities.
- Codeforces, ICPC, Olympiad, or other programming competition results.
- Public developer tools, libraries, systems, or technical projects.
- Technical publications, benchmarks, or engineering write-ups.
How to apply
- Submit your application, including details of your AI training experience and publicly verifiable technical contributions.
- Complete an AI interview involving technical and integrity checks.
- Undergo a Hiring Manager review.
- If selected, complete project onboarding and begin your first assigned task.
Y
K
A
O
W
H
Discussion
No comments yet. Ask a question or share a tip about this role.