About this role
Preparing for this role? See the best AI tools for developers, practise with ready-made prompts for developers, read Best AI Programming Assistant Tools for Developers 2026 and take the free 30-day AI challenge.
Pay: $60–$120/hour equivalent, paid per accepted task
Location: Global, fully remote
Job Type: Contractor
Time Commitment: 15+ hours per week
Schedule: Flexible
our partner is looking for Software Engineers with prior professional AI-training or model-evaluation experience to support a project focused on improving advanced AI systems.
You will use your software engineering expertise to create challenging technical tasks, evaluate model-generated code, identify failures, and establish objective standards for correct solutions. This role is intended for candidates who have already completed substantive AI-training work.
What You’ll Work On
- Create realistic software engineering tasks that challenge advanced AI models.
- Develop task instructions, reproducible environments, reference solutions, tests, and evaluation criteria.
- Evaluate model-generated code and technical deliverables for correctness, robustness, and completeness.
- Identify issues involving implementation, debugging, dependencies, performance, testing, or reproducibility.
Required Qualifications
- Prior paid experience training, evaluating, benchmarking, or improving AI systems.
- Strong practical proficiency with Python.
- Professional experience with at least one additional programming language, such as Java, JavaScript, TypeScript, Rust, C++, C#, or Go.
- Strong knowledge of software design, debugging, testing, version control, and dependency management.
Public Proof of Skill
Strong applicants should provide evidence such as:
- Substantial GitHub or GitLab repositories
- Meaningful open-source contributions
- Maintainer or core-contributor responsibilities
- Codeforces, ICPC, Olympiad, or other programming-competition results
- Public developer tools, libraries, systems, or technical projects
- Technical publications, benchmarks, or engineering write-ups
Preferred Qualifications
- Experience creating AI-training tasks, reference solutions, tests, graders, or benchmark problems.
- Experience reviewing other AI trainers’ technical submissions.
- Codeforces rating of approximately 1900 or higher, or strong results in another programming competition.
- Experience with Linux, Docker, CI/CD, command-line tools, or sandboxed environments.
- Prior experience with long-horizon or agentic coding evaluations.
- Availability to begin immediately.
Application Process
- Submit your application, AI-training experience, and public technical evidence.
- Complete an AI interview containing technical and integrity checks.
- Complete a hiring-manager review.
- If selected, complete project onboarding and begin your first assigned task.
Compensation
- Compensation is output-based. Experts are paid for tasks that meet project specifications and quality requirements. Task availability and completion time may vary.
This is a 100% remote contract role, open to applicants worldwide in the countries the employer accepts.