About this role
Preparing for this role? See the best AI tools for developers, practise with ready-made prompts for developers, read The 12 Best AI Tools for Developers in 2026 (Tested & Ranked) and take the free 30-day AI challenge.
our partner is working with a leading AI research lab on a project to improve how AI systems do real-world software engineering work. We're looking for senior software engineers who have deep experience at top technology, quantitative or financial firms. You'll use that expertise to train and evaluate frontier AI models. This role is open to candidates based in the United States, United Kingdom, Canada, Australia or New Zealand. Key Responsibilities
Designing realistic, challenging software engineering tasks drawn from real-world work, including but not limited to:
feature changes in large codebases
debugging and root-cause analysis
code review
system design and architecture decisions
codebase Q&A
technical documentation and planning
Writing clear task proposals that explain why each task is realistic, how it is verified and what makes it hard for current AI models
Illustrating "what good looks like" by writing reference solutions, tests and precise, task-specific criteria covering correctness, architecture, maintainability, style and scope
Assessing AI-generated code and engineering work, and working alongside AI coding agents to judge where they succeed and fail
Applying consistent, evidence-based judgment so that assessments are reproducible and defensible
Taking structured feedback from senior reviewers and iterating on submitted work
Ideal Profile
4+ years of professional software engineering experience in one of these settings:
A leading technology or quantitative firm (such as FAANG+, Magnificent 7+, top-tier quant funds or equivalent) with a Mid-senior level title. Machine Learning, Infrastructure, Data, Security and Quant Developer engineering roles may also qualify.
A top financial institution (a leading bank, hedge fund or trading firm), working as a software engineer
A high-growth technology company that is a household name, with a senior-level title
Experience may be split across more than one qualifying employer
Strong command of at least one major language (such as Python, Java, C++, Go, TypeScript or Rust), along with system design, testing, debugging and code review
Proven ability to work in large, complex production codebases and to judge code quality beyond "it passes the tests"
Excellent written English, including the ability to explain technical decisions and trade-offs clearly
Bachelor's degree from an English-speaking country
A detail-oriented, consistent approach to evaluating complex work
Comfortable receiving feedback and calibrating judgment against set standards
Location: United States.
