About this role
Preparing for this role? See the best AI tools for lawyers, practise with ready-made prompts for lawyers, read Free AI Tools for Lawyers: What's Actually Free in 2026 and take the free 30-day AI challenge.
Role Overview
our partner is hiring practicing attorneys to build and review evaluation tasks for AI agents.
Each task is set in a realistic simulated firm or legal department, with the case files, contracts, correspondence and matter records lawyers really work from.
You make sure each task reads like a real assignment, the reference answer holds up against the record and the governing law, and the grading rubric checks what a senior attorney would check.
Key Responsibilities
Refine AI-drafted task prompts so they read like a real assignment from a partner, general counsel or client, without giving the answer away.
Check the reference answer against the world's files and the applicable law, and correct what is wrong.
Write and tighten grading rubrics: pass/fail criteria tied to specific facts and rules, plus the judgment calls a non-lawyer would miss.
Read how leading AI models attempt each task and confirm they fail for fair, defensible reasons.
Work in your own practice area. Current areas include corporate and M&A, commercial contracts, civil litigation and arbitration, employment, real estate, intellectual property, estate planning, family, criminal and appellate, administrative and regulatory, antitrust, privacy, securities, environmental, health, and insurance coverage.
Ideal Qualifications
3+ years of practice as an attorney. Major-firm or in-house experience at a large company is a plus, but time in practice matters more.
JD or an international-equivalent law degree.
Active bar admission (or an international equivalent such as Barrister or Solicitor with advocacy rights).
US bar admission preferred.
Prior experience on a our partner RL project is a plus.
Details
Pay: $120/hour.
You start with a short onboarding session and a paid skills assessment. Once you pass, you can claim your first task right away.
Strong performers can move into reviewer and world lead roles.
This is a 100% remote contract role, open to applicants worldwide in the countries the employer accepts.
