AI tools, scored for your job
Get paid to train AI
Hiring now · posted today General Contract · Remote

AI Evaluation Rubric Reviewer

$40–$75/hr

Published pay. The final rate depends on your test, experience and country.

Remote · United States Language: English Posted 16 Nov 2026

AI Evaluation Rubric Reviewer is remote AI-training work for professionals, paying $40–$75/hr, open to applicants in United States. Posted 16 Nov 2026, still open on 16 Nov 2026. Zekai lists it for free; the application goes to the employer's platform.

About this role

Preparing for this role? See the best AI tools for professionals, practise with ready-made prompts for professionals, read What Is Generative Artificial Intelligence in 2026 and take the free 30-day AI challenge.

The work

We are testing AI agents on realistic shopping and delivery tasks. You'll review AI-drafted tasks (prompt, grading rubric) along with model attempts, and decide whether each failure is fair:

- Was the information available?

- Does the rubric accurately grade what the prompt asked for?

- Would a valid answer have been marked wrong?

- Is the rubric structure correct per the guidelines?

Fix the criteria that aren't fair, flag the tasks that can't be saved, and write a short note on each call.

Must-haves

- Substantial experience writing or reviewing rubrics for LLM evals. Please list the projects in your application.

- Can read a long agent transcript and find where it went wrong.

- Precise writing, and you check the data instead of trusting the reference answer.

This is a 100% remote contract role. Applicants must be located in: United States.

How applying works

  1. Click Apply now and enter your email.
  2. You go straight to the application. We email you a copy of this role and your link.
  3. Most platforms ask for a CV and a short skills test or AI interview in your field.
How to get this job

Your plan for “AI Evaluation Rubric Reviewer”

What to know before you apply

Pay
At the published rate, 10 hours a week would be about $400–$750 and 20 hours about $800–$1,500 before tax. Hours depend on the project, so treat this as a ceiling, not a promise.
Location
Only applicants located in United States can be paid for this role. Platforms usually verify where you are, so apply only if you are based there.
Timing
This listing is new today. Projects can close once they have enough people, so it is worth applying while it is open.
Test tips for professionals
  • Read the task guidelines fully before your first submission.
  • Explain your judgement in plain, complete sentences.
  • Keep a steady weekly schedule; reliable contributors get more work.
Prepare free with Zekai

3 steps to stand out as one of the best professionals

  1. 01

    Know the AI tools professionals use

    Reviewers judge AI answers in your field. Knowing the tools people in your profession actually use helps you spot what good output looks like.

    All 1,009 AI tools for professionals →
  2. 02

    Practise with real prompts

    Many tests ask you to write or judge prompts in your field. Practise on these first.

    System Incident Log Analysis Prompt

    Act as an IT operations engineer. Analyze the following system log text: [[PASTE SYSTEM LOGS HERE]]. Identify any metric anomalies, potential software integration failures, or security rule violations. Summarize your findings in …

    Document synthesis and executive summary

    Analyze the attached document regarding [[PROJECT NAME OR TOPIC]]. Extract the key strategic risks, financial assumptions, and operational recommendations. Format the output into a three-page executive brief with clear section he…

    Prompt library for professionals →
  3. 03

    Take the free 30-day AI challenge

    Five minutes a day to get confident with AI in your own work. Day 1 is open now, no card needed.

    Start Day 1 free →

    Guides for professionals

Professionals

More paid AI roles for professionals

Confidential clientAI training · Professionals

Chief of Staff

  • Remote · Worldwide
  • Contract · flexible hours

You'll operate at the intersection of strategy, operations, and execution, helping the CRO focus on what matters most…

$80k–$120k/yr
21d ago
Confidential clientAI training · Professionals

Gmail & Google Calendar AI Assistant Evaluator

50 open
  • Remote · United States
  • Contract · flexible hours

Our partner is engaging Gmail & Google Calendar AI Assistant Evaluators to collaborate on a customer-driven project e…

$15–$30/hr
52d ago
Confidential clientAI training · Professionals

Trainium (NKI) Kernel Expert

  • Remote · United States
  • Contract · flexible hours

Evaluate the quality, correctness, and hardware-appropriateness of Neuron Kernel Interface (NKI) development tasks us…

$70–$90/hr
83d ago
Confidential clientAI training · Professionals

GPU Kernel Expert

  • Remote · United States
  • Contract · flexible hours

Evaluate the quality, correctness, and completeness of GPU/accelerator kernel development tasks used to train and eva…

$70–$90/hr
83d ago
LovableJob at an AI company · Professionals

Product Experience Specialist (APAC)

  • Remote
  • Full-time

We're looking for Product Experience Specialists to deliver world-class support and shape the future of Lovable. You …

Salary on company site
364d ago
ModalJob at an AI company · Developers

Member of Technical Staff - SDK

  • Remote · United States
  • Full-time

Develop new primitives to meet demand for entirely new areas that arise as a result of the ongoing industry transform…

$150k–$350k/yr
451d ago
new in the last 24 hours

New roles in your inbox every morning

Pick your field. One short email a day, only when new roles open. Unsubscribe any time.

Roles come from public listings and change daily. Pay is what the listing shows; offers depend on your test, experience and country. If you join through a Zekai link, the platform may pay Zekai a fee. Your pay stays the same.

$40–$75/hr
Get paid to train AI →

See Zekai first in Google