AI tools, scored for your job
Get paid to train AI
Hiring now · posted today Finance Contract · Remote

AI Evaluation Rubric Auditor

$60–$85/hr

Published pay. The final rate depends on your test, experience and country.

Remote · Canada, United States, Argentina, Brazil, Chile, Colombia +2 Language: English Posted 16 Nov 2026

AI Evaluation Rubric Auditor is remote AI-training work for accountants, paying $60–$85/hr, open to applicants in Canada, United States, Argentina, Brazil, Chile, Colombia and more. Posted 16 Nov 2026, still open on 17 Nov 2026. Zekai lists it for free; the application goes to the employer's platform.

About this role

Preparing for this role? See the best AI tools for accountants, practise with ready-made prompts for accountants, read Best AI for Accounting Firms in 2026 and take the free 30-day AI challenge.

The work

We are testing AI agents on realistic shopping and delivery tasks. Your role as a Project Auditor is to audit the work done by our Project Reviewers. You will audit complete tasks (prompt and grading rubric) and determine if their rubric failures are fair:

- Was the information available?

- Does the rubric accurately grade what the prompt asked for?

- Would a valid answer have been marked wrong?

- Is the rubric structure correct per the guidelines?

This role includes providing feedback to reviewers regarding prompt and rubric issues.

Must-haves

- Substantial experience writing or reviewing rubrics for LLM evals. Please list the projects in your application.

- Experience providing feedback to writers/reviewers

- Consistent availability

- Can read a long agent transcript and find where it went wrong.

- Precise writing, and you check the data instead of trusting the reference answer.

This is a 100% remote contract role. Applicants must be located in: Canada, United States, Argentina, Brazil, Chile, Colombia, Peru, United Kingdom.

How applying works

  1. Click Apply now and enter your email.
  2. You go straight to the application. We email you a copy of this role and your link.
  3. Most platforms ask for a CV and a short skills test or AI interview in your field.
How to get this job

Your plan for “AI Evaluation Rubric Auditor”

What to know before you apply

Pay
At the published rate, 10 hours a week would be about $600–$850 and 20 hours about $1,200–$1,700 before tax. Hours depend on the project, so treat this as a ceiling, not a promise.
Timing
This listing is new today. Projects can close once they have enough people, so it is worth applying while it is open.
Test tips for accountants
  • Show the journal entries or the tax computation, not just the conclusion.
  • State the standard and country (IFRS, US GAAP, local tax code) before you judge an answer.
  • Check dates and thresholds: AI often uses last year's rates.
Prepare free with Zekai

3 steps to stand out as one of the best accountants

  1. 01
  2. 02

    Practise with real prompts

    Many tests ask you to write or judge prompts in your field. Practise on these first.

    Draft a Cash Flow Forecast Narrative

    Act as a financial analyst. I have a cash flow forecast for [[Client Name]] for the next six months. The key drivers are: • Revenue is projected to [[grow/decline]] by [[X]]% per month. • A large one-time expense of $[[Amount]] f…

    Structure a Business Valuation Memo

    Act as a business valuation expert. I need to prepare an internal memo outlining the approach to value a [[Business Type, e.g., local plumbing business]]. Structure a memo that covers the three main approaches to valuation: 1. As…

    Prompt library for accountants →
  3. 03

    Take the free 30-day AI challenge

    Five minutes a day to get confident with AI in your own work. Day 1 is open now, no card needed.

    Start Day 1 free →

    Guides for accountants

Accountants

More paid AI roles for accountants

Confidential clientAI training · Accountants

Senior Accountant Expert

Top pay30 open
  • Remote · United States
  • Contract · flexible hours

Our partner is engaging Senior Accountant Experts to contribute specialized knowledge and experience to a dynamic cus…

$60–$120/hr
54d ago
Confidential clientAI training · Accountants

Accounting Expert

  • Remote · Worldwide
  • Contract · flexible hours

Our partner is hiring experienced accountants and auditors to build and review evaluation tasks for AI agents.

$95/hr
3d ago
Confidential clientAI training · Accountants

Individual Tax Professional (CPA / EA)

Top pay
  • Remote · United States
  • Contract · flexible hours

Our partner is seeking experienced individual tax professionals to help evaluate advanced AI systems on realistic per…

$110–$130/hr
6d ago
Confidential clientAI training · Accountants

AI Developer Trace Task Auditor

  • Remote · United States
  • Contract · flexible hours

Evaluate the quality and correctness of AI-assisted software-development traces used to train and evaluate a frontier…

$70–$90/hr
80d ago
Confidential clientAI training · Accountants

Accounting & Audit Domain Expert

Top pay
  • Remote · United States
  • Contract · flexible hours

Join a leading AI lab's cutting-edge GenAI team to be at the core of the AI revolution, where your expertise fuels th…

$60–$100/hr
89d ago
new in the last 24 hours

New roles in your inbox every morning

Pick your field. One short email a day, only when new roles open. Unsubscribe any time.

Roles come from public listings and change daily. Pay is what the listing shows; offers depend on your test, experience and country. If you join through a Zekai link, the platform may pay Zekai a fee. Your pay stays the same.

$60–$85/hr
Get paid to train AI →

See Zekai first in Google