AI tools, scored for your job
Get paid to train AI
🤝 You're in the Free & Open Source library — community-built tools you can use without paying. A different concept from our scored AI directory.
Home › Free & Open Source › Marketing › llm-leaderboard
✦ Free & open sourceMarketing

llm-leaderboard

A structured open dataset mapping language model benchmark scores, context window limits, and provider pricing.

Compare LLM benchmark scores, context windows, and pricing

Access structured JSON datasets comparing performance metrics, provider costs, and model capabilities.

Free forever · source-available · Licence not stated in the repository
✓ Light setup✓ Browser-based✓ Real-time collaboration
5.4
ZEKAI
Deprecated dataset for manual analysis
Free & open source · Marketing
Ease of setup7
Usefulness4
Maintenance5.5
Community5
★ 356
GITHUB
Oct 2025
UPDATED
Licence not stated in the repository
LICENSE
🛠 Don't want to self-host? We can set it up →
⚡ Quick answer

This repository provides a structured JSON dataset for evaluating large language models. Content strategists and marketers can inspect model context windows, performance scores, and API pricing to select cost-effective models. Note that this project is officially deprecated and no longer receives active updates.

Runs on
Web
Hosting
Cloud or self-host
Languages
JavaScript
License
Licence not stated in the repository

What it is

It aggregates performance benchmark results (such as MMLU, GPQA, and MATH), parameter counts, context limits, and vendor pricing inside schema-validated JSON files. The dataset organizes records across dedicated directories for models, providers, and benchmarks. You can inspect these files directly to build custom comparison tools or analyze model efficiency.

Who it's for

It suits technical content creators, prompt engineers, and marketing operations specialists who need raw historical data to compare model performance against API costs. It does not suit non-technical users seeking a turnkey SaaS platform or teams requiring live, auto-updating benchmark feeds.

Key features

Structured JSON schemas

Model metadata, benchmark results, and pricing details are stored in clean, schema-validated JSON directories.

Provider cost mapping

Links specific language models to provider pricing, context window parameters, and hardware configurations.

Standardized evaluation tracking

Records model scores across established industry benchmarks including MMLU, GPQA, HumanEval, and MATH.

Verifiable source citations

Data entry schemas encourage verifiable citation links for all submitted performance metrics.

How to start

Up and running in 3 steps

1
Clone the repository to access the raw JSON dataset in the data folder.
2
Inspect data/provider_models and data/model_benchmarks to evaluate costs and scores.
3
Parse the schema-validated files into your internal reports or custom dashboard.
Real talk

Our honest verdict

What's great

  • Organized JSON directory structure makes data extraction straightforward
  • Combines provider pricing directly with performance benchmark metrics
  • Uses schema validation to maintain data consistency
  • Includes context window and output limits useful for content workflow planning

Where it falls short

  • The repository is deprecated and no longer receives updates or official maintenance
  • Requires custom scripts or JSON viewers to query and filter data effectively
  • Does not state a standard open-source license in its repository metadata
  • You must host your own web viewer if you want an interactive dashboard interface
Cost

Pricing & license

The repository content is free to download and inspect. The only cost involved is the time and hosting infrastructure required to run your own custom parsing scripts or web viewers.

Paid work

Open remote AI-training roles

880 open remote AI-training roles, updated November 15, 2026; most pay $20–$175 an hour. Zekai lists them free; the application goes to the employer’s platform.

See all open roles →
Compare

Alternatives

Get more

Use it, or have us set it up

📄 Free PDF: llm-leaderboard for Marketing
A plain-language guide to getting your first result. Sent to your inbox.
🔔 64 people want a full video tutorial — add your vote
🛠 Don't want to set it up yourself?
We'll install, host & configure llm-leaderboard for you — and train your team.
Get it done for you ↗
Questions

FAQ

Is this repository still actively maintained?

No, the repository is officially deprecated. Data corrections are now directed to the llm-stats community section.

How can content teams use this data?

You can compare context window sizes and pricing against benchmark scores to pick the right LLM for long-form generation.

Radar

Latest AI news

✉️ The weekly free toolkit
The best new free & open-source tools for Marketing — one short email a week, in your language.
Free · 1 email/week · unsubscribe anytime
Get paid to train AI →

See Zekai first in Google