Evaluator Bench
MLCommons, 2025-01-15; retrieved 2026-09-15 tier 3 self (mlcommons)confirmed

MLCommons Releases AILuminate Creative Commons DEMO Benchmark Prompt Dataset to Github

https://mlcommons.org/2025/01/ailuminate-demo-benchmark-prompt-dataset/

1,200-prompt DEMO set public; Practice set on request; Official Test dataset private to prevent gaming; public reports of 13 systems-under-test. Fetched 2026-09-15; spans present.

Signals citing this source

  • MLCommons (AILuminate), M (for): Open benchmark specification and methodology.“The Official Test dataset is private to avoid SUTs “gaming” the benchmark”
  • MLCommons (AILuminate), S (for): Benchmark design set by working group.“designed and developed by the MLCommons AI Risk and Reliability working group”
  • MLCommons (AILuminate), R (for): Publishes results.“Public reports of 13 systems-under-test (SUTS)”
  • MLCommons (AILuminate), X (for): Benchmarks free to use.“publicly released the AILuminate DEMO prompt dataset, a collection of 1,200 prompts”

Ledger rows citing this URL

  • none