Evaluator Bench
evaluator; distance from a lab: 1

MLCommons

Scorecard

Industry consortium producing the AILuminate safety benchmark.

Consortium, San Francisco, US. Confidence med. Domains: benchmarks, assurance. Scores by preset: Lab procurement 59; Regulator or auditor 60; Public trust 55; Equal weights 60. Weakest dimension: funding 1/4.

What would move the score. Published funding shares and a hazard suite the members do not see in advance.

Funding 1/4
Material revenue or investment from evaluated labs or their investors.
evidence: tier 3 (self)
Anchor 1 on funding given the cited signals. Curator disclosure (2026-09-15): the curator holds small public-market shareholdings in Google and Meta. This evaluator has a confirmed ledger tie to Google (founding participation row T63; not a grant). Per the disclosure Rule, this assessment is flagged for public review: the rationale names the holding so readers can weigh it, and any reader can file a correction through the contribution path.
Governance 2/4
For-profit or PBC with a published COI policy.
evidence: tier 3 (self)
Anchor 2 on governance given the cited signals.
Personnel 2/4
Frequent two-way hiring; recusal on request.
evidence: tier 3 (self)
Anchor 2 on personnel given the cited signals.
Access depth (lab-granted) 2/4
Pre-release with safeguards off or extended time.
evidence: tier 3 (self)
Anchor 2 on access given the cited signals.
Scope control 3/4
Evaluator sets scope and can add questions.
evidence: tier 3 (self)
Anchor 3 on scope given the cited signals.
Publication rights 3/4
Publishes; redaction limited to security; redactions disclosed.
evidence: tier 3 (self)
Anchor 3 on publication given the cited signals.
Method transparency 4/4
Open code, tasks, reproducible runs, factsheets.
evidence: tier 3 (self)
Anchor 4 on methods given the cited signals.
Role incompatibility 3/4
Tools are open or free to the ecosystem.
evidence: tier 3 (self)
Anchor 3 on products given the cited signals.

Traced money and ties

hop 0: 1 row. Confirmed rows: 1 of 1. Dollar sums: confirmed + unaudited USD rows only; imported figures are not re-derived and never summed. Second hop traced for 1 of 1 sources. Ties: none within two steps recorded. Bounded negatives on file: 1.

Ledger

Money in

Bounded negatives

Roles hosted

One hop away

Google / Google DeepMind, Peter Mattson

Funding graph, focused

Neighbours at full strength, everything else faded. Hover any name to move the focus; click a name to open its page.

The funding graph around MLCommons Teal lines are money (transfers); amber lines are roles. Solid: confirmed; dashed: imported. Hover a name to isolate it and its neighbours; hover a line for the row. Every element is a row in data/ledger/. Labs Direct lab ties Two steps Three or more Public or unattributed Evaluators Amazon Anthropic G42 Google / Google DeepMind Meta Microsoft OpenAI Thinking Machines Lab xAI AI Safety Fund (Frontier Model Alexandr Wang Andreessen Horowitz Anthropic employees (personal Ben Mann D.E. Shaw Ventures David Farhi Dustin Moskovitz Frontier-lab employees and alu Holden Karnofsky Jaan Tallinn Jeffrey Ladish Macroscopic Ventures (formerly Miles Brundage Neil Chowdhury Nvidia OpenAI employees (personal hol OpenAI Foundation Peter Mattson Sequoia Capital Zico Kolter Adam Gleave Artificial Intelligence Underw Clement Delangue Coefficient Giving (formerly O Good Ventures Foundation Jason Droege Longview Philanthropy Marco Mascorro Paul Christiano Rajiv Dattani Survival and Flourishing Fund Conrad Stosz Dan Hendrycks Mike McCormick Alec Radford Alignment Research Center Constellation Research Center ELMA Philanthropies Emerson Collective EU budget (Digital Europe Prog Fifty Years (50Y) Founders Pledge Founders Pledge (frontier AI f Gates Foundation Halcyon Futures Hillspire (Schmidt family offi Hudson River Trading Jacob Hilton MacKenzie Scott Magarac Venture Partners Obvious Ventures Redpoint Ventures Renaissance Philanthropy Salesforce Ventures Samsung Next Schmidt Sciences Silicon Valley Community Found Skoll Foundation Snowflake Ventures Swish Ventures Sympatico Ventures The Audacious Project (TED) UK government (DSIT) US government (NIST appropriat Valhalla Foundation Vanguard Charitable Wing Venture Capital Y Combinator Andon Labs Apollo Research AVERI Center for AI Safety Epoch AI EquiStamp EU AI Office FAR.AI Gray Swan Hugging Face Open Alignment In Irregular (formerly Pattern La METR Microsoft AI Red Team MLCommons Nemesys Insights Palisade Research RAND Corporation Redwood Research SaferAI Scale AI (SEAL) SecureBio Transluce UK AI Security Institute US CAISI (NIST)