Evaluator Bench
Kevin Bass (GitHub), 2026-09-14; retrieved 2026-09-15 tier 2 ledgerimported

evaluators.csv (24 evaluators: Coefficient and SFF totals, lab money, government contracts, leadership)

https://raw.githubusercontent.com/kevinnbass/metr-money-figure/master/research/evaluators.csv

Parallel scorecard with index totals per evaluator; imported as leads. Repository default branch is master (the earlier main path returned 404). Sibling files: coefficient_pages.csv, tallinn-donations-ledger-2026-09-13.csv, vanguard-fy*-schedule-i.csv, finances.csv. [audit 2026-09-15] URL repointed from the GitHub blob page (JavaScript-rendered) to the raw file so quoted spans can be checked; status set to imported, which is what it is (a row-level lead copied from another project, supporting a 2 and nothing else until re-derived).

Imported from kevinnbass/metr-money-figure:evaluators.csv.

Signals citing this source

  • Apollo Research, F (against): SFF recommendations of about $1.1M and Coefficient grants of about $4.4M per the index snapshot; both funders are one to two steps from an Anthropic investor.
  • Apollo Research, F (for): Self-reported philanthropic base: about $1.5M from Open Philanthropy and about $500K from SFF at the time of its Manifund listing; a later index total of $4.41M is recorded as an imported row and the two figures are not reconciled.
  • US CAISI (NIST), G (against): Appropriation of about $10M in FY2026 within NIST's AI line; the director resigned in July 2026 and the NIST director is acting head.
  • EquiStamp, F (for): Revenue from evaluators and governments; leads EU Lot 3 (EUR 1.17M to consortium) and Lot 4 (EUR 0.88M).
  • EquiStamp, F (against): Private company; no funding or ownership disclosed.
  • EquiStamp, G (against): No COI policy on the site.
  • EquiStamp, P (against): Founders' prior affiliations and any lab ties not established.
  • EquiStamp, A (against): Access is through client engagements; no pre-release lab access documented.
  • EquiStamp, M (against): Methods not public.
  • EquiStamp, X (for): No lab-facing products found; clients are evaluators and governments.
  • Hugging Face Open Alignment Initiative, F (against): Series D (2023) investors include Google, Amazon and Nvidia; Nvidia's reported acquisition would make a lab investor the owner.
  • Irregular (formerly Pattern Labs), F (for): A search of Coefficient Giving's grants pages under the name Irregular found no grant (snapshot 2026-09-15); the imported row citing $6.8M under Pattern Labs is unreconciled and marked differs.
  • MLCommons (AILuminate), P (against): President is a Google senior staff engineer; members include the frontier developers whose models are benchmarked.
  • Nemesys Insights, F (against): Paid assessor for Amazon per the Nova papers; private LLC with no funding disclosed.
  • Nemesys Insights, G (against): No policy published.
  • Nemesys Insights, P (against): Leadership from a university red-team center; lab ties not established.
  • Nemesys Insights, A (against): Access through client engagement.
  • Nemesys Insights, S (against): Scope set by the lab client.
  • Nemesys Insights, R (against): Findings appear inside the lab's papers.
  • Nemesys Insights, M (against): Methods described in the lab's papers only.
  • Nemesys Insights, X (against): Consults for a lab.
  • Palisade Research, P (against): Founder and executive director previously worked on Anthropic's security team; SFF recommendations of about $3.2M.
  • Palisade Research, F (against): Coefficient Giving's database records two grants totaling $2,123,463 for general support; the imported index total of $3,803,463 is marked differs pending reconciliation.“$2,123,463 2025-05”
  • RAND, F (against): Large share of AI work funded by Coefficient Giving.
  • RAND, F (for): FY2024 revenue of about $489M is mostly US government contract research; Coefficient's $87M since 2020 is a small share.
  • Redwood Research, F (against): Coefficient grants of about $63M across five grants dominate income; SFF about $2.4M and a $1.33M Tallinn gift; both funders sit one to two steps from an Anthropic investor.
  • SaferAI, X (against): Advisory work for G42 and for an unnamed major AI company, alongside public framework ratings; no lab-money policy on the site.“advised G42 and 'a major AI company' on frontier safety frameworks”
  • SaferAI, F (for): SFF recommendations of about $1.06M over 2022 to 2025; EU AI Office Lot 1 consortium member.
  • Scale AI (SEAL / Scale Labs), F (against): Meta paid $14.3B for a 49% non-voting stake in June 2025; the founder moved to Meta.

Ledger rows citing this URL

  • none