Evaluator Bench
evaluator; distance from a lab: 3

Center for AI Safety

Scorecard

Hazard benchmarks (WMDP, HLE); director advises xAI.

Nonprofit, San Francisco, US. Independent referee; listed with independent referees. Confidence med. Domains: benchmarks, misuse.

Conditional floor 8/8 evidenced · Retrieved & confirmed Scores by preset: Lab procurement –; Regulator or auditor –; Public trust –; Equal weights –. Weakest evidenced dimension: access depth (lab-granted) 0/4. At least one evidenced conflict dimension (funding, governance, personnel, role incompatibility, scope, publication) at 1: usable with conditions the card names.

The weighted numbers are hidden until you ask for them: two readers with different priorities see different numbers, and neither is the site's.

What would move the score. A public recusal policy covering the xAI relationship.

Dissent, lower. Personnel should be 0. The director advises xAI while CAIS benchmarks (HLE, WMDP, MASK) are used to grade xAI models, and no recusal statement, disclosure page, or conflict policy is published anywhere on safe.ai. P.1 sets 0 for a governance role without recusal; an unrecused standing advisory role at a graded lab, with the benchmark scoring under the same director, is the nearest thing to it.

Dissent, higher. Personnel should be 2. An advisory role is not a board or committee seat, so P.2's cap of 1 rather than P.1's 0 applies, and nothing above 1 was ever claimed. HLE and WMDP are scored mechanically on public leaderboards where xAI's results are visible to anyone, no equity, hiring flow, or second lab tie is documented, and a published recusal note would lift the value to 2.

Each value is the tightest admissible cap or, with no cap, the highest admissible floor, under RULES.md. The binding signal is highlighted. A dash means no admissible signal sets a bound under the Retrieved & confirmed policy.

Funding 3/4
Anchor 3: Mostly philanthropic or public money; some lab-linked pooled funds.
evidence: tier 3 (self)
Capped at 3 by cais.09 (F.6): SFF recommended $289,000 to CAIS in 2025 (about $1.1M cumulative per 80,000 Hours); SFF's funder Jaan Tallinn led Anthropic's Series A.. Floors up to 3 from cais.02 do not exceed the cap.
Governance 2/4
Anchor 2: For-profit or PBC with a published COI policy.
evidence: tier 3 (self)
Floored at 2 by cais.05 (G.1): Nonprofit.. No admissible signal caps it.
Personnel 1/4
Anchor 1: Leaders hold equity or advisory roles at labs; informal recusal.
evidence: tier 3 (self)
Capped at 1 by cais.01 (P.2): Director is a safety adviser to xAI while the organization's benchmarks are used to grade xAI models..
Access depth (lab-granted) 0/4
Anchor 0: Public API only.
evidence: tier 3 (self)
mechanism: lab-controlled
Capped at 0 by cais.06 (A.6): Public-model access..
Scope control 3/4
Anchor 3: Evaluator sets scope and can add questions.
evidence: tier 3 (self)
Floored at 3 by cais.07 (S.5): Sets own benchmark design.. No admissible signal caps it.
Publication rights 2/4
Anchor 2: Publishes; lab reviews with broad redaction.
evidence: tier 3 (self)
mechanism: self-imposed
Floored at 2 by cais.08 (R.1): Publishes results.. No admissible signal caps it.
Method transparency 3/4
Anchor 3: Tasks or code partly open.
evidence: tier 3 (self)
Floored at 3 by cais.03 (M.3): Open benchmarks widely used.. No admissible signal caps it.
Role incompatibility 3/4
Anchor 3: Tools are open or free to the ecosystem.
evidence: tier 3 (self)
Capped at 3 by cais.04 (X.6): Co-produced Humanity's Last Exam with Scale AI, a Meta-owned vendor..

Values under each evidence policy

DimensionLeads includedRetrieved & confirmedAgainst interestVerified spansPrimary only
Funding3333–
Governance22–2–
Personnel1111–
Access depth (lab-granted)0000–
Scope control33–3–
Publication rights22–2–
Method transparency33–3–
Role incompatibility3333–
Score, Lab procurement– Conditional floor, 8/8– Conditional floor, 8/8– Conditional floor, 4/8– Conditional floor, 8/8– Unevidenced, 0/8
Score, Regulator or auditor– Conditional floor, 8/8– Conditional floor, 8/8– Conditional floor, 4/8– Conditional floor, 8/8– Unevidenced, 0/8
Score, Public trust– Conditional floor, 8/8– Conditional floor, 8/8– Conditional floor, 4/8– Conditional floor, 8/8– Unevidenced, 0/8
Score, Equal weights– Conditional floor, 8/8– Conditional floor, 8/8– Conditional floor, 4/8– Conditional floor, 8/8– Unevidenced, 0/8

Traced money and ties

hop 2: 2 rows (recommendation $1.4M). Confirmed rows: 2 of 2 Dollar sums: confirmed + unaudited USD rows only; imported figures are not re-derived and never summed. Second hop traced for 1 of 1 sources. Ties: none within two steps recorded. Checked and not found: 0.

Ledger

Money in

Roles hosted

Funding graph, focused

Every one-way chain to a lab and to an evaluator at full strength, everything else faded. Hover any name to trace its connections; click a name to open its page.

The funding graph around Center for AI Safety Teal lines are money (transfers); amber lines are roles. Solid: confirmed; dashed: imported. Hover a name to trace its chains to the labs and evaluators — each chain runs one way across the columns, never doubling back; hover a line for the row. Every element is a row in data/ledger/. Labs Direct lab ties Two steps Three or more Public or unattributed Evaluators Amazon Anthropic G42 Google / Google DeepMind Meta Microsoft OpenAI Thinking Machines Lab xAI AI Safety Fund (Frontier Model Alexandr Wang Andreessen Horowitz Anthropic employees (personal Ben Mann D.E. Shaw Ventures David Farhi Dustin Moskovitz Frontier-lab employees and alu Holden Karnofsky Jaan Tallinn Jeffrey Ladish Macroscopic Ventures (formerly Miles Brundage Neil Chowdhury Nvidia OpenAI employees (personal hol OpenAI Foundation Peter Mattson Sequoia Capital Zico Kolter Adam Gleave Artificial Intelligence Underw Clement Delangue Coefficient Giving (formerly O Good Ventures Foundation Jason Droege Longview Philanthropy Marco Mascorro Paul Christiano Rajiv Dattani Survival and Flourishing Fund Conrad Stosz Dan Hendrycks Mike McCormick Alec Radford Alignment Research Center Constellation Research Center ELMA Philanthropies Emerson Collective EU budget (Digital Europe Prog Fifty Years (50Y) Founders Pledge Founders Pledge (frontier AI f Gates Foundation Halcyon Futures High Tide Foundation Hillspire (Schmidt family offi Hudson River Trading Jacob Hilton MacKenzie Scott Magarac Venture Partners Obvious Ventures Redpoint Ventures Renaissance Philanthropy Salesforce Ventures Samsung Next Schmidt Sciences Silicon Valley Community Found Skoll Foundation Snowflake Ventures Swish Ventures Sympatico Ventures The Audacious Project (TED) UK government (DSIT) US government (NIST appropriat Valhalla Foundation Vanguard Charitable Wing Venture Capital Y Combinator Andon Labs Apollo Research AVERI Center for AI Safety Epoch AI EquiStamp EU AI Office FAR.AI Gray Swan Hugging Face Open Alignment In Irregular (formerly Pattern La METR Microsoft AI Red Team MLCommons Nemesys Insights Palisade Research RAND Corporation Redwood Research SaferAI Scale AI (SEAL) SecureBio Transluce UK AI Security Institute US CAISI (NIST)