Evaluators
24 ranked organizations in three lists, scored on eight independence dimensions under the Retrieved & confirmed evidence policy (confirmed sources only). Band first: an evidenced 0 on a conflict dimension (funding, governance, personnel, role incompatibility, scope, publication) is a disqualifying floor, a 1 a conditional floor; access and methods never set a band. The band, the coverage, the weakest evidenced dimension, and the eight dimension values show by default. The weighted number is hidden until you choose a preset and press the button: two readers with different presets will see different numbers, and both are right. Open a row for the derivation, the binding signals, the ledger, and the focused graph. Independence only; not quality, coverage, or competence. A "no floor triggered" band means no disqualifying conflict is on file; it is not a pass.
Score with a weight preset
The number is yours, so it is hidden until you choose weights. Pick a preset, then press the button to reveal the weighted number for every row.
Independent referees 12
Nonprofits, public benefit corporations, and academic groups whose work is evaluation or research and who sell nothing to the labs they grade. Lab fees for the evaluation itself keep an organization here; they lower its funding value instead.
| Evaluator | Role | Band | Score | Evidenced | Floor | Confidence | Confirmed rows | Domains |
|---|---|---|---|---|---|---|---|---|
| METR Nonprofit, Berkeley, US | Independent referee | No floor triggered | – | 8/8 | personnel 2 | high | 5/7 | autonomy, scheming, incident, assurance |
| Redwood Research Nonprofit, Berkeley, US | Independent referee | No floor triggered | – | 7/8 | governance 2 | med | 1/2 | scheming, incident, assurance |
| RAND Nonprofit, Santa Monica, US | Independent referee | No floor triggered | – | 6/8 | governance 2 | med | 4/4 | bio, cyber, assurance |
| Palisade Research Nonprofit, Berkeley, US | Independent referee | No floor triggered | – | 8/8 | access depth (lab-granted) 1 | med | 3/3 | cyber, autonomy, scheming |
| Transluce Nonprofit, San Francisco, US | Independent referee | No floor triggered | – | 8/8 | funding 2 | high | 3/3 | misuse, autonomy, benchmarks, assurance |
| AVERI Nonprofit, Washington, US | Independent referee | No floor triggered | – | 8/8 | access depth (lab-granted) 0 | med | 4/4 | assurance |
| Apollo Research Public benefit corp, London, UK / San Francisco, US | Independent referee | No floor triggered | – | 8/8 | funding 2 | high | 4/5 | scheming, autonomy, assurance |
| FAR.AI Nonprofit, Berkeley, US | Independent referee | No floor triggered | – | 8/8 | access depth (lab-granted) 1 | high | 4/6 | jailbreak, bio, cyber, misuse |
| SaferAI Nonprofit, Paris, FR | Independent referee | No floor triggered | – | 8/8 | access depth (lab-granted) 1 | med | 2/3 | assurance, cyber, bio, misuse |
| Humane Intelligence Nonprofit, New York, US | Independent referee | No floor triggered | – | 8/8 | access depth (lab-granted) 0 | med | 0/0 | misuse, jailbreak |
| Center for AI Safety Nonprofit, San Francisco, US | Independent referee | Conditional floor | – | 8/8 | access depth (lab-granted) 0 | med | 2/2 | benchmarks, misuse |
| EquiStamp Private company, capital structure undisclosed, Remote | Independent referee | Conditional floor | – | 6/8 | governance 1 | low | 1/1 | assurance, autonomy |
Government institutes 3
Public bodies. Publication and access are scored on outcome and tagged statutory where the constraint is the law, not a lab.
| Evaluator | Role | Band | Score | Evidenced | Floor | Confidence | Confirmed rows | Domains |
|---|---|---|---|---|---|---|---|---|
| EU AI Office Government, Brussels, BE | Government institute | No floor triggered | – | 8/8 | publication rights 2 | med | 1/1 | bio, cyber, misuse, assurance |
| UK AI Security Institute Government, London, UK | Government institute | No floor triggered | – | 8/8 | publication rights 2 | high | 2/3 | cyber, bio, jailbreak, autonomy, assurance |
| US CAISI (NIST) Government, Gaithersburg, US | Government institute | No floor triggered | – | 7/8 | method transparency 2 | high | 1/2 | cyber, bio, misuse, assurance |
Commercial and first-party 9
Vendors, first-party lab teams, and consortia funded by the labs they benchmark. Competent testers can sit here; the rubric measures structure, not skill.
| Evaluator | Role | Band | Score | Evidenced | Floor | Confidence | Confirmed rows | Domains |
|---|---|---|---|---|---|---|---|---|
| SecureBio Nonprofit, Cambridge, US | Vendor / red-team co. | Conditional floor | – | 8/8 | funding 1 | high | 12/12 | bio, misuse |
| Andon Labs VC-backed, Stockholm, SE / San Francisco, US | Vendor / red-team co. | Conditional floor | – | 8/8 | governance 1 | low | 1/1 | autonomy, benchmarks |
| MLCommons (AILuminate) Consortium, San Francisco, US | Benchmark or consortium | Conditional floor | – | 8/8 | access depth (lab-granted) 0 | med | 1/1 | benchmarks, assurance |
| Dreadnode VC-backed, Remote | Vendor / red-team co. | Conditional floor | – | 7/8 | governance 1 | low | 0/0 | cyber |
| Irregular (formerly Pattern Labs) VC-backed, Tel Aviv, IL / San Francisco, US | Vendor / red-team co. | Conditional floor | – | 8/8 | funding 1 | high | 6/6 | cyber, misuse |
| Nemesys Insights Private company, capital structure undisclosed, Albany, US | Vendor / red-team co. | Conditional floor | – | 8/8 | funding 1 | low | 2/2 | bio, misuse, assurance |
| Microsoft AI Red Team Big Tech unit, Redmond, US | First-party lab team | – | 7/8 | governance 0 | med | 1/1 | jailbreak, cyber, misuse | |
| Gray Swan VC-backed, Pittsburgh, US | Vendor / red-team co. | – | 8/8 | role incompatibility 0 | high | 2/2 | jailbreak, cyber, misuse | |
| Scale AI (SEAL / Scale Labs) VC-backed, San Francisco, US | Vendor / red-team co. | – | 8/8 | funding 0 | high | 1/1 | benchmarks, cyber, bio, misuse |
Watchlist (not ranked)
Expected entrants scored on the same rubric but excluded from rankings and every statistic: initiatives announced with no evaluations yet. Hypothetical composites are not scored (RULES 11).
| Evaluator | Role | Band | Score | Evidenced | Floor | Confidence | Confirmed rows | Domains |
|---|---|---|---|---|---|---|---|---|
| Hugging Face Open Alignment Initiative VC-backed, New York, US / Paris, FR | Expected entrant | not ranked | – | 8/8 | funding 0 | low | 1/1 | assurance |
Out of scope, retained for the record
Organizations whose frontier work is capability benchmarking rather than safety evaluation (RULES 11, decision D-001). Records, bounds, and derivations are unchanged; they are unranked and excluded from every statistic.
| Evaluator | Role | Band | Score | Evidenced | Floor | Confidence | Confirmed rows | Domains |
|---|---|---|---|---|---|---|---|---|
| Holistic Agent Leaderboard (Princeton) Academic, Princeton, US | Benchmark or consortium | not ranked | – | 8/8 | access depth (lab-granted) 0 | med | 0/0 | benchmarks, autonomy |
| Epoch AI Nonprofit, Remote | Independent referee | not ranked | – | 8/8 | access depth (lab-granted) 1 | high | 4/8 | benchmarks |