evaluator; distance from a lab: 2
AVERI
Scorecard
Standards body for frontier AI auditing; ran the first double-blind enclave evaluation of a proprietary model.
Nonprofit, Washington, US. Independent referee; listed with independent referees. Confidence med. Domains: assurance.
No floor triggered 8/8 evidenced · Retrieved & confirmed Scores by preset: Lab procurement –; Regulator or auditor –; Public trust –; Equal weights –. Weakest evidenced dimension: access depth (lab-granted) 0/4. No evidenced dimension below 2 — not a pass. No floor triggered means no disqualifying conflict is on file, not a pass. Full evidence.
The weighted numbers are hidden until you ask for them: two readers with different priorities see different numbers, and neither is the site's.
What would move the score. Funder disclosure and a first audit under statutory terms.
- Funding 3 to 4: score –
- Governance 3 to 4: score –
- Personnel 3 to 4: score –
Dissent, lower. Access is at the floor; no lower notch exists. The public record supports 0: AVERI's only completed engagement queried Gemini 2.5 Flash-Lite, a model public since July 2025, inside an enclave that hid the weights from the auditors, and the report says the pilot addressed some but not all ways a developer could tamper with the evaluation. No pre-release checkpoint, safeguards-off run, log or on-site access appears anywhere on the record.
Dissent, higher. Access could be 1. AVERI reports API credits from six labs "as donations or compensation" and says it is pursuing pilots on whether companies follow their own safety and security frameworks; Google DeepMind ran a proprietary model for it inside a trusted execution environment it did not control. A nonpublic endpoint or pre-release checkpoint in the current pilots would meet anchor 1, and the enclave protocol is a documented route to deeper access.
Each value is the tightest admissible cap or, with no cap, the highest admissible floor, under RULES.md. The binding signal is highlighted. A dash means no admissible signal sets a bound under the Retrieved & confirmed policy.
Funding 3/4
Anchor 3: Mostly philanthropic or public money; some lab-linked pooled funds.
evidence: tier 3 (self)
Capped at 3 by averi.06 (F.12): Funding sources not fully itemized publicly.; capped at 3 by averi.11 (F.6): AIUC, a funder, was seeded by an Anthropic co-founder; Halcyon's CEO chairs Transluce's board; Coefficient's principal is an Anthropic investor. Two of the named funders are two steps from labs.; capped at 3 by averi.14 (F.7): AVERI accepts and has received donations from current and former non-executive employees and alumni of frontier AI companies; amounts are not disclosed.. Floors up to 3 from averi.09, averi.12 do not exceed the cap.
Curator: Anchor 3: philanthropic and diversified with named funders and a no-majority rule; lab-employee donations and two lab-adjacent funders keep it off anchor 4.
- Funding sources not fully itemized publicly. caps at 3 by F.12binding“Our funders include the AI Underwriting Company, Coefficient Giving, Craig Falls, Geoff Ralston, Good Forever Foundation”
About (AVERI, 2026) tier 3 self (averi)confirmed2026-09-14 - Funders disclosed by name (AIUC, Coefficient, Halcyon, Fathom, Sympatico Ventures, individuals) with no donor a majority; donations from non-executive lab employees disclosed; API credits offered by six labs and disclosed. floors at 3 by F.12“No donor makes up a majority of our funding”
About (funding and conflicts) (AVERI, 2026) tier 3 self (averi)confirmed2026-09-14 - AIUC, a funder, was seeded by an Anthropic co-founder; Halcyon's CEO chairs Transluce's board; Coefficient's principal is an Anthropic investor. Two of the named funders are two steps from labs. caps at 3 by F.6binding“Our funders include the AI Underwriting Company, Coefficient Giving”
About (funding and conflicts) (AVERI, 2026) tier 3 self (averi)confirmed Apollo Research is becoming a PBC (investment section) (Apollo Research, 2026-01-20) tier 3 self (apollo)confirmed2026-09-14 - Raised $7.5 million toward a $13 million goal at launch; funders named; a search of Coefficient's grants pages found no discrete AVERI grant page though AVERI names Coefficient as a funder. floors at 3 by F.12“AVERI said it has raised $7.5 million toward a goal of $13 million”
Former OpenAI policy chief debuts new institute called AVERI (Fortune, 2026-01-15) tier 4 press (primary)confirmed About (funding and conflicts) (AVERI, 2026) tier 3 self (averi)confirmed2026-01-15 - AVERI accepts and has received donations from current and former non-executive employees and alumni of frontier AI companies; amounts are not disclosed. caps at 3 by F.7binding“several non-executive employees and alumni of frontier AI companies”
About (funding and conflicts) (AVERI, 2026) tier 3 self (averi)confirmed Former OpenAI policy chief debuts new institute called AVERI (Fortune, 2026-01-15) tier 4 press (primary)confirmed2026-09-15 - AVERI has received API credits "as donations or compensation" from Amazon, Anthropic, Google DeepMind, Microsoft, OpenAI and Thinking Machines Lab; amounts are undisclosed. informational (F.11: undisclosed in-kind credits without a dependency statement set no funding bound; recorded on the access mechanism. Null pending a document on which credits were compensation for pilot work (F.3 would then cap at 2).)“We've received API credits as donations or compensation from Amazon, Anthropic, Google DeepMind, Microsoft, OpenAI”
About (funding and conflicts) (AVERI, 2026) tier 3 self (averi)confirmed2026-09-15 - Document request: Dollar shares per funder.
Governance 3/4
Anchor 3: Nonprofit or public body with a COI policy.
evidence: tier 3 (self)
Floored at 3 by averi.13 (G.2): Published conflict-of-interest rules: all employees, board members and contractors disclose conflicts on starting and biannually; material conflicts trigger mandatory recusal or declining the engagement; no cash donations or compensation are accepted from frontier AI companies or their executives; lab API credits and lab-employee donations are disclosed; the most significant conflicts are disclosed publicly.. No admissible signal caps it.
Curator: Anchor 3: 501(c)(3) nonprofit (averi.02). No tier-1 source for an independent board or external review, so 4 is unearned under the tier rule (C10).
- 501(c)(3); endorsed Illinois SB 315; co-authored AEF-1. floors at 2 by G.1“AVERI is a 501(c)(3), US-based non-profit think tank”
The Launch of AVERI (Miles Brundage (Substack), 2026-01-15) tier 3 self (averi)confirmed About (AVERI, 2026) tier 3 self (averi)confirmed2026-09-14 - Published conflict-of-interest rules: all employees, board members and contractors disclose conflicts on starting and biannually; material conflicts trigger mandatory recusal or declining the engagement; no cash donations or compensation are accepted from frontier AI companies or their executives; lab API credits and lab-employee donations are disclosed; the most significant conflicts are disclosed publicly. floors at 3 by G.2binding“mitigation steps may include recusing personnel from projects or declining an engagement entirely”
About (funding and conflicts) (AVERI, 2026) tier 3 self (averi)confirmed2026-09-15
Personnel 3/4
Anchor 3: Recusal policy and disclosure of lab ties.
evidence: tier 3 (self)
Capped at 3 by averi.04 (P.3): Founder and several staff come from labs and lab-adjacent institutions.. Floors up to 3 from averi.10 do not exceed the cap.
Curator: Anchor 3: recusal policy with a stated cooling-off for the director and disclosed ties; short of anchor 4 because the cooling-off is person-specific rather than a standing rule.
- Founder and several staff come from labs and lab-adjacent institutions. caps at 3 by P.3binding“I hope that AVERI will also benefit from the practical experiences I had at OpenAI”
The Launch of AVERI (Miles Brundage (Substack), 2026-01-15) tier 3 self (averi)confirmed2026-09-14 - Executive director recused from auditing OpenAI for at least two years after leaving and sold eligible shares; team members' individual holdings disclosed; mandatory recusal for material conflicts. floors at 3 by P.5“Miles is recused from directly auditing OpenAI until two years after his October 2024 departure”
About (funding and conflicts) (AVERI, 2026) tier 3 self (averi)confirmed2026-09-14
Access depth (lab-granted) 0/4
Anchor 0: Public API only.
evidence: tier 3 (self)
mechanism: lab-controlled
Capped at 0 by averi.16 (A.1): AVERI's only completed engagement evaluated Gemini 2.5 Flash-Lite, a publicly released model, inside a secure enclave that hid the weights; no pre-release, safeguards-off, helpful-only, log or on-site access has been documented.. Floors up to 0 from averi.01 do not exceed the cap.
- Secure-enclave double-blind protocol with DeepMind, OpenMined and MLCommons; neither party sees the other's confidential material. floors at 0 by A.1“The pilot tested Gemini 2.5 Flash-Lite using never-before-used prompts from the MLCommons safety benchmark family”
AVERI Pilot Report: The World's First Double-Blind Evaluation of a Proprietary Language Model (AVERI, 2026-08) tier 3 self (averi)confirmed2026-09-14 - AVERI's only completed engagement evaluated Gemini 2.5 Flash-Lite, a publicly released model, inside a secure enclave that hid the weights; no pre-release, safeguards-off, helpful-only, log or on-site access has been documented. caps at 0 by A.1binding“The pilot tested Gemini 2.5 Flash-Lite”
AVERI Pilot Report: The World's First Double-Blind Evaluation of a Proprietary Language Model (AVERI, 2026-08) tier 3 self (averi)confirmed About (AVERI, 2026) tier 3 self (averi)confirmed2026-09-15
Scope control 3/4
Anchor 3: Evaluator sets scope and can add questions.
evidence: tier 3 (self)
Capped at 3 by averi.05 (S.4): Pilot audits are voluntary and lab-selected; not yet a full-time auditor..
- Pilot audits are voluntary and lab-selected; not yet a full-time auditor. caps at 3 by S.4binding“some companies have chosen to conduct voluntary pilots with AVERI and other organizations”
About (AVERI, 2026) tier 3 self (averi)confirmed AVERI Pilot Report: The World's First Double-Blind Evaluation of a Proprietary Language Model (AVERI, 2026-08) tier 3 self (averi)confirmed2026-09-14
Publication rights 2/4
Anchor 2: Publishes; lab reviews with broad redaction.
evidence: tier 3 (self)
mechanism: lab-controlled
Capped at 2 by averi.15 (R.2): The findings of the only completed pilot were delivered to Google DeepMind as a confidential report; the public pilot report describes the protocol and the types of successes and failure modes without quantitative results, prompts or outputs.. Floors up to 2 from averi.07 do not exceed the cap.
- Publishes pilot findings. floors at 2 by R.2“We provided Google DeepMind with a confidential report of the findings”
AVERI Pilot Report: The World's First Double-Blind Evaluation of a Proprietary Language Model (AVERI, 2026-08) tier 3 self (averi)confirmed2026-09-14 - The findings of the only completed pilot were delivered to Google DeepMind as a confidential report; the public pilot report describes the protocol and the types of successes and failure modes without quantitative results, prompts or outputs. caps at 2 by R.2binding“We provided Google DeepMind with a confidential report of the findings”
AVERI Pilot Report: The World's First Double-Blind Evaluation of a Proprietary Language Model (AVERI, 2026-08) tier 3 self (averi)confirmed2026-09-15
Method transparency 3/4
Anchor 3: Tasks or code partly open.
evidence: tier 3 (self)
Floored at 3 by averi.03 (M.1): Publishes pilot reports and open tooling; framework paper with AI Assurance Levels.. No admissible signal caps it.
- Publishes pilot reports and open tooling; framework paper with AI Assurance Levels. floors at 3 by M.1binding“converts what we learn into auditing standards, policy analysis, and open source tools”
AVERI Pilot Report: The World's First Double-Blind Evaluation of a Proprietary Language Model (AVERI, 2026-08) tier 3 self (averi)confirmed Frontier AI Auditing: Toward Rigorous Third-Party Assessment (arXiv 2601.11699) (arXiv, 2026-02-07) tier 4 press (primary)confirmed2026-09-14
Role incompatibility 3/4
Anchor 3: Tools are open or free to the ecosystem.
evidence: tier 3 (self)
Floored at 3 by averi.08 (X.4): No commercial products; tools open source.. No admissible signal caps it.
- No commercial products; tools open source. floors at 3 by X.4binding“We convert insights from these processes into open source tools”
About (AVERI, 2026) tier 3 self (averi)confirmed2026-09-14
Values under each evidence policy
| Dimension | Leads included | Retrieved & confirmed | Against interest | Verified spans | Primary only |
|---|---|---|---|---|---|
| Funding | 3 | 3 | 3 | 3 | – |
| Governance | 3 | 3 | – | 3 | – |
| Personnel | 3 | 3 | 3 | 3 | – |
| Access depth (lab-granted) | 0 | 0 | 0 | 0 | – |
| Scope control | 3 | 3 | 3 | 3 | – |
| Publication rights | 2 | 2 | 2 | 2 | – |
| Method transparency | 3 | 3 | 3 | 3 | – |
| Role incompatibility | 3 | 3 | – | 3 | – |
| Score, Lab procurement | – No floor triggered, 8/8 | – No floor triggered, 8/8 | – No floor triggered, 6/8 | – No floor triggered, 8/8 | – Unevidenced, 0/8 |
| Score, Regulator or auditor | – No floor triggered, 8/8 | – No floor triggered, 8/8 | – No floor triggered, 6/8 | – No floor triggered, 8/8 | – Unevidenced, 0/8 |
| Score, Public trust | – No floor triggered, 8/8 | – No floor triggered, 8/8 | – No floor triggered, 6/8 | – No floor triggered, 8/8 | – Unevidenced, 0/8 |
| Score, Equal weights | – No floor triggered, 8/8 | – No floor triggered, 8/8 | – No floor triggered, 6/8 | – No floor triggered, 8/8 | – Unevidenced, 0/8 |
Traced money and ties
hop 1: 1 row; hop 2: 2 rows; unattributed: 1 row. Confirmed rows: 4 of 4 Dollar sums: confirmed + unaudited USD rows only; imported figures are not re-derived and never summed. Second hop traced for 3 of 4 sources: untraced Halcyon Futures. Ties: none within two steps recorded. Checked and not found: 2.
Ledger
Money in
- T49 from Coefficient Giving (formerly Open Philanthropy): grant undisclosed, 2025-2026. named funder; no donor is a majority confirmedtier 3 selfhttps://www.averi.org/about
- T50 from Halcyon Futures: grant undisclosed, 2025-2026. named funder confirmedtier 3 selfhttps://www.averi.org/about
- T51 from Artificial Intelligence Underwriting Company: grant undisclosed, 2025-2026. named funder confirmedtier 3 selfhttps://www.averi.org/about
- T65 from Frontier-lab employees and alumni (unnamed): grant undisclosed, 2025-2026. donations from current and former non-executive employees of frontier AI companies confirmedtier 3 selfhttps://www.averi.org/about
Checked and not found
- N17 Is any single donor a majority of AVERI's funding?
Searched organization's own funding disclosure, snapshot 2026-09-14: no single donor is a majority of funding. Prompted by AVERI's own funding page (averi.09). confirmedhttps://www.averi.org/about - N20 Is there a discrete Coefficient Giving grant page for AVERI?
Searched coefficientgiving.org/grants, name search, snapshot 2026-09-15: no discrete Coefficient Giving grant page for AVERI found, although AVERI names Coefficient as a funder. Prompted by AVERI names Coefficient as a funder (averi.12). unauditedhttps://coefficientgiving.org/
Roles hosted
- R47 Miles Brundage: principal, 2025. executive director; recused from auditing OpenAI for at least two years; sold eligible OpenAI shares confirmedhttps://www.averi.org/about
- R60 Coefficient Giving (formerly Open Philanthropy): pays, 2025. named funder confirmedhttps://www.averi.org/about
- R91 Sympatico Ventures: pays, 2025. named funder on AVERI's page confirmedhttps://www.averi.org/about
Funding graph, focused
Every one-way chain to a lab and to an evaluator at full strength, everything else faded. Hover any name to trace its connections; click a name to open its page.