Evaluator Bench
AVERI, 2026-08; retrieved 2026-09-14 tier 3 self (averi)confirmed

AVERI Pilot Report: The World's First Double-Blind Evaluation of a Proprietary Language Model

https://www.averi.org/ourwork/averi-pilot-report-the-worlds-first-double-blind-eval

Gemini 2.5 Flash-Lite in secure enclave with DeepMind, OpenMined, MLCommons; notes EU CoP and Illinois SB 315 requirements. Cited from a search snippet; not re-fetched in full. [verify 2026-09-15] fetched; page reachable, citation matches

Signals citing this source

  • AVERI, A (for): Secure-enclave double-blind protocol with DeepMind, OpenMined and MLCommons; neither party sees the other's confidential material.“The pilot tested Gemini 2.5 Flash-Lite using never-before-used prompts from the MLCommons safety benchmark family”
  • AVERI, M (for): Publishes pilot reports and open tooling; framework paper with AI Assurance Levels.“converts what we learn into auditing standards, policy analysis, and open source tools”
  • AVERI, S (against): Pilot audits are voluntary and lab-selected; not yet a full-time auditor.
  • AVERI, R (for): Publishes pilot findings.“We provided Google DeepMind with a confidential report of the findings”
  • AVERI, R (against): The findings of the only completed pilot were delivered to Google DeepMind as a confidential report; the public pilot report describes the protocol and the types of successes and failure modes without quantitative results, prompts or outputs.“We provided Google DeepMind with a confidential report of the findings”
  • AVERI, A (against): AVERI's only completed engagement evaluated Gemini 2.5 Flash-Lite, a publicly released model, inside a secure enclave that hid the weights; no pre-release, safeguards-off, helpful-only, log or on-site access has been documented.“The pilot tested Gemini 2.5 Flash-Lite”
  • MLCommons (AILuminate), M (for): Open benchmark specification and methodology.
  • MLCommons (AILuminate), A (for): Used as neutral ground in the AVERI double-blind pilot.“AVERI, OpenMined, and MLCommons were not able to see the model’s weights”
  • MLCommons (AILuminate), A (against): The only documented engagement evaluated a publicly released model inside an enclave; no pre-release access from a lab is documented.“AVERI, OpenMined, and MLCommons were not able to see the model’s weights”

Ledger rows citing this URL

  • none