Assurance regime; jurisdiction US/EU
Frontier AI evaluation
Subject of the study.
Path signature
V voluntary D delegation V voluntary T trigger S standards A access/publication T trigger M mandate A access/publication
Reference path.
Stages reached
| Voluntary | 2022 |
| Trigger | 2025 |
| Mandate | 2026 |
| Standards | 2025 |
| Oversight | not reached |
| Independence | not reached |
| Access | 2025 |
Mechanisms
| Before reform | Now | |
|---|---|---|
| pays | A | A |
| selects | A | A |
| access | shallow | shallow |
| publishes | summary | summary |
| oversees | none | none |
Mixed payers in practice (labs, philanthropy, governments); labs select and scope; embedded access pledged Sept 2026, not yet in force; nobody oversees evaluators.
Payer, access, publication
Who pays. Mixed: philanthropy, lab fees, government budgets. No accreditation.
Access. Voluntary pre-release API access, occasionally deeper; embedded access pledged September 2026.
Publication. System-card citations; some independent reports; redaction negotiated per engagement.
Milestones
| Year | Kind | Event | Strength | Harm | |
|---|---|---|---|---|---|
| 2020 | proposal | Toward Trustworthy AI Development calls for third-party auditing and red teaming. | source | ||
| 2022 | voluntary assurance | ARC Evals (later METR) tests GPT-4 pre-release; reported in the 2023 system card. | source | ||
| 2023 | delegation | Labs run their own safety evaluations and select, scope, and pay the external testers cited in system cards. | source | ||
| 2023 | voluntary assurance | White House voluntary commitments; UK AI Safety Institute founded. | source | ||
| 2024 | voluntary assurance | US AISI signs pre-deployment MOUs with OpenAI and Anthropic. | source | ||
| 2025 | access expansion | EU Code of Practice: at least 20 business days and minimally guardrailed versions for external evaluators (voluntary code supporting the AI Act; enforceable obligations land August 2026). | 2 | source | |
| 2025 | standards | AEF-1 (AI Evaluator Forum) published in December: minimum operating conditions for independent evaluation. | 2 | source | |
| 2025 | trigger | FrontierMath funding disclosure failure. | integrity_failure: Benchmark funder and data access undisclosed under contract | source | |
| 2026 | access expansion | Anthropic and OpenAI pledge embedded evaluators with employee-level access and publication rights. | 1 | source | |
| 2026 | mandate | Illinois SB 315 mandates annual independent audits from 2028 and bars the auditor and developer from holding a financial interest in each other (a financial-audit-style independence test embedded in the mandate). | 3 | source | |
| 2026 | trigger | OpenAI agent swarm breaches Hugging Face; first on-site third-party incident investigation. | integrity_failure: Evaluation agents breached a third party; safeguards and monitoring failed | source |
seed; secondary sources; verify each milestone against a primary source before citing in the paper Data: data/industries/frontier-ai.json