Evaluator Bench
evaluator; distance from a lab: 1

FAR.AI

Scorecard

Red-teaming and adversarial-robustness nonprofit; leads the EU AI Office CBRN risk-modelling contract.

Nonprofit, Berkeley, US. Independent referee; listed with independent referees. Confidence high. Domains: jailbreak, bio, cyber, misuse.

No floor triggered 8/8 evidenced · Retrieved & confirmed Scores by preset: Lab procurement –; Regulator or auditor –; Public trust –; Equal weights –. Weakest evidenced dimension: access depth (lab-granted) 1/4. No evidenced dimension below 2 — not a pass. No floor triggered means no disqualifying conflict is on file, not a pass. Full evidence.

The weighted numbers are hidden until you ask for them: two readers with different priorities see different numbers, and neither is the site's.

What would move the score. Disclose the AI Safety Fund share of budget and whether lab-funded pools are excluded from evaluation work.

Dissent, lower. Access could be 0. No cited source records the terms of any lab engagement: the wiki says FAR.AI began red-teaming leading models for frontier labs in late 2023 and that publicly confirmed engagements include OpenAI, but neither it nor the $30M post shows pre-release checkpoints, safeguards off, or logs. Anchor 0 is public API only, and that is all the record demonstrates; the EU contract is a regulator engagement, not lab access.

Dissent, higher. Access could be 2. OpenAI's GPT-5 system card credits FAR.AI with 80 hours of red-teaming before release, and pre-release red-teamers typically test early checkpoints under relaxed safeguards; the transparency page describes engagements with companies building frontier models under NDA. If the engagement terms show safeguards off or extended windows, anchor 2 applies, and helpful-only or log access would reach 3.

Each value is the tightest admissible cap or, with no cap, the highest admissible floor, under RULES.md. The binding signal is highlighted. A dash means no admissible signal sets a bound under the Retrieved & confirmed policy.

Funding 3/4
Anchor 3: Mostly philanthropic or public money; some lab-linked pooled funds.
evidence: tier 1 (filing/index)
Capped at 3 by farai.02 (F.6): Also funded by the AI Safety Fund, financed by the Frontier Model Forum, the labs' industry body.; capped at 3 by farai.12 (F.6): Largest single grant: $28,675,000 over three years from Coefficient Giving (September 2025); over $30 million in 2025 commitments from Coefficient, Schmidt Sciences, SFF, CSET and the Frontier Model Forum's AI Safety Fund.; capped at 3 by farai.17 (F.6): Founders Pledge, a donor-advised giving vehicle, granted $500,000 to the FAR.AI Integrity Team for EU AI Office manipulation evaluations.; capped at 3 by farai.16 (F.3): FAR.AI charges for-profit AI developers market rates for consulting and caps that revenue at 10% of total annual revenue; the transparency page reports TY2024 earned revenue of $429K against $24.3M total.. Floors up to 3 from farai.01, farai.10 do not exceed the cap. Not counted under this policy: farai.14 (no confirmed source (unaudited)).
Governance 2/4
Anchor 2: For-profit or PBC with a published COI policy.
evidence: tier 3 (self)
Floored at 2 by farai.05 (G.1): Nonprofit with diversified disclosed funders.. No admissible signal caps it.
Personnel 2/4
Anchor 2: Frequent two-way hiring; recusal on request.
evidence: tier 4 (press)
Capped at 2 by farai.07 (P.3): Runs field-building programs that place researchers into labs..
Access depth (lab-granted) 1/4
Anchor 1: Pre-release API with safeguards on.
evidence: tier 4 (press)
mechanism: lab-controlled
Floored at 1 by farai.06 (A.1): Pre- and post-deployment red teaming for labs and EU bodies.. No admissible signal caps it.
Scope control 3/4
Anchor 3: Evaluator sets scope and can add questions.
evidence: tier 1 (filing/index)
Floored at 3 by farai.03 (S.3): Selected by the European Commission to lead a three-year CBRN technical-assistance contract, with a regulator as client.; floored at 3 by farai.11 (S.3): Leads EU AI Office Lot 1 (CBRN) with SecureBio and SaferAI, EUR 1.43M to the consortium.. No admissible signal caps it.
Publication rights 2/4
Anchor 2: Publishes; lab reviews with broad redaction.
evidence: tier 3 (self)
mechanism: self-imposed
Floored at 2 by farai.04 (R.7): Publishes jailbreak and stress-test results across vendors.. No admissible signal caps it.
Method transparency 2/4
Anchor 2: Methods described in prose.
evidence: tier 3 (self)
Floored at 2 by farai.08 (M.2): Publishes methods and papers.. No admissible signal caps it.
Role incompatibility 2/4
Anchor 2: Consults for labs.
evidence: tier 3 (self)
floor and cap disagree
Conflict: floor 3 from farai.09 (X.4) against cap 2 from farai.15 (X.3); resolved at 2 by X.3: FAR.AI's transparency page records paid work for for-profit frontier developers at market rates; paid consulting for a lab caps role incompatibility at 2 whatever else is open or free.
Resolution: X.3 decides 2. FAR.AI's transparency page records paid work for for-profit frontier developers at market rates; paid consulting for a lab caps role incompatibility at 2 whatever else is open or free.
  • No commercial products; consulting limited to governments. floors at 3 by X.4“FAR AI conducts empirical technical AI safety research without developing or deploying AI products of its own”
    FAR AI (Longterm Wiki, 2026-02-26) tier 4 press (aggregator)confirmed2026-09-14
  • FAR.AI's transparency page publishes a consulting policy whose clients include companies building frontier AI models, charged at market rates, with revenue from for-profit developers capped at 10% of annual revenue; TY2024 earned revenue was $429K. caps at 2 by X.3binding“we charge market rates to for-profit developers to avoid subsidizing private actors”
    Transparency (FAR.AI, 2026) tier 3 self (farai)confirmed2026-09-15

Values under each evidence policy

DimensionLeads includedRetrieved & confirmedAgainst interestVerified spansPrimary only
Funding3333–
Governance22–2–
Personnel222––
Access depth (lab-granted)1111–
Scope control3333–
Publication rights22–2–
Method transparency22–2–
Role incompatibility2222–
Score, Lab procurement– No floor triggered, 8/8– No floor triggered, 8/8– No floor triggered, 5/8– No floor triggered, 7/8– Unevidenced, 0/8
Score, Regulator or auditor– No floor triggered, 8/8– No floor triggered, 8/8– No floor triggered, 5/8– No floor triggered, 7/8– Unevidenced, 0/8
Score, Public trust– No floor triggered, 8/8– No floor triggered, 8/8– No floor triggered, 5/8– No floor triggered, 7/8– Unevidenced, 0/8
Score, Equal weights– No floor triggered, 8/8– No floor triggered, 8/8– No floor triggered, 5/8– No floor triggered, 7/8– Unevidenced, 0/8

Traced money and ties

hop 1: 1 row; hop 2: 2 rows (recommendation $0.9M); public: 1 row; unattributed: 2 rows (grant $0.5M). Confirmed rows: 4 of 6 (2 excluded after a re-fetch disagreed or could not be verified: T03, T43) Dollar sums: confirmed + unaudited USD rows only; imported figures are not re-derived and never summed. Second hop traced for 5 of 6 sources: untraced Founders Pledge (frontier AI fund). Ties: none within two steps recorded. Checked and not found: 0.

Ledger

Money in

Roles held

Roles hosted

Funding graph, focused

Every one-way chain to a lab and to an evaluator at full strength, everything else faded. Hover any name to trace its connections; click a name to open its page.

The funding graph around FAR.AI Teal lines are money (transfers); amber lines are roles. Solid: confirmed; dashed: imported. Hover a name to trace its chains to the labs and evaluators — each chain runs one way across the columns, never doubling back; hover a line for the row. Every element is a row in data/ledger/. Labs Direct lab ties Two steps Three or more Public or unattributed Evaluators Amazon Anthropic G42 Google / Google DeepMind Meta Microsoft OpenAI Thinking Machines Lab xAI AI Safety Fund (Frontier Model Alexandr Wang Andreessen Horowitz Anthropic employees (personal Ben Mann D.E. Shaw Ventures David Farhi Dustin Moskovitz Frontier-lab employees and alu Holden Karnofsky Jaan Tallinn Jeffrey Ladish Macroscopic Ventures (formerly Miles Brundage Neil Chowdhury Nvidia OpenAI employees (personal hol OpenAI Foundation Peter Mattson Sequoia Capital Zico Kolter Adam Gleave Artificial Intelligence Underw Clement Delangue Coefficient Giving (formerly O Good Ventures Foundation Jason Droege Longview Philanthropy Marco Mascorro Paul Christiano Rajiv Dattani Survival and Flourishing Fund Conrad Stosz Dan Hendrycks Mike McCormick Alec Radford Alignment Research Center Constellation Research Center ELMA Philanthropies Emerson Collective EU budget (Digital Europe Prog Fifty Years (50Y) Founders Pledge Founders Pledge (frontier AI f Gates Foundation Halcyon Futures High Tide Foundation Hillspire (Schmidt family offi Hudson River Trading Jacob Hilton MacKenzie Scott Magarac Venture Partners Obvious Ventures Redpoint Ventures Renaissance Philanthropy Salesforce Ventures Samsung Next Schmidt Sciences Silicon Valley Community Found Skoll Foundation Snowflake Ventures Swish Ventures Sympatico Ventures The Audacious Project (TED) UK government (DSIT) US government (NIST appropriat Valhalla Foundation Vanguard Charitable Wing Venture Capital Y Combinator Andon Labs Apollo Research AVERI Center for AI Safety Epoch AI EquiStamp EU AI Office FAR.AI Gray Swan Hugging Face Open Alignment In Irregular (formerly Pattern La METR Microsoft AI Red Team MLCommons Nemesys Insights Palisade Research RAND Corporation Redwood Research SaferAI Scale AI (SEAL) SecureBio Transluce UK AI Security Institute US CAISI (NIST)