Nemesys Insights
Scorecard
Red-teaming firm led by the director of a university red-teaming center; paid assessor on Amazon's Nova model papers; subcontractor on the EU Lot 1 CBRN consortium. Paid contracting work for labs and governments makes this a vendor, not an independent referee.
Private company, capital structure undisclosed, Albany, US. Vendor / red-team co.; listed with commercial and first-party. Confidence low. Domains: bio, misuse, assurance.
Conditional floor 8/8 evidenced · Retrieved & confirmed Scores by preset: Lab procurement –; Regulator or auditor –; Public trust –; Equal weights –. Weakest evidenced dimension: funding 1/4. At least one evidenced conflict dimension (funding, governance, personnel, role incompatibility, scope, publication) at 1: usable with conditions the card names.
The weighted numbers are hidden until you ask for them: two readers with different priorities see different numbers, and neither is the site's.
What would move the score. Funding disclosure and a published policy on lab clients.
- Funding 1 to 2: score –
- Governance 1 to 2: score –
- Personnel 2 to 3: score –
Dissent, lower. Publication (R) at 1 could be 0. The public record shows no Nemesys-authored report on any model; every finding appears inside Amazon's own technical reports, which Amazon wrote, framed and released. Amazon "shares input prompts and corresponding model outputs" with the assessor and asks for a verdict against its own thresholds. Where the lab both commissions the work and holds the only pen, anchor 0 (lab approval required) is the closer description.
Dissent, higher. Publication (R) at 1 could be 2. Amazon's reports name Nemesys, describe its 120-prompt threshold analysis and its 800-participant uplift study, and print its conclusion; the assessor's method and verdict reached the public in the lab's paper rather than as a bare citation. That is nearer to a published finding with lab review than to lab-edited summaries only.
Each value is the tightest admissible cap or, with no cap, the highest admissible floor, under RULES.md. The binding signal is highlighted. A dash means no admissible signal sets a bound under the Retrieved & confirmed policy.
- Paid assessor for Amazon per the Nova papers; private LLC with no funding disclosed. caps at 1 by F.5binding“We share input prompts and corresponding model outputs with third-party assessors (e.g., Nemesys, METR)”
evaluators.csv (24 evaluators: Coefficient and SFF totals, lab money, government contracts, leadership) (Kevin Bass (GitHub), 2026-09-14) tier 2 ledgerimported Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework (arXiv 2507.06260, HTML) (Amazon (arXiv), 2025-07) tier 4 press (primary)confirmed2026-09-14 - EU Lot 1 subcontractor under the FAR.AI consortium. floors at 2 by F.3“Nemesys Insights (chemical, biological, radiological, and nuclear threats)”
TED 864574-2025 Technical Assistance for AI Safety (Tenders Electronic Daily, 2025-12) tier 1 filingconfirmed FAR.AI Selected to Lead EU AI Act CBRN Risk Consortium (FAR.AI, 2026-02-02) tier 3 self (farai)confirmed2026-09-14
- No policy published. caps at 1 by G.2binding“Nemesys Insights, LLC is a strategic analysis and advisory company”
evaluators.csv (24 evaluators: Coefficient and SFF totals, lab money, government contracts, leadership) (Kevin Bass (GitHub), 2026-09-14) tier 2 ledgerimported Nemesys Insights LLC | Strategic Analysis (Nemesys Insights, 2025) tier 3 self (nemesys)confirmed2026-09-14
- Leadership from a university red-team center; lab ties not established. caps at 2 by P.6binding“promote the tools and techniques developed by the Center for Advanced Red Teaming (CART)”
evaluators.csv (24 evaluators: Coefficient and SFF totals, lab money, government contracts, leadership) (Kevin Bass (GitHub), 2026-09-14) tier 2 ledgerimported Our Story (Nemesys Insights, 2025) tier 3 self (nemesys)confirmed Our Team (Nemesys Insights, 2025) tier 3 self (nemesys)confirmed2026-09-14
- Access through client engagement. caps at 1 by A.1binding“Nemesys Insights executed an Exploratory Critical-Capability Threshold Analysis”
evaluators.csv (24 evaluators: Coefficient and SFF totals, lab money, government contracts, leadership) (Kevin Bass (GitHub), 2026-09-14) tier 2 ledgerimported Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework (arXiv 2507.06260, HTML) (Amazon (arXiv), 2025-07) tier 4 press (primary)confirmed2026-09-14
- Scope set by the lab client. caps at 1 by S.1binding“their final assessment of whether the model is safe and remains within the defined thresholds for public release”
evaluators.csv (24 evaluators: Coefficient and SFF totals, lab money, government contracts, leadership) (Kevin Bass (GitHub), 2026-09-14) tier 2 ledgerimported Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework (arXiv 2507.06260, HTML) (Amazon (arXiv), 2025-07) tier 4 press (primary)confirmed2026-09-14
- Findings appear inside the lab's papers. caps at 1 by R.3binding“reviewed results on test sets, scoring rubrics, and safety-judge notes to verify our internal findings”
evaluators.csv (24 evaluators: Coefficient and SFF totals, lab money, government contracts, leadership) (Kevin Bass (GitHub), 2026-09-14) tier 2 ledgerimported Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework (arXiv 2507.06260, HTML) (Amazon (arXiv), 2025-07) tier 4 press (primary)confirmed2026-09-14
- Methods described in the lab's papers only. caps at 2 by M.2binding“They selected 120 "uplift indicator" prompts—60 in synthetic biology and 60 in microbiology/delivery systems”
evaluators.csv (24 evaluators: Coefficient and SFF totals, lab money, government contracts, leadership) (Kevin Bass (GitHub), 2026-09-14) tier 2 ledgerimported Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework (arXiv 2507.06260, HTML) (Amazon (arXiv), 2025-07) tier 4 press (primary)confirmed2026-09-14
- Consults for a lab. caps at 1 by X.2binding“we worked with Nemesys Insights to conduct a large-scale, independent uplift study”
evaluators.csv (24 evaluators: Coefficient and SFF totals, lab money, government contracts, leadership) (Kevin Bass (GitHub), 2026-09-14) tier 2 ledgerimported Evaluating Nova 2.0 Lite model under Amazon's Frontier Model Safety Framework (arXiv 2601.19134v1, HTML) (Amazon (arXiv), 2026-01) tier 4 press (primary)confirmed2026-09-14
Values under each evidence policy
| Dimension | Leads included | Retrieved & confirmed | Against interest | Verified spans | Primary only |
|---|---|---|---|---|---|
| Funding | 1 | 1 | 1 | 1 | 2 |
| Governance | 1 | 1 | 1 | 1 | – |
| Personnel | 2 | 2 | 2 | 2 | – |
| Access depth (lab-granted) | 1 | 1 | 1 | 1 | – |
| Scope control | 1 | 1 | 1 | 1 | – |
| Publication rights | 1 | 1 | 1 | 1 | – |
| Method transparency | 2 | 2 | 2 | 2 | – |
| Role incompatibility | 1 | 1 | 1 | 1 | – |
| Score, Lab procurement | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – No floor triggered, 1/8 |
| Score, Regulator or auditor | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – No floor triggered, 1/8 |
| Score, Public trust | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – No floor triggered, 1/8 |
| Score, Equal weights | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – No floor triggered, 1/8 |
Traced money and ties
hop 0: 1 row; public: 1 row. Confirmed rows: 2 of 2 Dollar sums: confirmed + unaudited USD rows only; imported figures are not re-derived and never summed. Second hop traced for 2 of 2 sources. Ties: none within two steps recorded. Checked and not found: 1.
Ledger
Money in
- T61 from Amazon: contract undisclosed, 2025. paid assessor for Amazon Nova models per Nova papers confirmedtier 3 selfhttps://arxiv.org/pdf/2507.06260
- T62 from EU budget (Digital Europe Programme): contract undisclosed, 2025-12-04. Lot 1 subcontractor to the FAR.AI consortium confirmedtier 3 selfhttps://www.far.ai/blog/far-ai-selected-to-lead-eu-ai-act-cbrn-risk-consortium
Checked and not found
- N15 Does Coefficient Giving fund Nemesys Insights?
Searched Coefficient grants index, 2,911 rows, snapshot 2026-09-11: no grant from Coefficient Giving in its index. Prompted by imported lead: kevinnbass ledger row EV08. importedhttps://coefficientgiving.org/grants/
One hop away
Funding graph, focused
Every one-way chain to a lab and to an evaluator at full strength, everything else faded. Hover any name to trace its connections; click a name to open its page.