evaluator; distance from a lab: 1
Irregular (formerly Pattern Labs)
Scorecard
Offensive-cyber evaluator cited in OpenAI and Anthropic system cards; SOLVE framework used by UK AISI. Pattern Labs and Irregular are the same company.
VC-backed, Tel Aviv, IL / San Francisco, US. Vendor / red-team co.; listed with commercial and first-party. Confidence high. Domains: cyber, misuse.
Conditional floor 8/8 evidenced · Retrieved & confirmed Scores by preset: Lab procurement –; Regulator or auditor –; Public trust –; Equal weights –. Weakest evidenced dimension: funding 1/4. At least one evidenced conflict dimension (funding, governance, personnel, role incompatibility, scope, publication) at 1: usable with conditions the card names.
The weighted numbers are hidden until you ask for them: two readers with different priorities see different numbers, and neither is the site's.
What would move the score. A published COI policy, disclosure of lab revenue share, and a firewall between evaluations and the security-products business.
- Funding 1 to 2: score –
- Governance 1 to 2: score –
- Personnel 2 to 3: score –
Dissent, lower. Role incompatibility (1 to 0). Irregular says it works side by side with OpenAI and Anthropic 'to develop the defenses needed before deployment' and describes a roadmap of runtime security controls for the same customers it evaluates. If any evaluated lab pays for those defenses, X.1 places the company at 0: it would be selling the fix to the party it grades.
Dissent, higher. Funding (1 to 2). The revenue disclosed is 'millions' from lab partnerships, but the UK government uses the SOLVE framework and the company markets to enterprises beyond the labs. If a non-lab revenue base is documented, per-engagement lab fees beside that base cap at 2 under F.3 rather than 1 under F.5.
Each value is the tightest admissible cap or, with no cap, the highest admissible floor, under RULES.md. The binding signal is highlighted. A dash means no admissible signal sets a bound under the Retrieved & confirmed policy.
Funding 1/4
Anchor 1: Material revenue or investment from evaluated labs or their investors.
evidence: tier 1 (filing/index)
Capped at 1 by irregular.01 (F.4): $80M raised in 2025 at a $450M valuation, led by Sequoia and Redpoint; Sequoia is also a major OpenAI investor.; capped at 1 by irregular.02 (F.5): Millions in annual revenue from the labs it evaluates.. Floors up to 1 from irregular.06 do not exceed the cap. Not counted under this policy: irregular.11 (superseded).
Curator: Anchor 1: a $6.8M Coefficient Giving grant appears in the index under the former name Pattern Labs (irregular.11; ledger T09 marked differs, unreconciled). A name-search under ‘Irregular’ found nothing (irregular.12), which does not rebut the old-name row — the project treats the two names as the same company. Material lab-adjacent funding with disputed attribution.
- $80M raised in 2025 at a $450M valuation, led by Sequoia and Redpoint; Sequoia is also a major OpenAI investor. caps at 1 by F.4binding“a round led by Sequoia Capital and Redpoint Ventures”
Irregular raises $80M to secure frontier AI models (TechCrunch, 2025-09-17) tier 4 press (primary)confirmed Partnering with Irregular (Sequoia Capital, 2025-09-17) tier 3 self (sequoia)confirmed Series B funding round (xAI, 2024-05-26) tier 3 self (xai)confirmed2026-09-14 - Millions in annual revenue from the labs it evaluates. caps at 1 by F.5binding“Already generating millions in revenue, Irregular partners with leading labs like OpenAI and Anthropic”
Irregular Raises $80 Million (Newswire, 2025-09-17) tier 4 press (aggregator)confirmed2026-09-14 - Government clients (UK) alongside labs. floors at 1 by F.5 (anchor text)“the UK government and Anthropic use Irregular's SOLVE framework”
Irregular Raises $80 Million (Newswire, 2025-09-17) tier 4 press (aggregator)confirmed2026-09-14 - A Coefficient Giving grant of about $6.8M appears in the index for a venture-backed company; purpose to be verified. informational (Superseded signals support nothing (RULES 0).)not counted under standard
Coefficient Giving grants index (Coefficient Giving, 2026-09-11) tier 1 indeximported2026-09-11 - A search of Coefficient Giving's grants pages under the name Irregular found no grant (snapshot 2026-09-15); the imported row citing $6.8M under Pattern Labs is unreconciled and marked differs. informational (Informational: neither floor nor cap; recorded as a gap.)
Coefficient Giving home (Coefficient Giving, 2026) tier 3 self (coefficient)confirmed evaluators.csv (24 evaluators: Coefficient and SFF totals, lab money, government contracts, leadership) (Kevin Bass (GitHub), 2026-09-14) tier 2 ledgerimported2026-09-15 - Good Ventures Foundation, Coefficient's principal funder and an Anthropic investor's foundation, paid Pattern Labs Tech Inc $4,533,333 (grant dated 5 March 2024) and $2,266,667 (FY2025) for developing software tools, products and analysis focused on global security, and held a $3,000,000 stake in the company at 30 June 2024 per its 990-PF. caps at 3 by F.6“PATTERN LABS TECH INC 108 W 13TH ST STE 100 WILMINGTON DE 19801 2024-03-05 4533333”
Good Ventures Foundation 990-PF, tax period 2023-07-01 to 2024-06-30 (IRS e-file XML) (IRS e-file via GivingTuesday 990 data lake mirror, 2025) tier 1 filingconfirmed Good Ventures Foundation 990-PF, fiscal year ending June 2025 (ProPublica full-text render) (ProPublica Nonprofit Explorer, 2026) tier 1 filingconfirmed Site search (Coefficient Giving, 2026) tier 3 self (coefficient)confirmed evaluators.csv (row EV07) (kevinnbass/metr-money-figure, 2026-09-14) tier 2 ledgerconfirmed2026-09-15 - Document request: Re-run the Coefficient index search under both Irregular and Pattern Labs and record the snapshot.
Governance 1/4
Anchor 1: Private for-profit with no published COI policy.
evidence: tier 4 (press)
Capped at 1 by irregular.03 (G.2): Venture-backed for-profit; no published COI or recusal policy found..
- Venture-backed for-profit; no published COI or recusal policy found. caps at 1 by G.2binding“a round led by Sequoia Capital and Redpoint Ventures”
Irregular raises $80M to secure frontier AI models (TechCrunch, 2025-09-17) tier 4 press (primary)confirmed2026-09-14
Personnel 2/4
Anchor 2: Frequent two-way hiring; recusal on request.
evidence: tier 3 (self)
Capped at 2 by irregular.10 (P.5): Markets itself as embedded with the labs; no recusal or cooling-off policy found..
- Markets itself as embedded with the labs; no recusal or cooling-off policy found. caps at 2 by P.5binding“By embedding directly with these cutting-edge research labs”
Partnering with Irregular (Sequoia Capital, 2025-09-17) tier 3 self (sequoia)confirmed2026-09-14
Access depth (lab-granted) 2/4
Anchor 2: Pre-release with safeguards off or extended time.
evidence: tier 3 (self)
mechanism: lab-controlled
Floored at 2 by irregular.05 (A.1): Deep, repeated pre-deployment access at OpenAI, Anthropic and Google DeepMind; embedded with labs.. No admissible signal caps it.
- Deep, repeated pre-deployment access at OpenAI, Anthropic and Google DeepMind; embedded with labs. floors at 2 by A.1binding“By embedding directly with these cutting-edge research labs”
Partnering with Irregular (Sequoia Capital, 2025-09-17) tier 3 self (sequoia)confirmed2026-09-14
Scope control 2/4
Anchor 2: Scope negotiated per engagement.
evidence: tier 3 (self)
Capped at 2 by irregular.09 (S.2; S.6): Scope set by lab engagements; no public account of evaluator-set scope..
- Scope set by lab engagements; no public account of evaluator-set scope. caps at 2 by S.2; S.6binding“The company also co-authored a whitepaper with Anthropic”
Partnering with Irregular (Sequoia Capital, 2025-09-17) tier 3 self (sequoia)confirmed Irregular Raises $80 Million (Newswire, 2025-09-17) tier 4 press (aggregator)confirmed2026-09-14
Publication rights 2/4
Anchor 2: Publishes; lab reviews with broad redaction.
evidence: tier 3 (self)
mechanism: lab-controlled
Capped at 2 by irregular.14 (R.3): Findings on lab models reach the public only as lab-authored system-card citations; the company publishes its own research on other work (a RAND co-authored paper and an Anthropic co-authored whitepaper).. Floors up to 1 from irregular.08 do not exceed the cap.
- Findings appear in system cards for o3, o4-mini, GPT-5 and Claude models. floors at 1 by R.3 (anchor 1)“Irregular’s evaluations are cited (under their former name, Pattern Labs) in system cards for GPT-4, o3, o4 mini and 5”
Irregular raises $80M to secure frontier AI models (TechCrunch, 2025-09-17) tier 4 press (primary)confirmed Partnering with Irregular (Sequoia Capital, 2025-09-17) tier 3 self (sequoia)confirmed2026-09-14 - Findings on lab models reach the public only as lab-authored system-card citations; the company publishes its own research on other work (a RAND co-authored paper and an Anthropic co-authored whitepaper). caps at 2 by R.3binding“Irregular’s evaluations are cited (under their former name, Pattern Labs) in system cards for GPT-4, o3, o4 mini and 5”
Partnering with Irregular (Sequoia Capital, 2025-09-17) tier 3 self (sequoia)confirmed Irregular Raises $80 Million (Newswire, 2025-09-17) tier 4 press (aggregator)confirmed2026-09-15
Method transparency 2/4
Anchor 2: Methods described in prose.
evidence: tier 3 (self)
Floored at 2 by irregular.07 (M.2): Publishes some method papers; SOLVE framework adopted by UK AISI and Anthropic.. No admissible signal caps it.
- Publishes some method papers; SOLVE framework adopted by UK AISI and Anthropic. floors at 2 by M.2binding“the UK government and Anthropic both use the company’s SOLVE framework”
Partnering with Irregular (Sequoia Capital, 2025-09-17) tier 3 self (sequoia)confirmed2026-09-14
Role incompatibility 1/4
Anchor 1: Sells products or services other than the evaluation itself to labs or to their customers.
evidence: tier 4 (press)
Capped at 1 by irregular.13 (X.2): Delivers paid evaluation services to OpenAI and Anthropic; lab partnerships are the only revenue the company discloses..
- Growth plan is to sell runtime security controls into the same customers it evaluates. caps at 2 by X.1 (second sentence)“to develop the defenses needed before deployment”
Irregular Raises $80 Million (Newswire, 2025-09-17) tier 4 press (aggregator)confirmed2026-09-14 - Delivers paid evaluation services to OpenAI and Anthropic; lab partnerships are the only revenue the company discloses. caps at 1 by X.2binding“Already generating millions in revenue, Irregular partners with leading labs like OpenAI and Anthropic”
Irregular Raises $80 Million (Newswire, 2025-09-17) tier 4 press (aggregator)confirmed2026-09-15
Values under each evidence policy
| Dimension | Leads included | Retrieved & confirmed | Against interest | Verified spans | Primary only |
|---|---|---|---|---|---|
| Funding | 1 | 1 | 1 | 1 | 3 |
| Governance | 1 | 1 | 1 | 1 | – |
| Personnel | 2 | 2 | 2 | 2 | – |
| Access depth (lab-granted) | 2 | 2 | 2 | 2 | – |
| Scope control | 2 | 2 | 2 | 2 | – |
| Publication rights | 2 | 2 | 2 | 2 | – |
| Method transparency | 2 | 2 | 2 | 2 | – |
| Role incompatibility | 1 | 1 | 1 | 1 | – |
| Score, Lab procurement | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – No floor triggered, 1/8 |
| Score, Regulator or auditor | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – No floor triggered, 1/8 |
| Score, Public trust | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – No floor triggered, 1/8 |
| Score, Equal weights | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – Conditional floor, 8/8 | – No floor triggered, 1/8 |
Traced money and ties
hop 0: 1 row; hop 1: 1 row (investment $80.0M); hop 2: 4 rows (recommendation $6.8M, grant $6.8M, investment $3.0M). Confirmed rows: 6 of 6 Dollar sums: confirmed + unaudited USD rows only; imported figures are not re-derived and never summed. Second hop traced for 4 of 4 sources. Ties: Sequoia Capital (investor, distance 1, confirmed). Checked and not found: 1.
Ledger
Money in
- T09 from Coefficient Giving (formerly Open Philanthropy): recommendation $6.80M, 2024-02-28. cumulative confirmedtier 1 indexhttps://coefficientgiving.org/
- T20 from Sequoia Capital: investment $80.00M; round total, not one investor's check, 2025-09. Series round with Redpoint and angels; $450M valuation confirmedtier 4 presshttps://techcrunch.com/2025/09/17/irregular-raises-80-million-to-secure-frontier-ai-models
- T25 from OpenAI: contract undisclosed, 2023-2025. paid evaluation engagements; millions in annual revenue across lab clients confirmedtier 4 presshttps://www.newswire.com/news/irregular-raises-80-million-to-set-the-security-standards-for-frontier-ai
- T88 from Good Ventures Foundation: grant $4.53M, 2024-03-05. developing software tools, products and analysis focused on global security (990-PF FY2024 Part XV, paid; expenditure-responsibility grant to a non-charity) confirmedtier 1 filinghttps://gt990datalake-rawdata.s3.amazonaws.com/EfileData/XmlFiles/202501349349105365_public.xml
- T89 from Good Ventures Foundation: grant $2.27M, FY2025 (2024-07-01 to 2025-06-30). developing software tools, products and analysis focused on global security (approved for future payment in FY2024; paid in FY2025 Part XV) confirmedtier 1 filinghttps://projects.propublica.org/nonprofits/full_text/202641359349102829/IRS990PF
- T90 from Good Ventures Foundation: investment $3.00M, 2024-06-30. Pattern Labs Tech Inc listed among Good Ventures' corporate stock holdings at $3,000,000 book and fair value (990-PF FY2024 Part II) confirmedtier 1 filinghttps://gt990datalake-rawdata.s3.amazonaws.com/EfileData/XmlFiles/202501349349105365_public.xml
Checked and not found
- N19 Did Coefficient Giving fund Irregular or Pattern Labs?
Searched coefficientgiving.org/grants and openphilanthropy.org/grants, name search, snapshot 2026-09-15: no grant page under Pattern Labs or Irregular among 2,737 archived openphilanthropy.org/grants/ slugs, and no grant record for either name in Coefficient's live site search; the grant itself is recorded in Good Ventures' 990-PF under Pattern Labs Tech Inc. Prompted by imported transfer T09 ($6.8M under Pattern Labs), marked differs. confirmedhttps://coefficientgiving.org/
Roles hosted
- R26 Sequoia Capital: investor, 2025-09. confirmedhttps://techcrunch.com/2025/09/17/irregular-raises-80-million-to-secure-frontier-ai-models
- R82 Redpoint Ventures: investor, 2025-09. research pass 2026-09-15; co-lead unauditedhttps://techcrunch.com/2025/09/17/irregular-raises-80-million-to-secure-frontier-ai-models/
- R83 Swish Ventures: investor, 2025-09. research pass 2026-09-15 unauditedhttps://techcrunch.com/2025/09/17/irregular-raises-80-million-to-secure-frontier-ai-models/
Funding graph, focused
Every one-way chain to a lab and to an evaluator at full strength, everything else faded. Hover any name to trace its connections; click a name to open its page.