Evaluator Bench
METR (GitHub), 2025; retrieved 2026-09-15 tier 3 self (metr)confirmed

METR/hcast-public (GitHub README)

https://github.com/METR/hcast-public

"This repo contains source code for a subset of the tasks used in the HCAST: Human-Calibrated Autonomy Software Tasks paper"; tasks conform to the METR Task Standard. Supports anchor 3 (tasks partly open); the rest of the suite is withheld, and no third-party re-run is documented, so M.1 does not reach 4.

Signals citing this source

  • METR, M (for): Open task suites and published time-horizon methodology; results appear in system cards.“This repo contains source code for a subset of the tasks used in the HCAST”

Ledger rows citing this URL

  • none