Benchmark Registry
Benchmarks are not universally comparable. Each entry tracks methodology, health, reproducibility, and contamination concerns.
Popular but increasingly saturated code generation benchmark.
General knowledge · v2024
Saturating Multi-task language understanding across 57 subjects. Widely used but near-ceiling for frontier models.
Software engineering · v2025
Healthy Real-world software engineering tasks. Currently the most informative coding benchmark.
Multi-environment agent evaluation. Growing in relevance for agentic workflows.
Static Phase 1 placeholder. Full benchmark registry with run records and independent results in Phase 4.