SWE-bench
Princeton NLP · v2025
Details
- Purpose
- Evaluate models on real-world software engineering tasks from GitHub issues
- Domain
- Software engineering
- Health
- Healthy
- Reproducibility
- Reproducible
- Contamination concern
- Low
- Saturation
- Not yet saturated
- Code available
- Yes
- Last reviewed
- 2026-07-01
Benchmark run records and model comparisons will be added in Phase 4.