Researchers argue benchmarks encode difficulty distributions and identify measure mismatch and coverage gaps

The paper develops a measure‑based view of benchmark evaluation, showing that benchmarks induce an empirical difficulty distribution and exposing two failure modes: measure mismatch, where the benchmark’s difficulty distribution differs from the intended one, and coverage gaps, where uncovered regions prevent detection of meaningful performance differences.