ALL-PERF1003 · benchmark_quality
Judge whether a benchmark supports the decision made from it.
This is a contextual rule for all languages. Read its implementation.
Definition
Section titled “Definition”Compare the decision, workload distribution, data size, environment, warmup, repetitions, variance, baseline, resources, end-to-end effects, and recency.
Evidence
Section titled “Evidence”Findings cite benchmark code, commands, environments, samples, statistics, and the claimed decision.
Exceptions
Section titled “Exceptions”Focused microbenchmarks are valid for isolated mechanism questions with bounded claims.
Examples
Section titled “Examples”A representative service workload with controlled comparisons is representative. Timing one
parser token and claiming end-to-end speedup is micro_only.
References
Section titled “References”- Cites “Systems Performance”
- Cites “The Python Standard Library”, timeit
- Cites “Beyond the Basic Stuff with Python”, Measuring Performance and Big O