Benchmark Results — June 2026
Source-quality-weighted sweep results across 216 published benchmarks (236 matched to a catalog bike). Metrics, per-cluster bias, and methodology. Last updated 13 June 2026.
Aggregate Metrics (236 matched benchmarks)
Reference Baselines (hand-calibrated)
Three reference bikes from defaults.py are hand-calibrated and excluded from the ALL_KNOWN_BIKES auto-reseed. They serve as regression anchors — if any code change moves these by more than ±0.02s, something is wrong.
Top 10 Most Accurate Bikes
The 10 smallest residuals from published (non-derived) benchmarks — mean absolute error ±0.006s. Sources: Cycle World, Sport Rider, MCN, Motorcyclist, BikeWale, MotoStatz. Conditions: race fuel, professional rider, ideal strip.
Per-Cluster Bias
Positive bias means the simulator predicts slower than reality. A positive bias in the small-cc clusters is a known residual from the per-bike calibration work not yet completed — it is not a regression.
Methodology
Benchmark sources: Published magazine timeslips from Cycle World, Sport Rider, MCN, Motorcyclist, and BikeWale. Indian community strip records from MotoStatz, Facebook drag groups, and drag event published results. All benchmarks use professional rider / race fuel / ideal strip conditions unless explicitly noted.
Simulation conditions: All simulations run at ISA sea-level conditions (15°C, 101.325 kPa, 0% RH) to match the implicit conditions of most magazine tests. Bikes use their stock BikeConfig from the seeded database — no manual per-bike tuning of Cd or μ.
Matching: Benchmark-to-bike matching uses word-boundary regex to prevent false positives (YZF-R1 ≠ YZF-R15). Year ranges are respected — a 2009 benchmark does not match a 2015 model that shares a name but has different internals.
Known limitations: Residual small-cc and 2T bias is a calibration gap, not a physics gap. The correct fix is per-bike Cd and μ calibration using real strip data — the auto-calibrator in /api/calibrate/dragy handles this once you supply a Dragy CSV for your specific bike.
Regression testing: 10 dedicated regression tests intests/test_power_boost_regression.py run on every commit to catch DB drift. The test helper copies motoquant.db to a temp directory to avoid WAL journal-mode errors on mounted filesystems.