nightly_run - CyrilB1531/lodestar GitHub Wiki

Nightly benchmark run

Generated. Do not edit. Produced by .github/workflows/bench-nightly.yml; every edit is overwritten by the next run. The curated figures, measured on a named machine, are in performance. The last known reading for a method quiet tonight is in benchmark_latest.

Read the ratios, not the means. These run on a GitHub hosted runner: a shared VM whose hardware differs from night to night and whose neighbours are unknown. An absolute figure here is not comparable to the performance page, and not reliably comparable to yesterday's. A ratio against a baseline measured in the same run, on the same VM, in the same minute, is.

This run

  • Commit: b7818719585e35e8c1448884e1c36a152a3cd170
  • Previous run: b7818719585e35e8c1448884e1c36a152a3cd170
  • Runner: Linux / X64 (GitHub hosted)

Classes re-run

Selected by tools/select_benchmarks.py from the sources that changed since the previous run, and what earlier nights owed:

  • CosineTopKSelectionBenchmarks

Lodestar.Gpu.Benchmarks.CosineTopKSelectionBenchmarks-report-github

BenchmarkDotNet v0.14.0, Ubuntu 24.04.5 LTS (Noble Numbat)
AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.402
  [Host]   : .NET 10.0.13 (10.0.1326.50504), X64 RyuJIT AVX2
  ShortRun : .NET 10.0.13 (10.0.1326.50504), X64 RyuJIT AVX2

Job=ShortRun  IterationCount=3  LaunchCount=1  
WarmupCount=3
Method Documents K Device Mean Error StdDev Ratio RatioSD Allocated Alloc Ratio
SimdBaseline 10000 10 CPUAc(...) GPU) [26] 8.479 ms 0.5641 ms 0.0309 ms 1.00 0.00 26.02 KB 1.00
GpuResident 10000 10 CPUAc(...) GPU) [26] 26,692.585 ms 4,250.1151 ms 232.9631 ms 3,147.95 25.79 1520.07 KB 58.43
SimdBaseline 10000 100 CPUAc(...) GPU) [26] 9.084 ms 0.5902 ms 0.0324 ms 1.00 0.00 37.27 KB 1.00
GpuResident 10000 100 CPUAc(...) GPU) [26] 27,129.368 ms 7,874.6844 ms 431.6380 ms 2,986.39 42.16 1541.63 KB 41.37
SimdBaseline 10000 1000 CPUAc(...) GPU) [26] 13.641 ms 0.1274 ms 0.0070 ms 1.00 0.00 149.77 KB 1.00
GpuResident 10000 1000 CPUAc(...) GPU) [26] 31,855.072 ms 9,890.5393 ms 542.1338 ms 2,335.19 34.43 1765.7 KB 11.79
SimdBaseline 100000 10 CPUAc(...) GPU) [26] 121.644 ms 15.2202 ms 0.8343 ms 1.00 0.01 26.2 KB 1.00
GpuResident 100000 10 CPUAc(...) GPU) [26] 261,675.572 ms 20,343.0269 ms 1,115.0700 ms 2,151.23 15.03 39279.2 KB 1,499.19
SimdBaseline 100000 100 CPUAc(...) GPU) [26] 127.896 ms 32.2741 ms 1.7691 ms 1.00 0.02 37.5 KB 1.00
GpuResident 100000 100 CPUAc(...) GPU) [26] 258,529.330 ms 62,412.6367 ms 3,421.0472 ms 2,021.66 33.38 39301.7 KB 1,048.05
SimdBaseline 100000 1000 CPUAc(...) GPU) [26] 132.921 ms 20.3016 ms 1.1128 ms 1.00 0.01 150.08 KB 1.00
GpuResident 100000 1000 CPUAc(...) GPU) [26] 266,315.941 ms 39,079.5304 ms 2,142.0809 ms 2,003.66 20.17 39525.48 KB 263.36

Asked for last

Each was cut at its phase's stop, tonight or on an earlier night and not run since, or left unsettled by a step that died. They are asked for after everything else, so one that does not fit the window takes only what the rest leave.

  • TiledCosineTopKBenchmarks
  • TiledSparseDenseProductBenchmarks

Ratios that moved

Read against bench/nightly/ratios.csv by tools/nightly_series.py, under the thresholds that script sets. A ratio moves when either side of it does: read its baseline before calling a movement a regression.

No ratio stepped past its noise or drifted over ten days tonight.