performance - ligurio/sqa-wiki GitHub Wiki
References
- Community wiki about performance testing: https://github.com/qaload/qaload.github.io
Performance results analysis
- (!) https://github.com/AndreyAkinshin/perfolizer
- Change Point Detection in Software Performance Testing
- Chrome: Regression Detection for Performance Tests
Methodology
- http://www.brendangregg.com/activebenchmarking.html
- http://www.brendangregg.com/ActiveBenchmarking/bonnie++.html
- http://www.brendangregg.com/blog/2014-04-26/the-noploop-cpu-benchmark.html
Tuning environment
- https://perf.readthedocs.io/en/latest/system.html#pin-cpu
- https://rt.wiki.kernel.org/index.php/HOWTO:_Build_an_RT-application
- How to Benchmark Code Execution Times on Intel® IA-32 and IA-64 Instruction Set Architectures My journey to stable benchmark, part 1 (system)
Microbenchmarks
- low-overhead timers https://github.com/jdmccalpin/low-overhead-timers
- https://github.com/ligurio/microb/
- https://github.com/leo-yuriev/cicro-benchmarks/blob/master/README.md
- https://github.com/jku-ssw/cicro-benchmarks
- http://www.serpentine.com/criterion/fibber.html#grokularation
- https://github.com/bos/criterion
- https://github.com/Gabriel439/bench
- https://docs.python.org/3/library/timeit.html
- https://github.com/sharkdp/hyperfine
- https://github.com/japaric/criterion.rs
- https://blog.anp.lol/rust/2018/09/29/lolbench/
- https://github.com/brendanashworth/bench
- https://github.com/clibs/bench/
- mhz: Anatomy of a micro-benchmark
- https://github.com/softdevteam/softdevbib/blob/master/softdev.bib
Vis
- http://hdrhistogram.org/
- airspeed https://github.com/airspeed-velocity/asv/
- https://wiki.jenkins.io/display/JENKINS/Plot+Plugin
- https://github.com/undera/perf-target-site
- https://sense.blazemeter.com/
- https://overload.yandex.net/login/?next=/
- A Better Way of Reporting Performance Test Results картинка
- https://gist.github.com/hellerbarde/2843375, viz
- jmh?
- BenchmarkDotNet
Caliperhttps://microbenchmarks.appspot.com/, https://github.com/google/caliper/wiki/OnlineResults- criterion.rs пример
- flamescope, https://medium.com/netflix-techblog/netflix-flamescope-a57ca19d47bb
- flamegraphs http://www.brendangregg.com/flamegraphs.html
- benchdraw, benchcmp, benchstat, benchmany
Форматы
- Go
- Jenkins
- JMeter
- Google Benchmark
- criterion.rs, CSV
- https://sense.blazemeter.com/wiki/help:supported_formats/
CodeSpeed- BenchmarkAnything
- Chrome Performance Dashboard Data Format
- Skia
- catapult
Анализ, выявление регрессии
stathttps://github.com/thorduri/ministatstathttps://github.com/nferraz/ststathttps://github.com/simonccarter/stastathttps://github.com/dpmcmlxxvi/clistatsstathttps://sourceforge.net/projects/average/stathttps://github.com/tonyfischetti/qstatsstathttp://www.numcommand.com/stathttp://web.cs.wpi.edu/%7Eclaypool/misc/stats/stats.htmlstathttps://www.gnu.org/software/datamash/mlhttps://github.com/blue-yonder/tsfreshmlhttps://github.com/lytics/anomalyzermlhttps://github.com/yzhao062/Pyodmlhttps://github.com/yui0/catseye/mlhttps://github.com/attractivechaos/kannmlhttps://github.com/attractivechaos/sann- k-means
- lolbench: automagically and empirically discovering Rust performance regressions
- criterion.rs анализ
Статьи:
- How not to lie with statistics: the correct way to summarize benchmark results
- Detecting Benchmark Regression (слайды)
- https://www.chromium.org/chromium-os/testing/perf-regression-detection
- https://github.com/yzhao062/anomaly-detection-resources
- Automatic Detection of Performance Deviations in the Load Testing of Large Scale Systems -- Haroon Malik, Hadi Hemmati, Ahmed E. Hassan