Benchmarks - aaronriekenberg/rust-parallel GitHub Wiki

Test System

  • 2017 MacBook Pro (my personal laptop)
  • 2.8 GHz Quad-Core Intel Core i7
    • 4 physical cores: hw.physicalcpu: 4
    • 2 threads per core, 8 threads total: hw.logicalcpu: 8
    • rust-parallel uses 8 parallel jobs by default on this machine
  • 16GB RAM
  • MacOS 13.1

TLDR

  • rust-parallel is more than an order of magnitude faster than GNU Parallel in tests below (~15 times faster)
  • rust-parallel is ~40% faster than command line xargs with 8 processes.

1000 invocations of md5 -s

rust-parallel 0.440 seconds real time elapsed (~41% faster than xargs -P8, ~14x faster than GNU Parallel)

$ rust-parallel -V
rust-parallel 0.3.0

$ time head -1000 /usr/share/dict/words | rust-parallel md5 -s

rust-parallel md5 -s  0.80s user 1.96s system 623% cpu 0.440 total

xargs with 8 processes 0.742 real time elapsed

$ time head -1000 /usr/share/dict/words | xargs -P8 -n1 md5 -s

xargs -P8 -n1 md5 -s  0.65s user 1.71s system 318% cpu 0.742 total

GNU Parallel 6.314 seconds real time elapsed

$ parallel -V
GNU parallel 20230222

$ time head -1000 /usr/share/dict/words | parallel md5 -s

parallel md5 -s  4.37s user 7.91s system 194% cpu 6.314 total

10,000 invocations of md5 -s

rust-parallel 4.253 seconds real time elapsed (~39% faster than xargs -P8, ~16x faster than GNU Parallel)

$ rust-parallel -V
rust-parallel 0.3.0

$ time head -10000 /usr/share/dict/words | rust-parallel md5 -s

rust-parallel md5 -s  7.72s user 19.03s system 630% cpu 4.253 total

xargs with 8 processes 6.993 seconds real time elapsed

$ time head -10000 /usr/share/dict/words | xargs -P8 -n1 md5 -s

xargs -P8 -n1 md5 -s  6.09s user 16.04s system 316% cpu 6.993 total

GNU Parallel 69.15 seconds real time elapsed

$ parallel -V
GNU parallel 20230222

$ time head -10000 /usr/share/dict/words | parallel md5 -s

parallel md5 -s  51.25s user 91.65s system 206% cpu 1:09.15 total