Parallel Alternatives - cllu/.rc GitHub Wiki

File compression/decompression

bzip2 vs pbunzip2

$ bzip2 -d ../enwiki-20150304-pages-articles-multistream.xml.bz2
$ # takes about 30min
$ pv ../enwiki-20150304-pages-articles-multistream.xml.bz2 |pbunzip2 > enwiki.xml
$ # takes 3 min(12 core)

wget vs arias2

http://sourceforge.net/projects/aria2/