Benchmarks - YadeWira/lz6 GitHub Wiki

Benchmarks

Harness. lzbench 2.x, one thread (-t1,1), memory-to-memory, per file, on 2x Xeon E5-2697A v4 @ 2.60GHz. lz6 and lz5 are built in as lzbench plugins (bakeoff/lzbench_lz6.patch), so every codec is timed by the same loop with the same round-trip verification. Rerun with bakeoff/run_vs_codecs.sh.

  • Silesia (212 MB, 12 files): external, never used for tuning.
  • AIT A-H (13 MB, 8 files): the training set.

Full sweep of 2026-10-01, v1.6.9-pre, pinned to one core (a first unpinned run on the shared machine was 10-25% slow for every codec and was discarded); the fastlzma2/uf-lzma2/memlz rows are from the 2026-09-22/23 runs on the same machine (uf-lzma2 1.3/1.4 give the same bytes on this path, per its author). The machine is shared, so treat decode speeds as ±5%.

Update 2026-10-09: v1.6.11-pre (new decoder, tight extra-bit widths)

Re-measured in one session, one pinned core (taskset -c 44), load average below 3 all the time (sampled every 30 s): lz6 v1.6.11-pre at every level, lz6 v1.6.10-pre (commit 14228ba) at levels 1/3/8/15 and every competitor row below in the same window (zstd 1.5.7, lizard 2.1, lz5 v1.5, misa77 0.6.0, lz4 1.10.0, memlz 0.2 beta; the lzbench plugin still labels lz6 "1.6.9-pre"). An idle core boosts higher than the loaded ones of the sweep below, so speeds in this update and in the 2026-10-01 tables are not comparable; brotli, xz, zlib, fastlzma2, uf-lzma2 and lz6 HC were not re-run and stay in the old tables. The bytes: the decoder is the same for old streams (the format change shrinks the output by 0.2-0.3%); the decoder speed-up on the same compressed streams (v1.6.10-pre -> v1.6.11-pre, size-weighted) is x1.65, x1.69, x1.66, x1.65 on Silesia and x1.30, x1.36, x1.37, x1.38 on AIT at levels 1/3/8/15. What changed is in Decoder Internals and NEWS.

  • Against zstd (Silesia, one thread, same session). lz6 now decodes faster than zstd at the same ratio from level 2 up and still encodes slower: lz6 -2 (31.58%) vs zstd -3 (31.21%): 1.11x the decode speed, 0.84x the encode speed; lz6 -3 (28.79%) vs zstd -6 (28.89%): 1.23x, 0.86x; lz6 -8 (26.99%) vs zstd -14 (27.03%): 1.19x, 0.71x; lz6 -12 (25.22%) vs zstd -17 (25.58%): 1.37x, 0.57x; lz6 -15 (25.12%) vs zstd -19 (24.96%): 1.40x decode, 3.8x slower to encode. zstd -1 (34.55%) still decodes 1.16x faster than lz6 -1 (33.09%).
  • Against lizard. lizard -49 (28.62%) decodes 1.11x faster than lz6 -3 (28.79%) and encodes at 1.9 MB/s; lz6 -5 (27.54%) is 1.1 points smaller, decodes at 0.95x its speed and encodes at 33 MB/s.
  • Against misa77 / lz4: unchanged in kind: memcpy-class decode (5.2 GB/s / 3.7 GB/s) at 42.65% / 47.60%.

Silesia (212 MB, 12 files)

codec size ratio enc MB/s dec MB/s
zstd 1.5.7 -19 52,891,946 24.96% 2.8 807
lz6 v1.6.11-pre -15 53,235,107 25.12% 0.7 1,129
lz6 v1.6.11-pre -14 53,251,388 25.13% 1.0 1,131
lz6 v1.6.11-pre -13 53,294,747 25.15% 1.5 1,134
lz6 v1.6.10-pre -15 53,416,718 25.20% 0.7 684
lz6 v1.6.11-pre -12 53,449,473 25.22% 2.9 1,144
lz6 v1.6.11-pre -11 53,940,387 25.45% 3.9 1,095
zstd 1.5.7 -17 54,207,531 25.58% 5.1 837
lz6 v1.6.11-pre -10 54,404,878 25.67% 4.4 1,120
lz6 v1.6.11-pre -9 56,587,531 26.70% 6.2 1,069
lz6 v1.6.11-pre -8 57,208,934 26.99% 8.4 1,051
zstd 1.5.7 -14 57,287,654 27.03% 11.8 881
lz6 v1.6.10-pre -8 57,390,652 27.08% 8.3 633
lz6 v1.6.11-pre -7 57,854,555 27.30% 17.1 1,119
zstd 1.5.7 -12 57,919,879 27.33% 27.8 862
lz6 v1.6.11-pre -6 58,101,483 27.41% 28.9 1,148
lz6 v1.6.11-pre -5 58,365,582 27.54% 33.0 1,144
zstd 1.5.7 -9 59,081,628 27.88% 54.1 857
lz6 v1.6.11-pre -4 59,719,496 28.18% 49.3 1,098
lizard 2.1 -49 60,667,327 28.62% 1.9 1,209
lz6 v1.6.11-pre -3 61,016,819 28.79% 65.1 1,092
lz6 v1.6.10-pre -3 61,178,469 28.87% 57.6 646
zstd 1.5.7 -6 61,239,847 28.89% 75.8 890
zstd 1.5.7 -5 62,751,501 29.61% 102 828
lz5 v1.5 HC -15 65,595,195 30.95% 2.2 793
zstd 1.5.7 -3 66,137,723 31.21% 171 874
lz6 v1.6.11-pre -2 66,927,072 31.58% 144 967
lz6 v1.6.11-pre -1 70,119,901 33.09% 170 1,043
lz6 v1.6.10-pre -1 70,270,844 33.16% 171 633
zstd 1.5.7 -1 73,229,468 34.55% 360 1,207
lizard 2.1 -30 85,765,250 40.47% 328 1,235
misa77 0.6.0 -1 90,385,225 42.65% 52.0 5,205
lz4 1.10.0 100,880,147 47.60% 555 3,663
memlz 0.2 beta 126,970,186 59.91% 1,129 1,096

AIT A-H (13 MB, 8 files)

codec size ratio enc MB/s dec MB/s
lz6 v1.6.11-pre -15 5,082,521 38.69% 1.9 834
lz6 v1.6.11-pre -14 5,083,762 38.70% 2.5 835
lz6 v1.6.10-pre -15 5,084,398 38.70% 1.9 603
lz6 v1.6.11-pre -13 5,087,488 38.73% 3.6 837
lz6 v1.6.11-pre -12 5,097,479 38.80% 6.5 843
lz6 v1.6.11-pre -11 5,100,551 38.83% 8.6 843
lz6 v1.6.11-pre -10 5,102,307 38.84% 8.7 842
lz6 v1.6.11-pre -9 5,117,684 38.96% 9.5 836
lz6 v1.6.11-pre -8 5,138,904 39.12% 11.3 810
lz6 v1.6.10-pre -8 5,140,591 39.13% 11.4 593
lz6 v1.6.11-pre -6 5,173,103 39.38% 52.7 847
lz6 v1.6.11-pre -5 5,174,147 39.39% 55.0 846
lz6 v1.6.11-pre -7 5,176,112 39.40% 21.7 824
lz6 v1.6.11-pre -4 5,187,030 39.49% 62.7 842
lz6 v1.6.11-pre -3 5,200,902 39.59% 101 898
lz6 v1.6.10-pre -3 5,204,721 39.62% 104 661
lz6 v1.6.11-pre -2 5,292,359 40.29% 183 896
lz6 v1.6.11-pre -1 5,334,221 40.61% 216 898
lz6 v1.6.10-pre -1 5,335,737 40.62% 216 690
zstd 1.5.7 -19 7,351,946 55.97% 5.7 1,097
zstd 1.5.7 -17 7,388,815 56.25% 8.3 1,104
zstd 1.5.7 -14 7,531,904 57.34% 15.2 1,037
zstd 1.5.7 -12 7,552,509 57.49% 38.3 1,033
zstd 1.5.7 -9 7,578,176 57.69% 69.8 1,036
zstd 1.5.7 -6 7,611,083 57.94% 98.0 1,037
zstd 1.5.7 -5 7,619,521 58.00% 129 1,000
zstd 1.5.7 -3 7,695,386 58.58% 196 1,084
zstd 1.5.7 -1 7,869,981 59.91% 445 1,383
lizard 2.1 -49 8,041,060 61.21% 5.8 1,560
lz5 v1.5 HC -15 8,603,507 65.49% 2.1 1,013
lizard 2.1 -30 8,756,123 66.66% 457 1,943
misa77 0.6.0 -1 9,628,088 73.29% 52.7 7,955
lz4 1.10.0 9,938,605 75.66% 791 5,156
memlz 0.2 beta 10,724,346 81.64% 1,910 1,713

The sections below are the 2026-10-01 sweep (v1.6.9-pre) and keep its numbers and its conclusions (e.g. "the zstd ladder is ahead"), which the update above supersedes for decode speed.

Silesia (2026-10-01 sweep, v1.6.9-pre)

codec size ratio enc MB/s dec MB/s
uf-lzma2 1.2.0 -11 (asm decoder) 48,483,163 22.88% 1.2 118
fastlzma2 1.0.1 -10 (C decoder) 48,673,262 22.97% 3.3 81.0
uf-lzma2 1.1.0 -10 (asm decoder) 48,673,262 22.97% 3.3 110
zstd 1.5.7 -19 52,891,946 24.96% 2.6 756
lz6 -15 53,416,718 25.20% 0.7 668
lz6 -13 53,479,947 25.23% 1.4 676
lz6 -12 53,636,281 25.31% 2.6 666
zstd 1.5.7 -17 54,207,531 25.58% 4.9 816
lz6 -10 54,594,576 25.76% 4.0 651
zstd 1.5.7 -14 57,287,654 27.03% 11.2 828
lz6 -8 57,390,652 27.08% 7.6 592
zstd 1.5.7 -12 57,919,879 27.33% 26.6 813
lz6 -7 58,046,491 27.39% 15.1 627
lz6 -6 58,272,166 27.49% 26.7 660
lz6 -5 58,534,476 27.62% 28.8 653
zstd 1.5.7 -9 59,081,628 27.88% 53.3 824
lz6 -4 59,879,274 28.25% 41.8 629
zstd 1.5.7 -7 60,258,866 28.43% 67.6 826
lizard 2.1 -49 60,667,327 28.62% 1.8 1,159
zstd 1.5.7 -6 61,239,847 28.90% 75.4 853
lz6 -3 61,401,238 28.97% 57.1 594
zstd 1.5.7 -5 62,751,501 29.61% 101 787
zstd 1.5.7 -4 64,958,218 30.65% 168 766
lz5 v1.5 HC -15 65,595,195 30.95% 2.0 769
zstd 1.5.7 -3 66,137,723 31.21% 170 850
lz6 -2 67,069,091 31.65% 135 536
zstd 1.5.7 -2 69,332,724 32.71% 252 949
lz6 -1 70,270,844 33.16% 164 620
zstd 1.5.7 -1 73,229,468 34.55% 348 1,166
lizard 2.1 -30 85,765,250 40.47% 318 1,197
misa77 0.6.0 -1 90,385,225 42.65% 51.2 4,933
lz4 1.10.0 100,880,147 47.60% 540 3,537
memlz 0.2 beta 126,970,186 59.91% 963 888

The zstd -2, -3, -4, -6, -7, -9, -12, -14 and -17 rows and lz6 -6 come from a complementary run on 2026-10-02 (same machine, harness and pinning; the first sweep only had zstd -1, -5 and -19, which left the ladder between them out of the comparison). In that run lz6 -1/-2/-3 and zstd -1/-5 repeated within 4% of the rows above. The instruction counts behind the decode gap are in bakeoff/SINGLE_THREAD_STUDY.md.

AIT A-H

codec size ratio enc MB/s dec MB/s
lz6 -15 5,084,398 38.70% 1.7 572
lz6 -13 5,089,348 38.74% 3.3 576
lz6 -8 5,140,591 39.13% 10.6 562
lz6 -3 5,219,589 39.73% 94.0 583
lz6 -2 5,294,924 40.31% 182 631
lz6 -1 5,335,737 40.62% 213 668
uf-lzma2 1.2.0 -11 (asm decoder) 6,914,570 52.64% 2.2 64.8
fastlzma2 1.0.1 -10 (C decoder) 7,067,576 53.80% 6.2 47.0
uf-lzma2 1.1.0 -10 (asm decoder) 7,067,576 53.80% 6.2 63.9
zstd 1.5.7 -19 7,351,946 55.97% 4.7 1,049
zstd 1.5.7 -1 7,869,981 59.91% 431 1,352
lizard 2.1 -49 8,041,060 61.21% 5.3 1,518
lz5 v1.5 HC -15 8,603,507 65.49% 2.1 969
lizard 2.1 -30 8,756,123 66.66% 441 1,864
misa77 0.6.0 -1 9,628,088 73.29% 51.1 7,957
lz4 1.10.0 9,938,605 75.66% 774 5,148
memlz 0.2 beta 10,724,346 81.64% 1,872 1,713

Read AIT with care: one file decides it. D is glibc random() output; lz6 regenerates it from its seed (2,000,000 → 7 bytes) and every other codec stores it raw. Without D:

codec without D A B C E F G H
lz6 -15 45.66% 55.27% 18.45% 27.04% 79.45% 79.44% 29.22% 38.72%
brotli -11 45.27% 50.85% 17.41% 24.92% 84.75% 79.28% 30.97% 36.26%
uf-lzma2 -10 45.50% 51.87% 17.78% 24.78% 85.52% 82.53% 28.53% 35.97%
xz -9 45.92% 53.34% 17.76% 24.72% 85.84% 82.38% 29.71% 35.87%
zstd -19 48.06% 51.77% 18.06% 26.66% 88.52% 87.83% 31.61% 38.49%
uf-lzma2 -11 44.13% 51.87% 17.78% 24.59% 78.01% 79.12% 28.51% 35.86%

lz6 has the best E among the codecs without a per-file parameter search (uf-lzma2 -11's lc/lp/pb search takes it: 78.01%); on F brotli -11 and uf-lzma2 -11 are slightly smaller (within 0.3 points); on A, B and C it still trails zstd -19, by 3.5/0.4/0.4 points (H, an x86 file, is now 0.2 points ahead) (6.0/1.2/2.2/2.5 before the entropy-priced parser). Without D lz6 -15 (45.66%) is ahead of xz -9 and zstd -19. Until 2026-09-22 these pages compared lz6's 7-file ratio (then 46.66%) with the other codecs' 8-file totals: a parser bug dropped D from lz6's denominator only (reported by uf-lzma2, fixed in bakeoff/parse_lzbench.py).

Reading

  • Against LZMA2 (uf-lzma2, fastlzma2). uf-lzma2 -11 has the best Silesia ratio in the table (22.88%), 2.6 points under lz6 -15, at ~110-118 MB/s decode; lz6 -15 decodes ~5.5x faster. uf-lzma2 and fastlzma2 produce byte-identical streams at levels 1-10; the uf-lzma2 rows use Igor Pavlov's assembler LZMA decoder (+36-48% decode over the C decoder in lzbench's fastlzma2 build).
  • Against lz4 / lz5, the codecs lz6 replaces: lz6 -15 is 5.5 points smaller than lz5 HC -15 on Silesia and 22 points smaller than lz4.
  • Against zstd, on one thread, the zstd ladder is ahead over most of the range. With every zstd level on the table: zstd -3 (31.21%) is smaller than lz6 -2 and -1 (31.65%, 33.16%), encodes as fast as lz6 -1 and decodes 1.3-1.5x faster; zstd -6 matches lz6 -3 (28.90% vs 28.97%) at 1.3x its encode speed and 1.4x its decode speed; zstd -12 is smaller than lz6 -5 and -6; zstd -14 matches lz6 -8; and lz6 -15 is 0.24 points behind zstd -19 (1.56 before v1.6.5-pre, 0.76 before v1.6.6-pre, 0.50 before v1.6.9-pre), 3.7x slower to encode and 1.13x slower to decode. (Earlier versions of this page compared only against zstd -1, -5 and -19, which made lz6 -1 and -3 look better than they are.) lz6 wins per file only where its transforms apply: ooffice (x86 filter: 38.18% vs 42.18% at the top, 41.27% vs 46.00% for lz6 -5 against zstd -12), x-ray (byte planes: 55.85% against 60.53% for zstd -19 and 62.68% for zstd -12); sao only at mid levels (71.18% for lz6 -5 against 71.69% for zstd -12; at the top lz6 -15 loses it, 69.90% against 68.95%). On the other files lz6 -5 is 0.3-1.7 points behind zstd -12 at a similar encode speed. Blocks are independent, so decoding also scales with -T (see Multithreaded CLI below).
  • Against brotli. lz6 -15 decodes 1.5x faster than brotli -11, which is 1.7 points smaller.
  • Against lizard. lz6 -15 is 3.4 points smaller on Silesia; lizard -49 decodes 1.7x faster.
  • Against misa77. A different trade: memcpy-class decode (5-8 GB/s) with no entropy coding, and ~17 points worse ratio than lz6 -15 on Silesia.
  • memlz is a compression-speed codec (~0.9 GB/s both ways on Silesia) with a worse ratio than lz4; it marks the fast end of the field.

Multithreaded CLI (v1.6.10-pre)

A different measurement from the tables above: the command line, not the library in memory. lz6 -N -B5 -T# -c silesia.tar on Silesia.tar (212 MB as one file), wall time, best of 3, threads pinned to distinct physical cores of one socket (2x Xeon E5-2697A v4, shared machine, treat as +-10%). The compressed bytes are identical at every thread count.

level ratio 1 thread 2 4 8 16 speed-up (16) decode, 1 -> 16 threads
1 33.91% 1.76 s 1.28 0.78 0.77 0.76 2.3x 445 -> 1,129 MB/s
3 29.56% 4.42 s 2.51 1.71 1.43 1.23 3.6x 445 -> 1,507 MB/s
5 28.39% 6.70 s 3.89 2.67 1.89 1.74 3.9x 468 -> 1,578 MB/s
9 27.49% 31.1 s 17.5 10.9 7.69 6.16 5.0x 458 -> 1,469 MB/s

v1.6.11-pre (2026-10-09, same method, CPU 0 vs CPUs 0-15 of a shared machine, one session with v1.6.10-pre for comparison): decode 1 -> 16 threads -1 638 -> 1,230, -3 680 -> 1,260, -5 675 -> 1,222, -9 621 -> 1,180 MB/s (v1.6.10-pre in that session: 454 -> 1,050, 481 -> 1,061, 473 -> 1,094, 449 -> 1,126): +38-49% on one thread, +5-20% at 16, where the 0.16-0.19 s wall time is mostly fixed costs (reading the file, page faults on the output, the checksum) and other tenants. Compression: -9 31.8 -> 5.8 s (5.5x), -5 6.5 -> 1.75 s (3.7x), -3 4.15 -> 1.32 s (3.1x), -1 1.82 -> 0.74 s (2.5x); ratios 27.41 / 28.31 / 29.56 / 33.84%.

Level 1 is limited by the main thread (reading, segmentation, XXH32, copy: ~270 MB/s); the other levels by the number of blocks. The block size decides how many there are, and it is a ratio / speed trade-off. 8 threads, same file:

block size -3 ratio -3, 1 -> 8 threads -9 ratio -9, 1 -> 8 threads decode at 8 threads
default (B7: segments up to 47 MB here) 29.15% 4.68 -> 1.82 s (2.6x) 27.11% 36.0 -> 15.0 s (2.4x) 770-840 MB/s
-B5 (16 MB) 29.56% 4.42 -> 1.43 s (3.1x) 27.49% 31.1 -> 7.69 s (4.0x) 1,120-1,190 MB/s
-B4 (4 MB) 29.71% 3.81 -> 1.22 s (3.1x) 27.65% 33.8 -> 6.01 s (5.6x) 1,260-1,320 MB/s

Smaller blocks cost 0.4-0.6 points here, and -B4 is also faster on one thread (-3: 3.81 s vs 4.68 s to compress, 458 vs 394 MB/s to decode) because a block stays in cache. The default stays B7 for ratio (see the Architecture note on match windows). Rows within one table come from one run; across tables the one-thread times differ by up to 10% (other jobs on the machine). --hc and -0 compress on one thread; decoding is multithreaded for every codec.

Progress

Same harness, same machine. lz6 seq on Silesia:

v1.6.3-pre v1.6.4-pre (decoder rewrite + seq v2) v1.6.5-pre (priced parser, ladder, BMI2) v1.6.6-pre (offset low bits, fast encode) v1.6.7-pre (row hash, rich decode entries) v1.6.8-pre (HUF 4X, fast1) v1.6.9-pre (x86 filter, 32-slot rows) v1.6.11-pre (decoder rewrite 2, tight extra bits)*
level 15, ratio 26.39% 26.52% 25.72% 25.43% 25.45% 25.46% 25.20% 25.12%
level 15, decode 274 MB/s 493 MB/s 540 MB/s 557 MB/s 601 MB/s 677 MB/s 668 MB/s 1,129 MB/s
level 2, ratio 32.00% 32.00% 31.85% 31.86% 31.83% 31.65% 31.58%
level 2, encode 39.3 MB/s 95.8 MB/s 96.6 MB/s 144 MB/s 135 MB/s 144 MB/s
level 2, decode 255 MB/s 446 MB/s 469 MB/s 464 MB/s 496 MB/s 543 MB/s 536 MB/s 967 MB/s
level 3, ratio 32.00% 31.73% 29.14% 29.16% 28.97% 28.79%

v1.6.10-pre did not change the engine (it adds -T# to the command line), so its one-thread numbers are those of v1.6.9-pre.

* v1.6.11-pre column: measured on an idle pinned core (see the update at the top of this page), the other columns on a busier machine, so the speed figures of that column are not directly comparable with them; in the same idle session v1.6.10-pre decoded level 15 at 684 MB/s (x1.65 slower than v1.6.11-pre).

AIT is left out of this table: its v1.6.3-pre numbers were measured without file D (the parser bug above), so they are not comparable.

What changed is described in Decoder Internals. The remaining gap is structural, see Roadmap.