Getting Started - YadeWira/lz6 GitHub Wiki

Getting Started

Build

Linux / macOS (gcc or clang, C99):

make -C programs lz6

Windows, cross-compiled from Linux with mingw-w64 (static, Windows 7 SP1+), x64 and x86:

./build_windows.sh

The seq codec sources (lib/entropy/lz6fse.c, lz6seq.c, lz6huf.c) are built at -O2. They miscompile at -O3; programs/Makefile handles this.

CLI

lz6 -2 file              # compress with the seq engine (good ratio/speed trade)
lz6 -15 file             # maximum compression
lz6 --hc -9 file         # classic LZ6-HC frame codec
lz6 -d file.lz6          # decompress (codec-agnostic, per-block dispatch)
lz6 -2 -c file | lz6 -d -c     # pipes
lz6 -m file1 file2       # multiple inputs
lz6 -5 -B5 -T0 file      # all cores: same bytes as one thread, several times faster on big files
lz6 -d -T4 file.lz6      # decompress on 4 threads (any file, any -T)
lz6 -b -15 file          # in-memory benchmark
  • Levels 1-15 select the seq engine: LZ match finder + FSE/rANS entropy stage.
  • --hc with levels 1-15 selects the classic HC frame codec: byte-wise LZ, no entropy stage, very fast decode.
  • Without a level, the fast LZ frame codec is used.
  • -B1..-B7 set the block size (64KB, 256KB, 1MB, 4MB, 16MB, 64MB, 256MB). Match windows go up to 32MB.
  • -T# sets the worker threads (0 = every core, default 1). Compression is threaded for the seq levels 1-15 (not --hc, -0); decompression for any file. The output does not depend on -T. Each thread works on one block, so a file only speeds up if it has several: the default block size of seq levels on large inputs is B7 (few, large blocks); -B5 or -B4 make more of them at a cost of 0.4-0.6 points of ratio, and -B4 is also faster on a single thread. Memory grows with -B and -T (about two blocks per thread). Numbers in Benchmarks.

Levels

The seq engine (levels 1-15) and the HC frame codec (--hc) have separate level tables. The HC table is the frozen portable profile. The seq ladder is tuned to be strictly monotonic: every level compresses better and slower than the one before it.

level parser Silesia encode MB/s*
1 single-pass fast (2^15 hash, 2^20 window) 33.09% 170
2 single-pass fast (2^20 hash, 2^25 window) 31.58% 144
3 row hash, greedy, 8 candidates 28.79% 65
4 row hash, lazy, 8 candidates 28.18% 49
5 row hash, 32-slot rows, lazy 2, 24 candidates 27.54% 33
6 row hash, 32-slot rows, lazy 2, 32 candidates 27.41% 29
7 lowest_price, 8 candidates 27.30% 17
8 optimal, 8 searches 26.99% 8.4
9 optimal, 8 searches, full 26.70% 6.2
10 optimal, 16 searches 25.67% 4.4
11 optimal, 32 searches 25.45% 3.9
12 optimal, 64 searches 25.22% 2.9
13 optimal, 64 + 1 refinement 25.15% 1.5
14 optimal, 64 + 2 refinements 25.13% 0.99
15 optimal, 64 + 3 refinements 25.12% 0.74

* lzbench, one thread, full Silesia, v1.6.11-pre, one pinned core with the machine idle (2026-10-09); all 15 levels were measured in that sweep.

Levels 1-2 use a single-pass fast parser and levels 3-5 the row-hash match finder (see Architecture). The optimal levels (8-15) price every choice with the entropy costs the seq coder will actually charge. Those costs come from a cheap level-3 pre-parse; levels 13-15 then re-parse with refined prices.

Library API (seq codec)

#include "lib/entropy/lz6seq.h"

size_t LZ6_compress_seq(const char* src, size_t srcSize,
                        char* dst, size_t dstCap, int level);   /* 1..15 */
size_t LZ6_decompress_seq(const char* src, size_t srcSize,
                          char* dst, size_t dstCap);            /* 0 on error */

LZ6_decompress_seq never writes outside dst[0 .. original size), and it validates every length, offset and stream bound, so it is safe on corrupt input. Frames (lib/lz6frame.h) wrap seq, light-LZ and raw blocks with optional content checksums.