Update on LZMA Compression Memory Usage - pete4abw/lrzip-next GitHub Wiki
LZMA2 Style Dictionary Sizes Now in Effect
lrzip-next
Version 0.7.50 will mark a departure from the way Dictionary Sizes can be specified. lrzip-next
will still automatically determine lzma
dictionary sizes. But if the --dictsize=
option is used, instead of 12-30, the options will range from 0-40. The table below will show the new mappings.
DictionaryCode | 2s Exponent | 2*2^nDict Size | lrzip-next LevelDefault Dict Size | Per ThreadMemory Overhead | DictionaryCode | 3*2^nDict Size | Per ThreadMemory Overhead |
---|---|---|---|---|---|---|---|
0 | 11 | 4,096 | 6,354,944 | 1 | 6,144 | 6,378,496 | |
2 | 12 | 8,192 | 6,402,048 | 3 | 12,288 | 6,449,152 | |
4 | 13 | 16,384 | 6,496,256 | 5 | 24,576 | 6,590,464 | |
6 | 14 | 32,768 | 6,684,672 | 7 | 49,152 | 6,873,088 | |
8 | 15 | 65,536 | 1 | 7,061,504 | 9 | 98,304 | 7,438,336 |
10 | 16 | 131,072 | 7,815,168 | 11 | 196,608 | 8,568,832 | |
12 | 17 | 262,144 | 2 | 9,322,496 | 13 | 393,216 | 10,829,824 |
14 | 18 | 524,288 | 12,337,152 | 15 | 786,432 | 15,351,808 | |
16 | 19 | 1,048,576 | 3 | 18,366,464 | 17 | 1,572,864 | 24,395,776 |
18 | 20 | 2,097,152 | 30,425,088 | 19 | 3,145,728 | 42,483,712 | |
20 | 21 | 4,194,304 | 4 | 54,542,336 | 21 | 6,291,456 | 78,659,584 |
22 | 22 | 8,388,608 | 102,776,832 | 23 | 12,582,912 | 151,011,328 | |
24 | 23 | 16,777,216 | 5 | 199,245,824 | 25 | 25,165,824 | 295,714,816 |
26 | 24 | 33,554,432 | 6,7 | 392,183,808 | 27 | 50,331,648 | 585,121,792 |
28 | 25 | 67,108,864 | 8 | 778,059,776 | 29 | 100,663,296 | 1,163,935,744 |
30 | 26 | 134,217,728 | 9 | 1,549,811,712 | 31 | 201,326,592 | 2,321,563,648 |
32 | 27 | 268,435,456 | 3,093,315,584 | 33 | 402,653,184 | 4,636,819,456 | |
34 | 28 | 536,870,912 | 6,180,323,328 | 35 | 805,306,368 | 9,267,331,072 | |
36 | 29 | 1,073,741,824 | 12,354,338,816 | 37 | 1,610,612,736 | 18,528,354,304 | |
38 | 30 | 2,147,483,648 | 24,702,369,792 | 39 | 3,221,225,472 | 37,050,400,768 | |
40 | 31 | 4,294,967,295 | 49,398,431,733 |
Memory Overhead per thread is computed as follows:
Dictionary Size * 11.5 + 6MB + 16KB
Dictionary Size will be automatically reduced if sufficient memory is not available