lib/zstd
1
0
Fork 0
mirror of https://github.com/facebook/zstd.git synced 2025-07-29 11:21:22 +03:00
Code Activity

[lazy] Skip over incompressible data

Browse Source 
Every 256 bytes the lazy match finders process without finding a match,
they will increase their step size by 1. So for bytes [0, 256) they search
every position, for bytes [256, 512) they search every other position,
and so on. However, they currently still insert every position into
their hash tables. This is different from fast & dfast, which only
insert the positions they search.

This PR changes that, so now after we've searched 2KB without finding
any matches, at which point we'll only be searching one in 9 positions,
we'll stop inserting every position, and only insert the positions we
search. The exact cutoff of 2KB isn't terribly important, I've just
selected a cutoff that is reasonably large, to minimize the impact on
"normal" data.

This PR only adds skipping to greedy, lazy, and lazy2, but does not
touch btlazy2.

| Dataset | Level | Compiler     | CSize ∆ | Speed ∆ |
|---------|-------|--------------|---------|---------|
| Random  |     5 | clang-14.0.6 |    0.0% |   +704% |
| Random  |     5 | gcc-12.2.0   |    0.0% |   +670% |
| Random  |     7 | clang-14.0.6 |    0.0% |   +679% |
| Random  |     7 | gcc-12.2.0   |    0.0% |   +657% |
| Random  |    12 | clang-14.0.6 |    0.0% |  +1355% |
| Random  |    12 | gcc-12.2.0   |    0.0% |  +1331% |
| Silesia |     5 | clang-14.0.6 | +0.002% |  +0.35% |
| Silesia |     5 | gcc-12.2.0   | +0.002% |  +2.45% |
| Silesia |     7 | clang-14.0.6 | +0.001% |  -1.40% |
| Silesia |     7 | gcc-12.2.0   | +0.007% |  +0.13% |
| Silesia |    12 | clang-14.0.6 | +0.011% | +22.70% |
| Silesia |    12 | gcc-12.2.0   | +0.011% |  -6.68% |
| Enwik8  |     5 | clang-14.0.6 |    0.0% |  -1.02% |
| Enwik8  |     5 | gcc-12.2.0   |    0.0% |  +0.34% |
| Enwik8  |     7 | clang-14.0.6 |    0.0% |  -1.22% |
| Enwik8  |     7 | gcc-12.2.0   |    0.0% |  -0.72% |
| Enwik8  |    12 | clang-14.0.6 |    0.0% | +26.19% |
| Enwik8  |    12 | gcc-12.2.0   |    0.0% |  -5.70% |

The speed difference for clang at level 12 is real, but is probably
caused by some sort of alignment or codegen issues. clang is
significantly slower than gcc before this PR, but gets up to parity with
it.

I also measured the ratio difference for the HC match finder, and it
looks basically the same as the row-based match finder. The speedup on
random data looks similar. And performance is about neutral, without the
big difference at level 12 for either clang or gcc.
This commit is contained in:
Nick Terrell
2023-03-09 17:26:07 -08:00
committed by Nick Terrell
parent 3b001a38fe
commit a3c3a38b9b
4 changed files with 186 additions and 131 deletions
Download Patch File Download Diff File Expand all files Collapse all files

230
tests/regression/results.csv
View File

@ -6,10 +6,10 @@ silesia.tar, level 0, compress
silesia.tar, level 1, compress simple, 5327717
silesia.tar, level 3, compress simple, 4854086
silesia.tar, level 4, compress simple, 4791503
silesia.tar, level 5, compress simple, 4679219
silesia.tar, level 6, compress simple, 4614874
silesia.tar, level 7, compress simple, 4579642
silesia.tar, level 9, compress simple, 4555245
silesia.tar, level 5, compress simple, 4679004
silesia.tar, level 6, compress simple, 4614561
silesia.tar, level 7, compress simple, 4579828
silesia.tar, level 9, compress simple, 4555448
silesia.tar, level 13, compress simple, 4502956
silesia.tar, level 16, compress simple, 4360546
silesia.tar, level 19, compress simple, 4265911
@ -40,10 +40,10 @@ silesia, level 0, compress
silesia, level 1, compress cctx, 5306632
silesia, level 3, compress cctx, 4842075
silesia, level 4, compress cctx, 4779186
silesia, level 5, compress cctx, 4667825
silesia, level 6, compress cctx, 4604587
silesia, level 7, compress cctx, 4569976
silesia, level 9, compress cctx, 4545538
silesia, level 5, compress cctx, 4667668
silesia, level 6, compress cctx, 4604351
silesia, level 7, compress cctx, 4570271
silesia, level 9, compress cctx, 4545850
silesia, level 13, compress cctx, 4493990
silesia, level 16, compress cctx, 4360041
silesia, level 19, compress cctx, 4296055
@ -53,7 +53,7 @@ silesia, multithreaded long distance mode, compress
silesia, small window log, compress cctx, 7082951
silesia, small hash log, compress cctx, 6526141
silesia, small chain log, compress cctx, 4912197
silesia, explicit params, compress cctx, 4794128
silesia, explicit params, compress cctx, 4794318
silesia, uncompressed literals, compress cctx, 4842075
silesia, uncompressed literals optimal, compress cctx, 4296055
silesia, huffman literals, compress cctx, 6172202
@ -104,10 +104,10 @@ silesia, level 0, zstdcli,
silesia, level 1, zstdcli, 5306680
silesia, level 3, zstdcli, 4842123
silesia, level 4, zstdcli, 4779234
silesia, level 5, zstdcli, 4667873
silesia, level 6, zstdcli, 4604635
silesia, level 7, zstdcli, 4570024
silesia, level 9, zstdcli, 4545586
silesia, level 5, zstdcli, 4667716
silesia, level 6, zstdcli, 4604399
silesia, level 7, zstdcli, 4570319
silesia, level 9, zstdcli, 4545898
silesia, level 13, zstdcli, 4494038
silesia, level 16, zstdcli, 4360089
silesia, level 19, zstdcli, 4296103
@ -117,7 +117,7 @@ silesia, multithreaded long distance mode, zstdcli,
silesia, small window log, zstdcli, 7095048
silesia, small hash log, zstdcli, 6526189
silesia, small chain log, zstdcli, 4912245
silesia, explicit params, zstdcli, 4795654
silesia, explicit params, zstdcli, 4795840
silesia, uncompressed literals, zstdcli, 5120614
silesia, uncompressed literals optimal, zstdcli, 4319566
silesia, huffman literals, zstdcli, 5321417
@ -129,10 +129,10 @@ silesia.tar, level 0, zstdcli,
silesia.tar, level 1, zstdcli, 5329010
silesia.tar, level 3, zstdcli, 4854164
silesia.tar, level 4, zstdcli, 4792352
silesia.tar, level 5, zstdcli, 4680075
silesia.tar, level 6, zstdcli, 4615668
silesia.tar, level 7, zstdcli, 4581604
silesia.tar, level 9, zstdcli, 4555249
silesia.tar, level 5, zstdcli, 4679860
silesia.tar, level 6, zstdcli, 4615355
silesia.tar, level 7, zstdcli, 4581791
silesia.tar, level 9, zstdcli, 4555452
silesia.tar, level 13, zstdcli, 4502960
silesia.tar, level 16, zstdcli, 4360550
silesia.tar, level 19, zstdcli, 4265915
@ -143,7 +143,7 @@ silesia.tar, multithreaded long distance mode, zstdcli,
silesia.tar, small window log, zstdcli, 7100701
silesia.tar, small hash log, zstdcli, 6529264
silesia.tar, small chain log, zstdcli, 4917022
silesia.tar, explicit params, zstdcli, 4820917
silesia.tar, explicit params, zstdcli, 4821112
silesia.tar, uncompressed literals, zstdcli, 5122571
silesia.tar, uncompressed literals optimal, zstdcli, 4310145
silesia.tar, huffman literals, zstdcli, 5342074
@ -235,18 +235,18 @@ silesia, level 0, advanced
silesia, level 1, advanced one pass, 5306632
silesia, level 3, advanced one pass, 4842075
silesia, level 4, advanced one pass, 4779186
silesia, level 5 row 1, advanced one pass, 4667825
silesia, level 5 row 2, advanced one pass, 4670136
silesia, level 5, advanced one pass, 4667825
silesia, level 6, advanced one pass, 4604587
silesia, level 7 row 1, advanced one pass, 4569976
silesia, level 7 row 2, advanced one pass, 4564868
silesia, level 7, advanced one pass, 4569976
silesia, level 9, advanced one pass, 4545538
silesia, level 11 row 1, advanced one pass, 4505351
silesia, level 11 row 2, advanced one pass, 4503116
silesia, level 12 row 1, advanced one pass, 4505351
silesia, level 12 row 2, advanced one pass, 4503116
silesia, level 5 row 1, advanced one pass, 4667668
silesia, level 5 row 2, advanced one pass, 4670326
silesia, level 5, advanced one pass, 4667668
silesia, level 6, advanced one pass, 4604351
silesia, level 7 row 1, advanced one pass, 4570271
silesia, level 7 row 2, advanced one pass, 4565169
silesia, level 7, advanced one pass, 4570271
silesia, level 9, advanced one pass, 4545850
silesia, level 11 row 1, advanced one pass, 4505658
silesia, level 11 row 2, advanced one pass, 4503429
silesia, level 12 row 1, advanced one pass, 4505658
silesia, level 12 row 2, advanced one pass, 4503429
silesia, level 13, advanced one pass, 4493990
silesia, level 16, advanced one pass, 4360041
silesia, level 19, advanced one pass, 4296055
@ -257,7 +257,7 @@ silesia, multithreaded long distance mode, advanced
silesia, small window log, advanced one pass, 7095000
silesia, small hash log, advanced one pass, 6526141
silesia, small chain log, advanced one pass, 4912197
silesia, explicit params, advanced one pass, 4795654
silesia, explicit params, advanced one pass, 4795840
silesia, uncompressed literals, advanced one pass, 5120566
silesia, uncompressed literals optimal, advanced one pass, 4319518
silesia, huffman literals, advanced one pass, 5321369
@ -269,18 +269,18 @@ silesia.tar, level 0, advanced
silesia.tar, level 1, advanced one pass, 5327717
silesia.tar, level 3, advanced one pass, 4854086
silesia.tar, level 4, advanced one pass, 4791503
silesia.tar, level 5 row 1, advanced one pass, 4679219
silesia.tar, level 5 row 2, advanced one pass, 4682161
silesia.tar, level 5, advanced one pass, 4679219
silesia.tar, level 6, advanced one pass, 4614874
silesia.tar, level 7 row 1, advanced one pass, 4579642
silesia.tar, level 7 row 2, advanced one pass, 4575393
silesia.tar, level 7, advanced one pass, 4579642
silesia.tar, level 9, advanced one pass, 4555245
silesia.tar, level 11 row 1, advanced one pass, 4514753
silesia.tar, level 11 row 2, advanced one pass, 4513604
silesia.tar, level 12 row 1, advanced one pass, 4514309
silesia.tar, level 12 row 2, advanced one pass, 4513797
silesia.tar, level 5 row 1, advanced one pass, 4679004
silesia.tar, level 5 row 2, advanced one pass, 4682334
silesia.tar, level 5, advanced one pass, 4679004
silesia.tar, level 6, advanced one pass, 4614561
silesia.tar, level 7 row 1, advanced one pass, 4579828
silesia.tar, level 7 row 2, advanced one pass, 4575602
silesia.tar, level 7, advanced one pass, 4579828
silesia.tar, level 9, advanced one pass, 4555448
silesia.tar, level 11 row 1, advanced one pass, 4514962
silesia.tar, level 11 row 2, advanced one pass, 4513816
silesia.tar, level 12 row 1, advanced one pass, 4514517
silesia.tar, level 12 row 2, advanced one pass, 4514007
silesia.tar, level 13, advanced one pass, 4502956
silesia.tar, level 16, advanced one pass, 4360546
silesia.tar, level 19, advanced one pass, 4265911
@ -291,7 +291,7 @@ silesia.tar, multithreaded long distance mode, advanced
silesia.tar, small window log, advanced one pass, 7100655
silesia.tar, small hash log, advanced one pass, 6529206
silesia.tar, small chain log, advanced one pass, 4917041
silesia.tar, explicit params, advanced one pass, 4807078
silesia.tar, explicit params, advanced one pass, 4807274
silesia.tar, uncompressed literals, advanced one pass, 5122473
silesia.tar, uncompressed literals optimal, advanced one pass, 4310141
silesia.tar, huffman literals, advanced one pass, 5341705
@ -553,18 +553,18 @@ silesia, level 0, advanced
silesia, level 1, advanced one pass small out, 5306632
silesia, level 3, advanced one pass small out, 4842075
silesia, level 4, advanced one pass small out, 4779186
silesia, level 5 row 1, advanced one pass small out, 4667825
silesia, level 5 row 2, advanced one pass small out, 4670136
silesia, level 5, advanced one pass small out, 4667825
silesia, level 6, advanced one pass small out, 4604587
silesia, level 7 row 1, advanced one pass small out, 4569976
silesia, level 7 row 2, advanced one pass small out, 4564868
silesia, level 7, advanced one pass small out, 4569976
silesia, level 9, advanced one pass small out, 4545538
silesia, level 11 row 1, advanced one pass small out, 4505351
silesia, level 11 row 2, advanced one pass small out, 4503116
silesia, level 12 row 1, advanced one pass small out, 4505351
silesia, level 12 row 2, advanced one pass small out, 4503116
silesia, level 5 row 1, advanced one pass small out, 4667668
silesia, level 5 row 2, advanced one pass small out, 4670326
silesia, level 5, advanced one pass small out, 4667668
silesia, level 6, advanced one pass small out, 4604351
silesia, level 7 row 1, advanced one pass small out, 4570271
silesia, level 7 row 2, advanced one pass small out, 4565169
silesia, level 7, advanced one pass small out, 4570271
silesia, level 9, advanced one pass small out, 4545850
silesia, level 11 row 1, advanced one pass small out, 4505658
silesia, level 11 row 2, advanced one pass small out, 4503429
silesia, level 12 row 1, advanced one pass small out, 4505658
silesia, level 12 row 2, advanced one pass small out, 4503429
silesia, level 13, advanced one pass small out, 4493990
silesia, level 16, advanced one pass small out, 4360041
silesia, level 19, advanced one pass small out, 4296055
@ -575,7 +575,7 @@ silesia, multithreaded long distance mode, advanced
silesia, small window log, advanced one pass small out, 7095000
silesia, small hash log, advanced one pass small out, 6526141
silesia, small chain log, advanced one pass small out, 4912197
silesia, explicit params, advanced one pass small out, 4795654
silesia, explicit params, advanced one pass small out, 4795840
silesia, uncompressed literals, advanced one pass small out, 5120566
silesia, uncompressed literals optimal, advanced one pass small out, 4319518
silesia, huffman literals, advanced one pass small out, 5321369
@ -587,18 +587,18 @@ silesia.tar, level 0, advanced
silesia.tar, level 1, advanced one pass small out, 5327717
silesia.tar, level 3, advanced one pass small out, 4854086
silesia.tar, level 4, advanced one pass small out, 4791503
silesia.tar, level 5 row 1, advanced one pass small out, 4679219
silesia.tar, level 5 row 2, advanced one pass small out, 4682161
silesia.tar, level 5, advanced one pass small out, 4679219
silesia.tar, level 6, advanced one pass small out, 4614874
silesia.tar, level 7 row 1, advanced one pass small out, 4579642
silesia.tar, level 7 row 2, advanced one pass small out, 4575393
silesia.tar, level 7, advanced one pass small out, 4579642
silesia.tar, level 9, advanced one pass small out, 4555245
silesia.tar, level 11 row 1, advanced one pass small out, 4514753
silesia.tar, level 11 row 2, advanced one pass small out, 4513604
silesia.tar, level 12 row 1, advanced one pass small out, 4514309
silesia.tar, level 12 row 2, advanced one pass small out, 4513797
silesia.tar, level 5 row 1, advanced one pass small out, 4679004
silesia.tar, level 5 row 2, advanced one pass small out, 4682334
silesia.tar, level 5, advanced one pass small out, 4679004
silesia.tar, level 6, advanced one pass small out, 4614561
silesia.tar, level 7 row 1, advanced one pass small out, 4579828
silesia.tar, level 7 row 2, advanced one pass small out, 4575602
silesia.tar, level 7, advanced one pass small out, 4579828
silesia.tar, level 9, advanced one pass small out, 4555448
silesia.tar, level 11 row 1, advanced one pass small out, 4514962
silesia.tar, level 11 row 2, advanced one pass small out, 4513816
silesia.tar, level 12 row 1, advanced one pass small out, 4514517
silesia.tar, level 12 row 2, advanced one pass small out, 4514007
silesia.tar, level 13, advanced one pass small out, 4502956
silesia.tar, level 16, advanced one pass small out, 4360546
silesia.tar, level 19, advanced one pass small out, 4265911
@ -609,7 +609,7 @@ silesia.tar, multithreaded long distance mode, advanced
silesia.tar, small window log, advanced one pass small out, 7100655
silesia.tar, small hash log, advanced one pass small out, 6529206
silesia.tar, small chain log, advanced one pass small out, 4917041
silesia.tar, explicit params, advanced one pass small out, 4807078
silesia.tar, explicit params, advanced one pass small out, 4807274
silesia.tar, uncompressed literals, advanced one pass small out, 5122473
silesia.tar, uncompressed literals optimal, advanced one pass small out, 4310141
silesia.tar, huffman literals, advanced one pass small out, 5341705
@ -871,18 +871,18 @@ silesia, level 0, advanced
silesia, level 1, advanced streaming, 5306388
silesia, level 3, advanced streaming, 4842075
silesia, level 4, advanced streaming, 4779186
silesia, level 5 row 1, advanced streaming, 4667825
silesia, level 5 row 2, advanced streaming, 4670136
silesia, level 5, advanced streaming, 4667825
silesia, level 6, advanced streaming, 4604587
silesia, level 7 row 1, advanced streaming, 4569976
silesia, level 7 row 2, advanced streaming, 4564868
silesia, level 7, advanced streaming, 4569976
silesia, level 9, advanced streaming, 4545538
silesia, level 11 row 1, advanced streaming, 4505351
silesia, level 11 row 2, advanced streaming, 4503116
silesia, level 12 row 1, advanced streaming, 4505351
silesia, level 12 row 2, advanced streaming, 4503116
silesia, level 5 row 1, advanced streaming, 4667668
silesia, level 5 row 2, advanced streaming, 4670326
silesia, level 5, advanced streaming, 4667668
silesia, level 6, advanced streaming, 4604351
silesia, level 7 row 1, advanced streaming, 4570271
silesia, level 7 row 2, advanced streaming, 4565169
silesia, level 7, advanced streaming, 4570271
silesia, level 9, advanced streaming, 4545850
silesia, level 11 row 1, advanced streaming, 4505658
silesia, level 11 row 2, advanced streaming, 4503429
silesia, level 12 row 1, advanced streaming, 4505658
silesia, level 12 row 2, advanced streaming, 4503429
silesia, level 13, advanced streaming, 4493990
silesia, level 16, advanced streaming, 4360041
silesia, level 19, advanced streaming, 4296055
@ -893,7 +893,7 @@ silesia, multithreaded long distance mode, advanced
silesia, small window log, advanced streaming, 7111103
silesia, small hash log, advanced streaming, 6526141
silesia, small chain log, advanced streaming, 4912197
silesia, explicit params, advanced streaming, 4795672
silesia, explicit params, advanced streaming, 4795857
silesia, uncompressed literals, advanced streaming, 5120566
silesia, uncompressed literals optimal, advanced streaming, 4319518
silesia, huffman literals, advanced streaming, 5321370
@ -905,18 +905,18 @@ silesia.tar, level 0, advanced
silesia.tar, level 1, advanced streaming, 5327708
silesia.tar, level 3, advanced streaming, 4859271
silesia.tar, level 4, advanced streaming, 4797470
silesia.tar, level 5 row 1, advanced streaming, 4679226
silesia.tar, level 5 row 2, advanced streaming, 4682169
silesia.tar, level 5, advanced streaming, 4679226
silesia.tar, level 6, advanced streaming, 4614873
silesia.tar, level 7 row 1, advanced streaming, 4579641
silesia.tar, level 7 row 2, advanced streaming, 4575394
silesia.tar, level 7, advanced streaming, 4579641
silesia.tar, level 9, advanced streaming, 4555246
silesia.tar, level 11 row 1, advanced streaming, 4514754
silesia.tar, level 11 row 2, advanced streaming, 4513604
silesia.tar, level 12 row 1, advanced streaming, 4514309
silesia.tar, level 12 row 2, advanced streaming, 4513797
silesia.tar, level 5 row 1, advanced streaming, 4679020
silesia.tar, level 5 row 2, advanced streaming, 4682355
silesia.tar, level 5, advanced streaming, 4679020
silesia.tar, level 6, advanced streaming, 4614558
silesia.tar, level 7 row 1, advanced streaming, 4579823
silesia.tar, level 7 row 2, advanced streaming, 4575601
silesia.tar, level 7, advanced streaming, 4579823
silesia.tar, level 9, advanced streaming, 4555445
silesia.tar, level 11 row 1, advanced streaming, 4514959
silesia.tar, level 11 row 2, advanced streaming, 4513810
silesia.tar, level 12 row 1, advanced streaming, 4514514
silesia.tar, level 12 row 2, advanced streaming, 4514003
silesia.tar, level 13, advanced streaming, 4502956
silesia.tar, level 16, advanced streaming, 4360546
silesia.tar, level 19, advanced streaming, 4265911
@ -927,7 +927,7 @@ silesia.tar, multithreaded long distance mode, advanced
silesia.tar, small window log, advanced streaming, 7117559
silesia.tar, small hash log, advanced streaming, 6529209
silesia.tar, small chain log, advanced streaming, 4917021
silesia.tar, explicit params, advanced streaming, 4807102
silesia.tar, explicit params, advanced streaming, 4807288
silesia.tar, uncompressed literals, advanced streaming, 5127423
silesia.tar, uncompressed literals optimal, advanced streaming, 4310141
silesia.tar, huffman literals, advanced streaming, 5341712
@ -1189,10 +1189,10 @@ silesia, level 0, old stre
silesia, level 1, old streaming, 5306388
silesia, level 3, old streaming, 4842075
silesia, level 4, old streaming, 4779186
silesia, level 5, old streaming, 4667825
silesia, level 6, old streaming, 4604587
silesia, level 7, old streaming, 4569976
silesia, level 9, old streaming, 4545538
silesia, level 5, old streaming, 4667668
silesia, level 6, old streaming, 4604351
silesia, level 7, old streaming, 4570271
silesia, level 9, old streaming, 4545850
silesia, level 13, old streaming, 4493990
silesia, level 16, old streaming, 4360041
silesia, level 19, old streaming, 4296055
@ -1207,10 +1207,10 @@ silesia.tar, level 0, old stre
silesia.tar, level 1, old streaming, 5327708
silesia.tar, level 3, old streaming, 4859271
silesia.tar, level 4, old streaming, 4797470
silesia.tar, level 5, old streaming, 4679226
silesia.tar, level 6, old streaming, 4614873
silesia.tar, level 7, old streaming, 4579641
silesia.tar, level 9, old streaming, 4555246
silesia.tar, level 5, old streaming, 4679020
silesia.tar, level 6, old streaming, 4614558
silesia.tar, level 7, old streaming, 4579823
silesia.tar, level 9, old streaming, 4555445
silesia.tar, level 13, old streaming, 4502956
silesia.tar, level 16, old streaming, 4360546
silesia.tar, level 19, old streaming, 4265911
@ -1291,10 +1291,10 @@ silesia, level 0, old stre
silesia, level 1, old streaming advanced, 5306388
silesia, level 3, old streaming advanced, 4842075
silesia, level 4, old streaming advanced, 4779186
silesia, level 5, old streaming advanced, 4667825
silesia, level 6, old streaming advanced, 4604587
silesia, level 7, old streaming advanced, 4569976
silesia, level 9, old streaming advanced, 4545538
silesia, level 5, old streaming advanced, 4667668
silesia, level 6, old streaming advanced, 4604351
silesia, level 7, old streaming advanced, 4570271
silesia, level 9, old streaming advanced, 4545850
silesia, level 13, old streaming advanced, 4493990
silesia, level 16, old streaming advanced, 4360041
silesia, level 19, old streaming advanced, 4296055
@ -1305,7 +1305,7 @@ silesia, multithreaded long distance mode, old stre
silesia, small window log, old streaming advanced, 7111103
silesia, small hash log, old streaming advanced, 6526141
silesia, small chain log, old streaming advanced, 4912197
silesia, explicit params, old streaming advanced, 4795672
silesia, explicit params, old streaming advanced, 4795857
silesia, uncompressed literals, old streaming advanced, 4842075
silesia, uncompressed literals optimal, old streaming advanced, 4296055
silesia, huffman literals, old streaming advanced, 6172207
@ -1317,10 +1317,10 @@ silesia.tar, level 0, old stre
silesia.tar, level 1, old streaming advanced, 5327708
silesia.tar, level 3, old streaming advanced, 4859271
silesia.tar, level 4, old streaming advanced, 4797470
silesia.tar, level 5, old streaming advanced, 4679226
silesia.tar, level 6, old streaming advanced, 4614873
silesia.tar, level 7, old streaming advanced, 4579641
silesia.tar, level 9, old streaming advanced, 4555246
silesia.tar, level 5, old streaming advanced, 4679020
silesia.tar, level 6, old streaming advanced, 4614558
silesia.tar, level 7, old streaming advanced, 4579823
silesia.tar, level 9, old streaming advanced, 4555445
silesia.tar, level 13, old streaming advanced, 4502956
silesia.tar, level 16, old streaming advanced, 4360546
silesia.tar, level 19, old streaming advanced, 4265911
@ -1331,7 +1331,7 @@ silesia.tar, multithreaded long distance mode, old stre
silesia.tar, small window log, old streaming advanced, 7117562
silesia.tar, small hash log, old streaming advanced, 6529209
silesia.tar, small chain log, old streaming advanced, 4917021
silesia.tar, explicit params, old streaming advanced, 4807102
silesia.tar, explicit params, old streaming advanced, 4807288
silesia.tar, uncompressed literals, old streaming advanced, 4859271
silesia.tar, uncompressed literals optimal, old streaming advanced, 4265911
silesia.tar, huffman literals, old streaming advanced, 6179056

1 Data Config Method Total compressed size
6 silesia.tar level 1 compress simple 5327717
7 silesia.tar level 3 compress simple 4854086
8 silesia.tar level 4 compress simple 4791503
9 silesia.tar level 5 compress simple 4679219 4679004
10 silesia.tar level 6 compress simple 4614874 4614561
11 silesia.tar level 7 compress simple 4579642 4579828
12 silesia.tar level 9 compress simple 4555245 4555448
13 silesia.tar level 13 compress simple 4502956
14 silesia.tar level 16 compress simple 4360546
15 silesia.tar level 19 compress simple 4265911
40 silesia level 1 compress cctx 5306632
41 silesia level 3 compress cctx 4842075
42 silesia level 4 compress cctx 4779186
43 silesia level 5 compress cctx 4667825 4667668
44 silesia level 6 compress cctx 4604587 4604351
45 silesia level 7 compress cctx 4569976 4570271
46 silesia level 9 compress cctx 4545538 4545850
47 silesia level 13 compress cctx 4493990
48 silesia level 16 compress cctx 4360041
49 silesia level 19 compress cctx 4296055
53 silesia small window log compress cctx 7082951
54 silesia small hash log compress cctx 6526141
55 silesia small chain log compress cctx 4912197
56 silesia explicit params compress cctx 4794128 4794318
57 silesia uncompressed literals compress cctx 4842075
58 silesia uncompressed literals optimal compress cctx 4296055
59 silesia huffman literals compress cctx 6172202
104 silesia level 1 zstdcli 5306680
105 silesia level 3 zstdcli 4842123
106 silesia level 4 zstdcli 4779234
107 silesia level 5 zstdcli 4667873 4667716
108 silesia level 6 zstdcli 4604635 4604399
109 silesia level 7 zstdcli 4570024 4570319
110 silesia level 9 zstdcli 4545586 4545898
111 silesia level 13 zstdcli 4494038
112 silesia level 16 zstdcli 4360089
113 silesia level 19 zstdcli 4296103
117 silesia small window log zstdcli 7095048
118 silesia small hash log zstdcli 6526189
119 silesia small chain log zstdcli 4912245
120 silesia explicit params zstdcli 4795654 4795840
121 silesia uncompressed literals zstdcli 5120614
122 silesia uncompressed literals optimal zstdcli 4319566
123 silesia huffman literals zstdcli 5321417
129 silesia.tar level 1 zstdcli 5329010
130 silesia.tar level 3 zstdcli 4854164
131 silesia.tar level 4 zstdcli 4792352
132 silesia.tar level 5 zstdcli 4680075 4679860
133 silesia.tar level 6 zstdcli 4615668 4615355
134 silesia.tar level 7 zstdcli 4581604 4581791
135 silesia.tar level 9 zstdcli 4555249 4555452
136 silesia.tar level 13 zstdcli 4502960
137 silesia.tar level 16 zstdcli 4360550
138 silesia.tar level 19 zstdcli 4265915
143 silesia.tar small window log zstdcli 7100701
144 silesia.tar small hash log zstdcli 6529264
145 silesia.tar small chain log zstdcli 4917022
146 silesia.tar explicit params zstdcli 4820917 4821112
147 silesia.tar uncompressed literals zstdcli 5122571
148 silesia.tar uncompressed literals optimal zstdcli 4310145
149 silesia.tar huffman literals zstdcli 5342074
235 silesia level 1 advanced one pass 5306632
236 silesia level 3 advanced one pass 4842075
237 silesia level 4 advanced one pass 4779186
238 silesia level 5 row 1 advanced one pass 4667825 4667668
239 silesia level 5 row 2 advanced one pass 4670136 4670326
240 silesia level 5 advanced one pass 4667825 4667668
241 silesia level 6 advanced one pass 4604587 4604351
242 silesia level 7 row 1 advanced one pass 4569976 4570271
243 silesia level 7 row 2 advanced one pass 4564868 4565169
244 silesia level 7 advanced one pass 4569976 4570271
245 silesia level 9 advanced one pass 4545538 4545850
246 silesia level 11 row 1 advanced one pass 4505351 4505658
247 silesia level 11 row 2 advanced one pass 4503116 4503429
248 silesia level 12 row 1 advanced one pass 4505351 4505658
249 silesia level 12 row 2 advanced one pass 4503116 4503429
250 silesia level 13 advanced one pass 4493990
251 silesia level 16 advanced one pass 4360041
252 silesia level 19 advanced one pass 4296055
257 silesia small window log advanced one pass 7095000
258 silesia small hash log advanced one pass 6526141
259 silesia small chain log advanced one pass 4912197
260 silesia explicit params advanced one pass 4795654 4795840
261 silesia uncompressed literals advanced one pass 5120566
262 silesia uncompressed literals optimal advanced one pass 4319518
263 silesia huffman literals advanced one pass 5321369
269 silesia.tar level 1 advanced one pass 5327717
270 silesia.tar level 3 advanced one pass 4854086
271 silesia.tar level 4 advanced one pass 4791503
272 silesia.tar level 5 row 1 advanced one pass 4679219 4679004
273 silesia.tar level 5 row 2 advanced one pass 4682161 4682334
274 silesia.tar level 5 advanced one pass 4679219 4679004
275 silesia.tar level 6 advanced one pass 4614874 4614561
276 silesia.tar level 7 row 1 advanced one pass 4579642 4579828
277 silesia.tar level 7 row 2 advanced one pass 4575393 4575602
278 silesia.tar level 7 advanced one pass 4579642 4579828
279 silesia.tar level 9 advanced one pass 4555245 4555448
280 silesia.tar level 11 row 1 advanced one pass 4514753 4514962
281 silesia.tar level 11 row 2 advanced one pass 4513604 4513816
282 silesia.tar level 12 row 1 advanced one pass 4514309 4514517
283 silesia.tar level 12 row 2 advanced one pass 4513797 4514007
284 silesia.tar level 13 advanced one pass 4502956
285 silesia.tar level 16 advanced one pass 4360546
286 silesia.tar level 19 advanced one pass 4265911
291 silesia.tar small window log advanced one pass 7100655
292 silesia.tar small hash log advanced one pass 6529206
293 silesia.tar small chain log advanced one pass 4917041
294 silesia.tar explicit params advanced one pass 4807078 4807274
295 silesia.tar uncompressed literals advanced one pass 5122473
296 silesia.tar uncompressed literals optimal advanced one pass 4310141
297 silesia.tar huffman literals advanced one pass 5341705
553 silesia level 1 advanced one pass small out 5306632
554 silesia level 3 advanced one pass small out 4842075
555 silesia level 4 advanced one pass small out 4779186
556 silesia level 5 row 1 advanced one pass small out 4667825 4667668
557 silesia level 5 row 2 advanced one pass small out 4670136 4670326
558 silesia level 5 advanced one pass small out 4667825 4667668
559 silesia level 6 advanced one pass small out 4604587 4604351
560 silesia level 7 row 1 advanced one pass small out 4569976 4570271
561 silesia level 7 row 2 advanced one pass small out 4564868 4565169
562 silesia level 7 advanced one pass small out 4569976 4570271
563 silesia level 9 advanced one pass small out 4545538 4545850
564 silesia level 11 row 1 advanced one pass small out 4505351 4505658
565 silesia level 11 row 2 advanced one pass small out 4503116 4503429
566 silesia level 12 row 1 advanced one pass small out 4505351 4505658
567 silesia level 12 row 2 advanced one pass small out 4503116 4503429
568 silesia level 13 advanced one pass small out 4493990
569 silesia level 16 advanced one pass small out 4360041
570 silesia level 19 advanced one pass small out 4296055
575 silesia small window log advanced one pass small out 7095000
576 silesia small hash log advanced one pass small out 6526141
577 silesia small chain log advanced one pass small out 4912197
578 silesia explicit params advanced one pass small out 4795654 4795840
579 silesia uncompressed literals advanced one pass small out 5120566
580 silesia uncompressed literals optimal advanced one pass small out 4319518
581 silesia huffman literals advanced one pass small out 5321369
587 silesia.tar level 1 advanced one pass small out 5327717
588 silesia.tar level 3 advanced one pass small out 4854086
589 silesia.tar level 4 advanced one pass small out 4791503
590 silesia.tar level 5 row 1 advanced one pass small out 4679219 4679004
591 silesia.tar level 5 row 2 advanced one pass small out 4682161 4682334
592 silesia.tar level 5 advanced one pass small out 4679219 4679004
593 silesia.tar level 6 advanced one pass small out 4614874 4614561
594 silesia.tar level 7 row 1 advanced one pass small out 4579642 4579828
595 silesia.tar level 7 row 2 advanced one pass small out 4575393 4575602
596 silesia.tar level 7 advanced one pass small out 4579642 4579828
597 silesia.tar level 9 advanced one pass small out 4555245 4555448
598 silesia.tar level 11 row 1 advanced one pass small out 4514753 4514962
599 silesia.tar level 11 row 2 advanced one pass small out 4513604 4513816
600 silesia.tar level 12 row 1 advanced one pass small out 4514309 4514517
601 silesia.tar level 12 row 2 advanced one pass small out 4513797 4514007
602 silesia.tar level 13 advanced one pass small out 4502956
603 silesia.tar level 16 advanced one pass small out 4360546
604 silesia.tar level 19 advanced one pass small out 4265911
609 silesia.tar small window log advanced one pass small out 7100655
610 silesia.tar small hash log advanced one pass small out 6529206
611 silesia.tar small chain log advanced one pass small out 4917041
612 silesia.tar explicit params advanced one pass small out 4807078 4807274
613 silesia.tar uncompressed literals advanced one pass small out 5122473
614 silesia.tar uncompressed literals optimal advanced one pass small out 4310141
615 silesia.tar huffman literals advanced one pass small out 5341705
871 silesia level 1 advanced streaming 5306388
872 silesia level 3 advanced streaming 4842075
873 silesia level 4 advanced streaming 4779186
874 silesia level 5 row 1 advanced streaming 4667825 4667668
875 silesia level 5 row 2 advanced streaming 4670136 4670326
876 silesia level 5 advanced streaming 4667825 4667668
877 silesia level 6 advanced streaming 4604587 4604351
878 silesia level 7 row 1 advanced streaming 4569976 4570271
879 silesia level 7 row 2 advanced streaming 4564868 4565169
880 silesia level 7 advanced streaming 4569976 4570271
881 silesia level 9 advanced streaming 4545538 4545850
882 silesia level 11 row 1 advanced streaming 4505351 4505658
883 silesia level 11 row 2 advanced streaming 4503116 4503429
884 silesia level 12 row 1 advanced streaming 4505351 4505658
885 silesia level 12 row 2 advanced streaming 4503116 4503429
886 silesia level 13 advanced streaming 4493990
887 silesia level 16 advanced streaming 4360041
888 silesia level 19 advanced streaming 4296055
893 silesia small window log advanced streaming 7111103
894 silesia small hash log advanced streaming 6526141
895 silesia small chain log advanced streaming 4912197
896 silesia explicit params advanced streaming 4795672 4795857
897 silesia uncompressed literals advanced streaming 5120566
898 silesia uncompressed literals optimal advanced streaming 4319518
899 silesia huffman literals advanced streaming 5321370
905 silesia.tar level 1 advanced streaming 5327708
906 silesia.tar level 3 advanced streaming 4859271
907 silesia.tar level 4 advanced streaming 4797470
908 silesia.tar level 5 row 1 advanced streaming 4679226 4679020
909 silesia.tar level 5 row 2 advanced streaming 4682169 4682355
910 silesia.tar level 5 advanced streaming 4679226 4679020
911 silesia.tar level 6 advanced streaming 4614873 4614558
912 silesia.tar level 7 row 1 advanced streaming 4579641 4579823
913 silesia.tar level 7 row 2 advanced streaming 4575394 4575601
914 silesia.tar level 7 advanced streaming 4579641 4579823
915 silesia.tar level 9 advanced streaming 4555246 4555445
916 silesia.tar level 11 row 1 advanced streaming 4514754 4514959
917 silesia.tar level 11 row 2 advanced streaming 4513604 4513810
918 silesia.tar level 12 row 1 advanced streaming 4514309 4514514
919 silesia.tar level 12 row 2 advanced streaming 4513797 4514003
920 silesia.tar level 13 advanced streaming 4502956
921 silesia.tar level 16 advanced streaming 4360546
922 silesia.tar level 19 advanced streaming 4265911
927 silesia.tar small window log advanced streaming 7117559
928 silesia.tar small hash log advanced streaming 6529209
929 silesia.tar small chain log advanced streaming 4917021
930 silesia.tar explicit params advanced streaming 4807102 4807288
931 silesia.tar uncompressed literals advanced streaming 5127423
932 silesia.tar uncompressed literals optimal advanced streaming 4310141
933 silesia.tar huffman literals advanced streaming 5341712
1189 silesia level 1 old streaming 5306388
1190 silesia level 3 old streaming 4842075
1191 silesia level 4 old streaming 4779186
1192 silesia level 5 old streaming 4667825 4667668
1193 silesia level 6 old streaming 4604587 4604351
1194 silesia level 7 old streaming 4569976 4570271
1195 silesia level 9 old streaming 4545538 4545850
1196 silesia level 13 old streaming 4493990
1197 silesia level 16 old streaming 4360041
1198 silesia level 19 old streaming 4296055
1207 silesia.tar level 1 old streaming 5327708
1208 silesia.tar level 3 old streaming 4859271
1209 silesia.tar level 4 old streaming 4797470
1210 silesia.tar level 5 old streaming 4679226 4679020
1211 silesia.tar level 6 old streaming 4614873 4614558
1212 silesia.tar level 7 old streaming 4579641 4579823
1213 silesia.tar level 9 old streaming 4555246 4555445
1214 silesia.tar level 13 old streaming 4502956
1215 silesia.tar level 16 old streaming 4360546
1216 silesia.tar level 19 old streaming 4265911
1291 silesia level 1 old streaming advanced 5306388
1292 silesia level 3 old streaming advanced 4842075
1293 silesia level 4 old streaming advanced 4779186
1294 silesia level 5 old streaming advanced 4667825 4667668
1295 silesia level 6 old streaming advanced 4604587 4604351
1296 silesia level 7 old streaming advanced 4569976 4570271
1297 silesia level 9 old streaming advanced 4545538 4545850
1298 silesia level 13 old streaming advanced 4493990
1299 silesia level 16 old streaming advanced 4360041
1300 silesia level 19 old streaming advanced 4296055
1305 silesia small window log old streaming advanced 7111103
1306 silesia small hash log old streaming advanced 6526141
1307 silesia small chain log old streaming advanced 4912197
1308 silesia explicit params old streaming advanced 4795672 4795857
1309 silesia uncompressed literals old streaming advanced 4842075
1310 silesia uncompressed literals optimal old streaming advanced 4296055
1311 silesia huffman literals old streaming advanced 6172207
1317 silesia.tar level 1 old streaming advanced 5327708
1318 silesia.tar level 3 old streaming advanced 4859271
1319 silesia.tar level 4 old streaming advanced 4797470
1320 silesia.tar level 5 old streaming advanced 4679226 4679020
1321 silesia.tar level 6 old streaming advanced 4614873 4614558
1322 silesia.tar level 7 old streaming advanced 4579641 4579823
1323 silesia.tar level 9 old streaming advanced 4555246 4555445
1324 silesia.tar level 13 old streaming advanced 4502956
1325 silesia.tar level 16 old streaming advanced 4360546
1326 silesia.tar level 19 old streaming advanced 4265911
1331 silesia.tar small window log old streaming advanced 7117562
1332 silesia.tar small hash log old streaming advanced 6529209
1333 silesia.tar small chain log old streaming advanced 4917021
1334 silesia.tar explicit params old streaming advanced 4807102 4807288
1335 silesia.tar uncompressed literals old streaming advanced 4859271
1336 silesia.tar uncompressed literals optimal old streaming advanced 4265911
1337 silesia.tar huffman literals old streaming advanced 6179056