NEWS line for the fix of #1917, by @arshsmith1.
The multithreaded decoder hands out its output buffers in order,
tracking positions with two counters used modulo PBUFFERS_NB.
They were int : after INT_MAX buffers, the increment overflowed,
the index became negative, and BufPool_getBuffer() returned
bp->buffers[-2] : decoded data was written out of bounds.
Each frame takes at least one buffer, so a stream of 2^31 empty
or skippable frames (8 bytes each, ~17 GB) reaches it. Measured on dev :
1M frames take ~18 s, so ~11 h through a pipe. Bug since v1.10.0.
The counters are now unsigned long long : they never wrap in practice,
and the asserts comparing them still hold. A 32-bit unsigned would not
work : 2^32 is not a multiple of PBUFFERS_NB (3).
Checked under ASan, with both counters starting at INT_MAX - 5 :
dev overflows (heap-buffer-overflow in BufPool_getBuffer()),
this change decodes correctly.
33 tests, in 2 new suites : lz4 (decoding of invalid input, one-shot and streaming compression)
and lz4hc (streaming, dictionaries, destSize). Tests depending on the HC level run on both levels
previously selected by fuzzer.
UT_sample(), KB, MB and ARRAY_SIZE() are now provided by unittest.h, for all suites.
LZ4_compress_HC_continue() copied the tables of an attached dictionary
only above 4 KB, so 4 KB pages, the most common size, used the dictCtx
search. They now use the copy.
At 4096 bytes, with a 64 KB dictionary, compared to the dictCtx search :
level 2 : -2.6% compressed size (it only searched the dictionary
when the page had no match), levels 3-9 : same output,
levels 10-11 : -0.3% / -0.1%, level 12 : same output.
Speed, AMD EPYC Genoa (Zen 4), x86-64, gcc 11 / gcc 15 / clang 22 :
level 2 : -1 to +2%, levels 3-9 : -3.3 to +4.7%, level 10 : -6 to -11%,
level 11 : +12 to +23%, level 12 : +95 to +130%.
x86 (gcc 12 -m32) : level 2 : -2.7%, levels 3-9 : -0.3 to -3.1%,
level 10 : -2.5%, level 11 : +27%, level 12 : +172%.
The manual says that -t neither creates nor removes files, and that
--rm deletes the source after a successful compression or
decompression. The CLI did not follow it for two cases :
- `lz4 -t --rm file.lz4` tested the file, then deleted it.
- With stdin as source, the output was complete, but remove("stdin")
failed and the command ended with "Remove error : stdin"
(exit code 50 when compressing, 45 when decompressing).
A file named `stdin` in the working directory would have been deleted.
One helper, LZ4IO_mustRemoveSrcFile(), now decides. It answers no in
test mode and for stdin, and both remove sites use it :
LZ4IO_finishCompressedFile() and LZ4IO_decompressSrcFile().
A regular source file is still removed.
Each test is a function with its own context : resources are released at the end of the test,
even when it fails, and the random generator restarts from the same seed.
A failed assertion ends the test, the runner continues with the next one.
First suite : 18 lz4frame unit tests, moved from frametest.c
The DLL was compiled directly from sources, without the objects' cache keyed by flags :
after make lib-nomt, make lib-mt kept the DLL without threads (and conversely).
It now uses c_dynamic_library(), as the Unix shared library, with the multi-threading flags
in the cache key ; the resource object is an additional dependency. The import library
is the same for both variants (same exports). With BUILD_SHARED=no, liblz4 builds nothing, as before.
Checked with mingw-w64 : MULTITHREAD=0 / 1 / back to 1 (cached) switch the DLL's thread imports ;
a program linked to the multi-threaded DLL compresses with nbWorkers=4 under wine.
FUZ_rand() cycles can be short : 31487 values for seed 2145. Bytes taken directly from it
then repeat every ~31 KB, and blocks expected to be stored uncompressed compress (to ~50%) :
testMixedUpdates() and testFavorDecSpeedAfterRawBlock() failed with seed 2145, also on dev.
make test picks a random seed : ~1 run in 240 failed here.
FUZ_fillNoise() generates incompressible bytes as a hash (XXH32) of their position, keyed by FUZ_rand(),
for the 8 buffers or segments filled with noise. Seeds 1 to 2400 pass.
dev (#1918) split file compression : LZ4IO_compressFilename_extRess() opens files and prepares preferences,
frame compressors only compress, and LZ4IO_finishCompressedFile() ends files for frame and legacy formats.
During the rebase, the branch's frame compression (liblz4 multi-threading, memory-mapped input,
detection of a source modified or replaced) kept its single function ; it now follows this split :
- LZ4IO_compressFrame() compresses, returning the frame tail (content checksum) ;
- LZ4IO_compressFilename_extRess() maps input, adjusts nbWorkers, then checks whether the source changed
(inconsistent memory-mapped output : removed, error 54) ;
- LZ4IO_finishCompressedFile() keeps a source modified or replaced during compression (--rm not honored),
and doesn't copy the attributes of a replaced one (legacy : never).
The processed size is 0 when a file can't be opened (dev fix, lost in the rebase).
lz4frame : worker states initialized by LZ4F_initLZ4State() (dev #1921), as the cctx's state.
Note : LZ4F_makeBlock() keeps explicit state parameters, instead of the cctx (dev #1921) : worker threads have no cctx.
lz4frame :
- LZ4F_uncompressedUpdate() accepts linked blocks (dev and v1.10.0 could silently corrupt such frames) :
single-threaded mode drops the LZ4 state's history when an uncompressed segment starts
(the decoder's history contains the uncompressed data) ; multi-threaded mode needs nothing,
jobs reload the 64 KB actually preceding them. LZ4F_compressBegin() now starts frames in compressed mode :
the mode of the previous frame's end was kept, which skipped the history drop (frame corrupted with a dictionary).
- Windows : worker stacks reserved instead of committed (STACK_SIZE_PARAM_IS_A_RESERVATION) ;
#error when <windows.h> was already included without the Vista API (amalgamated sources) ;
doc : no multi-threaded context use nor release under the loader lock (DllMain, DLL global objects).
lib/Makefile, programs/Makefile : threads detection skipped by goals building nothing threaded (clean, uninstall, ...),
run once per command (sub-makes inherit it) : make install probes once instead of 2-4 times, make clean never.
lz4 : explicit -T# also limited to the jobs of each file (-T1000 on a 10 MB file : 3 workers, RSS 298 -> 27 MB at -9),
and by the memory budget on all systems (-T99999 : 1024 liblz4 maximum, 331 threads within the 4 GB default budget) ;
an explicit nb of threads reduced triggers a warning (automatic : only with -v).
tests : frametest testUncompressedLinked (linked blocks, dictionary made of input, frames ending uncompressed),
mixed updates and small-dst scripts and fuzzer with linked blocks ; frameMT uncompressed segments with linked blocks ;
ossfuzz stream and uncompressed round-trip fuzzers with linked blocks ; test-lz4-multithread budget warnings.
On a new or reset stream (prefixStart == NULL), LZ4_saveDictHC() installed @safeBuffer as prefix,
which skipped the initialization done by the next compression (LZ4HC_init_internal()).
The stream then started at index 0, which the hash table uses for empty entries :
positions matched themselves (offset 0), and the block decoded into wrong data, at all HC levels,
also with an attached dictionary. Present in v1.10.0.
It now saves nothing, leaving the stream to be initialized by the next compression.
Found by lz4frame with uncompressed then compressed segments in linked blocks.
tests : fuzzer.c saveDictHC test #926 compresses after LZ4_saveDictHC(), without and with an attached dictionary.
LZ4F_compressBound(), without autoFlush :
- single-threaded : (srcSize / blockSize + 1) blocks, stored uncompressed in worst case, plus the frame footer.
It covers up to (blockSize-1) buffered bytes, a buffered block emitted separately (mode switch),
or remaining from an incomplete LZ4F_flush(). srcSize == 0 : one block + footer, so that LZ4F_flush() and
LZ4F_compressEnd() complete in a single invocation, strictly below the bound (v1.10.0 guarantee kept).
- nbWorkers >= 1 : the job replaces the block : ceil(srcSize / jobSize) jobs (1 for srcSize == 0) + footer,
independently of nbWorkers. Updates defer completed jobs not fitting into dstBuffer : output is only required
to start a job while all slots are busy. Flush and end emit jobs in flight in several invocations.
nbWorkers is new : no existing user depends on a single invocation there.
Multi-threaded LZ4F_uncompressedUpdate() : uncompressed segments are jobs storing blocks uncompressed ;
a mode switch submits the job being filled (same output as single-threaded mode). A single LZ4F_mt_update()
serves both updates : uncTail, LZ4F_mt_emitUncTail() and LZ4F_uncompressedUpdate_MT() are removed.
lz4 : output buffer sized by LZ4F_compressBound(chunk) (~4 MB, instead of 72 MB with 8 threads),
end of frame in a loop ; more threads within a memory budget (--memory=300M : 13 -> 23 threads) ;
releaseLag = (2*nbWorkers+1) jobs + 64 KB.
tests : frameMT (bound structure, flush / end loops), stream fuzzer (reference loops with nbWorkers).
LZ4F_flush() and LZ4F_compressEnd() : each invocation fills dstBuffer, or completes :
while it returns dstCapacity, output may remain, and the operation is invoked again, with any dstCapacity.
Output remains strictly below LZ4F_compressBound(0) (+1 for autoFlush), so that callers sizing
dstBuffer with the bound see no change : a single invocation completes.
- the first invocation sets the flush point : buffered input is compressed (all jobs waited for),
so input is no longer read after it returns (LZ4F_srcStableUntilFlush lifetime unchanged) ;
- single-threaded : a block not fitting is compressed into stageBuff (allocated on first need), same output ;
multi-threaded : the oldest job can be emitted partially (mt->emitted) ; frame footer staged (8 bytes) ;
- compressUpdate() / uncompressedUpdate() may follow an incomplete flush : remaining output comes first ;
until an end completes, only LZ4F_compressEnd() continues it (LZ4F_compressBegin() abandons it) ;
- dstCapacity 0 is rejected when output remains, without modifying the frame ;
- contentSize mismatch is detected when the end starts, before writing anything ;
- LZ4F_compressFrame*() keep failing with dstCapacity_tooSmall when the frame doesn't fit.
lz4file : LZ4F_writeClose() loops on LZ4F_compressEnd().
lz4 : releaseLag = LZ4F_compressBound(0) + 64 KB (bound of input retained within cctx).
tests : frametest (#1817 streaming probes, fuzzer), frameMT (validatePartialFlush, small dst, strict bound),
stream fuzzer : flush and end in pieces.
lz4hc levels >= 10 allocate a temporary buffer (LZ4HC_HEAPMODE) within worker threads :
documented in lz4frame.h (nbWorkers), lz4frame.c (pool) and lz4.c (LZ4_USER_MEMORY_FUNCTIONS
must then be thread-safe).
lz4 : the automatic nb of workers no longer shrinks from one file to the next
(cRess_t.nbWorkers high-water mark) : worker threads and buffers are kept
(-m on 300 files of 64 KB - 8 MB : 1156 -> 21 thread creations).
lz4 : with independent blocks, a file fitting into a single job is compressed by the calling
thread (nbWorkers 0, same output). Not with linked blocks : single-threaded output then depends
on stableSrc (last partial block referencing history in place), so memory-mapped input would differ.
lz4frame : without LZ4F_MULTITHREAD, LZ4F_compressBound() covers a single job slot
(jobs processed serially) : 4 MB instead of 68 MB with linked blocks and nbWorkers 8.
tests : frameMT validateCompressBound ; test-lz4-multithread : single-job files and -m.
LZ4F_getMaxNbWorkers() (static linking only, experimental like nbWorkers) :
0 when liblz4 is built without LZ4F_MULTITHREAD, otherwise the cap of nbWorkers,
now 1024 on 64-bit systems and 128 on 32-bit ones (was 200).
lz4 : build macro LZ4_NBWORKERS_MAX removed ; worker threads of frame and legacy
formats are capped by LZ4F_getMaxNbWorkers(), 1 without liblz4 multi-threading
(restores a cap for the legacy format on POSIX). lz4 -V reports "multithread"
only when both lz4 and liblz4 support it ; -T# warning and -v note accordingly.
VS2022 _test.bat checks the version line, catching a CLI built without LZ4F_MULTITHREAD
(same output, so no other observable difference).
frameMT : validateMaxNbWorkers ; test builds use LZ4F_NBWORKERS_MAX=128.
LZ4F_compressBegin*() validates its arguments first (dictSize included) :
rejecting them leaves a frame in progress untouched, jobs in flight included
(contract of #1817, tested by frametest). Past validation, the frame in progress
is abandoned before cctx is modified : an allocation failure invalidates it
(previously, cStage remained 1 with an inconsistent cctx, also with nbWorkers == 0).
lz4frame.h : LZ4F_srcStableUntilFlush lifetime and LZ4F_compressBegin() failures clarified.
Without autoFlush, LZ4F_compressUpdate() defers completed jobs not fitting into dst,
instead of failing : it only fails when output must be emitted (all job slots busy).
VS2022 : lz4.vcxproj x64 configurations define LZ4F_MULTITHREAD=1.
frameMT : validateDeferredEmission, validateBeginFailures.
lz4io.c : the legacy compression pipeline no longer carries frame-only code
(ReadTracker.prefix and .xxh32, prefixSize parameter, CompressJobDesc.lastBlock).
xxhash.h remains included, for the content checksum of a memory-mapped source.
lz4frame.c, frameMT.c : nbWorkers == 1 means one worker thread, in addition to
the calling thread. Comments only, behavior unchanged.
Prototype, not intended for a PR as is.
v1.10.1 entries for LZ4F_preferences_t.nbWorkers, LZ4F_srcStableUntilFlush, scalar reserved
fields, the CLI relying on liblz4 (memory-mapped input), -T# semantics, memory budget
(--memory, LZ4_MEMORY), detection of sources modified during compression, build policy
(lib-mt / lib-nomt, cmake and meson options) and make install.
The former "multithreaded compression honors custom block sizes" becomes : custom block
sizes are honored, and compressed single-threaded.
Prototype, not intended for a PR as is.
- lz4 CLI : the automatic nb of workers (no -T, or -T0) is limited by a memory budget
(4 GB, 1 GB on 32-bit systems), estimated from liblz4's job slots, input buffers and
LZ4 states, plus the CLI's buffers (within ~5% of actual allocations when all are used).
It's also limited to the nb of jobs of each file, when its size is known
(ex: 3 threads for a 10 MB file, instead of 145 on a 166-core machine).
-9 on 844 MB, 166 cores : unchanged auto (145 workers, ~3 GB budget estimate).
- --memory=# (suffixes K, M, G) and environment variable LZ4_MEMORY set the budget ;
an explicitly set budget also limits an explicit -T#, as does the default one on 32-bit
systems. A budget too small for 1 worker triggers a warning.
- lz4 CLI : no more fixed maximum nb of workers for frames (LZ4_NBWORKERS_MAX now only
bounds the legacy format's thread pool) ; liblz4 keeps a sanity limit, raised to 1024
(200 on 32-bit systems, where it keeps LZ4F_compressBound() within address space),
overridable at compile time (tests : 200, keeping the capped test case affordable).
- liblz4 without LZ4F_MULTITHREAD (ex: default static library) : independent blocks take
the single-threaded path (same output, no job buffers, no extra copies) ; linked blocks
keep jobs processed serially (same output as multi-threaded, ~3% slower at level 1).
- liblz4 : frames cut into jobs no longer allocate tmpBuff (-4.1 MB, LZ4F_cctx_size()).
- liblz4 : LZ4F_uncompressedUpdate() with workers now cuts blocks as single-threaded mode :
an incomplete last block waits for next uncompressed input (it was written immediately,
producing different blocks for consecutive uncompressed invocations). With independent
blocks, output is now identical to nbWorkers == 0 in all cases, and doesn't depend on
multi-threading support. tests/frameMT.c compares uncompressed segments with
single-threaded mode too. Full digest reference : 899dd7e3d21cad9a.
- Documented in man (-T#, --memory, LZ4_MEMORY, LZ4_NBWORKERS), programs/README.md, lz4frame.h.
Validated : full tests MT / no-MT (same digest), ASAN+UBSAN, TSAN, make test, wine,
CLI == dev CLI, lz4-nomt == lz4, memory budget scenarios, modification scenarios,
build policies, ~88k libFuzzer executions.
Prototype, not intended for a PR as is.
Since the previous checkpoint, any modification of a memory-mapped source (default mode)
was an error, so compressing a log file growing meanwhile, which used to work, failed.
A file only grown leaves its mapped part unchanged, and the frame is consistent.
When a modification is detected, the mapped content is hashed again (XXH32) and compared
with the frame's content checksum, computed while reading it :
- equal : the content read is unchanged (ex: log file growing meanwhile) : the output is
valid, holding the file as it was when opened : warning only, source kept with --rm ;
- different, or can't be verified (--no-frame-crc, or file shrunk) : error 54, output removed.
Unmodified files cost nothing more.
Documented in man (--[no-]mmap), -H help and lz4io.h.
Validated : growing log (default mode : valid output), rewrite of content already read
(error 54), rewrite ahead of reading (consistent : accepted), --no-frame-crc (error 54),
--no-mmap (always valid), also with a CLI built with ASAN+UBSAN ; make test, CLI == dev CLI.
Prototype, not intended for a PR as is.
- lz4 CLI : a source file modified during its compression is detected, comparing its size
and times (mtime, ctime) before and after, like tar ("file changed as we read it").
The source is kept, even with --rm.
- memory-mapped input is read several times (checksum, compression, history of next job) :
the frame can be inconsistent : error 54, output removed.
- otherwise (--no-mmap), input is read once : the frame holds the content read,
ex: up to the end of a log file growing meanwhile : warning only.
Documented in man (--[no-]mmap, --rm), -H help and lz4io.h.
- lz4frame.c : a missing or incomplete worker pool (ex: transient allocation failure) is
created again at next frame, instead of staying serial for the context's lifetime.
- lz4frame.c : abandoning a frame (error, LZ4F_compressBegin(), LZ4F_freeCompressionContext())
cancels jobs not started yet, instead of compressing them (and the calling thread helping) :
only running jobs are waited for. Level 12, 4 MB jobs : CPU spent abandoning a frame
1.58 s -> 0.80 s (2 workers), 4.03 s -> 3.50 s (8 workers).
- lz4 CLI : memory-mapped input released behind (2*nbWorkers+1) jobs of 4 MB + 64 KB of history,
instead of nbWorkers+2 chunks (fewer refaults : ~5-8% fewer minor faults at -9 -BD).
- tests/frameMT.c : validatePoolRetry (each absorbed allocation failure, then next frame
creates worker threads again ; fails with the previous lib).
Validated : full tests MT / no-MT, ASAN+UBSAN, TSAN, make test, wine, CLI == dev CLI,
in-place rewrite / growing log / truncation scenarios, ~88k libFuzzer executions.
Prototype, not intended for a PR as is.
programs/Makefile probed pthread with a fixed have_pthread.c in its directory
(pre-existing in dev) : concurrent invocations collided (24 / 192 concurrent probes
wrongly reported no threads, producing an lz4 CLI without multi-threading),
and read-only source trees made it fail.
- build/make/lz4defs.make : PTHREAD_PROBE, evaluated with the CC and ALLFLAGS of
the invoking Makefile, within a private temporary directory (mktemp -d).
- lib/Makefile and programs/Makefile employ it (lib/'s copy from previous commit removed).
Validated : probes of lib/ and programs/ (192 concurrent, read-only trees), full tests
MT / no-MT, ASAN+UBSAN, TSAN, make test, wine, CLI build (lz4 multithread, lz4-nomt not),
build policies, install scenarios + test_install.sh.
Prototype, not intended for a PR as is.
- lz4frame.c (Win32) : _WIN32_WINNT raised to 0x0600 (condition variables) when lower,
instead of only when undefined : <_mingw.h>, included by standard headers, defines it
to 0x502 on older mingw-w64, where CONDITION_VARIABLE was then undeclared.
A higher user-defined value is preserved.
- lib/Makefile : the pthread probe runs within a private temporary directory (mktemp -d),
instead of a fixed have_pthread.c in the source tree : concurrent invocations no longer
collide (35 / 320 concurrent probes wrongly reported no threads, producing liblz4.so
without multi-threading), and read-only source trees are supported.
The build message mentions "(threads not detected)" when the probe fails.
- make install, as zstd : installs the libraries already built (lib, lib-mt, lib-nomt),
and only builds them, with current settings, when absent :
`make lib-mt && make install` now installs the multi-threaded libraries.
liblz4.pc declares -pthread (Libs.private) when the installed liblz4.a requires it,
deduced from its symbols (nm) ; when they can't be listed, whenever threads are available.
Root Makefile : install is ordered after lib goals of the same command line
(make -j all install, make -j lib-mt install), instead of always rebuilding lib.
- lib/README.md documents it.
Validated : full tests MT / no-MT, ASAN+UBSAN, TSAN, make test, wine, build policies,
Windows DLL through lib/Makefile (imports CreateThread), mingw with _WIN32_WINNT=0x0502,
10 install scenarios + test_install.sh, concurrent and read-only probes.
Prototype, not intended for a PR as is.
- LZ4F_preferences_t : reserved array replaced by explicit scalar fields
(nbWorkers, reserved1, reserved0), and LZ4F_compressOptions_t likewise
(stableSrc, reserved2, reserved1, reserved0), as LZ4F_decompressOptions_t.
Same sizes and offsets (ABI unchanged). A future field takes the place of a
reserved one, without changing positional initializers.
One-time source change : the former positional form { ..., { 0, 0, 0 } }
no longer compiles in C++ (warnings in C) ; examples/frameCompress.c updated.
LZ4F_INIT_PREFERENCES : { LZ4F_INIT_FRAMEINFO, 0, 0u, 0u, 0u, 0u, 0u }.
- LZ4F_srcStableUntilFlush (2) is stronger than LZ4F_srcStable (1) : in addition
to the last 64 KB of input (which remain, including after LZ4F_flush()),
all of src remains until LZ4F_flush() / LZ4F_compressEnd() returns
(with autoFlush : until the invocation returns). Docs only : the code already
relied on this (history read in place after a flush, single-threaded path).
- tests/frameMT.c : validateFlushContract : one buffer, 2 invocations separated by
a flush, everything but the last 64 KB scrambled after the flush with content
resembling upcoming data ; nbWorkers 0 and 2, levels 1 and 9, with and without
autoFlush. Scrambling the last 64 KB too is detected (contentChecksum_invalid).
Streaming test : LZ4F_srcStableUntilFlush slices come from a working copy,
scrambled the same way after each flush.
Output unchanged (same digests). Validated : full tests MT / no-MT, ASAN+UBSAN, TSAN,
make test, wine, examples (C, C++), cxxtest, CLI == dev CLI.
Prototype, not intended for a PR as is.
A memory-mapped file truncated during its compression, or unreadable (I/O error),
raises SIGBUS in the thread reading it. Worker threads were blocking all signals,
so a fault in a worker killed the process, bypassing any handler.
- lz4frame.c : worker threads still block signals intended for the application,
but not synchronous ones (SIGBUS, SIGSEGV, SIGFPE, SIGILL) : the application's
handlers are invoked on faults (documented in lz4frame.h, nbWorkers).
- lz4 CLI : while a file is memory-mapped, a SIGBUS handler reports
"Error 53 : <file> : file truncated or unreadable during compression (memory-mapped) ;
retry with --no-mmap", then terminates the process (async-signal-safe : write, _exit).
The message is prepared beforehand ; when several threads fault simultaneously,
only the first one reports. Previous handler restored once unmapped.
- -H help, man page (--[no-]mmap) and lz4io.h updated.
Truncation test (silesia x4 truncated during -9 -T4) : error 53 with one message,
including --no-frame-crc (workers read the mapping first) ; with the former signal
mask, the process was killed by SIGBUS. --no-mmap : unchanged.
Output and speed unchanged. Validated : full tests MT / no-MT, ASAN+UBSAN, TSAN,
make test, wine (frameMT, lz4.exe == Linux CLI), CLI == dev CLI.
Prototype, not intended for a PR as is.
Historically, stableSrc only guarantees that the last 64 KB of input remain
present (linked blocks reference them instead of copying them). The MT engine
was reading whole jobs in place after LZ4F_compressUpdate() returns, silently
extending this contract : code correct with nbWorkers == 0 (buffer reused with
independent blocks, alternating buffers with linked blocks) became racy once
nbWorkers is enabled.
- lz4frame.h : enum LZ4F_stableSrc_t, the field remains unsigned (C++ compatibility) :
- LZ4F_srcUnstable (0) : default ;
- LZ4F_srcStable (1) : historical meaning, last 64 KB of input ;
- LZ4F_srcStableUntilFlush (2) : all of src remains until LZ4F_flush() / LZ4F_compressEnd()
returns, or the frame is abandoned : MT jobs read it in place.
- single-threaded behavior is unchanged : any non-zero value means stable.
- MT : only exactly LZ4F_srcStableUntilFlush enables in-place reading after return ;
otherwise job content is copied. autoFlush still reads in place (jobs complete within the call).
- one-shot compression and the CLI (memory-mapped input) employ LZ4F_srcStableUntilFlush.
- tests/frameMT.c : the three values ; with LZ4F_srcStable and workers, input is scrambled
after each invocation (a mutant reading it in place is detected, also by TSan).
- compress_frame_stream_fuzzer : stableSrc in 0..2.
Output unchanged (same digests), CLI mmap speed unchanged.
Validated : full tests MT / no-MT, ASAN+UBSAN, TSAN, make test, wine, CLI == dev CLI,
~88k libFuzzer executions.
Prototype, not intended for a PR as is.
- FORWARD_IF_ERROR() used to evaluate its argument twice ; applied to LZ4F_mt_begin(),
an allocation failure re-ran it, and LZ4F_compressBegin() reported success without
starting the frame. Found by allocation failure injection.
Fixed in dev by #1877 (single evaluation) ; FORWARD_IF_ERROR_MT() now follows the same pattern.
- tests/frameMT.c :
- each allocation fails in turn (one-shot, streaming copy / stableSrc, levels 1 and 9) :
clean allocation_failed or correct output, no leak, context remains usable ;
- LZ4F_write() / LZ4F_read() with workers : same frame as one-shot ;
- quick mode (-q), step labels, FRAMEMT_DUMP diagnostics ;
- CDict now created after its dictionary is filled (found by ASAN).
- make test runs test-frameMT-quick ; CI sanitizers : ASan and TSan steps.
- ossfuzz : compress_frame_stream_fuzzer employs different nbWorkers for reference and test,
comparing whole frames when output lags behind input ; round_trip_frame_fuzzer : random nbWorkers.
- LZ4F_MT_JOBSIZE_* can be overridden (testing only), to cross job boundaries with small inputs.
- lz4frame.h : per-invocation output depends on scheduling (without autoFlush).
Validated : full tests MT / no-MT (same digest), ASAN+UBSAN, TSAN, make test,
mingw-w64 build under wine (same digest as Linux), ~160k libFuzzer executions.
Prototype, not intended for a PR as is.
- regular files are memory-mapped (POSIX 64-bit, frames cut into jobs only) and given to
LZ4F_compressUpdate() with stableSrc : compression jobs read input in place, removing
two copies from the main thread (fread, then copy into job buffers).
- on Linux, input far behind jobs in flight is released (madvise MADV_DONTNEED) :
only a memory hint, released pages are read again from the file if still needed.
- --no-mmap disables it : a file truncated while being compressed makes lz4 crash (SIGBUS).
- output is identical with and without mmap.
- -T# : # worker threads ; the main thread also compresses while waiting (up to #+1 cores).
- custom block sizes (-B# other than 64KB / 256KB / 1MB / 4MB) remain single-threaded :
documented in -H help, man page (-T#, -B#), and a note with -v.
Level 1, -T16, silesia.tar : 82 ms (previous CLI 118 ms, --no-mmap 162 ms).
Prototype, not intended for a PR as is.
- input buffers form a pool (nbWorkers+1), shared by job slots :
a buffer is reclaimed as soon as its job is compressed, instead of when its output is emitted;
when none is free, the calling thread waits for any job to complete (job durations vary).
- job slots : 2*nbWorkers+1 (was +2), which also shrinks LZ4F_compressBound().
- stableSrc : when an input directly follows the previous one, history is read in place,
so jobs starting at an invocation boundary aren't copied anymore.
- the calling thread runs queued jobs while waiting only if enough jobs remain queued
to keep worker threads busy (it can't feed them while compressing), or once input is over.
CLI max RSS, level 1 : -T1 44 -> 29 MB, -T8 129 -> 80 MB, -T64 280 -> 107 MB (previous CLI : 13 / 55 / 212 MB);
HC speed back to previous CLI's level (-9 -T16 : 583 ms vs 582 ms).
Prototype, not intended for a PR as is.
- persistent thread pool within LZ4F_cctx (pthread / Win32), replaces the per-call fork-join;
nbWorkers is now the nb of worker threads (the calling thread also runs jobs while waiting).
- streaming : LZ4F_compressUpdate() / LZ4F_flush() / LZ4F_compressEnd() cut input into jobs
(4 MB linked, 1 MB independent), at fixed positions of the content; outputs emitted in order;
input read in place with stableSrc or autoFlush, copied otherwise; autoFlush respected;
LZ4F_compressBound() covers jobs in flight; LZ4F_uncompressedUpdate() drains pending jobs.
- one-shot compression goes through the same engine.
- CLI : frame compression goes through LZ4F streaming with nbWorkers (output identical to
previous CLI, and lz4 == lz4-nomt); custom block sizes fall back to single-thread.
- build systems compile the CLI's liblz4 sources with LZ4F_MULTITHREAD.
- tests/frameMT.c : streaming validation (slicing, stableSrc, flushes, autoFlush,
uncompressed segments, exact bounds, abandoned frames).
Prototype, not intended for a PR as is.
- LZ4F_preferences_t.nbWorkers (replaces reserved[0], ABI preserved):
LZ4F_compressFrame() and LZ4F_compressFrame_usingCDict() cut input into jobs,
compressed by up to nbWorkers threads (inline fork-join, threads created per call).
Output only depends on parameters, never on nbWorkers (>= 1) nor scheduling;
identical to single-threaded with independent blocks, identical to the CLI -T output.
- LZ4F_MULTITHREAD build macro (default 0) : without it, jobs run serially, same output.
- build policy (zstd-like) : dynamic library multi-threaded by default, static library not
(make lib-mt / lib-nomt / MULTITHREAD=, cmake LZ4_SHARED/STATIC_MULTITHREAD,
meson shared-/static-multithread, VS2022 liblz4-dll).
- internal LZ4F_compressBlocks_internal() extracted from LZ4F_compressUpdateImpl().
- tests/frameMT.c (make -C tests test-frameMT) : validation + bench mode.
Known gaps : Win32 thread path never compiled; no persistent thread pool; no streaming.
LZ4MID_compress() checked ip+1 for a longer match only after a match
found by the 4-byte table within the prefix, never after a match found
by the 8-byte table. The check now follows any match shorter than 8 bytes,
also within an external dictionary, and skips the candidate ip itself:
inside runs, ip+1 would find ip at offset 1, and periodic data would
lock onto one literal and one offset 1 per run (4 KB to ~1.6 KB).
The previous check is removed.
Compressed size : linux-6.12 tar -1.3%, Silesia -0.6%, enwik9 -0.5%
(4 MB blocks), -0.25% (4 KB blocks), -0.9% (64 KB linked blocks),
-0.2% (4 KB pages with a 64 KB dictionary).
Speed, AMD EPYC Genoa (Zen 4), x86-64, gcc 11 / gcc 15 / clang 22 :
-0.5 to -1.3% (256 KB blocks), -1.3 to -2% (4 KB blocks),
-1.8 to -2.5% (64 KB linked blocks), -0.2 to -1.2% (4 KB pages with a dictionary).
x86 (gcc 12 -m32) : +1.4% (4 KB pages with a dictionary) to -4.9% (64 KB linked blocks).
After each match, the fast parser only tested the most recent occurrence
of the sequence at ip. On periodic data with long runs, it is a short
match, and such matches can chain until the end of the input :
an 8-byte dictionary made a 4 KB page compress to 1543 bytes instead
of 49 (#1805). It also happens without dictionary.
The position following the previous occurrence of ip-1 is now tested
first, also within the dictionary, and taken if it matches 8+ bytes.
The table is filled with ip-1 instead of ip-2.
Both candidates share the same dictionary / current-block selection.
Silesia, compressed size : -0.6% (256 KB blocks), -0.3% (4 KB blocks),
-0.5% (4 KB pages with a 64 KB dictionary).
Speed, AMD EPYC Genoa (Zen 4), x86-64, gcc 11 / gcc 15 / clang 22 :
-0.8 to -1.2% (256 KB blocks), -1.7 to +1.5% (4 KB blocks),
-6 to +1% (4 KB pages with a dictionary).
x86 (gcc 12 -m32) : +1% (4 KB blocks) to -11% (dictionary).
Closes#1805
LZ4_compress_fast_continue() copied the tables of an attached dictionary
(one table to look into) only above 4 KB, and used the dictCtx loop
(two tables) below. The copy is now faster from 1/8 of the table size:
from 2 KB with the default table, capped at 4 KB for large tables.
The compressed output is unchanged.
AMD EPYC Genoa (Zen 4), x86-64, 2-4 KB inputs:
gcc 11 +3-5%, gcc 15 +5-8%, clang 22 +16-31%.
The automatic output name of decompression removed the .lz4 extension
with an obscure loop, comparing the last 5 characters (terminating zero
included) while zeroing them : it's now a plain suffix check.
