Prototype, not intended for a PR as is.
- lz4frame.c (Win32) : _WIN32_WINNT raised to 0x0600 (condition variables) when lower,
instead of only when undefined : <_mingw.h>, included by standard headers, defines it
to 0x502 on older mingw-w64, where CONDITION_VARIABLE was then undeclared.
A higher user-defined value is preserved.
- lib/Makefile : the pthread probe runs within a private temporary directory (mktemp -d),
instead of a fixed have_pthread.c in the source tree : concurrent invocations no longer
collide (35 / 320 concurrent probes wrongly reported no threads, producing liblz4.so
without multi-threading), and read-only source trees are supported.
The build message mentions "(threads not detected)" when the probe fails.
- make install, as zstd : installs the libraries already built (lib, lib-mt, lib-nomt),
and only builds them, with current settings, when absent :
`make lib-mt && make install` now installs the multi-threaded libraries.
liblz4.pc declares -pthread (Libs.private) when the installed liblz4.a requires it,
deduced from its symbols (nm) ; when they can't be listed, whenever threads are available.
Root Makefile : install is ordered after lib goals of the same command line
(make -j all install, make -j lib-mt install), instead of always rebuilding lib.
- lib/README.md documents it.
Validated : full tests MT / no-MT, ASAN+UBSAN, TSAN, make test, wine, build policies,
Windows DLL through lib/Makefile (imports CreateThread), mingw with _WIN32_WINNT=0x0502,
10 install scenarios + test_install.sh, concurrent and read-only probes.
Prototype, not intended for a PR as is.
- LZ4F_preferences_t.nbWorkers (replaces reserved[0], ABI preserved):
LZ4F_compressFrame() and LZ4F_compressFrame_usingCDict() cut input into jobs,
compressed by up to nbWorkers threads (inline fork-join, threads created per call).
Output only depends on parameters, never on nbWorkers (>= 1) nor scheduling;
identical to single-threaded with independent blocks, identical to the CLI -T output.
- LZ4F_MULTITHREAD build macro (default 0) : without it, jobs run serially, same output.
- build policy (zstd-like) : dynamic library multi-threaded by default, static library not
(make lib-mt / lib-nomt / MULTITHREAD=, cmake LZ4_SHARED/STATIC_MULTITHREAD,
meson shared-/static-multithread, VS2022 liblz4-dll).
- internal LZ4F_compressBlocks_internal() extracted from LZ4F_compressUpdateImpl().
- tests/frameMT.c (make -C tests test-frameMT) : validation + bench mode.
Known gaps : Win32 thread path never compiled; no persistent thread pool; no streaming.
`install` builds the library and the CLI from its own sub-makes,
in lib/ and programs/, with no ordering relative to the `lib` and `lz4` targets.
When both are requested (`make all install`, parallel by default),
2 concurrent sub-makes build the same files in each directory's cachedObjs/,
and one of them can use an object, library or binary that the other one is still writing :
the link fails, or an empty or incomplete file gets installed.
Building `lib` and `lz4` first leaves nothing to rebuild for the install sub-makes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
and make it build the MT version of lz4
with an option that is enabled by default but can be disabled on demand.
Also :
- added make target mesonbuild, for easy local build
- made huge tests friendlier to MT variant, for faster testing time
To support older compiler which doesn't have explicit dialect option
for C90 (gcc-4.4), this change set split "c_standards" into multiple
part.
Original "make c_standards" still works as intended. But this change
gives extra freedom of choice for external program For example, CI
can choose test for standards which is supported by specimen compiler.
With this separation, we can also introduce C17 smoothly.
The following command doesn't work as intended
```
cd
git clone https://github.com/lz4/lz4.git
cd lz4
make usan MOREFLAGS='-Wcomma -Werror'
```
```
Cleaning completed
cc: error: unrecognized command line option ‘-Wcomma’; did you mean ‘-Wcomment’?
make[3]: *** [<builtin>: ../lib/lz4.o] Error 1
make[2]: *** [Makefile:65: lz4] Error 2
make[1]: *** [Makefile:133: test] Error 2
make: *** [Makefile:158: usan] Error 2
```
Because the following part of the `Makefile` doesn't propagate `CC`, `CFLAGS` and `LDFLAGS` to child `$(MAKE)` properly.
```
.PHONY: usan
usan: CC = clang
usan: CFLAGS = -O3 -g -fsanitize=undefined -fno-sanitize-recover=undefined -fsanitize-recover=pointer-overflow
usan: LDFLAGS = $(CFLAGS)
usan: clean
$(MAKE) test FUZZER_TIME="-T30s" NB_LOOPS=-i1
```
We need explicit propagation for child process
```
- $(MAKE) test FUZZER_TIME="-T30s" NB_LOOPS=-i1
+ CC=$(CC) CFLAGS=$(CFLAGS) LDFLAGS=$(LDFLAGS) $(MAKE) test FUZZER_TIME="-T30s" NB_LOOPS=-i1
```
After that, `make usan` works and it shows expected runtime error.
```
$ make usan MOREFLAGS='-Wcomma -Werror'
Cleaning completed
../lib/lz4frame.c:907:25: runtime error: applying non-zero offset 65536 to null pointer
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../lib/lz4frame.c:907:25 in
../lib/lz4frame.c:907:58: runtime error: applying zero offset to null pointer
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../lib/lz4frame.c:907:58 in
...
```
Please note that `make usan` is working at travis-ci. Because `.travis.yml` has the following [explicit `compiler:` setting](7a966c1511/.travis.yml (L66)).
```
- name: (Trusty) USan test
dist: trusty
compiler: clang
script:
- make usan MOREFLAGS=-Wcomma -Werror
```
the way `base` is used must allow 2-complement address overflow.
`base` is effectively a virtual pointer, never dereferenced directly.
What matters is `base + index`.
This is the address that will be dereference and must be valid (it is properly validated).
- only play listTest with `make test`, not `make all` which is limited to build
- update `clangtest`, so that it's possible to disable O3 optimization, for faster processing
