decompress_frame_fuzzer, round_trip_frame_fuzzer and compress_frame_stream_fuzzer
now enable the skipChecksums decompression option for about 1 input in 8.
state_attachDictRoundTrip() and state_attachDictHCRoundTrip() used the result
of LZ4_createStream() / LZ4_createStreamHC() without checking it.
On allocation failure, the NULL stream was dereferenced inside lz4.c / lz4hc.c,
showing up as a library crash instead of a failed assert in the harness.
Same issue as #1813, in the dictionary paths.
state_create() asserted state.cstream a second time instead of the
new state.cstreamHC, so a failed LZ4_createStreamHC() went unnoticed and
state_reset() then passed NULL to LZ4_resetStreamHC_fast().
Fixes#1813
It consumed 4 bytes, but only returned the first one (`*(data + size - 4)` is a byte),
since 8edc5879 (2019).
As a consequence, all values derived from it were limited to 0-255 :
most capacities, offsets and dictionary sizes chosen by the fuzzers
only covered a small fraction of their intended range.
It now returns the 4 bytes, combined in little-endian order.
The low byte is unchanged and the number of consumed bytes is the same,
so values within power-of-2 ranges <= 256 (such as boolean flags) are unaffected,
limiting the impact on existing corpora.
Also : FUZZ_dataProducer_range32() stores the seed as uint32_t (was size_t).
`make -C ossfuzz test` checks that FUZZ_dataProducer_retrieve32()
reads 32-bit values from the end of input,
and that FUZZ_dataProducer_range32() can produce any value of its range.
This test fails on current dev :
FUZZ_dataProducer_retrieve32() only returns a single byte,
so range32() never produces values > 255.
Bounds should guarantee success, without being requirements :
compression must succeed if and only if its output fits,
and then produce the same output as with a bound-sized buffer.
frametest unit tests :
- LZ4F_compressBegin*() into exactly the header size (7-19 bytes),
and a failed begin doesn't disturb a frame in progress
- LZ4F_compressFrame() and LZ4F_compressFrame_usingCDict() :
exact frame size succeeds with identical output, any smaller capacity fails
(every capacity for small frames, element boundaries otherwise)
- streaming : random scripts of updates, uncompressed updates and flushes;
each operation into exactly its output size, failures just before element boundaries,
frame invalidated after a failure, context reusable for a new frame
- LZ4F_flush() and LZ4F_compressEnd() require a frame in progress
- mixed updates : rewrite the insufficient capacity test
frametest fuzzer : random undersized capacities for compressFrame, updates, flushes and compressEnd
ossfuzz :
- compress_frame_fuzzer : success if and only if the frame fits, identical output
- new compress_frame_stream_fuzzer : streaming with random capacities,
in lockstep with a reference context receiving bound-sized buffers
- round_trip_frame_uncompressed_fuzzer : pass remaining capacity
Note : byte-exact comparisons start from fresh contexts,
because output of linked blocks at fast levels depends on context history
(pre-existing behavior, unrelated to capacity).
These tests fail on current dev.
* replace assert with test for LZ4F_uncompressedUpdate
* update documentation to incldue correct docstring
* remove unecessary entry point
* remove compress_linked_block_mode from fuzzing test
Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
Summary: Consuming bytes from the end of data instead of from the front to prevent "all-in-one" decisions.
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
* Run more decompression variants
* Round trip the compression fuzzer and do partial decompression as well
* Add a compression fuzzer that compresses into a smaller output buffer
and test the destSize variant
These fuzzers caught 2 bugs that were fixed in the previous commit.
* Input buffer over-read in partial decompress
* Partial decompress fails if output size is 0
