1
0
mirror of https://github.com/facebook/zstd.git synced 2025-04-19 00:44:19 +03:00

27 Commits

Author SHA1 Message Date
Yann Collet
72ce56b527 fixed another invalid scenario
compressSequencesAndLiterals() doesn't support sequence validation
2024-12-23 21:15:50 -08:00
Yann Collet
7b294caf46 add one valid test case
ZSTD_compressSequencesAndLiterals() may return a specific error code
when data to compress is non-compressible.
2024-12-23 19:43:17 -08:00
Yann Collet
f8725e80cc added fuzzer test for compressSequencesAndLiterals()
piggy-backing onto existing compressSequences() fuzzer test
2024-12-23 18:42:51 -08:00
Yann Collet
c97522f7fb codemod: ZSTD_sequenceFormat_e -> ZSTD_SequenceFormat_e
since it's a type name.

Note: in contrast with previous names, this one is on the Public API side.
So there is a #define, so that existing programs using ZSTD_sequenceFormat_e still work.
2024-12-20 10:36:56 -08:00
Nick Terrell
81a5e5d438 [fuzz] Turn off -Werror by default
This was causing OSS-Fuzz errors, due to compiler differences.
* Fix the issue
* Also turn off -Werror so we don't fail fuzzer builds for warnings
* Turn on -Werror in our CI
2024-03-26 16:34:36 -07:00
Nick Terrell
43118da8a7 Stop suppressing pointer-overflow UBSAN errors
* Remove all pointer-overflow suppressions from our UBSAN builds/tests.
* Add `ZSTD_ALLOW_POINTER_OVERFLOW_ATTR` macro to suppress
  pointer-overflow at a per-function level. This is a superior approach
  because it also applies to users who build zstd with UBSAN.
* Add `ZSTD_wrappedPtr{Diff,Add,Sub}()` that use these suppressions.
  The end goal is to only tag these functions with
  `ZSTD_ALLOW_POINTER_OVERFLOW`. But we can start by annoting functions
  that rely on pointer overflow, and gradually transition to using
  these.
* Add `ZSTD_maybeNullPtrAdd()` to simplify pointer addition when the
  pointer may be `NULL`.
* Fix all the fuzzer issues that came up. I'm sure there will be a lot
  more, but these are the ones that came up within a few minutes of
  running the fuzzers, and while running GitHub CI.
2023-09-28 17:35:05 -04:00
Elliot Gorokhovsky
a810e1eeb7 Provide an interface for fuzzing sequence producer plugins 2023-03-28 12:02:57 -07:00
Danielle Rozenblit
7d600c628a fix bound check for ZSTD_copySequencesToSeqStoreNoBlockDelim() 2023-01-24 06:40:40 -08:00
Danielle Rozenblit
7fc00c18b8 calloc dictionary in sequence compression fuzzer rather than generating a random buffer 2023-01-23 10:42:09 -08:00
Danielle Rozenblit
638d502002 modify sequence compression api fuzzer 2023-01-23 07:55:11 -08:00
W. Felix Handte
8927f985ff Update Copyright Headers 'Facebook' -> 'Meta Platforms'
```
for f in $(find . \( -path ./.git -o -path ./tests/fuzz/corpora \) -prune -o -type f);
do
  sed -i 's/Facebook, Inc\./Meta Platforms, Inc. and affiliates./' $f;
done
```
2022-12-20 12:37:57 -05:00
Yann Collet
678bfff4fe fix minor bug in sequence_compression_api tester
margin was merely slightly too short for extra splitting.
2022-03-29 16:45:09 -07:00
Dominique Pelle
b772f53952 Typo and grammar fixes 2022-03-12 08:58:04 +01:00
Yann Collet
9a758ce520 update sequence_compression_api fuzzer test
to check for under-sized dstCapacity.
2022-01-31 16:17:11 -08:00
Yann Collet
637b2d7a24 fixed bug 44168
discovered by oss-fuzz

It's a bug in the test itself :
ZSTD_compressBound() as an upper bound of the compress size
only works for data compressed "normally".
But in situations where many flushes are forcefully introduced,
this creates many more blocks,
each of which has a potential to increase the size by 3 bytes.
In extreme cases (lots of small incompressible blocks), the expansion can go beyond ZSTD_compressBound().

This situation is similar when using the CompressSequences() API
with Explicit Block Delimiters.
In which case, each explicit block acts like a deliberate flush.
When employed by a fuzzer, it's possible to generate scenarios like the one described above,
with tons of incompressible blocks of small sizes,
thus going beyond ZSTD_compressBound().

fix : when using Explicit Block Delimiters, use a larger bound, to account for this scenario.
2022-01-29 16:36:20 -08:00
Yann Collet
d64d5ddc57 fix 44122 test error
It's a bug in the test itself, in exceptional circumstances (no more space for additional sequence).

There should be enough room for all cases to work fine from now on,
and if not, we have an additional `assert()` to catch that situation.
2022-01-27 14:54:18 -08:00
Yann Collet
fc2ea97442 refactored fuzzer tests for sequence compression api
add explicit delimiter mode to libfuzzer test
2022-01-26 00:19:35 -08:00
Yann Collet
8da414231d found a few more places which were dependent on seqStore offcode sumtype numeric representation 2021-12-28 17:03:24 -08:00
Nick Terrell
a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
Nick Terrell
66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
senhuang42
4c5f337248 Use cctx's minMatch instead of global MINMATCH, make fuzzer use validation 2020-11-30 15:41:20 -05:00
senhuang42
23554ff25f Force CCtx minmatch to be same as generated minmatch 2020-11-23 13:29:20 -05:00
senhuang42
c502cd33e5 Fix generating 1 too few characters in random string generator 2020-11-20 16:58:25 -05:00
senhuang42
5b0c8f0a7c Add appropriate bound to matchlengths, and reduce srcSize max 2020-11-20 16:58:25 -05:00
senhuang42
a73a07b189 Add a bound for matchlength dependent on window size 2020-11-20 16:58:25 -05:00
senhuang42
5c68c5e31e Variety of minor fixups, reduce allocation, make deterministic 2020-11-20 16:58:25 -05:00
senhuang42
ed575963c5 Implement new fuzzer for sequence compression 2020-11-20 16:58:25 -05:00