lib/zstd
1
0
Fork 0
mirror of https://github.com/facebook/zstd.git synced 2025-08-05 19:15:58 +03:00
Code Activity
10,385 Commits 30 Branches 77 Tags
solaris_test
Commit Graph

10385 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet
ba50807029 make the bitstream generate only 0-value bits after an overflow 2023-06-14 15:42:37 -07:00
Yann Collet
bd00ec6804 Merge pull request #3674 from facebook/zeroSeq_noExtra 
detect extraneous bytes in the Sequences section
 2023-06-13 16:17:07 -07:00
Yann Collet
b46236278a detect extraneous bytes in the Sequences section 
when nbSeq == 0.

Reported by @ip7z
 2023-06-13 11:43:45 -07:00
Yann Collet
c8888d9bf4 Merge pull request #3671 from facebook/dependabot/github_actions/actions/checkout-3.5.3 
Bump actions/checkout from 3.5.2 to 3.5.3
 2023-06-12 09:58:53 -07:00
dependabot[bot]
6579f6c452 Bump actions/checkout from 3.5.2 to 3.5.3 
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.2 to 3.5.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](8e5e7e5ab8...c85c95e3d7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-06-12 06:00:14 +00:00
Yann Collet
ba12054273 Merge pull request #3669 from facebook/zeroSeq_2B 
fixed decoder behavior when nbSeqs==0 is encoded using 2 bytes
 2023-06-06 00:06:15 -04:00
Yann Collet
3732a08f5b fixed decoder behavior when nbSeqs==0 is encoded using 2 bytes 
The sequence section starts with a number, which tells how sequences are present in the section.
If this number if 0, the section automatically ends.

The number 0 can be represented using the 1 byte or the 2 bytes formats.
That's because the 2-bytes formats fully overlaps the 1 byte format.

However, when 0 is represented using the 2-bytes format,
the decoder was expecting the sequence section to continue,
and was looking for FSE tables, which is incorrect.

Fixed this behavior, in both the reference decoder and the educational behavior.

In practice, this behavior never happens,
because the encoder will always select the 1-byte format to represent 0,
since this is more efficient.

Completed the fix with a new golden sample for tests,
a clarification of the specification,
and a decoder errata paragraph.
 2023-06-05 16:03:00 -07:00
Yann Collet
3e815f5b3a Merge pull request #3664 from facebook/llu 
changed LLU suffix into ULL for Visual 2012 and lower
 2023-06-05 15:03:27 -04:00
Yann Collet
8030342eea Merge pull request #3659 from facebook/fixHarness 
Fixed a bug in the educational decoder
 2023-06-05 15:03:14 -04:00
Yann Collet
f60100dd5f Merge pull request #3668 from facebook/fix3667 
fix a minor inefficiency in compress_superblock
 2023-06-05 15:02:55 -04:00
Yann Collet
1f83b7cfc4 fix a minor inefficiency in compress_superblock 
and in `decodecorpus`:
the specific case `nbSeq=127` can be represented using the 1-byte format.
Note that both the 1-byte and the 2-bytes formats are valid to represent this case,
so there was no "error", produced data remains valid,
it's just that the 1-byte format is more efficient.

fix #3667

Credit to @ip7z for finding this issue.
 2023-06-05 09:51:52 -07:00
Gregor Jasny
7e09f07b32 Fix Intel Xcode builds with assembly 
When forcing the source file language to `C`, Xcode enforces
the file to be compiled as `C` by appending `-x c` to the
compiler command line.

For now try to limit the damage and only enforce the language
if the ASM and C compilers differ.

Reproducer (CMake `3.26.4`, Xcode `14.3`):
```
cmake -S build/cmake -B _b -GXcode -DCMAKE_OSX_ARCHITECTURES=x86_64
cmake --build _b
```

Fix: #3622
 2023-06-02 12:43:59 +02:00
Yann Collet
94a2f2791f changed LLU suffix into ULL for Visual 2012 and lower 
both suffixes are supposed to be valid,
but for some reason, Visual 2012 and lower only support ULL.
 2023-05-31 13:29:53 -07:00
Yann Collet
5108c9ac97 Fixed a bug in the educational decoder 
Credit to Igor Pavlov
 2023-05-27 11:22:30 -07:00
Tao He
5059618295 Add options in Makefile to cmake 
Signed-off-by: Tao He <sighingnow@gmail.com>
 2023-05-26 13:45:21 -07:00
Nick Terrell
d01a2c6929 Fix UBSAN issue (zero addition to NULL) 
Fix UBSAN issue that came up internally.
 2023-05-26 13:43:47 -07:00
Yann Collet
23a0643ef1 Merge pull request #3631 from facebook/example_streaming_mt 
minor : update streaming_compression example
 2023-05-23 20:38:28 -04:00
Duncan Horn
1b994cbc57 Get zstd working with ARM64EC on Windows 2023-05-23 18:40:31 -04:00
Felix Handte
ed6953772f Merge pull request #3623 from felixhandte/compile-out-compressors 
Allow Build-Time Exclusion of Individual Compression Strategies
 2023-05-22 17:38:10 -04:00
W. Felix Handte
1b65803fe7 Reorder Definitions in zstd_opt.c to Group Under Macro Guards (Slightly) 2023-05-22 12:41:48 -04:00
W. Felix Handte
59c7b2a492 Reorder Definitions in zstd_lazy.c to Group Under Macro Guards 2023-05-22 12:37:03 -04:00
W. Felix Handte
5490c75dda Also Allow/Document/Test Excluding dfast and Up 2023-05-04 12:31:41 -04:00
W. Felix Handte
cc1ffe0bd6 Add Documentation to lib/README.md 2023-05-04 12:20:02 -04:00
W. Felix Handte
eb9227935e Also Reorganize Zstd Opt Declarations 2023-05-04 12:18:58 -04:00
W. Felix Handte
d09f195ceb Remove blockCompressor NULL Checks 2023-05-04 12:18:58 -04:00
W. Felix Handte
b7add1dd67 Abort if Unsupported Parameters Used 2023-05-04 12:18:58 -04:00
W. Felix Handte
f242f5be8f Re-Order Lazy Declarations; Minimize ifndefs 2023-05-04 12:18:58 -04:00
W. Felix Handte
698af84fcf Add CI Test for Excluding Matchfinders 2023-05-04 12:18:58 -04:00
W. Felix Handte
bae174960b Add ZSTD_LIB_EXCLUDE_COMPRESSORS_DFAST_AND_UP Build Variable 2023-05-04 12:18:58 -04:00
W. Felix Handte
39b7946b95 Define Macros for Possibly-Present Functions; Use Them Rather than Ifdef Guards 2023-05-04 12:18:58 -04:00
W. Felix Handte
b12e8cb3e7 Merge Ultra and Ultra2 Exclusion 
Ultra2 does not exist for dict compression, and so uses ultra. So ultra must
be present if ultra2 is.
 2023-05-04 12:18:58 -04:00
W. Felix Handte
6761e1c949 Tweak Ultra/Opt Guards 2023-05-04 12:18:58 -04:00
W. Felix Handte
16bbd7437c Avoid Ratio Regression Tests When Compressors are Excluded 2023-05-04 12:18:58 -04:00
W. Felix Handte
5a75956001 Adjust Strategy in CParams to Avoid Using Excluded Block Compressors 2023-05-04 12:18:58 -04:00
W. Felix Handte
50cdf84f58 Macro-Exclude Block Compressors from Declaration/Definition 2023-05-04 12:18:58 -04:00
W. Felix Handte
81b86a2024 NULL Out Block Compressor Table Entries When Excluded 
Don't check about excluding `ZSTD_fast`. It's always included so that we know
we can resolve downwards and hit a strategy that's present.
 2023-05-04 12:18:58 -04:00
W. Felix Handte
cbf3e26316 Allow ZSTD_selectBlockCompressor() to Return NULL 
Return an error rather than segfaulting.
 2023-05-04 12:18:58 -04:00
Felix Handte
7806d80338 Merge pull request #3634 from facebook/dependabot/github_actions/github/codeql-action-2.3.2 
Bump github/codeql-action from 2.3.0 to 2.3.2
 2023-05-01 08:06:07 -07:00
dependabot[bot]
2a5076d264 Bump github/codeql-action from 2.3.0 to 2.3.2 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.3.0 to 2.3.2.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](b2c19fb9a2...f3feb00acb)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-05-01 06:04:02 +00:00
Yann Collet
6ec18aed31 minor : update streaming_compression example 
display a warning when requesting multi-threading
while linking to a library that doesn't support multi-threading.
 2023-04-26 12:45:23 -07:00
Elliot Gorokhovsky
0525d1cec6 Merge pull request #3629 from facebook/dependabot/github_actions/github/codeql-action-2.3.0 
Bump github/codeql-action from 2.2.11 to 2.3.0
 2023-04-25 22:58:25 -04:00
dependabot[bot]
be489f78df Bump github/codeql-action from 2.2.11 to 2.3.0 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.2.11 to 2.3.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](d186a2a36c...b2c19fb9a2)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-04-24 06:03:24 +00:00
Yann Collet
e256e43274 Merge pull request #3624 from danlark1/patch-4 
Disable unused variable warning in msan configurations
 2023-04-20 10:46:20 -07:00
Daniel Kutenin
4c25ea329b Disable unused variable warning in msan configurations 2023-04-20 11:14:08 +01:00
Yann Collet
189653a9c1 Merge pull request #3614 from mredig/darwin-fat 
add makefile entry to build fat binary on macos
 2023-04-19 15:06:36 -07:00
Yann Collet
504c4a1f36 Merge pull request #3620 from facebook/errata_128k 
[doc] add decoder errata paragraph
 2023-04-19 11:31:16 -07:00
Yann Collet
0d6954b4cc added golden file for the new decompressor erratum 2023-04-19 00:24:35 -07:00
Nick Terrell
61efb2a047 Add ZSTD_d_maxBlockSize parameter 
Reduces memory when blocks are guaranteed to be smaller than allowed by
the format. This is useful for streaming compression in conjunction with
ZSTD_c_maxBlockSize.

This PR saves 2 * (formatMaxBlockSize - paramMaxBlockSize) when streaming.
Once it is rebased on top of PR #3616 it will save
3 * (formatMaxBlockSize - paramMaxBlockSize).
 2023-04-17 22:06:44 -07:00
Yann Collet
ed313342f1 Merge pull request #3621 from facebook/remove_CIs 
removed travis & appveyor scripts
 2023-04-17 17:54:40 -07:00
Nick Terrell
0abf2baef9 Reduce streaming decompression memory by 128KB 
The split literals buffer patch increased streaming decompression memory
by 64KB (shrunk lit buffer from 128KB to 64KB, and added 128KB). This
patch removes the added 128KB buffer, because it isn't necessary.

The buffer was there because the literals compression code didn't know
the true `blockSizeMax` of the frame, and always put split literals so
they ended 128KB - 32 from the beginning of the block. Instead, we can
pass down the true `blockSizeMax` and ensure that the split literals
end up at `blockSizeMax - 32` from the beginning of the block. We
already reserve a full `blockSizeMax` bytes in streaming mode, so we
won't be overwriting the extDict window.
 2023-04-17 16:31:02 -07:00