1
0
mirror of https://github.com/facebook/zstd.git synced 2025-09-15 11:41:25 +03:00

8657 Commits

Author SHA1 Message Date
Nick Terrell
b53da1f6f4 Add extra space to match linux kernel v1.4.10 2022-02-22 19:58:18 -08:00
Nick Terrell
693aa7bad2 Add comment to unused variable suppression 2021-11-17 13:46:09 -08:00
Nick Terrell
1eba76a2d1 [linux-kernel] Don't add -O3 to CFLAGS
It is no longer necessary to get good performance, there is only a small
speed difference between -O2 and -O3, so just stick to the default of
-O2.

This also fixes the stack space usage on parisc. The compiler was buggy
for -O3 and used ~3KB of stack space for several functions. With -O2 the
problem is completely resolved, and stack space is back to a few hundred
bytes.

Additionally, we get a large code size win on gcc:

| Compiler | Before (Bytes) | After (Bytes) | Delta (Bytes) |
|----------|----------------|---------------|---------------|
| gcc-11   |         952754 |        738954 |       -213800 |
| clang-12 |         976290 |        938826 |        -37464 |
2021-11-16 14:29:35 -08:00
Nick Terrell
b9302410bf [linux-kernel] Don't inline function in zstd_opt.c
The optimal parser is unlikely to be used in the linux kernel in
practice. There is no reason these functions should be force inlined,
since we aren't gaining anything, and are losing build size.

| Compiler | Before (Bytes) | After (Bytes) | Delta (Bytes) |
|----------|----------------|---------------|---------------|
| gcc-11   |        1142090 |        952754 |       -189336 |
| clang-12 |        1228402 |        976290 |       -252112 |

This is a temporary solution pending the resolution of Issue #2862 in
the `dev` branch.
2021-11-15 20:37:58 -08:00
Nick Terrell
0118fe65ff Fix unused variable warning
`litLengthSum` is unused when asserts are disabled. Already fixed in `dev` by
PR#2838. Found by the Kernel test robot [0].

[0] https://lore.kernel.org/linux-mm/202111120312.833wII4i-lkp@intel.com/T/

Reported-by: kernel test robot <lkp@intel.com>
2021-11-15 16:55:59 -08:00
Nick Terrell
608bacf6cb Backport zstd patch from LKML
Credit to Nathan Chancellor for the bug fix and Nick Desaulniers for the
bug report.

Link: ClangBuiltLinux/linux#1486
Link: https://lore.kernel.org/all/20211021202353.2356400-1-nathan@kernel.org/
2021-11-11 12:18:35 -08:00
Nick Terrell
c0c38ba1db [binary-tree] Fix underflow of nbCompares
Fix underflow of `nbCompares` by switching to an `int` and comparing
`nbCompares > 0`. This is a minimal fix, because I don't want to change
the logic. These loops seem to be doing `nbCompares + 1` comparisons.

The bug was reported by Dan Carpenter and found by Smatch static
checker.

https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-11 15:57:35 -07:00
Nick Terrell
5a3e16f0c2 [ldm] Fix ZSTD_c_ldmHashRateLog bounds check
There is no minimum value check, so the parameter could be negative.
Switch to the standard pattern of using `BOUNDCHECK()`.

The bug was reported by Dan Carpenter and found by Smatch static
checker.

https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-11 15:43:57 -07:00
Nick Terrell
2c94f9fc61 [nit] Fix buggy indentation
The bug was reported by Dan Carpenter and found by Smatch static
checker.

https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-11 15:43:33 -07:00
Nick Terrell
695181c2e0 [multiple-ddicts] Fix NULL checks
The bug was reported by Dan Carpenter and found by Smatch static
checker.

https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-11 15:42:45 -07:00
Nick Terrell
20821a46f4 [lib] Make lib compatible with -Wfall-through excepting legacy
Switch to a macro `ZSTD_FALLTHROUGH;` instead of a comment. On supported
compilers this uses an attribute, otherwise it becomes a comment.

This is necessary to be compatible with clang's `-Wfall-through`, and
gcc's `-Wfall-through=2` which don't support comments. Without this the
linux build emits a bunch of warnings.
2021-09-23 11:54:14 -07:00
Nick Terrell
1715601e55 [contrib][linux] Reduce stack usage by 80 bytes
Instead of calling `ZSTD_compress_advanced()` and
`ZSTD_initCStream_advanced()`, which each take a `ZSTD_parameters` by
value, use the new advanced API.

Stack usage went from 2024 -> 1944.
2021-09-22 18:18:20 -07:00
Nick Terrell
07d7ebe448 [contrib][linux] Fix up SPDX license identifiers
Correctly identify that we are GPL v2+ or BSD 3 clause, as pointed out
in issue #2663.
2021-09-22 15:06:08 -07:00
Nick Terrell
67a426c322 [linux-kernel] Replace kernel-style comments
Replace kernel-style comments with regular comments.

E.g.

```
/** Before */

/* After */

/**
 * Before
 */

/*
 * After
 */

/***********************************
 * Before
 ***********************************/

/* *********************************
 * After
 ***********************************/
```
2021-04-29 15:53:38 -07:00
Nick Terrell
4432dac93b [contrib][linux-kernel] Add zstd_min_clevel() and zstd_max_clevel() 2021-03-30 10:37:45 -07:00
Nick Terrell
de9de869a3 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:37:39 -07:00
Nick Terrell
c1a244d534 Merge pull request #2539 from terrelln/linux-kernel-fixes
Fixes for the next linux kernel patch version
2021-03-24 16:29:38 -07:00
sen
c48889f097 Merge pull request #2538 from senhuang42/monotonicity_test
Add memory monotonicity test over srcSize
2021-03-22 16:54:34 -04:00
Nick Terrell
ebc2dfa821 Merge pull request #2524 from terrelln/huf-stack-reduction
[huf] Reduce stack usage of HUF_readDTableX2 by ~972 bytes
2021-03-22 12:37:54 -07:00
Nick Terrell
634bfd339f [FSE] Clean up workspace using dynamically sized struct 2021-03-22 11:07:07 -07:00
Yann Collet
8b97931ae1 Merge pull request #2550 from facebook/vvv_srcSize
fix #2549
2021-03-22 10:44:44 -07:00
Sen Huang
dff4a0e867 Make ZSTD_estimateCCtxSize_internal() loop through all srcSize parameter sets as well 2021-03-21 16:15:31 -07:00
Yann Collet
9fb4a42c7b fix #2549 2021-03-20 17:29:41 -07:00
Yann Collet
515807182f Merge pull request #2548 from SupervisedThinking/build_fix
meson: fix build by adding missing files
2021-03-19 15:20:14 -07:00
SupervisedThinking
edf2b1176d meson: fix build by adding missing files
fixes https://github.com/facebook/zstd/issues/2519
2021-03-19 19:52:45 +01:00
sen
eace4abc25 Merge pull request #2540 from senhuang42/fix_dds_supported
Fix dedicated dict search isSupported() requirements.
2021-03-17 23:25:53 -04:00
Nick Terrell
756bd59322 [huf][fse] Clean up workspaces
* Move `counting` to a struct in `FSE_decompress_wksp_body()`
* Fix error code in `FSE_decompress_wksp_body()`
* Rename a variable in `HUF_ReadDTableX2_Workspace`
2021-03-17 16:50:37 -07:00
Sen Huang
77ae664ba6 Fix ZSTD_dedicatedDictSearch_isSupported() requirements 2021-03-16 17:36:05 -07:00
Nick Terrell
ea288e0d8e [lib] Bump zstd version number 2021-03-16 11:47:27 -07:00
Nick Terrell
cd1551d261 [lib][tracing] Add ZSTD_NO_TRACE macro
When defined, it disables tracing, and avoids including the header.
2021-03-16 11:47:27 -07:00
Nick Terrell
7222614a19 [contrib][freestanding] Remove tracing support
Remove tracing support from `freestanding.py` to keep things simple.
2021-03-16 11:47:27 -07:00
Nick Terrell
e4b914e663 [contrib][linux] Expose zstd headers to avoid duplication
Expose the zstd headers in `include/linux` to avoid struct duplication.
This makes the member names not follow Kernel style guidelines, and
exposes the zstd symbols. But, the LMKL reviewers are okay with that.
2021-03-16 11:47:22 -07:00
Nick Terrell
49a9e070f5 [contrib][linux-kernel] Update test include stubs
Update the test include stubs so they are able to run the current zstd
version in the kernel, so I can compare stack usage.
2021-03-16 11:40:24 -07:00
Nick Terrell
d2dd35ae65 [contrib][linux-kernel] Fix unaligned.h
Fix the `unaligned.h` shim in the tests that was causing corruption in
the tests. Note that this is a problem with the test shim, not the
kernel code.
2021-03-16 11:40:24 -07:00
sen
413b3198b0 Merge pull request #2534 from foxeng/fix-seek-descriptor-check
Fix seek table descriptor check when loading
2021-03-16 13:09:00 -04:00
Sen Huang
b9dd821441 Add mem monotonicity test over srcSize 2021-03-16 08:24:26 -07:00
Felix Handte
d1a6d080cf Merge pull request #2535 from felixhandte/gha-release-artifacts
Add GitHub Action to Automatically Publish Release Tarballs
2021-03-15 13:58:27 -04:00
W. Felix Handte
d2b7f2e27a Allow a Passphrase on the Key 2021-03-15 12:48:53 -04:00
W. Felix Handte
eed64d75f4 Maintain Artifact Name Backwards Compatibility
When the tag is `v1.2.3`, name the artifacts `zstd-1.2.3.tar*` rather than
`zstd-v1.2.3.tar*`. When the tag doesn't match, use the full tag.
2021-03-15 11:59:31 -04:00
Fotis Xenakis
316f3d2224 Run generic-dev:gcc-8-asan-ubsan-testzstd on latest Ubuntu 2021-03-13 11:42:47 +02:00
W. Felix Handte
a51511e7a7 Remove CircleCI Artifact Generation 2021-03-12 17:35:11 -05:00
Fotis Xenakis
3c6f5d5eca Fix seekable test to provide valid descriptor 2021-03-13 00:00:08 +02:00
Fotis Xenakis
21697b9c9e Fix seek table descriptor check when loading 2021-03-12 23:07:15 +02:00
W. Felix Handte
5d1fec8ce1 Add GitHub Action to Automatically Publish Release Tarballs
This commit introduces a GitHub action that is triggered on release creation,
which creates the release tarball, compresses it, hashes it, signs it, and
attaches all of those files to the release.
2021-03-12 12:33:58 -05:00
sen
a3feed8dcd Merge pull request #2517 from PaulBone/num_cores
Make the number of physical CPU cores detection more robust
2021-03-08 11:14:34 -05:00
Yann Collet
3d6c903056 Merge pull request #2521 from animalize/doc_free
doc: ZSTD_free*() functions accept NULL pointer
2021-03-06 21:33:28 -08:00
Nick Terrell
3b1aba42cc [fse] Reduce stack usage of FSE_decompress_wksp() by 512 bytes
* Move `counting` into the workspace
* Inrease `HUF_DECOMPRESS_WORKSPACE_SIZE` by 512 bytes
2021-03-05 13:24:27 -08:00
Nick Terrell
0f18059a4e [huf] Reduce stack usage of HUF_readDTableX2 by ~460 bytes
* Use `HUF_readStats_wksp()`
* Use workspace in `HUF_fillDTableX2*()`
* Clean up workspace usage to use a workspace struct
2021-03-05 12:39:46 -08:00
Nick Terrell
b5fd348a85 Merge pull request #2523 from terrelln/huf-stack-reduction
Add HUF_writeCTable_wksp() function
2021-03-05 12:35:09 -08:00
Nick Terrell
5df2a21f1e Add HUF_writeCTable_wksp() function
This saves ~700 bytes of stack space in HUF_writeCTable.
2021-03-05 10:29:18 -08:00