1
0
mirror of https://github.com/facebook/zstd.git synced 2025-09-15 11:41:25 +03:00
Commit Graph

361 Commits

Author SHA1 Message Date
Yann Collet
76c2fdc7b7 better naming
and more narrow scope of local variables
2025-03-28 09:18:17 -07:00
Yann Collet
eb168a0afc add --patch-apply command
as an equivalent for `-d --patch-from`.
Requested by @sergeevabc in #2173.
2025-03-25 14:50:39 -07:00
Yann Collet
9a57bdc0bd attempt to reduce length of long cli tests by invoking -T0 2025-03-11 14:10:35 -07:00
Yann Collet
3c4096c83e fixed ShellCheck warning 2025-03-10 19:11:44 -07:00
Yann Collet
2ff87aefac fix FreeBSD
use an alias instead of a function

also: added more traces and updated version nb to v1.5.8
2025-03-10 19:04:41 -07:00
Yann Collet
c18374bb16 add test
checks that ZSTD_NBTHREADS triggers the expected verbose message

Also: checked that the new test script fails on current `dev` branch, and is fixed by this branch
2025-03-10 13:40:47 -07:00
Yann Collet
9b8b414833 updated playTests.sh to employ the new commands 2025-03-04 13:22:01 -08:00
Yann Collet
78275149ea Added a final success message at end of tests
so that a human reading the test log can determine everything was fine without consulting the shell error code.

Also: made `make check` slightly shorter by moving one longer test to `make test`
2025-01-17 19:30:55 -08:00
Diego Nieto Cid
4a4786bef0 tests: fix non-regular file test on GNU/Hurd
Since commit b21b03ca6 [1] the behaviour of writes to /dev/zero has been
fixed and now the non-regular file removal test no longer needs to be
done on /dev/random (which no longer works as random is not writable now).

[1] https://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?id=b21b03ca624b89caeedfe58430cea4b40317d39f
2024-12-10 10:30:52 -03:00
Po-Chuan Hsieh
103a85e6f6 Use md5sum rather than gmd5sum for FreeBSD
Reference:	https://man.freebsd.org/cgi/man.cgi?query=md5sum
2024-03-27 20:53:52 +08:00
daniellerozenblit
5a66afa051 Add common file extensions to --exclude-compressed (#3951) 2024-03-12 13:49:06 -04:00
Yann Collet
8688c6ed92 Merge pull request #3942 from facebook/fix3719
Fix #3719 : mixing -c, -o and --rm
2024-03-12 09:44:52 -07:00
Yann Collet
74e856a195 add tests inspired from #2927
centered around -T# and --fast=# arguments
2024-03-11 17:57:57 -07:00
Yann Collet
fbd9e628ae added tests 2024-03-11 12:17:34 -07:00
Yann Collet
81f444f4f9 made playTests.sh more compatible with older versions of grep
replaced `\+` by `*`.
`\+` means `[1-N]`,
while `*` means `[0-N]`,
so it's not strictly equivalent
but `\+` happens to be badly supported on some flavors of grep,
and for the purpose of these tests, `*` is good enough.
2024-01-16 12:14:35 -08:00
Yann Collet
e6f4b46493 playTests.sh does no longer needs grep -E
it makes the test script more portable across posix systems
because `grep -E` is not guaranteed
while `grep` is fairly common.
2024-01-15 11:16:46 -08:00
jysh1214
e99d554903 Fixed typo 2023-08-02 11:29:35 +08:00
Yann Collet
b46236278a detect extraneous bytes in the Sequences section
when nbSeq == 0.

Reported by @ip7z
2023-06-13 11:43:45 -07:00
W. Felix Handte
698af84fcf Add CI Test for Excluding Matchfinders 2023-05-04 12:18:58 -04:00
W. Felix Handte
bae174960b Add ZSTD_LIB_EXCLUDE_COMPRESSORS_DFAST_AND_UP Build Variable 2023-05-04 12:18:58 -04:00
Yann Collet
9cabd155fd return error code when benchmark fails
such scenario can happen, for example,
when trying a decompression-only benchmark on invalid data.
Other possibilities include an allocation error in an intermediate step.

So far, the benchmark would return immediately, but still return 0.
On command line, this would be confusing, as the program appears successful (though it does not display any successful message).

Now it returns !0, which can be interpreted as an error by command line.
2023-02-07 00:35:51 -08:00
Yann Collet
af09777b24 ensure that benchmark mode can only be invoked with zstd format
fix #3463
2023-01-31 09:04:29 -08:00
Yann Collet
8c85b29e32 disable --rm on -o command
make it more similar to -c (aka `stdout`) convention.
2023-01-25 16:09:25 -08:00
Yann Collet
cee6bec9fa refactor : --rm is ignored with stdout
`zstd` CLI has progressively moved to the policy of
ignoring `--rm` command when the output is `stdout`.
The primary drive is to feature a behavior more consistent with `gzip`,
when `--rm` is the default, but is also ignored when output is `stdout`.
Other policies are certainly possible, but would break from this `gzip` convention.

The new policy was inconsistenly enforced, depending on the exact list of commands.
For example, it was possible to circumvent it by using `-c --rm` in this order,
which would re-establish source removal.

- Update the CLI so that it necessarily catch these situations and ensure that `--rm` is always disabled when output is `stdout`.
- Added a warning message in this case (for verbosity 3 `-v`).
- Added an `assert()`, which controls that `--rm` is no longer active with `stdout`
- Added tests, which control the behavior, even when `--rm` is added after `-c`
- Removed some legacy code which where trying to apply a specific policy for the `stdout` + `--rm` case, which is no longer possible
2023-01-20 18:04:55 -08:00
W. Felix Handte
1e3eba65a6 Copy Permissions from Source File 2023-01-18 11:57:35 -08:00
Chris Burgess
2b9fde932b Fix small file passthrough (#3215) 2022-07-29 12:22:46 -07:00
Qiongsi Wu
b1bbb0eb4c [AIX] Fix Compiler Flags and Bugs on AIX to Pass All Tests (#3219)
* Fixing compiler warnings

* Replace the old -s flag with the -Wl,-s flag

* Fixing compiler warnings

* Fixing the linker strip flag and tests/code not working as expected on AIX
2022-07-29 12:21:59 -07:00
udayanbapat
43f21a600e Intial commit to address 3090. Added support to decompress empty block. (#3118)
* Intial commit to address 3090. Added support to decompress empty block

* Update zstd_decompress_block.c

Addressed review comments for the case of 'set_basic'

* Update lib/decompress/zstd_decompress_block.c

Co-authored-by: Nick Terrell <nickrterrell@gmail.com>

* Update lib/decompress/zstd_decompress_block.c

Co-authored-by: Nick Terrell <nickrterrell@gmail.com>

Co-authored-by: Nick Terrell <nickrterrell@gmail.com>
2022-07-14 11:54:34 -07:00
Dimitris Apostolou
cf1894b324 Fix typos 2022-03-05 23:47:25 +02:00
Xi Ruoyao
d109cef201 fix the assertion in readLinesFromFile (#3084)
* fix the assertion in readLinesFromFile

When the file is not terminated by endline, readLineFromFile will append
a '\0' for the last line.  In this case pos + lineLength == dstCapacity.

* test: don't print very long text garbage
2022-03-04 14:56:44 -05:00
binhdvo
936ae8a3a0 Move zstdgrep and zstdless tests to cli-tests (#3057)
* Move zstdgrep and zstdless tests to cli-tests

Co-authored-by: Binh Vo <binhvo@fb.com>
2022-02-09 11:21:33 -05:00
Yonatan Komornik
4bba97b4cb Macos playtest envvars fix (#3035)
* playtests.sh: fix for a bug in macos' /bin/sh that persists temporary env vars when introduced before function calls
* cli-tests/run.py: Do not use existing ZSTD* envvars
2022-02-03 18:42:20 -08:00
Yonatan Komornik
cc0657f27d AsyncIO compression part 2 - added async read and asyncio to compression code (#3022)
* Compression asyncio:
- Added asyncio functionality for compression flow
- Added ReadPool for async reads, implemented in both comp and decomp flows
2022-01-31 15:43:41 -08:00
Yann Collet
c9072dd60a Merge pull request #3027 from brailovich/dev
fix for -r on empty directory
2022-01-29 15:40:43 -08:00
brailovich
9f37d1fede Update playTests.sh
combination of -r with empty folder simplified to comply with sh compatibility tests
2022-01-27 08:22:05 -08:00
brailovich
501a353b91 Update playTests.sh 2022-01-26 18:56:52 -08:00
Nick Terrell
e60eba58bf Print zlib/lz4/lzma library versions in verbose version output
Knowing the version of zlib/lz4/lzma we're linking against is very
useful for debugging issues with those libraries, so print it out in the
verbosity 4 version output.

Also print this information at the top of `playTests.sh`.
2022-01-26 18:25:58 -08:00
brailovich
5e7523385b Update playTests.sh 2022-01-26 16:53:11 -08:00
binhdvo
17017ac8db Change zstdless behavior to align with zless (#2909)
* Change zstdless behavior to align with zless
2022-01-21 19:57:19 -05:00
Yonatan Komornik
1598e6c634 Async write for decompression (#2975)
* Async IO decompression:
- Added --[no-]asyncio flag for CLI decompression.
- Replaced dstBuffer in decompression with a pool of write jobs.
- Added an ability to execute write jobs in a separate thread.
- Added an ability to wait (join) on all jobs in a thread pool (queued and running).
2022-01-21 13:55:41 -08:00
Yann Collet
05e884f288 Merge pull request #2956 from sunwire/fix-tar-test-cases
Fix tar test cases
2022-01-11 10:16:34 -08:00
W. Felix Handte
ff5d1daf33 Clean Up Debugging Statements 2022-01-05 16:13:00 -05:00
W. Felix Handte
35208f702f Add Test Validating Stack is not Executable in playTests.sh 2022-01-04 16:57:35 -05:00
Paweł Marciniak
666372c7bf Fix tar test cases 2021-12-24 15:05:26 +01:00
Yann Collet
a9e43b37d0 Revert "Limit ZSTD_maxCLevel to 21 for 32-bit binaries." 2021-12-20 11:43:14 -08:00
Elliot Gorokhovsky
c5f1e826ca Merge pull request #2925 from embg/dict_training_sample_limit_size
Allow user to specify memory limit for dictionary training
2021-12-15 15:58:17 -05:00
Elliot Gorokhovsky
71c0c07c19 Allow user to specify memory limit for dictionary training 2021-12-14 14:29:01 -05:00
Nick Terrell
3e2a70b6fb Merge pull request #2905 from 15596858998/dev_1205
add test case
2021-12-13 13:45:23 -08:00
zx123123
c69d13eb99 Update playTests.sh 2021-12-13 08:58:42 +08:00
Felix Handte
0c26d98c0d Merge pull request #2910 from felixhandte/reject-irregular-dicts
Reject Irregular Dictionary Files
2021-12-09 11:44:37 -05:00