1
0
mirror of https://github.com/facebook/zstd.git synced 2025-07-05 18:41:19 +03:00

531 Commits

Author SHA1 Message Date
4bd5654e72 update --rm cmd help info
Starting from cee6bec9fa, --rm is ignored when the output is `stdout`.
2025-05-30 06:49:52 +08:00
76c2fdc7b7 better naming
and more narrow scope of local variables
2025-03-28 09:18:17 -07:00
eb168a0afc add --patch-apply command
as an equivalent for `-d --patch-from`.
Requested by @sergeevabc in #2173.
2025-03-25 14:50:39 -07:00
19541b4d1a fix minor unused variable warning 2025-03-11 00:17:45 -07:00
c583c2c39e warn when requesting decompression with multiple threads
restore #2918 fix
2025-03-10 22:50:00 -07:00
2ff87aefac fix FreeBSD
use an alias instead of a function

also: added more traces and updated version nb to v1.5.8
2025-03-10 19:04:41 -07:00
56e2ebf5c3 removed useless assert() 2025-03-10 09:54:06 -07:00
d5986f235f fix #4332: setting ZSTD_NBTHREADS=0 via environment variables 2025-03-10 00:12:34 -07:00
0de4991942 Add a method for checking if ZSTD was compiled with flags that impact determinism 2025-03-07 10:31:19 -05:00
f5a0e047cb fix typo 2025-03-04 15:12:35 -08:00
0b40c513fd update fileio to employ jobSize 2025-03-04 14:55:25 -08:00
0298df50f9 update cli-tests 2025-03-04 14:43:06 -08:00
19ffcf81b8 changed -B# command into --jobsize
to reduce confusion with the term "block".

-B# remains supported for existing scripts,
but it's no longer documented, so it's effectively a hidden shortcut.
2025-03-04 13:17:10 -08:00
fcfb3160dc convert benchmark unit to use the new naming scheme
chunks instead of blocks
2025-03-04 13:04:59 -08:00
d5dbdd6ece changed command --blocksize into --split
to reduce confusion with the concept of "blocks" inside a Zstandard frame.

We are now talking about "independent chunks" being produced by a `split` operation.

updated documentation accordingly.

Note: old commands "-B#` and `--blocksize=#` remain supported,
to maintain compatibility with existing scripts.
2025-03-04 12:53:23 -08:00
5b8575adaa Merge pull request #4289 from facebook/autoultra
--ultra automatically triggered with --long and --patch-from
2025-02-10 10:38:28 -08:00
468e1453a5 disable --max in 32-bit mode 2025-02-09 23:16:14 -08:00
39d1d82fa8 adjusted mml heuristic 2025-02-09 17:24:12 -08:00
630b47a158 added a new --max command
set parameters to maximum compression
(even more than -22)
2025-02-09 11:38:44 -08:00
aebffd66ec --ultra automatically triggered with --long and --patch-from
the purpose of --ultra is to make the user explicitly opt-in
to generate very large window size (> 8 MB).

The agreement to generate very large window size is already implicit
when selecting --long or --patch-from.

Consequently, `--ultra ` is automatically enabled when `--long` or `--patch-from` is set.
2025-02-08 22:47:05 -08:00
56500044c4 bench: nb threads specified with -v 2025-01-12 12:01:57 -08:00
56cfb7816a codemod: ZSTD_paramSwitch_e -> ZSTD_ParamSwitch_e 2024-12-20 10:36:58 -08:00
125f05282b publish new symbol ZSTD_compressSequencesAndLiterals() 2024-12-20 10:36:04 -08:00
17beeb5d1a Change CLI to employ multithreading by default (#4211)
* Change CLI to employ multithreading by default

* Document changes to benchmarking, print number of threads for display level >= 4, and add lower bound of 1 for the default number of threads
2024-12-12 13:09:29 -05:00
194062a4e7 Fix printing of filesize if >4GB 2024-11-20 16:11:17 +01:00
f34bc9cee6 improve man page on benchmark mode
update the man page in troff format,
and the README with latest `--help` content and complementary details about benchmark mode.

also: display level 0 when doing decompression benchmark
2024-10-23 00:16:13 -07:00
0079d515b1 Modify benchmark to only load sources once
After a regrettable update,
the benchmark module ended up reloading sources for every compression level.

While the delay itself is likely torelable,
the main issue is that the `--quiet` mode now also displays a loading summary between each compression line.
This wasn't the original intention, which is to produce a compact view of all compressions.

This is fixed in this version,
where sources are loaded only once, for all compression levels,
and loading summary is only displayed once.
2024-10-22 02:18:48 -07:00
039f404faa update documentation to specify that Dictionary can be used for benchmark
fix #4139
2024-09-25 16:56:01 -07:00
fbd9e628ae added tests 2024-03-11 12:17:34 -07:00
c610a01d7d fix #3719
only disable `--rm` at end of command line parsing,
so that `-c` only disables `--rm` if it's effectively selected,
and not if it's overriden by a later `-o FILE` command.
2024-03-11 11:38:55 -07:00
4d2bf7f0f2 removed sprintf usage from zstdcli.c
some static analyzers flag this standard C90 function as unsafe.
2024-02-24 23:03:40 -08:00
68a232c591 benchmark more can test targetCBlockSize 2024-02-23 13:13:03 -08:00
d0b7da30e2 add a lorem ipsum generator
this generator replaces the statistical generator
for the general case when no statistic is requested.

Generated data features a compression level speed / ratio curve
which is more in line with expectation.
2024-01-29 15:00:32 -08:00
8052cd0131 cli: better errors on arguent parsing 2023-12-18 13:59:33 +00:00
c28031df8f Add new line + [no-] to mmap-dict help output (#3601) 2023-04-06 13:01:58 -04:00
e769da1645 Merge pull request #3526 from facebook/bench_zstd_api
Simplify benchmark unit invocation API from CLI
2023-03-09 13:11:11 -08:00
e0fc9fd90b Merge pull request #3486 from daniellerozenblit/patch-from-low-memory-mode
Mmap large dictionaries in patch-from mode
2023-03-09 15:30:09 -05:00
96e55c14f2 ability to disable mmap + struct to manage FIO dictionary 2023-03-08 08:06:10 -08:00
1e38e07b3d simplified BMK_benchFilesAdvanced() 2023-03-06 12:34:13 -08:00
9efc14804e minor: fixed zlib wrapper internal benchmark
another possibility could be to link it to programs/benchfn .
Not worth the effort.
2023-03-06 12:20:06 -08:00
db79219f70 simplify BMK_syntheticTest() 2023-03-06 12:15:22 -08:00
547794ef40 Fix typos found by codespell 2023-02-18 10:31:48 +01:00
2d8afd9ce1 add manual flag to mmap dictionary 2023-02-14 09:42:23 -08:00
610c8b9e33 initial commit 2023-02-09 07:37:37 -08:00
9cabd155fd return error code when benchmark fails
such scenario can happen, for example,
when trying a decompression-only benchmark on invalid data.
Other possibilities include an allocation error in an intermediate step.

So far, the benchmark would return immediately, but still return 0.
On command line, this would be confusing, as the program appears successful (though it does not display any successful message).

Now it returns !0, which can be interpreted as an error by command line.
2023-02-07 00:35:51 -08:00
af09777b24 ensure that benchmark mode can only be invoked with zstd format
fix #3463
2023-01-31 09:04:29 -08:00
82ca00811a change logic when stderr is not console : don't update progress status
but keep warnings and final operation statement.

updated tests/cli-tests/ accordingly
2023-01-26 13:00:52 -08:00
7b3f03bc9d Merge pull request #3457 from yoniko/fix-rowhash-cli
[Bugfix] CLI row hash flags set the wrong values


`--[no-]row-match-finder` do the opposite of what they are supposed to.
In effect the no option would activate row hash while the other option will disable it.
This commit fixes the issue and changes the code to use the more readable enum values.
2023-01-25 22:40:25 -08:00
6422d1d7a8 Bugfix: --[no-]row-match-finder do the opposite of what they are supposed to 2023-01-25 17:59:35 -08:00
8c85b29e32 disable --rm on -o command
make it more similar to -c (aka `stdout`) convention.
2023-01-25 16:09:25 -08:00