1
0
mirror of https://github.com/facebook/zstd.git synced 2025-07-29 11:21:22 +03:00

2230 Commits

Author SHA1 Message Date
8edd147686 fix missing fclose()
fix #4151
2024-10-01 09:52:45 -07:00
039f404faa update documentation to specify that Dictionary can be used for benchmark
fix #4139
2024-09-25 16:56:01 -07:00
bc96d4b077 Merge pull request #4119 from xionghul/dev
Fix zstd-pgo run error
2024-09-24 17:55:43 -07:00
bf4a43fcd4 Fix zstd-pgo run error
The -Werror=missing-profile caused thread/zlib/lzma/lz4 detection failure
when build with profile-use, thus caused ZSTD_MULTITHREAD etc. is not
defined for profile-use, then there will be many profile mismatch information
in output and the final binary reports run error sometimes as below:

Error : ZSTD_CCtx_setParameter(ctx, ZSTD_c_nbWorkers, adv->nbWorkers) failed : Unsupported parameter

Signed-off-by: Xionghu Luo <xionghuluo@tencent.com>
2024-08-08 15:13:27 +08:00
14b8d398fd fix c90 comment style 2024-08-06 12:47:30 -07:00
89451cafbd formatString_u() can display numbers > 100
fixes #4110
2024-08-06 11:44:37 -07:00
b320d096a4 clarify when to use '--single-thread' with '--patch-from'
Using '--single-thread' with '--patch-from' on compression levels above 15 will lead to significantly worse compression ratios.

Corrected the man page not suggest anymore to do this.
2024-07-07 19:43:49 +02:00
f19c98228f Fix $filter and Msys/Cygwin
- switched the patter and input of $filter into the right places
- added pattern wildcard to MSYS_NT & CYGWIN_NT as they change with windows versions
- correctly identify MSYS2, even in an env like MINGW64
2024-06-05 18:37:27 +02:00
4c6a519fdd [fix] Add newline when file exceeds 128KB 2024-05-26 11:34:24 +02:00
a2f145f059 decompression errors always display the full origin filename
instead of the truncated size-limited version.
2024-04-01 11:49:21 -07:00
0fcdc62c2d Merge pull request #3969 from facebook/v156_prep
bump version number
2024-03-15 10:26:49 -07:00
686e7e4b4b updated version to v1.5.6 2024-03-14 15:38:14 -07:00
490163ac73 Merge pull request #3963 from felixhandte/also-handle-hidden-files-output-dir-mirror
Remove Erroneous Exclusion of Hidden Files and Folders in `--output-dir-mirror`
2024-03-14 10:21:00 -04:00
04d91d5219 Merge pull request #3960 from felixhandte/use-utimensat-on-posix-2001
Use `utimensat()` on FreeBSD
2024-03-13 17:10:35 -04:00
86b8e39a84 Remove Erroneous Exclusion of Hidden Files and Folders in --output-dir-mirror 2024-03-13 16:33:30 -04:00
d6ee2d5d24 Use utimensat() on FreeBSD
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on #3952.

Further addresses #3748.

[0] 937a005585/sys/sys/unistd.h (L96)

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
2024-03-13 13:26:21 -07:00
7c2aa73a8e Merge pull request #3958 from facebook/doc3698
updated documentation
2024-03-13 07:15:45 -07:00
ff6713fd72 fix duplicated paragraph
reported by @zougloub
2024-03-12 13:09:10 -07:00
5473b72a05 updated documentation
following recommendations by @zougloub at #3698
2024-03-12 12:27:33 -07:00
83ec3d0164 no longer truncate file name in verbose mode
fix #3702
2024-03-12 11:27:42 -07:00
5a66afa051 Add common file extensions to --exclude-compressed (#3951) 2024-03-12 13:49:06 -04:00
a6ff1194d8 Merge pull request #3952 from felixhandte/stop-hardcoding-posix-version-on-bsd
Stop Hardcoding the POSIX Version on BSDs
2024-03-12 12:49:19 -04:00
8688c6ed92 Merge pull request #3942 from facebook/fix3719
Fix #3719 : mixing -c, -o and --rm
2024-03-12 09:44:52 -07:00
8ba5bc4729 chore: fix some typos (#3949)
Signed-off-by: acceptacross <csqcqs@gmail.com>
2024-03-12 11:44:42 -04:00
edab9eed66 Fix AsyncIO reading seed queueing (#3940)
Fixes a bug in AsyncIO where we queue reads after opening a file so our queue will always be saturated (or as saturated as possible).
Previous code was looping up to `availableJobsCount` not realizing `availableJobsCount` was also decreasing in each iteration, so instead of queueing 10 jobs we'd queue 5 (and instead of 2 we'd queue 1).
This PR fixes the loop to queue as long as `availableJobsCount` is not 0.
2024-03-11 16:28:32 -07:00
f99a450ca4 Stop Hardcoding the POSIX Version on BSDs
BSDs should all have a `unistd.h` header.
2024-03-11 17:25:51 -04:00
1362699e87 minor man page clarification 2024-03-11 12:23:37 -07:00
fbd9e628ae added tests 2024-03-11 12:17:34 -07:00
c610a01d7d fix #3719
only disable `--rm` at end of command line parsing,
so that `-c` only disables `--rm` if it's effectively selected,
and not if it's overriden by a later `-o FILE` command.
2024-03-11 11:38:55 -07:00
4fb0a77314 update -V documentation
to answer #3727 comment
2024-03-03 23:24:40 -08:00
e385c3dd46 Merge pull request #3753 from facebook/make2
minor Makefile refactoring
2024-03-03 19:13:00 -08:00
4d2bf7f0f2 removed sprintf usage from zstdcli.c
some static analyzers flag this standard C90 function as unsafe.
2024-02-24 23:03:40 -08:00
cc4530924b speed optimized version of targetCBlockSize
note that the size of individual compressed blocks will vary more wildly with this modification.
But it seems good enough for a first test, and fix the speed regression issue.
Further refinements can be attempted later.
2024-02-23 14:03:26 -08:00
68a232c591 benchmark more can test targetCBlockSize 2024-02-23 13:13:03 -08:00
621a263fb2 Merge pull request #3903 from gruenich/feature/reduce-scope-of-variables
Reduce scope of variables
2024-02-22 09:42:02 -08:00
7170f51dd2 fix include order 2024-02-20 23:36:04 -08:00
e62e15df19 fix clangbuild
notably -Wconversion and -Wdocumentation
2024-02-20 22:43:22 -08:00
588dfbcc97 fix c89 compatibility by removing snprintf()
note that this function has been in the code for a long while,
so why does it only start failing CI tests now ?
2024-02-20 19:24:50 -08:00
7a225c0c46 internal benchmark: can select size of generated synthetic sample 2024-02-20 15:47:09 -08:00
83598aa106 datagen generates lorem ipsum by default 2024-02-20 15:24:25 -08:00
7003c9905e increase word dictionary
for higher variety of messages.
Now, level 5 compresses better than level 4 (by a hair).
2024-02-20 13:27:36 -08:00
3dbd861b7d runtime weight distribution table
and made small words a bit more common.
2024-02-20 12:26:37 -08:00
5a1bb4a4e0 add question marks
and (slightly) longer sentences.
2024-02-20 00:37:21 -08:00
40874d4aea enriched vocabulary again
using real latin sentences from Cicero.

Compression ratio lower again, closer to "real" text,

now level 6 is way better than level 4.

level 5 is still lower than level 4,
but at least it's now higher than level 3.
2024-02-20 00:30:29 -08:00
1e046ce7fa increase vocabulary size
makes compression a bit less good,
hence a bit more comparable with real text (though still too easy to compress).
level 6 is now stronger than level 4, by a hair.
However, there is still a ratio dip at level 5.
2024-02-20 00:12:32 -08:00
b921f1aad6 Reduce scope of variables
This improves readability, keeps variables local, and
prevents the unintended use (e.g. typo) later on.
Found by Cppcheck (variableScope)
2024-02-11 22:00:03 +01:00
1f87c88ecf [Zstd] Less verbose log for patch mode. 2024-02-06 12:07:11 -08:00
d0b7da30e2 add a lorem ipsum generator
this generator replaces the statistical generator
for the general case when no statistic is requested.

Generated data features a compression level speed / ratio curve
which is more in line with expectation.
2024-01-29 15:00:32 -08:00
e6f4b46493 playTests.sh does no longer needs grep -E
it makes the test script more portable across posix systems
because `grep -E` is not guaranteed
while `grep` is fairly common.
2024-01-15 11:16:46 -08:00
8052cd0131 cli: better errors on arguent parsing 2023-12-18 13:59:33 +00:00