1
0
mirror of https://github.com/facebook/zstd.git synced 2025-08-05 19:15:58 +03:00
Commit Graph

2060 Commits

Author SHA1 Message Date
Arpad Panyik
07cd78d366 AArch64: Add Neon path for convertSequences_noRepcodes
Add a 4-way Neon implementation for the convertSequences_noRepcodes
function. Remove 'static' keywords from all of its implementations to
be able to add unit tests.

Relative performance to Clang-18 using: `./fullbench -b18 -l5 enwik5`

Neoverse-V2   before     after
Clang-18:    100.000%  311.703%
Clang-19:    100.191%  311.714%
Clang-20:    100.181%  311.723%
GCC-13:      107.520%  252.309%
GCC-14:      107.652%  253.158%
GCC-15:      107.674%  253.168%

Cortex-A720   before     after
Clang-18:    100.000%  204.512%
Clang-19:    102.825%  204.600%
Clang-20:    102.807%  204.558%
GCC-13:      110.668%  203.594%
GCC-14:      110.684%  203.978%
GCC-15:      102.864%  204.299%

Co-authored by, Thomas Daubney <Thomas.Daubney@arm.com>
2025-07-10 18:20:57 +00:00
Arpad Panyik
8e4400463a Improve ZSTD_get1BlockSummary
Add a faster scalar implementation of ZSTD_get1BlockSummary which
removes the data dependency of the accumulators in the hot loop to
leverage the superscalar potential of recent out-of-order CPUs.
The new algorithm leverages SWAR (SIMD Within A Register) methodology
to exploit the capabilities of 64-bit architectures. It achieves this
by packing two 32-bit data elements into a single 64-bit register,
enabling parallel operations on these subcomponents while ensuring
that the 32-bit boundaries prevent overflow, thereby optimizing
computational efficiency.

Corresponding unit tests are included.

Relative performance to GCC-13 using: `./fullbench -b19 -l5 enwik5`

Neoverse-V2   before     after
GCC-13:      100.000%  290.527%
GCC-14:      100.000%  291.714%
GCC-15:       99.914%  291.495%
Clang-18:    148.072%  264.524%
Clang-19:    148.075%  264.512%
Clang-20:    148.062%  264.490%

Cortex-A720   before     after
GCC-13:      100.000%  235.261%
GCC-14:      101.064%  234.903%
GCC-15:      112.977%  218.547%
Clang-18:    127.135%  180.359%
Clang-19:    127.149%  180.297%
Clang-20:    127.154%  180.260%

Co-authored by, Thomas Daubney <Thomas.Daubney@arm.com>
2025-07-10 18:20:49 +00:00
Yann Collet
d77a7b6895 added test-largeDictionary to dev-long CI script 2025-06-21 11:34:10 -07:00
jinyaoguo
878be1c8f0 fix 2025-06-21 13:43:47 -04:00
jinyaoguo
16e13ebdeb delete 2025-06-21 13:03:13 -04:00
jinyaoguo
a74f7fcabd merge 2025-06-21 12:57:12 -04:00
Yann Collet
354cede369 Merge pull request #4412 from Cyan4973/rm_bd
remove duplicate
2025-06-19 14:32:32 -07:00
Yann Collet
e315155cc2 removed duplicate
this file is already present as `largeDictionary.c`
2025-06-18 15:07:32 -07:00
Yann Collet
429dc891b2 Merge pull request #4411 from arpadpanyik-arm/hist_sve2
AArch64: Add SVE2 implementation of histogram computation
2025-06-18 13:48:54 -07:00
Arpad Panyik
d28a737750 Add unit tests for HIST_count_wksp
The following tests are included:
- Empty input scenario test.
- Workspace size and alignment tests.
- Symbol out-of-range tests.
- Cover multiple input sizes, vary permitted maximum symbol
  values, and include diverse symbol distributions.

These tests verifies count table correctness, maxSymbolValuePtr
updates, and error-handling paths. It enables automated regression
of core histogram logic as well.
2025-06-13 22:55:53 +00:00
jinyaoguo
cad0b72ad8 Ensure BMK_timedFnState is always freed in benchMem
When an error occurs in BMK_isSuccessful_runOutcome, the code
previously skipped the call to BMK_freeTimedFnState(tfs),
leaking the allocated tfs object.
Fiexed by calling BMK_freeTimedFnState(tfs) before goto _cleanOut.
2025-06-12 19:52:58 -04:00
Yann Collet
88bea95d11 Merge pull request #4403 from dloidolt/fix_FUZZ_malloc_rand
fuzz: Fix FUZZ_malloc_rand() to return non-NULL for zero-size allocations
2025-06-09 10:57:59 -07:00
Yann Collet
39c091bc9e Merge pull request #4397 from xiaoge1001/free
Fix several locations with potential memory leak
2025-06-09 10:06:36 -07:00
shixuantong
de8d9e8914 Fix several locations with potential memory leak 2025-06-09 21:23:23 +08:00
Dominik Loidolt
4be08ba122 fuzz: Fix FUZZ_malloc_rand() to return non-NULL for zero-size allocations
The FUZZ_malloc_rand() function was incorrectly always returning NULL for
zero-size allocations. The random offset generated by
FUZZ_dataProducer_int32Range() was not being added to the pointer variable,
causing the function to always return (void *)0.
2025-06-05 17:28:30 +02:00
jinyaoguo
a81ffe11d4 Release resources in error paths via cleanup
Replace direct returns in error-handling branches with a unified
cleanup block that frees allocated resources before returning,
improving code quality and robustness.
2025-06-04 18:08:11 -04:00
shixuantong
4bd5654e72 update --rm cmd help info
Starting from cee6bec9fa, --rm is ignored when the output is `stdout`.
2025-05-30 06:49:52 +08:00
Dave Vasilevsky
f9938c217d lz4: Remove ancient test helpers
Building lz4 as root was causing `make clean` to fail with permission
errors.

We used to have to install lz4 from source back in Ubuntu 14.04, but
nowadays the installed lz4 is fine. Get rid of ancient helpers and
cruft!
2025-05-07 22:01:49 -07:00
Yann Collet
7f907d5c23 fix minor warning in zstreamtest 2025-04-29 09:56:00 -07:00
Li-Yu Yu
2a12bbaf90 Update cli-tests/file-stat tests 2025-03-28 22:46:26 +00:00
Yann Collet
2f9627863f update error message 2025-03-28 10:02:37 -07:00
Yann Collet
0bdeb1d204 fix test 2025-03-28 09:18:17 -07:00
Yann Collet
a293cdcb85 added CI test 2025-03-28 09:18:17 -07:00
Yann Collet
76c2fdc7b7 better naming
and more narrow scope of local variables
2025-03-28 09:18:17 -07:00
Yann Collet
eb168a0afc add --patch-apply command
as an equivalent for `-d --patch-from`.
Requested by @sergeevabc in #2173.
2025-03-25 14:50:39 -07:00
Yann Collet
9a57bdc0bd attempt to reduce length of long cli tests by invoking -T0 2025-03-11 14:10:35 -07:00
Yann Collet
c583c2c39e warn when requesting decompression with multiple threads
restore #2918 fix
2025-03-10 22:50:00 -07:00
Yann Collet
3c4096c83e fixed ShellCheck warning 2025-03-10 19:11:44 -07:00
Yann Collet
2ff87aefac fix FreeBSD
use an alias instead of a function

also: added more traces and updated version nb to v1.5.8
2025-03-10 19:04:41 -07:00
Yann Collet
c18374bb16 add test
checks that ZSTD_NBTHREADS triggers the expected verbose message

Also: checked that the new test script fails on current `dev` branch, and is fixed by this branch
2025-03-10 13:40:47 -07:00
Yann Collet
d5986f235f fix #4332: setting ZSTD_NBTHREADS=0 via environment variables 2025-03-10 00:12:34 -07:00
Nick Terrell
b16d193512 [test] Add tests for determinism
Run compression & validate the compressed file matches a known checksum.

To update the output run:

```
make -C tests update-cli-tests
```
2025-03-07 10:31:19 -05:00
Yann Collet
5ae1cb9fa1 added a cli test for new command --jobsize 2025-03-04 15:24:43 -08:00
Yann Collet
0298df50f9 update cli-tests 2025-03-04 14:43:06 -08:00
Yann Collet
9b8b414833 updated playTests.sh to employ the new commands 2025-03-04 13:22:01 -08:00
Yann Collet
339bca6606 update ldm compression results 2025-02-10 10:46:37 -08:00
Yann Collet
72406b71c3 update hrlog rule to favor compression ratio a bit more at low levels 2025-02-10 10:46:37 -08:00
Yann Collet
7c5b6002c9 update results from the --long mode 2025-02-10 10:46:37 -08:00
Yann Collet
613901b6d3 modifies command echoing for macos / linux compatibility 2025-02-09 23:41:32 -08:00
Yann Collet
1603cbe83e update test for 32-bit mode
--max doesn't work in 32-bit mode, due to address space limitation
2025-02-09 23:02:14 -08:00
Yann Collet
41b7193757 added --max to test suite 2025-02-09 12:26:36 -08:00
Yann Collet
34ba14437a minor boundary change
improves compression ratio at low levels
2025-02-05 18:41:59 -08:00
Yann Collet
2949252923 fix minor conversion warnings 2025-02-05 17:01:19 -08:00
Yann Collet
e87d15938c more %zu warnings fixes 2025-02-05 16:48:19 -08:00
Yann Collet
590c22454e fix more %zu warnings 2025-02-05 16:36:54 -08:00
Yann Collet
f9c1850aa2 fix minor printf argument limitation on older mingw
which do not support `%zu` fields
2025-02-05 16:28:52 -08:00
Yann Collet
78275149ea Added a final success message at end of tests
so that a human reading the test log can determine everything was fine without consulting the shell error code.

Also: made `make check` slightly shorter by moving one longer test to `make test`
2025-01-17 19:30:55 -08:00
Yann Collet
8eb2587432 added benchmark for get1BlockSummary() 2025-01-15 17:11:27 -08:00
Yann Collet
bfc58f5ba2 generalize validation function 2025-01-15 17:11:27 -08:00
Yann Collet
d1f0e5fb97 fullbench can run a verification function
compressSequencesAndLiterals: fixed long lengths in scalar mode
2025-01-15 17:11:27 -08:00