2f8088f9df
fixed one race scenario
2025-02-01 21:23:36 -08:00
8803d1497a
switch to a single flush mutex+cond
...
which is transferred to the current oldest unfinished job.
2025-02-01 18:37:23 -08:00
5ef6e51e43
minor boundary change
...
improves compression ratio at low levels
2025-01-31 14:47:02 -08:00
78275149ea
Added a final success message at end of tests
...
so that a human reading the test log can determine everything was fine without consulting the shell error code.
Also: made `make check` slightly shorter by moving one longer test to `make test`
2025-01-17 19:30:55 -08:00
8eb2587432
added benchmark for get1BlockSummary()
2025-01-15 17:11:27 -08:00
bfc58f5ba2
generalize validation function
2025-01-15 17:11:27 -08:00
d1f0e5fb97
fullbench can run a verification function
...
compressSequencesAndLiterals: fixed long lengths in scalar mode
2025-01-15 17:11:27 -08:00
886720442f
initial implementation (incomplete)
...
needs to take care of long lengths > 65535
2025-01-15 17:11:27 -08:00
04a2a0219c
update type names
...
naming convention: Type names should start with a Capital letter (after the prefix)
2024-12-29 14:25:33 -08:00
a2ff6ea784
improve ZSTD_getFrameHeader on skippable frames
...
now reports:
- the header size
- the magic variant (within @dictID field)
2024-12-29 12:26:04 -08:00
f8a2b352d6
clarify doc on Frame-level methods when invoked on a skippable frame
...
following discussion at #4226
2024-12-29 02:41:09 -08:00
72ce56b527
fixed another invalid scenario
...
compressSequencesAndLiterals() doesn't support sequence validation
2024-12-23 21:15:50 -08:00
7b294caf46
add one valid test case
...
ZSTD_compressSequencesAndLiterals() may return a specific error code
when data to compress is non-compressible.
2024-12-23 19:43:17 -08:00
f8725e80cc
added fuzzer test for compressSequencesAndLiterals()
...
piggy-backing onto existing compressSequences() fuzzer test
2024-12-23 18:42:51 -08:00
b7a9e69d8d
added parameter litCapacity
...
to ZSTD_compressSequencesAndLiterals()
to enforce the litCapacity >= litSize+8 condition.
2024-12-20 10:37:01 -08:00
52a9bc6fca
fixed minor error in preparation of one fullbench scenario
2024-12-20 10:37:00 -08:00
ab0f1798e8
ensure that srcSize is controlled
2024-12-20 10:37:00 -08:00
a80f55f47d
added a test for ZSTD_compressSequencesAndLiterals
...
checks that srcSize is present in the frame header
and bounds the window size.
2024-12-20 10:37:00 -08:00
0a54f6f288
ZSTD_compressSequencesAndLiterals requires srcSize as parameter
...
this makes it possible to adjust windowSize to its tightest.
2024-12-20 10:37:00 -08:00
ad023b392f
fixed minor error in one benchmark scenario
2024-12-20 10:37:00 -08:00
12c47d3262
improved speed of the Sequences converter
2024-12-20 10:37:00 -08:00
95ad9e47ff
added benchmark for ZSTD_convertBlockSequences_wBlockDelim()
2024-12-20 10:37:00 -08:00
61ac8311e0
attempt to silence Visual Studio warning about fopen()
2024-12-20 10:37:00 -08:00
f617e86b71
fixed incorrect assert
2024-12-20 10:37:00 -08:00
f0d0d95234
added tests
...
check that ZSTD_compressAndLiterals() also controls that the `srcSize` field is exact.
2024-12-20 10:37:00 -08:00
31b5ef2539
ZSTD_compressSequencesAndLiterals() now supports multi-blocks frames.
2024-12-20 10:36:59 -08:00
5164d44dab
change advanced parameter name: ZSTD_c_repcodeResolution
...
and updated its documentation.
Note: older name ZSTD_c_searchForExternalRepcodes remains supported via #define
2024-12-20 10:36:59 -08:00
f176514467
minor doc update
2024-12-20 10:36:59 -08:00
f281497aef
fullbench: new scenario: compressSequencesAndLiterals()
2024-12-20 10:36:59 -08:00
ac05ea89a5
fullbench: switch default generator to lorem ipsum
...
which creates more "realistic" scenarios than former compressible noise.
The legacy data generator remains accessible,
it is triggered when requesting an explicit compressibility factor (-P#).
2024-12-20 10:36:59 -08:00
8ab04097ed
add the compressSequences() benchmark scenario
2024-12-20 10:36:59 -08:00
c050ae4fb8
fullbench: preparation functions are now in charge of allocating buffers
2024-12-20 10:36:59 -08:00
8b7e1b795d
minor variable renaming
2024-12-20 10:36:59 -08:00
c540976a4b
minor: test reordering
2024-12-20 10:36:59 -08:00
4c097b4939
refactor fullbench
...
to make it easier to add new scenarios
2024-12-20 10:36:58 -08:00
09964c6276
prepare fullbench for a refactor
2024-12-20 10:36:58 -08:00
0b013b2688
added unit tests to ZSTD_compressSequencesAndLiterals()
...
seems to work as expected,
correctly control that `litSize` and `srcSize` are exactly correct.
2024-12-20 10:36:58 -08:00
c97522f7fb
codemod: ZSTD_sequenceFormat_e -> ZSTD_SequenceFormat_e
...
since it's a type name.
Note: in contrast with previous names, this one is on the Public API side.
So there is a #define, so that existing programs using ZSTD_sequenceFormat_e still work.
2024-12-20 10:36:56 -08:00
477a01067f
codemod: symbolEncodingType_e -> SymbolEncodingType_e
2024-12-20 10:36:56 -08:00
a2245721ca
codemod: seqStore_t -> SeqStore_t
...
same idea, SeqStore_t is a type name, it should start with a Capital letter.
2024-12-20 10:36:55 -08:00
9671813375
codemod: seqDef -> SeqDef
...
SeqDef is a type name, so it should start with a Capital letter.
It's an internal symbol, no impact on public API.
2024-12-20 10:36:55 -08:00
4a4786bef0
tests: fix non-regular file test on GNU/Hurd
...
Since commit b21b03ca6 [1] the behaviour of writes to /dev/zero has been
fixed and now the non-regular file removal test no longer needs to be
done on /dev/random (which no longer works as random is not writable now).
[1] https://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?id=b21b03ca624b89caeedfe58430cea4b40317d39f
2024-12-10 10:30:52 -03:00
bbaba45589
change experimental parameter name
...
from ZSTD_c_useBlockSplitter to ZSTD_c_splitAfterSequences.
2024-10-31 13:43:40 -07:00
4f93206d62
changed variable name to ZSTD_c_blockSplitterLevel
...
suggested by @terrelln
2024-10-29 11:12:09 -07:00
f593ccda04
removed trace left over
2024-10-28 16:57:01 -07:00
37706a677c
added a test
...
test both that the new parameter works as intended,
and that the over-split protection works as intended
2024-10-28 16:31:15 -07:00
4e1a87976f
update regression results
2024-10-25 16:33:00 -07:00
c63b5d2294
update regression results
2024-10-24 14:47:55 -07:00
ea082692aa
added block splitter variant for greedy & lazy (levels 5 to 7)
2024-10-24 13:27:01 -07:00
70c77d20d6
update regression results
...
first block is no longer splitted since adding the @savings over-split protection
2024-10-24 11:43:16 -07:00
