1
0
mirror of https://github.com/facebook/zstd.git synced 2025-07-29 11:21:22 +03:00
Commit Graph

788 Commits

Author SHA1 Message Date
9009b59f09 [linux-kernel] Add missing semicolon in zstd_deps.h 2020-09-23 23:26:10 -07:00
a70fa342f0 [CI][linux-kernel] Enable -Wunused-const-variable and -Wunused-but-set-variable 2020-09-23 12:59:57 -07:00
978659d34f [linux-kernel] Fix unused variable warnings with malloc, calloc, and free 2020-09-23 12:59:57 -07:00
5651fe9b49 [contrib][linux-kernel] Add decompress_sources.h
Add decompress_sources.h, which includes all the decompression .c files.
This is used for kernel decompression.

Also, add a test which checks that including decompress_sources.h works.
2020-09-14 12:49:23 -07:00
80053bdae3 updating cold benchmark 2020-09-10 18:51:52 -04:00
3420c202af [linux-kernel] Update README with initial instructions on how to update zstd in the kernel 2020-09-09 14:36:22 -07:00
aab4bf7b0d [linux-kernel] Add test that checks the ifdef hardwiring 2020-09-09 14:36:19 -07:00
29c5de8780 Add linux-kernel freestanding 2020-09-09 14:35:57 -07:00
1c3cb2c05c [contrib] Add preprocessor hardwiring to freestanding.py 2020-09-09 14:35:39 -07:00
ac3a136b0a [lib] Replace 64-bit divisions with ZSTD_div64() 2020-09-09 14:35:39 -07:00
f91ed5c766 [lib] s/current/curr because it collides with Linux Kernel macro 2020-09-09 14:35:39 -07:00
ae455dde08 [contrib] Add freestanding translator prototype
This is the idea, some of the functionality isn't yet implemented.
2020-08-26 12:26:05 -07:00
80f577baa2 Move standard includes to zstd_deps.h 2020-08-26 12:25:08 -07:00
5d81d44e40 Fixed VS variable shadowing warning (and added test) 2020-07-29 12:33:39 +02:00
9bfd371d9f Remove superfluous build-time options 2020-07-08 11:39:17 +02:00
bc5fe33243 Change Emscripten docker image to the official one which is used for testing the single-file feature. 2020-07-07 19:10:12 +09:00
a035654ab8 Enable docker based Emscripten build on single file library test if emcc is not available. 2020-07-05 22:25:20 +09:00
4a9b7d136f Initial implementation (files added, macros fixed)
Hashing functions still to fix.
2020-06-22 10:31:36 +02:00
11a392ce23 minor markdown formatting fix 2020-05-26 13:15:35 -07:00
5aa5aa4df7 [pzstd] Fix compilation error in MinGW 2020-05-22 22:26:02 -07:00
1a8767cbda Minor tidy 2020-05-18 12:33:44 +02:00
c6b03fa69c Reduced generated sized on macOS (and other envs where realpath is missing)
Added Python fallback for when realpath fails.
2020-05-18 10:12:02 +02:00
9b8f337357 [contrib] Support seek table-only API
Memory constrained use cases that manage multiple archives benefit from
retaining multiple archive seek tables without retaining a ZSTD_seekable
instance for each.

* New opaque type for seek table: ZSTD_seekTable.
* ZSTD_seekable_copySeekTable() supports copying seek table out of a
  ZSTD_seekable.
* ZSTD_seekTable_[eachSeekTableOp]() defines seek table API that mirrors
  existing seek table operations.
* Existing ZSTD_seekable_[eachSeekTableOp]() retained; they delegate to
  ZSTD_seekTable the variant.

These changes allow the above-mentioned use cases to initialize a
ZSTD_seekable, extract its ZSTD_seekTable, then throw the ZSTD_seekable
away to save memory. Standard ZSTD operations can then be used to
decompress frames based on seek table offsets.

The copy and delegate patterns are intended to minimize impact on
existing code and clients. Using copy instead of move for the infrequent
operation extracting a seek table ensures that the extraction does not
render the ZSTD_seekable useless. Delegating to *new* seek
table-oriented APIs ensures that this is not a breaking change for
existing clients while supporting all meaningful operations that depend
only on seek table data.
2020-05-07 09:31:43 -04:00
5163778c02 Yay It Works, Clean Up Logging A Bit 2020-05-05 15:27:27 -04:00
005ffd3bdb Fix Sed Regex Escaping for Extended Regex Rules...... 2020-05-05 14:54:04 -04:00
bdfb50f7c4 Uhhh It's Still Failing, Let's Try Printing Debug Info? 2020-05-05 13:58:22 -04:00
229cbd056e Fix Fallbacks to Work on FreeBSD 2020-05-05 13:18:45 -04:00
6381b7ee3f Add Fallback When realpath Doesn't Support --relative-to 2020-05-04 18:59:47 -04:00
c7cba8e87c Mark zstddeclib-in.c Not Executable 2020-05-04 18:59:26 -04:00
b48f6c7d26 Try to Fix Single File Library Combiner Script to Handle Relative Includes 2020-05-04 15:20:26 -04:00
0301ef5d04 [bench] Extending largeNbDicts to compression (#2089)
* adding cdict_collection_t

* adding shuffleCDictionaries()

* adding compressInstructions

* adding compress()

* integrating compression into bench()

* copy paste error fix

* static analyzer uninit value complaint fix

* changing to control

* removing assert

* changing to control

* moving memcpy to seperate function

* fixing static analyzer complaint

* another hacky solution attempt

* Copying createbuffer logic
2020-05-04 10:42:22 -07:00
3eaa525138 Fixed clash when projects are already using xxHash
Undefing XXH_* macros allows the `.c` to build standalone without clashes. Removing `xxhash.c` and only including the header is the correct usage (according to `XXH_PRIVATE_API`).
2020-04-07 18:17:59 +02:00
d01e7c2fad Renamed directory 2020-04-07 13:34:19 +02:00
59812afac7 Improved documentation 2020-04-07 13:19:52 +02:00
7af7735fa3 Merge remote-tracking branch 'upstream/dev' into single-file-lib 2020-04-07 11:13:02 +02:00
edd9a07322 Code replicated in compression and decompression moved to shared headers
`CHECK_F` macro moved to `error_private.h` (shared between `fse_compress.c` and `fse_decompress.c`). `ZSTD_limitCopy()` moved to `zstd_internal.h` (shared between `zstd_compress.c` and `zstd_decompress.c`). Erroneous build artefact `zstd.h` removed from repo.
2020-04-07 11:02:06 +02:00
1d267dc5d6 returning on null check 2020-04-03 13:39:02 -07:00
d598c88fb3 adding fclose before return 2020-04-03 13:31:47 -07:00
88da79b42e Minor typo 2020-04-03 20:50:54 +02:00
0442be5aa4 Added missing -pthread flag (was breaking on BSD) 2020-04-03 20:34:47 +02:00
7c420344d2 Single-file decoder script can now (optionally) create an encoder
To complement the single-file decoder a new script was added to create an amalgamated single-file of all of the Zstd source, along with examples and (simple) tests.
2020-04-03 19:07:46 +02:00
dbdeca1412 Edit distance match finder 2020-03-13 09:39:33 -05:00
fe454c0a76 Add Support for Dictionaries 2019-12-18 11:54:39 -08:00
18f03516b3 Print Summary of Errors Encountered 2019-12-18 11:29:53 -08:00
4584c88a99 Don't Rely on Implicit Return in Main 2019-12-17 11:26:18 -08:00
4be9b4b9bb Merge pull request #1893 from felixhandte/check-flipped-bits
Add Tool to Diagnose Whether Corrupt Blobs are Plausibly Bit-Flips
2019-12-03 15:35:07 -08:00
9a3de0a535 changed name from createX to assembleX
shows that the resulting object just takes ownership of provided buffer.
2019-11-25 15:34:55 -08:00
b5fb2e7c12 Add Tool to Diagnose Whether Corrupt Blobs are Plausibly Bit-Flips
I spend an increasing amount of my time looking at "Corrupted block detected"
failures in decompression. Not infrequently, I suspect that it is the result
of hardware failure, and that the blob has become bit-flipped or otherwise
corrupted somewhere along the line.

For that reason I was motivated to write a little tool to inspect blobs that
fail to decompress, to try modifying them, and then check whether they
decompress successfully. This seems like potentially a generally useful tool,
so I figured it might be worth putting in `contrib/`.
2019-11-19 01:53:36 -05:00
485fec5665 removed experimental_dict_builders
no longer required,
can still be accessed by going to back to an earlier commit or release (<= v1.4.4)
2019-11-06 14:43:14 -08:00
31a0abbfda updated pzstd and largeNbDicts to use the new FileNamesTable* abstraction 2019-11-06 09:10:05 -08:00