lib/zstd
1
0
Fork 0
mirror of https://github.com/facebook/zstd.git synced 2025-07-29 11:21:22 +03:00
Code Activity

created ZSTD_compress2() and ZSTD_compressStream2()

Browse Source 
ZSTD_compress_generic() is renamed ZSTD_compressStream2().

Note that, for the time being,
the "stable" API and advanced one use different parameter planes :
setting parameters using the advanced API does not influence ZSTD_compressStream()
and using ZSTD_initCStream() does not influence parameters for ZSTD_compressStream2().
This commit is contained in:
Yann Collet
2018-11-30 11:16:26 -08:00
parent d3a0c71259
commit d8e215cbee
12 changed files with 133 additions and 141 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
doc/zstd_manual.html
View File

@ -484,10 +484,11 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
</b>/* These parameters are only useful if multi-threading is enabled (compiled with build macro ZSTD_MULTITHREAD).<b>
* They return an error otherwise. */
ZSTD_p_nbWorkers=400, </b>/* Select how many threads will be spawned to compress in parallel.<b>
* When nbWorkers >= 1, triggers asynchronous mode :
* ZSTD_compress_generic() consumes input and flush output if possible, but immediately gives back control to caller,
* When nbWorkers >= 1, triggers asynchronous mode when used with ZSTD_compressStream2() :
* ZSTD_compressStream2() consumes input and flush output if possible, but immediately gives back control to caller,
* while compression work is performed in parallel, within worker threads.
* (note : a strong exception to this rule is when first invocation sets ZSTD_e_end : it becomes a blocking call).
* (note : a strong exception to this rule is when first invocation sets ZSTD_e_end :
* in which case, ZSTD_compressStream2() delegates to ZSTD_compress2(), which is always a blocking call).
* More workers improve speed, but also increase memory usage.
* Default value is `0`, aka "single-threaded mode" : no worker is spawned, compression is performed inside Caller's thread, all invocations are blocking */
ZSTD_p_jobSize=401, </b>/* Size of a compression job. This value is enforced only when nbWorkers >= 1.<b>
@ -630,6 +631,18 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
</p></pre><BR>
<pre><b>size_t ZSTD_compress2( ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize);
</b><p> Behave the same as ZSTD_compressCCtx(), but compression parameters are set using the advanced API.
- Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*()
- The function is always blocking.
Hint : compression runs faster if `dstCapacity` >= `ZSTD_compressBound(srcSize)`.
@return : compressed size written into `dst` (<= `dstCapacity),
or an error code if it fails (which can be tested using ZSTD_isError()).
</p></pre><BR>
<pre><b>typedef enum {
ZSTD_e_continue=0, </b>/* collect more data, encoder decides when to output compressed result, for optimal compression ratio */<b>
ZSTD_e_flush=1, </b>/* flush any data provided so far,<b>
@ -640,11 +653,11 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
* each frame is independent (does not reference any content from previous frame). */
} ZSTD_EndDirective;
</b></pre><BR>
<pre><b>size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
ZSTD_outBuffer* output,
ZSTD_inBuffer* input,
ZSTD_EndDirective endOp);
</b><p> Behave about the same as ZSTD_compressStream. To note :
<pre><b>size_t ZSTD_compressStream2( ZSTD_CCtx* cctx,
ZSTD_outBuffer* output,
ZSTD_inBuffer* input,
ZSTD_EndDirective endOp);
</b><p> Behave about the same as ZSTD_compressStream, with additional control on end directive.
- Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*()
- Compression parameters cannot be changed once compression is started (save a list of exceptions in multi-threading mode)
- outpot->pos must be <= dstCapacity, input->pos must be <= srcSize
@ -653,7 +666,7 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
- In multi-thread mode, function is non-blocking : it just acquires a copy of input, and distribute job to internal worker threads,
and then immediately returns, just indicating that there is some data remaining to be flushed.
The function nonetheless guarantees forward progress : it will return only after it reads or write at least 1+ byte.
- Exception : in multi-threading mode, if the first call requests a ZSTD_e_end directive, it is blocking : it will complete compression before giving back control to caller.
- Exception : if the first call requests a ZSTD_e_end directive, the function delegates to ZSTD_compress2() which is always blocking.
- @return provides a minimum amount of data remaining to be flushed from internal buffers
or an error code, which can be tested using ZSTD_isError().
if @return != 0, flush is not fully completed, there is still some data left within internal buffers.
@ -666,13 +679,13 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
</p></pre><BR>
<pre><b> ZSTDLIB_API size_t ZSTD_DCtx_setMaxWindowSize(ZSTD_DCtx* dctx, size_t maxWindowSize);
<pre><b>size_t ZSTD_DCtx_setMaxWindowSize(ZSTD_DCtx* dctx, size_t maxWindowSize);
</b><p> Refuses allocating internal buffers for frames requiring a window size larger than provided limit.
This protects a decoder context from reserving too much memory for itself (potential attack scenario).
This parameter is only useful in streaming mode, since no internal buffer is allocated in single-pass mode.
By default, a decompression context accepts all window sizes <= (1 << ZSTD_WINDOWLOG_LIMIT_DEFAULT)
@return : 0, or an error code (which can be tested using ZSTD_isError()).
</p></pre><BR>
<pre><b>size_t ZSTD_DCtx_loadDictionary(ZSTD_DCtx* dctx, const void* dict, size_t dictSize);
@ -1018,7 +1031,7 @@ size_t ZSTD_freeCCtxParams(ZSTD_CCtx_params* params);
an existing CCtx.
These parameters will be applied to
all subsequent compression jobs.
- ZSTD_compress_generic() : Do compression using the CCtx.
- ZSTD_compressStream2() : Do compression using the CCtx.
- ZSTD_freeCCtxParams() : Free the memory.
This can be used with ZSTD_estimateCCtxSize_advanced_usingCCtxParams()
@ -1068,12 +1081,12 @@ size_t ZSTD_freeCCtxParams(ZSTD_CCtx_params* params);
</p></pre><BR>
<pre><b>size_t ZSTD_compress_generic_simpleArgs (
<pre><b>size_t ZSTD_compressStream2_simpleArgs (
ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos,
ZSTD_EndDirective endOp);
</b><p> Same as ZSTD_compress_generic(),
</b><p> Same as ZSTD_compressStream2(),
but using only integral types as arguments.
This variant might be helpful for binders from dynamic languages
which have troubles handling structures containing memory pointers.