From da7b7ebdda18fd703c03f63d0d6e3cf82efa3fec Mon Sep 17 00:00:00 2001 From: Yann Collet Date: Wed, 21 Oct 2026 15:30:40 -0700 Subject: [PATCH] copy the tables of an attached dictionary from 2 KB inputs LZ4_compress_fast_continue() copied the tables of an attached dictionary (one table to look into) only above 4 KB, and used the dictCtx loop (two tables) below. The copy is now faster from 1/8 of the table size: from 2 KB with the default table, capped at 4 KB for large tables. The compressed output is unchanged. AMD EPYC Genoa (Zen 4), x86-64, 2-4 KB inputs: gcc 11 +3-5%, gcc 15 +5-8%, clang 22 +16-31%. --- NEWS | 1 + lib/lz4.c | 7 ++++++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/NEWS b/NEWS index af284f34..4244afee 100644 --- a/NEWS +++ b/NEWS @@ -24,6 +24,7 @@ perf : faster decompression on aarch64 and x86-64 (up to +19% on Neoverse-V2 and perf : `LZ4_FAST_DEC_LOOP` enabled on aarch64 with clang, including Android (up to +9% on Cortex-A5xx), by @Nicoshev and @arpadpanyik-arm perf : `LZ4_FAST_DEC_LOOP` enabled on riscv64 when unaligned accesses are fast (`-mno-strict-align`, or a `-mcpu` with fast unaligned accesses) perf : faster compression with reused contexts, for blocks >= 1 KB : linked frames (up to +69%), `LZ4_resetStream_fast()` (up to +94%) +perf : faster compression of 2-4 KB blocks with `LZ4_attach_dictionary()` (+3% to +30% on Zen 4) perf : level 2 compresses slightly better (~0.1-0.5%) and faster (~+1-2%) perf : faster levels 11 (~+25%) and 12 (~+10%) perf : `--favor-decSpeed` : faster decompression (~+2-4% at level 12), with fewer sequences diff --git a/lib/lz4.c b/lib/lz4.c index 4969f10a..0b13e035 100644 --- a/lib/lz4.c +++ b/lib/lz4.c @@ -1734,6 +1734,11 @@ static int LZ4_isDictSmall(const LZ4_stream_t_internal* streamPtr) return (streamPtr->dictSize < 64 KB) && (streamPtr->dictSize < streamPtr->currentOffset); } +/* from this input size (2 KB by default, at most 4 KB), + * copying the tables of an attached dictionary is faster than looking into both tables. + * The copy cost follows the table size. */ +#define LZ4_DICTCTX_COPY_THRESHOLD ((LZ4_HASHTABLESIZE / 8 < 4 KB) ? (LZ4_HASHTABLESIZE / 8) : 4 KB) + int LZ4_compress_fast_continue (LZ4_stream_t* LZ4_stream, const char* source, char* dest, @@ -1794,7 +1799,7 @@ int LZ4_compress_fast_continue (LZ4_stream_t* LZ4_stream, * dictCtx->currentOffset - dictCtx->dictSize. This makes it safe * to use noDictIssue even when the dict isn't a full 64 KB. */ - if (inputSize > 4 KB) { + if (inputSize >= LZ4_DICTCTX_COPY_THRESHOLD) { /* For compressing large blobs, it is faster to pay the setup * cost to copy the dictionary's tables into the active context, * so that the compression loop is only looking into one table.