1
0
mirror of https://github.com/redis/go-redis.git synced 2026-10-24 07:59:20 +03:00
Files
go-redis/pipeline_buffer_test.go
Nedyalko Dyakov bd0cea4ea8 chore(pipeline): Create pipeline pool even without specific buffer passed (#3959)
* fix(pipeline): PipelinePoolSize alone should create the pipeline pool

The dedicated pipeline connection pool was created only when
PipelineReadBufferSize or PipelineWriteBufferSize was set. Setting just
PipelinePoolSize therefore produced no pipeline pool at all, with no error and
no log line, and every pipelined operation silently kept using the main
connection pool — competing with ordinary commands for the same connections,
which is the opposite of what the option asks for.

The gating was inverted. The pool size is the primary intent ("give pipelines
their own N connections"); the buffer sizes are tuning. And the buffer sizes
already supplied a default for the size (10), so the dependency only ran one
way.

Any of the three options now creates the pool. Enabling it by size inherits the
regular ReadBufferSize/WriteBufferSize, so a caller who only wants to bound
pipeline connections no longer has to know that a buffer size is secretly
required.

Found while measuring client-side caching over a high-latency link: the CSC
background batched reads go through withPipelineConn, which falls back to the
regular pool when none exists, so they were contending with the very reader
misses they exist to prevent.

Tests cover all four option combinations and were verified to fail with the
gate reverted.

* feat(pipeline): give autopipelined clients a dedicated pipeline pool by default

Autopipelining is pipeline-heavy by definition, yet the dedicated pipeline
pool still had to be enabled separately; without it every batch competes
with regular commands for main-pool connections. Two paths now default it in:

Config time: Options.init defaults PipelinePoolSize to DefaultPipelinePoolSize
(new exported const, 10) when AutoPipelineOptions is set. Cluster node clients
do not inherit AutoPipelineOptions (the cluster autopipeliner lives at cluster
level and dispatches into the node clients' pipeline hooks), so clientOptions
applies the same default there. Sentinel and failover node clients inherit
AutoPipelineOptions and are covered by Options.init; Ring does not support
autopipelining. A negative PipelinePoolSize opts out explicitly.

Runtime: the common pattern configures the autopipeliner after NewClient
(AutoPipelineWithOptions on a plain client), where pool creation has already
happened. ensurePipelinePool now builds the pool lazily when an autopipeliner
is created. baseClient.pipelinePool becomes a pointer to an atomic slot
(pool + pool-name ref) so the lazy publication is race-free against
concurrent Pipelined callers on the hot path, and WithTimeout clones share
the slot — two sharers cannot install two different pools (CAS; the loser
closes its candidate). Streaming-credentials and maintnotifications hooks are
wired before publication; otel registration happens for the winner. Conn, Tx
and SentinelClient never allocate the slot and keep using the main pool.

The extraction into buildPipelinePool also surfaced that NewFailoverClient
kept a duplicated creation block with the pre-fix gate (buffer sizes only),
so PipelinePoolSize alone silently created no pipeline pool on failover
clients; the shared helper fixes that, with a regression test. Stale
RingOptions/FailoverOptions doc lines still describing the old gate are
corrected.

* feat(pipeline): 64KiB default buffers and no pre-dial for the pipeline pool

The options the dedicated pipeline pool is built with are now resolved by
pipelinePoolOptions, a pure function with pinned tests:

Buffers default to DefaultPipelineBufferSize (new exported const, 64 KiB)
rather than inheriting the regular 32 KiB. Pipeline connections move whole
batches per round trip and earn bigger buffers than per-command traffic:
measured on the autopipeline engine, throughput plateaus around 64 KiB,
gains nothing past ~128 KiB, and very large buffers can regress it. An
explicitly larger regular buffer is kept rather than shrunk, and explicit
pipeline buffer sizes always win. The RESP3 minimum clamp applies as on the
main pool.

MinIdleConns is forced to 0 on the pipeline pool. It previously inherited
the main pool's value through the options clone, so a client with
MinIdleConns set pre-dialed that many pipeline connections at pool creation
— a silent doubling of the idle footprint. The pipeline pool is burst
capacity: its connections dial on demand, and the pool size
(PipelinePoolSize or DefaultPipelinePoolSize) is only a cap, so an unused
pipeline pool holds zero connections.

* test: make pipeline-conn specs aware of the dedicated pipeline pool

The autopipeline-subjects job builds its suite subject by calling
AutoPipelineWithOptions on the shared rawClient, which now (correctly)
creates the dedicated pipeline pool lazily - so client.Pipelined and
client.Pipeline run on pipeline-pool connections, not the main pool's.
Two specs assumed the old sharing:

should Auth asserted main-pool stats (Hits=2/Misses=1/TotalConns=1) after
two Pipelined calls. It now branches on PoolStats.PipelineStats: when the
dedicated pool exists, the two pipelines account there (first dials,
second reuses) and the main pool holds only the BeforeEach FlushDB dial;
in the default subject mode PipelineStats is nil and the original
assertions run unchanged.

should ClientSetInfo issued CLIENT SETINFO through a pipeline and then read
CLIENT INFO through rawClient - with a pipeline pool those are different
connections, and SETINFO is per-connection state. It now reads CLIENT INFO
through the same pipeline that issued the SETINFOs, which is equivalent on
a single-pool client and correct on a split-pool one.

Verified against redis:8 in all three GOREDIS_TEST_SUBJECT modes
(default, ap-blocking, ap-async): Ran 2 of 1543 Specs, 2 Passed.

* test: warm the pipeline-pool conn in AP tests that instrument connections

Two autopipeline unit tests assumed batches execute on the main pool's
already-initialized connection; with the dedicated pipeline pool now
created lazily at autopipeliner creation, a batch's first dispatch dials
and initializes a fresh pipeline-pool connection instead. Both tests now
warm that connection with a real two-command batch first (a lone command
takes the solo fast path through Process on the main pool and would not
touch the pipeline pool):

TestAutoPipelineRetriesOnNetworkError counted the pipeline pool's first
dial as if it were a retry redial (3 dials observed, 2 expected). The
warm-up runs before the write failure is armed, so the count again
measures exactly the one redial the retry performs.

TestAutoPipelineHookPostNextErrorPartialBatch installed a hook that
unconditionally injects a post-next error before the pipeline-pool
connection existed. Connection init runs its handshake pipeline through
the client's hook chain (newConn shares hooksMixin), so the hook failed
the pipeline conn's init itself — retries exhausted, the injected error
was stamped on every command, and the test measured init poisoning
rather than the post-next outcome rule it pins. The warm-up initializes
the connection before the hook is installed.

* docs: TODO(v10) — ClientTracking* and ClientMaintNotifications are stateful

CLIENT TRACKING and CLIENT MAINT_NOTIFICATIONS mutate per-connection
state, exactly like CLIENT SETNAME/SETINFO, but live on cmdable: on a
pooled client they land on an arbitrary pool connection that is
immediately returned to the pool, so the flag applies to a connection
the caller cannot address again. They belong on statefulCmdable, next
to ClientSetName — but the move removes the methods from
Client/UniversalClient/Cmdable (and defining them on both cmdable and
statefulCmdable makes every Pipeline selector ambiguous), so it is a
breaking change deferred to v10. Documented now; the move happens in a
follow-up PR. No behavior change.

* docs: drop the TODO(v10) notes on the per-connection commands

The ClientTracking*/ClientMaintNotifications move to statefulCmdable ships
as its own PR against master (feature/stateful-conn-commands) rather than
waiting for v10, so the TODO notes here are obsolete. Docs only.

* feat(pipeline): always create the pipeline pool; spill on exhaustion

The dedicated pipeline pool is now created unconditionally at
NewClient/NewFailoverClient — like the pubsub pool — instead of behind a
matrix of triggers (pipeline options set, AutoPipelineOptions declared,
or lazily when an autopipeliner was built). It is pure burst capacity:
no pre-dialing (MinIdleConns forced to 0), a small cap
(DefaultPipelinePoolSize) and 64 KiB buffers, so an unused pipeline pool
holds zero connections and costs nothing, while pipelines stop competing
with regular commands for main-pool connections. PipelinePoolSize < 0
opts out and restores the old pipelines-on-the-main-pool behavior.

To keep this from capping heavy concurrent Pipelined callers — who
previously shared the much larger main pool — withPipelineConn spills to
the main pool when the pipeline pool is exhausted (ErrPoolTimeout):
bursts wider than the cap keep their old capacity, spilled pipelines
simply run with regular buffers, and total connections stay bounded by
PoolSize + PipelinePoolSize.

Pipeline-pool connections are excluded from CLIENT TRACKING during init.
Pipelined commands never consult or populate the built-in client-side
cache (only the single-command cached path on main-pool connections
does), so tracking pipeline reads only grew the server's tracking table
and produced invalidation pushes for keys the cache does not hold.

With creation unconditional, the lazy machinery is deleted: the atomic
slot and CAS publication, ensurePipelinePool, the autopipeliner-factory
hooks, the Options.init AutoPipelineOptions defaulting and its cluster
clientOptions mirror. pipelinePool is a plain field set before the
client is visible to any goroutine; WithTimeout clones share it. Net
diff of this commit is negative.

Tests: the creation-rule table becomes always-on plus opt-out; new e2e
coverage for the spill (a pipeline pool of one, held busy by a pipelined
BLPOP, must not queue a second pipeline behind PoolTimeout) and for the
tracking exclusion (CLIENT LIST shows the main-pool connection tracked
and the pipeline-pool connection untracked).

* test: keep HImport mock choreography on the main pool; doc the pool default

The HImport mock tests choreograph exact per-connection sequences — a
pool of one, booms armed to fire on the Nth replayed PREPARE, session
counts pinned per connection. The always-on pipeline pool adds a second
connection whose init replays fieldsets (correct production behavior:
HIMPORT pipelines work on pipeline-pool connections precisely because of
that replay), which fired the armed booms during the pipeline conn's
init instead of the reissue under test and shifted the session counts.
Opt every client in the file out with PipelinePoolSize: -1: the tests
pin HIMPORT reissue/session semantics, not pool routing.

Also states the PipelinePoolSize default plainly in the option docs:
DefaultPipelinePoolSize (10) connections (review feedback).

* test: skip conn-init handshakes in the cluster shard pipeline hooks

The shard-level pipeline hooks in the cluster hook-order specs asserted
Expect(cmds).To(HaveLen(...)) unconditionally. With the always-on
pipeline pool, a node's first pipeline dials a pipeline-pool connection
whose init runs its handshake pipeline through the node client's hook
chain, so the assertion fired on the handshake and the resulting Ginkgo
panic aborted the whole suite. Guard both hooks with the idiom
ring_test.go already uses for the same situation: defer GinkgoRecover()
and pass connection-initialization pipelines (hello/client) through.
The cluster-level hooks are untouched — node connection init never runs
through the cluster client's chain.

* test: adapt plain-pipeline tests to the always-on pipeline pool

TestNoPipelinePoolStats pinned the stats shape of a client without a
pipeline pool by creating a client without pipeline options — a premise
the always-on pool inverts. Having no pool now requires the explicit
opt-out, so the test sets PipelinePoolSize: -1, which is exactly the
configuration whose stats shape it verifies.

TestPipelineRetriesOnNetworkError is the plain-pipeline twin of the AP
dial-count test adapted earlier: pipelines now run on the dedicated
pipeline pool, so the first Exec dialed a fresh pipeline-pool connection
and the test counted it as a phantom retry redial. Warm the pipeline
connection with one Pipelined round trip before arming the write
failure, so the count again measures exactly the one redial the retry
performs.

* fix(pipeline): address #3959 review (spill, caps, UniversalOptions)

- UniversalOptions now exposes PipelineReadBufferSize / PipelineWriteBufferSize /
  PipelinePoolSize and maps them through Cluster()/Failover()/Simple(), so
  UniversalClient users can size or opt out (PipelinePoolSize < 0) of the now
  always-created pipeline pool (ofek's CHANGES_REQUESTED blocker).
- withPipelineConn no longer double-counts the Limiter on spill: it acquires the
  main-pool conn directly (not via withConn) and records which pool owns the
  conn, so the single deferred ReportResult+release runs once, in order (cursor
  HIGH).
- The pipeline pool no longer inherits MaxActiveConns (reset to 0, so the ceiling
  is MaxActiveConns + PipelinePoolSize, not ~2x) and uses a short
  DefaultPipelinePoolTimeout so a saturated pipeline pool spills promptly instead
  of queueing; spill also triggers on ErrPoolExhausted (defensive) and the doc is
  corrected. Stale "lazy ensurePipelinePool" comments reworded (eager, set-once).

Adds tests: spill accounts the Limiter exactly once, the clone's cap/timeout
overrides, and the UniversalOptions passthrough.

* fix(pipeline): review follow-ups (docs, de-flake, init-spill, stats)

- Options/FailoverOptions/RingOptions docs: pipeline pool is created by default
  (opt out PipelinePoolSize < 0); buffers default to max(regular, 64 KiB).
- pipeline_pool_gate_test.go: poll PoolStats instead of a fixed Sleep (de-flake).
- withPipelineConn now spills to the main pool on a pipeline-conn INIT failure
  too (e.g. a fresh dial refused with maxclients), not just Get saturation;
  non-saturation Get errors (ctx cancelled / pool closed) still do not spill.
- ClusterClient.PoolStats / Ring.PoolStats aggregate each node/shard pipeline
  pool into PoolStats.PipelineStats.

* fix(pipeline): review follow-ups (docs, de-flake, init-spill, stats)

- Options/FailoverOptions/RingOptions docs: pipeline pool is created by default
  (opt out PipelinePoolSize < 0); buffers default to max(regular, 64 KiB).
- pipeline_pool_gate_test.go: poll PoolStats instead of a fixed Sleep (de-flake).
- withPipelineConn now spills to the main pool on a pipeline-conn INIT failure
  too (e.g. a fresh dial refused with maxclients), not just Get saturation;
  non-saturation Get errors (ctx cancelled / pool closed) still do not spill.
- ClusterClient.PoolStats / Ring.PoolStats aggregate each node/shard pipeline
  pool into PoolStats.PipelineStats.
- Ring.PoolStats now folds WaitCount/WaitDurationNs/StaleConns into both the
  main-pool and pipeline-pool accumulators, matching the cluster fold (were
  dropped before).
- pipelinePoolOptions caps the pipeline PoolTimeout at DefaultPipelinePoolTimeout
  but honors a caller's SHORTER PoolTimeout (min), so a client tuned to spill
  faster is not forced to wait the full default before falling back.

* fix(pipeline): accept pipeline pool settings in URL options

ParseURL/ParseClusterURL/ParseFailoverURL now consume pipeline_pool_size,
pipeline_read_buffer_size and pipeline_write_buffer_size, so a URL such as
redis://host:6379?pipeline_pool_size=-1 applies the documented opt-out instead of
being rejected as an unexpected option. Adds TestParseURLPipelinePoolOptions.

* ci(govulncheck): use stable Go to pick up security patches

setup-go's "1.26.x" resolved to go1.26.5, which govulncheck flags for two
standard-library vulnerabilities fixed in go1.26.6: GO-2026-6090 (crypto/tls)
and GO-2026-5972 (encoding/asn1). Track the latest stable toolchain so future
security patches are picked up automatically instead of pinning a patch.

* fix(pool): size maintnotif defaults for the pipeline pool too

Handoff worker and queue defaults were derived from PoolSize alone, but
the manager hooks the dedicated pipeline pool as well, so its
connections competed for capacity sized as if only the main pool
existed. Options.init now derives the defaults from the combined
ceiling (PoolSize + effective PipelinePoolSize) and widens the
MaxActiveConns bound the same way, since the pipeline pool sits outside
it.

---------

Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com>
2026-08-24 14:42:40 +03:00

252 lines
8.5 KiB
Go

package redis_test
import (
"context"
"testing"
"github.com/redis/go-redis/v9"
)
// skipWithoutRedis skips the test when no server answers at apTestAddr(), so a
// unit-only run doesn't turn missing integration infrastructure into a
// failure. It probes with a DEFAULT-options client, not the test's configured
// one: probing with the client under test would convert a regression in the
// very options being exercised (buffer sizes, pipeline pool) into a silent
// skip while the server is up.
func skipWithoutRedis(t *testing.T, ctx context.Context, _ *redis.Client) {
t.Helper()
probe := redis.NewClient(&redis.Options{Addr: apTestAddr()})
defer probe.Close()
if err := probe.Ping(ctx).Err(); err != nil {
t.Skipf("no redis at %s: %v", apTestAddr(), err)
}
}
// TestPipelineBufferSizes verifies that enabling the pipeline buffer options keeps
// pipelining working end-to-end. It does not assert the actual socket buffer sizes
// (not observable here) - only that the configured client still runs pipelines.
func TestPipelineBufferSizes(t *testing.T) {
ctx := context.Background()
// Create client with custom pipeline buffer sizes
client := redis.NewClient(&redis.Options{
Addr: apTestAddr(),
ReadBufferSize: 64 * 1024, // 64 KiB for regular connections
WriteBufferSize: 64 * 1024, // 64 KiB for regular connections
PipelineReadBufferSize: 512 * 1024, // 512 KiB for pipeline connections
PipelineWriteBufferSize: 512 * 1024, // 512 KiB for pipeline connections
PipelinePoolSize: 5, // Small pool for pipelining
})
defer client.Close()
skipWithoutRedis(t, ctx, client)
cleanupBufferTestKeys(t, client)
// Test that regular commands work
err := client.Set(ctx, "test_key", "test_value", 0).Err()
if err != nil {
t.Fatalf("Failed to set key: %v", err)
}
val, err := client.Get(ctx, "test_key").Result()
if err != nil {
t.Fatalf("Failed to get key: %v", err)
}
if val != "test_value" {
t.Fatalf("Expected 'test_value', got '%s'", val)
}
// Test that pipeline works
pipe := client.Pipeline()
pipe.Set(ctx, "pipe_key1", "value1", 0)
pipe.Set(ctx, "pipe_key2", "value2", 0)
pipe.Get(ctx, "pipe_key1")
pipe.Get(ctx, "pipe_key2")
cmds, err := pipe.Exec(ctx)
if err != nil {
t.Fatalf("Pipeline execution failed: %v", err)
}
if len(cmds) != 4 {
t.Fatalf("Expected 4 commands, got %d", len(cmds))
}
// Verify results
if cmds[2].(*redis.StringCmd).Val() != "value1" {
t.Fatalf("Expected 'value1', got '%s'", cmds[2].(*redis.StringCmd).Val())
}
if cmds[3].(*redis.StringCmd).Val() != "value2" {
t.Fatalf("Expected 'value2', got '%s'", cmds[3].(*redis.StringCmd).Val())
}
t.Log("Pipeline with custom buffer sizes works correctly")
}
// TestNoPipelinePool verifies that client works without pipeline pool (backward compatibility)
func TestNoPipelinePool(t *testing.T) {
ctx := context.Background()
// Create client WITHOUT custom pipeline buffer sizes
client := redis.NewClient(&redis.Options{
Addr: apTestAddr(),
ReadBufferSize: 64 * 1024, // 64 KiB for all connections
WriteBufferSize: 64 * 1024, // 64 KiB for all connections
// No PipelineReadBufferSize or PipelineWriteBufferSize
})
defer client.Close()
skipWithoutRedis(t, ctx, client)
cleanupBufferTestKeys(t, client)
// Test that pipeline still works (using regular pool)
pipe := client.Pipeline()
pipe.Set(ctx, "no_pipe_pool_key1", "value1", 0)
pipe.Set(ctx, "no_pipe_pool_key2", "value2", 0)
pipe.Get(ctx, "no_pipe_pool_key1")
pipe.Get(ctx, "no_pipe_pool_key2")
cmds, err := pipe.Exec(ctx)
if err != nil {
t.Fatalf("Pipeline execution failed: %v", err)
}
if len(cmds) != 4 {
t.Fatalf("Expected 4 commands, got %d", len(cmds))
}
// Verify results
if cmds[2].(*redis.StringCmd).Val() != "value1" {
t.Fatalf("Expected 'value1', got '%s'", cmds[2].(*redis.StringCmd).Val())
}
if cmds[3].(*redis.StringCmd).Val() != "value2" {
t.Fatalf("Expected 'value2', got '%s'", cmds[3].(*redis.StringCmd).Val())
}
t.Log("Pipeline without custom buffer sizes (backward compatibility) works correctly")
}
// TestPipelinePoolStats verifies that PoolStats includes pipeline pool stats
func TestPipelinePoolStats(t *testing.T) {
ctx := context.Background()
// Create client with custom pipeline buffer sizes
client := redis.NewClient(&redis.Options{
Addr: apTestAddr(),
ReadBufferSize: 64 * 1024, // 64 KiB for regular connections
WriteBufferSize: 64 * 1024, // 64 KiB for regular connections
PipelineReadBufferSize: 512 * 1024, // 512 KiB for pipeline connections
PipelineWriteBufferSize: 512 * 1024, // 512 KiB for pipeline connections
PipelinePoolSize: 5, // Small pool for pipelining
})
defer client.Close()
skipWithoutRedis(t, ctx, client)
cleanupBufferTestKeys(t, client)
// Execute some pipeline commands
pipe := client.Pipeline()
for i := 0; i < 10; i++ {
pipe.Set(ctx, "stats_key", "value", 0)
}
_, err := pipe.Exec(ctx)
if err != nil {
t.Fatalf("Pipeline execution failed: %v", err)
}
// Get pool stats
stats := client.PoolStats()
if stats == nil {
t.Fatal("PoolStats returned nil")
}
// Verify pipeline stats are included
if stats.PipelineStats == nil {
t.Fatal("PipelineStats is nil - pipeline pool stats not included")
}
t.Logf("Regular pool stats: TotalConns=%d, IdleConns=%d, Hits=%d, Misses=%d",
stats.TotalConns, stats.IdleConns, stats.Hits, stats.Misses)
t.Logf("Pipeline pool stats: TotalConns=%d, IdleConns=%d, Hits=%d, Misses=%d",
stats.PipelineStats.TotalConns, stats.PipelineStats.IdleConns,
stats.PipelineStats.Hits, stats.PipelineStats.Misses)
// Verify pipeline pool has connections
if stats.PipelineStats.TotalConns == 0 {
t.Error("Pipeline pool has no connections")
}
t.Log("PoolStats includes pipeline pool stats correctly")
}
// TestNoPipelinePoolStats verifies that PoolStats works without pipeline pool.
// The pool is now created by default, so having none requires the explicit
// opt-out (PipelinePoolSize < 0).
func TestNoPipelinePoolStats(t *testing.T) {
ctx := context.Background()
// Opt out of the dedicated pipeline pool: pipelines run on the main pool.
client := redis.NewClient(&redis.Options{
Addr: apTestAddr(),
ReadBufferSize: 64 * 1024, // 64 KiB for all connections
WriteBufferSize: 64 * 1024, // 64 KiB for all connections
PipelinePoolSize: -1,
})
defer client.Close()
skipWithoutRedis(t, ctx, client)
cleanupBufferTestKeys(t, client)
// Execute some commands
err := client.Set(ctx, "test_key", "test_value", 0).Err()
if err != nil {
t.Fatalf("Failed to set key: %v", err)
}
// Get pool stats
stats := client.PoolStats()
if stats == nil {
t.Fatal("PoolStats returned nil")
}
// Verify pipeline stats are nil (no pipeline pool)
if stats.PipelineStats != nil {
t.Error("PipelineStats should be nil when no pipeline pool is configured")
}
t.Logf("Regular pool stats: TotalConns=%d, IdleConns=%d, Hits=%d, Misses=%d",
stats.TotalConns, stats.IdleConns, stats.Hits, stats.Misses)
t.Log("PoolStats works correctly without pipeline pool")
}
// cleanupBufferTestKeys removes this file's fixed test keys before and after a
// test. These tests share a Redis instance with the rest of the package (and
// "test_key" is also used by autopipeline_test.go), so leaving state behind
// makes results order-dependent — the same class of leak that produced a real
// CI flake in the search suite (Copilot review on #3942).
//
// Call it AFTER skipWithoutRedis: it talks to the server, so running it first
// turns a no-Redis environment into a hard failure instead of the intended
// skip (also Copilot, on the first cut of this cleanup).
func cleanupBufferTestKeys(t *testing.T, client *redis.Client) {
t.Helper()
keys := []string{
"test_key", "stats_key",
"pipe_key1", "pipe_key2",
"no_pipe_pool_key1", "no_pipe_pool_key2",
}
ctx := context.Background()
if err := client.Del(ctx, keys...).Err(); err != nil {
t.Fatalf("cleanup keys: %v", err)
}
// The post-test sweep gets its OWN client: t.Cleanup runs after the test
// function's defers, and every caller here defers client.Close(), so a
// cleanup issued on the passed-in client would land on a closed one and
// silently do nothing — leaving exactly the keys this helper exists to
// remove (Copilot review on #3942).
t.Cleanup(func() {
c := redis.NewClient(&redis.Options{Addr: apTestAddr()})
defer c.Close()
if err := c.Del(context.Background(), keys...).Err(); err != nil {
t.Errorf("post-test key cleanup failed, later tests may see stale keys: %v", err)
}
})
}