feat(csc): Refresh and miss coalescing (#3989)

* feat(csc): refresh-on-invalidate and reader-miss coalescing

Two additions over the shared-tracking client-side cache, both inert
unless CSC is enabled.

Refresh-on-invalidate re-fetches recently-read keys as soon as their
invalidation push arrives, instead of leaving them to be reloaded by the
next reader that misses. Invalidated cache keys that were valid and read
within the recency horizon are collected under the shard lock as they are
deleted, batched over a fixed window from the first collect, and flushed
when the window expires, when a size cap is hit, or on demand the moment a
reader misses a key still sitting in the window. Refetches reserve each
cache key before touching the network and release any reservation they do
not publish, so a reader never blocks on an orphaned in-progress entry.
Reads run on a background pipelined connection. Enabled via
Options.ClientSideCacheRefreshOnInvalidate; window and demand behaviour
tunable through GOREDIS_CSC_REFRESH_WINDOW_MS and GOREDIS_CSC_REFRESH_DEMAND.

Reader-miss coalescing pipelines concurrent cache-miss reads onto a single
CSC-tracked main-pool connection. Each miss is sent as the caller's own
per-key command (not rewritten to MGET, so it is cluster-safe); every reply
is applied to its caller's command and published to the cache under the
connection's captured tracking generation, so entries stay invalidatable.
initConn issues CLIENT TRACKING ON for every pooled connection while CSC is
active, so the main pool is tracked; using it explicitly (rather than the
pipeline pool) keeps the path correct even if a later change makes a
dedicated pipeline pool deliberately untracked, since publishing on an
untracked connection would serve stale until TTL. Enabled via
GOREDIS_CSC_COALESCE_MISSES; worker count via GOREDIS_CSC_COALESCE_WORKERS
(default 8).

Adds LocalCache.deleteByRedisKeyCollectingHot / LRUClock / InvalidationStats
and per-cache invalidation counters to support the above, with unit coverage
of the recency filter and an integration test asserting a refreshed entry is
itself invalidatable.

* feat(csc): full-duplex reader-miss coalescing + invalidation batching (#3965)

* docs: point security reports at Redis VDP (#3949)

* fix(pool): log buffered push data at debug level (#3948)

The buffered-push-data notice in isHealthyConn fired unconditionally.
  With client-side caching every tracked write parks an invalidate frame
  on idle pooled conns, so at default log level the line floods the log
  on nearly every pool Get under write-heavy workloads. It documents the
  normal healthy path, not a problem - gate it behind LogLevelDebug like
  the equivalent notices in maintnotifications.

* feat(csc): reader-miss coalescing modes (workers + full-duplex) and invalidation batching

Builds on the CSC refresh-on-invalidate + base miss-coalescing PR. Adds:

- Coalescing MODES for the reader-miss path, selected by config:
  * "workers" (default): a small pinned worker pool fetches reserved misses.
  * "fullduplex": one held connection with a writer + reader goroutine pair
    pipelines the reserved misses (concurrent, replies streamed back). Ordering
    is preserved: the miss-coalescer's reservation dedups concurrent misses of
    the same key to a single fetch, and each caller blocks on its own request's
    completion — the caching client is blocking per goroutine, so no goroutine
    ever sees its own reads reordered; full-duplex only overlaps independent
    goroutines' fetches on the wire.
- Invalidation batching: coalesce invalidation-driven cache deletes within a
  configurable window instead of one delete per push.

Config (no environment variables): new AutoPipeline-free Options fields —
ClientSideCacheCoalesceMisses (enable), ClientSideCacheCoalesceMode
("workers"/"fullduplex"), ClientSideCacheCoalesceWorkers, and
ClientSideCacheInvalidationBatchWindow. Prototype-only read-path telemetry
(READPATH_LOG + LocalCache stat counters) removed; no stats surface added.

Measured on a 50ms-RTT WAN proxy under invalidation churn, the coalescing fixes
turn published v9.22.0's miss-stampede collapse (p99 up to ~1.1s, throughput
floored) into ~5x throughput at ~1 RTT p99, matching rueidis; the full-duplex
mode gives the tighter mid-range tail at fewer connections. See
AP_CSC_TWOCLIENT_VS_RUEIDIS.md.

Depends on: feature/csc-refresh-and-miss-coalescing.

* fix(csc): address #3965 review and lint in coalescing modes

- lint (the only red CI check): drop the redundant `chan struct{}` type from the
  recycle declaration (ST1023) and //nolint:unused the deliberately-off, measured
  cscRefreshCooldown knob.
- Reject the "pinned" PROTOTYPE engine from the public ClientSideCacheCoalesceMode
  option: it holds a connection with no idle invalidation drain and can serve
  stale values (cursor HIGH). It now falls back to "workers" and is reachable
  only via an internal benchmark hook (cscForcePinned).
- Pass the new CSC miss-coalescing / invalidation-batching knobs through
  UniversalOptions.Simple() (ClientSideCacheRefreshOnInvalidate, CoalesceMisses,
  CoalesceMode, CoalesceWorkers, InvalidationBatchWindow) so UniversalClient
  users can enable them (codex P2).
- Give the invalidation batcher a stop path: it is stopped and cleared when the
  last user releases the binding (releaseLocked), and ensureBatcher refuses to
  start one for an already-released binding, so its goroutine no longer lives
  past the binding re-arming its timer forever; a later re-acquire starts fresh.
- Full-duplex session: acquire the pool connection only after the first miss
  arrives, and release it if CSC serving was disabled meanwhile, so an idle
  session no longer starves a small pool (PoolSize:1) until PoolTimeout.

Adds unit tests for the pinned-mode rejection and the UniversalOptions passthrough.

* fix(csc): FD apply-and-gate on id/gen mismatch; drain buffered pushes; doc

- Full-duplex readOne now applies the read reply and lets fulfillCached gate
  only the cache publish on the captured conn id/generation -- matching the
  workers/pinned engines -- instead of failing the caller with ErrClosed and
  losing a good reply on a mid-flight id/gen change.
- peekAndProcessPushNotifications also drains when the reader has buffered bytes
  (HasBufferedData), not only when the socket is readable (MaybeHasData), so a
  buffered invalidation is processed on the idle tick.
- Workers-mode doc corrected: a tracked connection is acquired/released per
  batch, not held across batches.

* fix(search): read the full RESP3 map reply in FTHybridCmd (#3956)

FTHybridCmd.readReply used ReadSlice, which reads a RESP3 map header's
length as an element count and consumes only half its frames, leaving the
connection desynced. Peek the type and read a map with ReadReply, like the
other FT.* parsers.

* fix(csc): retry uncached on mid-miss disable; guard cancelled fetches

- A coalesced miss re-runs uncached when CSC serving is disabled after the miss
  was reserved (RESP3 downgrade / CLIENT TRACKING loss during a conn re-init),
  via an internal retry-uncached sentinel that processCached catches, instead of
  surfacing a spurious pool.ErrClosed for a valid cacheable read.
- A caller whose context cancels mid-fetch no longer races the coalescer: a CAS
  interlock (claimAbandon/claimApply) hands the Cmder to exactly one of the caller
  or the applying worker, so the worker never writes a Cmder the caller has taken
  back. The reply is still classified and published to the shared cache, and a
  cancelled Get returns the context error deterministically (matching the
  non-coalesced path).
- A fetch after coalescer shutdown returns pool.ErrClosed instead of hanging on a
  post-drain enqueue race.
Adds TestClassifyCachedReply, TestCSCMissReqClaimInterlock,
TestCSCMissCoalesceAbandonedFetchNoRace, TestFullDuplexDisabledMidMissRetriesUncached.

* fix(csc): drop batched deletes on flush; rebuild batcher on window change

- A full cache flush (FLUSHDB/FLUSHALL) cleared the cache but left the
  invalidation batcher's queued per-key deletes, which then fired and evicted
  entries repopulated after the flush (an extra miss within the window). Drop the
  batcher's pending queue on flush.
- A running batcher's window is fixed at creation, so a second client binding to
  the same shared handler with a stricter window kept the old cadence and its
  staleness bound did not hold. setInvalBatchWindow now drops the running batcher
  on a window change so the next invalidation starts a fresh one with the new
  window.

* ci(govulncheck): use stable Go to pick up security patches

setup-go's "1.26.x" resolved to go1.26.5, which govulncheck flags for two
standard-library vulnerabilities fixed in go1.26.6: GO-2026-6090 (crypto/tls)
and GO-2026-5972 (encoding/asn1). Track the latest stable toolchain so future
security patches are picked up automatically instead of pinning a patch.

* fix(csc): preserve queued deletes on rebuild; honor configured timeouts

- The invalidation batcher's stop path drains keys still buffered in its channel
  into the final flush, so a window-change rebuild loses no queued deletes
  (they would otherwise serve pre-invalidation values until TTL/MaxStaleness).
  Adds TestInvalBatchStopAppliesQueuedDeletes.
- ensureBatcher reads the window under the handler lock (not the caller's
  pre-lock snapshot) and returns nil at window 0, so a concurrent tighten or
  disable cannot be undone by a batcher rebuilt with the stale cadence.
- setInvalBatchWindow folds windows strictest-wins across attached clients
  (explicit 0/inline strictest, then smaller nonzero): a later, looser attach
  can no longer lengthen batching past an earlier client's staleness bound.
  The effective window resets when the last user releases.
- The miss-coalescer flush budget derives from the configured WriteTimeout +
  ReadTimeout (plus a 5s pool-Get floor) instead of a fixed 5s, so clients with
  deliberately long timeouts do not see only coalesced misses clipped early.

* fix(csc): batcher stop/release safety; FD idle release; flush budget

- enqueue/stop interlock: enqueue sends under stopMu.RLock and stop flips a
  stopped flag under stopMu.Lock before closing stopCh, so a key sent by a
  handler holding a stale batcher pointer lands before the close (applied by the
  stop-drain) and a post-stop enqueue applies inline — no delete is parked in a
  channel nothing drains. Adds TestInvalBatchEnqueueAfterStopAppliesInline.
- The batcher snapshots cache/refresh at creation and apply uses the snapshot,
  so the release-time stop-drain still lands queued deletes on a shared cache
  after releaseLocked has cleared the binding (a successor reusing that cache
  would otherwise serve pre-invalidation values until TTL/MaxStaleness).
- Full-duplex coalescer sessions release their held connection after an idle
  grace (cscFullDuplexSessionIdle, 1s) instead of holding it toward the 30s
  recycle age: at PoolSize:1 a non-cacheable command otherwise blocks until
  PoolTimeout. The next miss re-acquires; an idle coalescer holds zero
  connections. Adds TestFullDuplexSessionReleasesIdleConn.
- batchBudget includes the configured PoolTimeout alongside Write/ReadTimeout
  (5s floor for scheduling only), so a coalesced flush's pool Get waits the same
  saturation budget as the normal path, including ReadTimeout: -1 setups.

* fix(csc): close coalescer lifecycle races; honor pool budget on acquire

- cscMissCoalescer is an atomic.Pointer: processCached loads it once (a cache
  miss racing Client.Close could observe non-nil then call fetch on nil — a
  panic and a data race); stop Swaps to nil so exactly one closer wins.
- Connection acquisition in all three engines goes through acquireCtx:
  max(5s, PoolTimeout) so a configured pool budget above 5s is honored under
  temporary saturation (matching the normal command path), and cancelled when
  the coalescer stops so Close is not stalled behind a blocked pool Get. The
  workers engine's per-batch Get is split out of the batch write/read budget
  (batchBudget covers write+read only again).
- The runtime cleanup for a client dropped without Close also stops the
  coalescer workers (idempotent stopWorkers), so a forgotten client no longer
  leaks the worker goroutines and everything they retain (cache, pools).
- Document that ClientSideCacheCoalesceMisses requires the built-in LocalCache;
  with a custom Cache implementation the option is ignored.

* fix(csc): coalescer latency, shutdown and clone fixes

- fullDuplexLoop backs off only after an error end; a clean idle/recycle end
  starts the next session immediately instead of adding a flat 5ms to a miss
  already queued.
- flushBatch no longer routes the engine's bounded batch ctx through
  c.context(): with the default ContextTimeoutEnabled=false that stripped the
  deadline, and with ReadTimeout disabled a non-replying server wedged the
  worker in ReadRawReply and Close in wg.Wait.
- baseClient.clone carries the miss coalescer (atomic.Pointer Store of the
  owner's Load), so WithTimeout clones keep coalescing instead of silently
  falling back to per-caller fetches; lifecycle stays owner-only.
- Document that ClientSideCacheInvalidationBatchWindow requires the built-in
  LocalCache (matching the miss-coalescing option).

* fix(csc): make FD session I/O interruptible; refresh rebuild; stat CAS

- The full-duplex session writer/reader pass the engine's cancellable session
  ctx directly instead of through c.context(), which stripped it under the
  default ContextTimeoutEnabled=false; and since no context can interrupt a
  deadline-less socket read, the supervisor's stop path now bounds the graceful
  drain — after mc.stop it waits up to the batch budget and then closes the held
  connection. Close always unblocks: promptly on a healthy drain, within the
  budget against a server that never replies.
- setRefreshQueue drops a running batcher when the refresh binding changes
  (like setInvalBatchWindow), so a client attaching refresh-on-invalidate later
  does not leave batched deletes permanently feeding a nil refresher.
- countBatch's max-batch statistic uses a CAS loop, so concurrent workers cannot
  commit a smaller maximum over a larger one.

* fix(csc): join FD supervisor before release; drain queue in GC cleanup

- The FD session joins its supervisor goroutine (supDead) after closing
  superDone and BEFORE releasing the connection: the deferred scancel makes
  sctx.Done() ready at return, and a still-parked supervisor could pick it over
  the equally-ready superDone and close a healthy connection already returned to
  the pool — possibly under an unrelated command.
- The GC cleanup for a client dropped without Close drain-cancels the miss queue
  after stopping the workers (non-blocking), so a ctx-cancelled caller's queued
  request cannot strand an IN_PROGRESS reservation that would block another
  client sharing the injected cache until StaleTimeout.

* fix(csc): clear the push-probe deadline for negative ReadTimeout

A negative ReadTimeout means every WithReader skips SetReadDeadline, so the 1ms
deadline peekAndProcessPushNotifications installs for its probe stayed on the
socket and poisoned the next reply read (a spurious i/o timeout on a held
full-duplex connection, or on a pooled conn reused right after a drain). Clear
the deadline after the probe for negative-ReadTimeout clients; timeout>=0
callers re-arm their own on the next read. Fixed at the source, covering the
FD coalescer tick, the autopipeline paths and the CSC hit-drain.

* refactor(csc): full-duplex is the only miss-coalescer engine

Misses are caller-blocking — a real request waits on every fetch — so the
latency-first engine is the right one, always: a lone miss is written
immediately, batching stays opportunistic (packs only what is already queued),
and new misses stream out while earlier replies are in flight (~1 RTT per
miss, no batch phase-lock, no pool Get on the hot path).

Remove the alternatives and their configuration surface:
- the "workers" engine (N pooled conns, half-duplex batches): its per-batch
  round trip added a full RTT of tail latency to misses arriving mid-flight,
  and its batching advantage is preserved by the FD writer's opportunistic
  packing;
- the "pinned" benchmark prototype (no idle invalidation drain; could serve
  stale after an invalidation while idle; never publicly selectable);
- Options.ClientSideCacheCoalesceMode and ClientSideCacheCoalesceWorkers (both
  unreleased experimental fields) and their UniversalOptions mirrors:
  ClientSideCacheCoalesceMisses now selects the one engine, no tuning needed.

CSCMissCoalesceStats.Mode becomes Active. Background invalidation traffic is
unaffected: it keeps the batching-first windowed batcher and refresh queue.

* fix(csc): held-conn probe safety and lifetime bounds

- checkForData no longer resets the connection deadline: the probe is a raw
  MSG_PEEK|MSG_DONTWAIT syscall that never touches net.Conn deadlines, and on a
  held full-duplex connection the reader's idle tick ran it CONCURRENTLY with
  the writer inside WithWriter — stripping the armed write deadline and letting
  a blocked write hang past its timeout. connCheck's own reset stays (it runs
  on idle pool-owned conns).
- peekAndProcessPushNotifications falls back to a throttled timed drain
  (TakeCscPeriodicReadPending, the CSC drainer's opaque-transport discipline)
  when socket readiness cannot be inspected (Windows; dialer wrappers exposing
  neither syscall.Conn nor NetConn), so a push addressed to the held connection
  is consumed within the fallback interval instead of sitting until the next
  reply.
- The full-duplex session's hold is bounded by the connection's REMAINING
  absolute lifetime via a new pool.Conn.ExpiresAt() accessor (jitter included),
  floored at 10ms — a near-expiry pooled connection is no longer held for a
  fresh full recycle age past the expiry the pool's reaper enforces.

* fix(csc): scope idle-drain fallback to held conns; drainer deadline hygiene

- The opaque-transport timed-drain fallback moves out of the shared
  peekAndProcessPushNotifications and into the full-duplex session tick, the
  one place a connection has no alternative drain coverage. The blanket
  fallback made every push path run throttled timed drains on uninspectable
  transports (including mock connections), perturbing sequenced push handling
  — the maintnotifications downgrade test caught it in CI. Pooled push paths
  keep their drainer/health-check coverage. The timed drain itself is factored
  into timedPushDrain, shared by both callers.
- drainPushNotifications explicitly clears the socket deadline after its
  hard-deadline probe/drain for clients with a negative ReadTimeout (which
  never re-arm): the SetDeadline reset removed from checkForData — because it
  clobbered a concurrent writer's deadline on held full-duplex connections —
  had been clearing this residue incidentally on the next drainer tick. This
  is the explicit, correctly-scoped replacement.

* refactor(csc): drop the public coalescer stats API

Observability rides the client's normal telemetry — coalesced misses fire the
otel operation-duration and error callbacks like any other command path — so
CSCMissCoalesceStats (an unreleased experimental type) duplicated that surface
for no consumer. The engine's internal counters stay for the in-package tests,
which read them directly off the coalescer.

* refactor(csc): drop the public coalescer stats API

Observability rides the client's normal telemetry — coalesced misses fire the
otel operation-duration and error callbacks like any other command path — so
CSCMissCoalesceStats (an unreleased experimental type) duplicated that surface
for no consumer. The engine's internal counters stay for the in-package tests,
which read them directly off the coalescer.

Also: acquireCtx folds in the pool's full dial budget (DialerRetries x
(DialTimeout + DialerRetryTimeout) + DialTimeout) so a slow endpoint gets its
configured dial-retry sequence instead of being cancelled mid-sequence; and the
ClientSideCacheCoalesceMisses GoDoc documents pool sizing for the held
connection (PoolSize 1 + sustained misses starves non-cacheable commands).

Also compacts the review-round comments in the touched files (invariants and
contracts preserved; narration removed).

* fix(csc): bound the recycle-path drain like the stop path

Reaching the recycle age signaled the recycle and exited the supervisor while a
reader could stay blocked in a deadline-less read on a non-replying server —
with the supervisor gone nothing would ever close the conn, and a later Close
hung in wg.Wait. The recycle path now waits up to the batch budget for the
session to drain and then closes the connection, mirroring the stop path. Also
documents that a custom DialerRetryBackoff exceeding DialerRetryTimeout should
raise PoolTimeout (an opaque callback's total delay is not observable from
options).

* fix(csc): progress-based drain backstop; departial test flake

- The stop/recycle drain backstop force-closed the conn after one fixed
  batch budget, which could cut a HEALTHY drain of a deep in-flight
  pipeline (up to cscFullDuplexDepth replies, ~in-flight x RTT,
  possibly under maintenance-relaxed timeouts). The supervisor now
  samples the in-flight queue per budget interval: any consumed reply
  extends the wait; only a zero-progress interval closes the conn.
- TestReleaseConnRemovesConnectionAfterPartialPushRead skips the benign
  case where the drain probe times out before consuming any byte: the
  frame is intact and re-pooling is safe; only a Put after bytes moved
  into the reader is the desync bug the test pins.

* fix(pool): clear residual read deadline in checkForData

Removing the SetDeadline reset left a prior WithReader's read deadline
armed; once expired, rawConn.Read fails fast with i/o timeout before
the non-blocking peek runs, the CSC drainer treats the error as fatal
and removes an idle-but-healthy conn, evicting its cache coverage. The
full-duplex coalescer concentrates coverage on one held conn, so one
spurious removal wiped the whole cache (WAN: hit rate 25%, throughput
cut to a third). Clear only the READ deadline: the write deadline stays
untouched because checkForData runs concurrently with command writes on
a held full-duplex connection - which is what the old full SetDeadline
reset clobbered. Regression test pins the expired-deadline case.

* fix(csc): claim FD writes; progress-aware drain backstop

- The abandon interlock gated only the reply side, so the session
  writer could serialize an abandoned caller's cmd args - a
  use-after-return on mutable args (e.g. a []byte key), with the reply
  publishable under the original cache key. The writer now claims each
  request (PENDING->WRITING) around arg serialization, releases before
  the flush, and drops abandoned requests with their reservation
  cancelled; abandonOrWait yields through a WRITING claim, which spans
  one in-memory batch encode.
- The drain backstop measures progress as completed reads (readsDone)
  instead of len(inflight), which was blind to the reader's active
  read, and its interval honors the connection's effective
  (maintenance-relaxed) read timeout via pool.Conn.EffectiveReadTimeout
  so a legitimately deep read is not cut short.
- The partial-push-read test completes the frame and requires a whole-
  frame parse before skipping: an empty reader buffer alone does not
  prove the probe consumed nothing.

* fix(csc): snapshot miss wire form at enqueue

Replace the WRITING ownership claim with a wire snapshot taken in
fetch, while the caller still owns cmd: the session writer writes only
engine-owned bytes and never reads cmd, so no ownership window spans
arg serialization (which could implicitly flush and block on the
socket), abandoning callers return immediately, and a post-abandon arg
mutation can neither reach the wire nor publish under the original
cache key. Abandoned fetches complete again, so their reservations
settle instead of stranding IN_PROGRESS behind the miss backlog.

Also: fetch's Close-race branch drains the queue after cancelling (a
request landing after the shutdown drain is not retained via a live
WithTimeout clone), and the recycle backstop honors explicitly
disabled read deadlines - it waits for the session instead of force-
closing on a zero-progress interval, escalating only once stop fires
so Close still terminates.

* fix(csc): harden FD session recycle and probe paths

- timedPushDrain reads under a hard deadline: WithReader let an active
  maintenance-relaxed timeout replace the 1ms probe cap, parking the FD
  session reader for the relaxed duration on a no-data probe while it
  held the connection. WithReaderHardDeadline bypasses relaxation and
  clears the deadline on exit (the negative-ReadTimeout cleanup block
  is now redundant and removed).
- The supervisor also watches the recycle channel, so a
  reader-triggered recycle (handoff/close-on-put) gets the same bounded
  progress-based drain backstop as the age-triggered path - a stalled
  in-flight reply can no longer postpone a requested handoff until
  Close.
- The deadline-less recycle carve-out tests rt <= 0: Options.init
  normalizes ReadTimeout -1 (indefinite) to 0, so the indefinite mode
  was still force-closed after one budget interval.

* fix(csc): epoch inval drop; refresh ownership; close order

- Queued invalidations carry the batcher epoch they were enqueued
  under; drop() bumps the epoch instead of draining, and apply skips
  stale-epoch items (queue and in-progress batch alike, so the dropCh
  signal is gone). The flush handler bumps before cache.Flush(), so a
  per-key invalidation racing in from another tracked connection after
  the flush keeps its delete - the drain-based drop could discard it
  and leave a repopulated entry stale. Dedup is per key+epoch so a
  post-flush re-arrival is not swallowed by the duplicate check.
- stopCSCRefresher clears the shared handler's refresh binding only
  while it still points at the closing client's own queue: a sibling
  client sharing the cache/processor keeps its refresher fed.
- Teardown deactivates cscActive before stopping the coalescer, and
  fetch's stop paths settle with errCSCRetryUncached so a clone's miss
  racing the teardown window re-runs uncached instead of failing with
  ErrClosed on an open pool; the mid-apply branch returns the settle
  result itself rather than discarding a successfully applied reply.

* fix(csc): serialize inval apply with drop; FD handler adapter

- apply/drop are serialized by a batcher mutex: apply snapshotting the
  epoch once let a concurrent FLUSH land mid-batch and stale-epoch
  deletes still run post-flush, evicting repopulations - the case the
  epoch exists to prevent. drop() holding the mutex means an in-flight
  batch finishes before the flush (harmless: the flush wipes it) and
  every later apply sees the new epoch.
- Both FD-session push paths (the reader's per-reply drain and
  timedPushDrain) hand handlers the nonblocking cscHandlerClient
  adapter like the background drainer does: a custom push handler
  calling Close() signaled-and-deferred instead of deadlocking on
  mc.wg.Wait while the session reader is parked inside that handler.

* fix(csc): retry-uncached drains; refresh stack; weak close

- The shutdown drain and the GC cleanup settle queued misses with
  errCSCRetryUncached, matching fetch's stop paths: a caller woken
  during the teardown window re-runs its read on the still-open pool
  instead of surfacing ErrClosed.
- The shared handler tracks refresh bindings in a stack: clearing the
  ACTIVE binding restores the next-newest live sibling instead of
  nil, so closing the newest owner no longer severs an older client's
  still-running refresher (the identity check alone only protected the
  reverse order).
- The push-handler adapter closes through the canonical *Client via a
  weak back-pointer: Client.Close also stops the cached autopipeliners,
  which baseClient.Close bypassed, leaving flush goroutines running
  against closed pools. Weak so the wrapper stays collectible and the
  drop-without-Close cleanup still fires.

* fix(csc): write-side backstop; probe gate; clone bypass

- The drain backstop honors the write side too: the interval covers the
  connection's effective write timeout (pool.Conn.EffectiveWriteTimeout)
  and the deadline-less carve-out fires when either read or write
  deadlines are disabled - a deadline-free write blocked flushing a
  large request shows no read progress by construction and must not be
  cut by an age/handoff recycle.
- The opaque-transport speculative probe runs only with the built-in
  push processor: a custom processor may surface the empty-probe
  timeout per its contract, which would remove and redial a healthy
  session on every idle probe.
- withTimeout clones whose timeouts diverge from the owner's bypass
  miss coalescing: the shared engine reads the owner's options, so a
  coalesced miss would honor the owner's deadlines, defeating
  WithTimeout. Hits and uncached fetch caching are unchanged.
- releaseLocked clears refresh/refreshStack with the binding: a client
  dropped without Close must not leave a dead queue for a successor to
  inherit or restore.
- The drop-without-Close cleanup drain settles with errCSCRetryUncached
  like every other stop path (the round-22 hunk for this block was a
  silent patch no-op; applied for real).

* fix(csc): honor lifetime jitter in the FD recycle age

Capping the recycle age at the raw ConnMaxLifetime before consulting
ExpiresAt collapsed every positive-jitter connection back to the
unjittered lifetime, re-synchronizing session recycles across clients
started together - the herd ConnMaxLifetimeJitter exists to prevent.
Bound the age by the connection's actual absolute expiry only (jitter
included), subject to the independent 30s session cap.

* fix(csc): fail the FD session on push-processor errors

Log-and-continue past a surfaced push-processor error let the reader
proceed into ReadRawReply on a possibly desynchronized stream: a
processor that consumed part of a push before failing leaves the next
bytes mid-frame, and a fragment could be applied as a caller's reply
and published to the cache. Treat the error as session-fatal (close and
remove the connection), matching drainPushNotifications - the built-in
processor surfaces only mid-frame failures, and a custom processor's
contract cannot prove no bytes were consumed.

* feat(csc): native OTel attribution for coalesced misses

A coalesced miss returned from processCached without touching
processState, so the native recorder saw zero attempts and a nil
connection (empty server.address), and engine-failed requests bypassed
the error callback that processWithRetry emits. fetch now returns the
serving session connection (recorded by the reader before the done
settle, which is the happens-before edge) and processCached stamps it
into processState as one attempt; applyAndSettle and settleErr emit the
native error callback with parity to processWithRetry and the FD
autopipeline engine's failReqs - the retry-uncached sentinel is
excluded, since its command re-runs on the fully instrumented path.

Refs the deferred-observability thread on #3965.

* fix(csc): FD miss-coalescer review fixes

- Attribute write-side failures to the held session connection: the miss
  coalescer sets req.servedBy for the whole batch before the write, so a
  WithWriter error (or ctx cancel during the inflight handoff) settles with the
  connection attribution the reply path records, not a nil conn / zero attempts.

- Tolerate fragmented push frames on held sessions: once readiness is
  established, the drain uses the longer cscDrainHardReadCap budget (as the
  background drainer does) instead of the 1ms probe cap, which could time out
  mid-frame over TLS and fatally close a healthy session. The 1ms cap is
  reserved for speculative no-data probes (opaque transports).

- Route a WithTimeout clone's CSC push-handler Close through the owner wrapper:
  the clone's cscClientWeak now points at the canonical owner, so closeCanonical
  calls owner.Close() (full CSC + autopipeliner teardown) instead of
  baseClient.Close closing shared pools while the owner's drainer runs.

* fix(csc): harden refresh-on-invalidate push drain

Two fixes to the refresh-on-invalidate reply reader, matching the
miss-coalescer / background-drainer policy:

- Propagate push-processor errors instead of logging and continuing. A
  surfaced error means bytes may have been consumed mid-frame; reading the next
  reply on a desynchronized stream could publish a push fragment under the
  wrong cache key. Returning the error aborts the batch so withConn retires the
  connection.

- Use the nonblocking cscHandlerClient adapter for the drain. This reader is
  part of the refresher's waitgroup, so a custom push handler calling Close()
  on the raw client would deadlock shutdown (Close waits on the refresher
  goroutine parked in the handler).

* fix(maintnotifications): skip endpoint DNS detect when mode is disabled (#3969)

* Update options.go

* Update options_test.go

* Update options.go

* fix flaky test

* fix(maintnotifications): skip endpoint DNS detect when mode is disabled

* perf(proto): add zero-copy semantics to scan, remove redundant data c… (#3972)

* perf(proto): add zero-copy semantics to scan, remove redundant data conversions in reader

* chore: unpin govulncheck version

* chore: require govulncheck to use latest go version

* chore: pin govulncheck and resolve go-version to stable

---------

Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>

* fix(csc): record cancel metric on coalesced miss

A caller that cancels its context while waiting on a coalesced miss returned
early without recording an error metric: the serving session still completes
the background fetch (applyAndSettle success or settleErr), so neither records
THIS caller's cancellation, undercounting the cancellation rate whenever miss
coalescing is enabled. Report it here, attributed to the caller's ctx, matching
processWithRetry.

The conn is passed as nil deliberately: req.servedBy is written by the session
and only safe to read after the req.done receive (the happens-before edge),
which has not happened on the cancellation branch, and is nil anyway when the
cancel beats the serve. The metric recorder treats a nil conn as "no peer
attributes".

* fix(csc): retry-uncached on coalescer stop-acquire

When getConn fails because acquireCtx was cancelled by mc.stop during coalescer
teardown, the session settled the pulled miss and drained the queue with the raw
context.Canceled. processCached only retries uncached on the errCSCRetryUncached
sentinel, so in-flight coalesced reads surfaced a spurious cancellation even
though the caller's context was still valid, instead of re-running on the
still-open pool like the other stop paths.

acquireCtx derives from context.Background() and is cancelled only by mc.stop
(its sole non-deadline cancel source), so a context.Canceled here uniquely means
teardown: settle first and the queue drain with errCSCRetryUncached and report
stopped so the loop exits without backing off. Genuine acquire failures (dial
error, deadline) keep the raw error and the backoff path.

* fix(csc): detect coalescer stop by signal, not error

The full-duplex miss coalescer mapped a getConn cancellation to
errCSCRetryUncached by testing errors.Is(err, context.Canceled). But getConn runs
the dialer, credentials callbacks, and init hooks, any of which can return
context.Canceled from their OWN context while the coalescer is not stopping.
Treating the error value as proof that mc.stop fired permanently exited the sole
full-duplex worker; the current miss retried uncached, but later misses still
enqueued through the still-live coalescer with no worker to settle them, hanging
deadline-free callers and leaking IN_PROGRESS reservations.

Detect teardown by checking mc.stop directly instead. mc.stop is closed only by
stopWorkers, and close(mc.stop) is ordered before acquireCtx cancels its context
(before getConn returns), so a real teardown is always visible. On stop, settle
retry-uncached and exit; on any other acquire failure — including an unrelated
context.Canceled — fail the batch and back off so the worker stays alive. This
supersedes the errors.Is check added in the previous round.

* docs(csc): note per-session Limiter for miss coalescing

ClientSideCacheCoalesceMisses holds one connection per coalescer session and
serves many misses on it, so Options.Limiter is admitted (Allow/ReportResult)
once per session — per held connection — not per coalesced miss, unlike the
plain per-command path. This is inherent to the held-connection model; a
per-miss Allow would defeat the coalescing and re-admit a connection already
held. Document it on the option so callers that need strict per-command
admission or circuit-breaking know not to enable miss coalescing.

* chore(deps): bump rojopolis/spellcheck-github-actions (#3967)

Bumps [rojopolis/spellcheck-github-actions](https://github.com/rojopolis/spellcheck-github-actions) from 0.63.0 to 0.66.0.
- [Release notes](https://github.com/rojopolis/spellcheck-github-actions/releases)
- [Changelog](https://github.com/rojopolis/spellcheck-github-actions/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rojopolis/spellcheck-github-actions/compare/0.63.0...0.66.0)

---
updated-dependencies:
- dependency-name: rojopolis/spellcheck-github-actions
  dependency-version: 0.66.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(csc): propagate drain errors on fragmented attributes

The held-session push drain (pushDrainWithin) dispatched the built-in processor
through ProcessPendingNotifications, which swallows an error that occurs AFTER
partially consuming a frame — e.g. a RESP3 attribute whose remainder arrives
slower than the hard read cap. A swallowed mid-frame error left the connection
desynchronized, so later fragments could be misread as command replies and cached
under the wrong key, or fail unrelated misses.

Use ProcessPendingNotificationsBuffered for the built-in processor, exactly as
drainPushNotifications already does, so a post-consumption error is propagated and
the caller retires the desynced session. Custom processors keep the unbuffered
call (their contract is "invoked only when notifications are known present").

* fix(csc): limit bare-socket push drain to built-in processor

Round-8 review fix (codex r3809020181).

peekAndProcessPushNotifications fired on MaybeHasData (raw socket
readiness). Over TLS, maybeHasData unwraps the transport. It then
returns true for a post-handshake control record that has zero RESP
bytes. (conn_check.go's connCheck refuses to unwrap TLS for this reason.
maybeHasData does unwrap it.) A custom NotificationProcessor then ran
ProcessPendingNotifications under the 50ms hard cap. The empty read
timed out. The FD session reader treats that timeout as fatal. It closed
a healthy connection, and its in-flight misses failed. The built-in
buffered processor accepts the empty read.

Now the bare-socket-readiness path runs only for the built-in processor.
A custom processor drains only when the reader already holds decoded
bytes (HasBufferedData), which means any TLS is already unwrapped.
Tradeoff: a push notification that arrives as bare socket readiness on a
custom and idle session waits for the next reply read, the session
recycle, or the MaxStaleness backstop. The connection stays open. This
mirrors the built-in-only guard on the opaque-transport probe in the FD
session tick.

* fix(csc): peek frame type before a custom push drain

Round-9 review fix (cursor r3811858390).

The round-8 D fix let a custom push processor drain on HasBufferedData. But
the buffered bytes can be a coalesced reply, not a push. A custom processor
invoked then could consume the reply, cache it under the wrong key, and
desync the stream. The built-in processor peeks the frame type and consumes
only push frames, but the NotificationProcessor interface does not promise
that. pushDrainWithin now peeks the frame type for a custom processor and
returns without a read when the frame is not a push. The peek is bounded by
the closure's hard read deadline.

* fix(csc): propagate custom push-drain peek errors

Round-10 review fix (codex r3812520847).

The round-9 fix peeked the frame type before it handed a buffered frame to a
custom push processor, but it returned nil on a peek error. PeekReplyType can
partially consume a fragmented RESP3 attribute (DiscardNext) before it errors,
which desyncs the stream. Swallowing the error kept the desynced connection,
so later fragments could be read as a command reply and cached under the wrong
key. The peek error now propagates, so the session fails and drops the
connection, as the built-in buffered path does. A cleanly-peeked non-push
frame still returns nil (a real reply left for the caller's read).

* fix(csc): drain miss-reader pushes through the shared safe helper

Round-11 review fix (cursor r3813795126).

The full-duplex miss reader (readOne) drained pushes with the unbuffered
ProcessPendingNotifications for both processor kinds -- a third drain site the
earlier desync fix did not reach. The built-in processor there swallowed a
mid-frame DiscardNext error on a fragmented attribute, and a custom processor
got no frame-type peek, so either could leave the stream misaligned; readOne
then read the shifted bytes as the reply and cached them under the wrong key.
Both drain paths now share drainPushFrames: the built-in uses the Buffered
variant (mid-frame errors propagate) and a custom processor is handed only a
confirmed push frame (peek first, propagate the peek error). The helper sets no
read deadline, so readOne's following ReadRawReply is unaffected.

* fix(csc): retire conn on pre-command drain desync; refresh drain helper

Two review fixes.

Pre-command push-drain desync (round-13 audit): _process drains pending
pushes before executing a command, then reads the reply on the same conn and,
on the CSC-miss capture path, caches it. A fragmented RESP3 frame straddling
the drain's short hard cap left the reader mid-frame; the error was logged and
ignored, so the reply read consumed the residue and could cache it under the
wrong key. _process now closes the conn and retries on a fresh one instead of
ignoring the error (peekAndProcessPushNotifications already propagated
mid-frame errors via drainPushFrames).

Refresh reader drain (cursor r3813795126 / r3815669446): the refresh reply
reader still drained pushes with the unbuffered ProcessPendingNotifications,
a fourth drain site that could swallow a mid-frame desync before ReadRawReply
and publish a push fragment under the wrong cache key. It now routes through
the shared drainPushFrames helper, like the idle-tick and miss-reader paths.

* fix(csc): stop the refresh worker on GC cleanup

Round-14 review fix (codex r3817196217).

When refresh-on-invalidate is enabled and the caller drops the client without
Close, the runtime cleanup stopped the miss coalescer and the invalidation
drainer but never signaled the refresh worker. runCSCRefresher then stayed
parked on its ticker/queue, holding the baseClient, cache, and pools -- a
goroutine and memory leak that defeated the drop-without-Close safety net.
cscRegisterCleanups now captures the refresh handle and signals it, idempotent
and non-blocking (a new cscRevalidateHandle.signalStop guarded by sync.Once, so
Close and the cleanup cannot double-close). stopCSCRefresher uses the same
signal.

* feat(csc): add refresh-failure metric

Adds RefreshFailed to CSCRefreshStats -- the refresh-failure counter the HLD
lists (go-redis section 4.1). A refresh round trip that errors increments it;
those keys stay evicted and a later read repopulates them, so a rising count is
the signal that refresh-on-invalidate is degrading to plain eviction. Counted
per errored batch, not per key.

* fix(csc): drain pending pushes before reading the reply

The CSC miss and refresh readers drained push notifications with the
Buffered variant, which stops the instant the reader buffer empties. If a
second invalidation was still on the socket ahead of the command reply,
ReadRawReply read that push as the reply and cached it under the wrong
key -- a one-frame shift that then cascaded to later replies. The readers
now drain in BLOCKING mode: they block on the socket and skip push frames
until a non-push frame (the reply) is next, the same non-buffered
discipline the full-duplex reader already uses. PeekReplyType is
attribute-aware, so a fragmented RESP3 attribute needs no separate
buffered scan. A swallowed boundary-peek TIMEOUT is caught by the reader's
shared read deadline (ReadRawReply hits the same expired deadline and
fails the session); a swallowed non-timeout peek error would need a
malformed attribute mid-push (a server protocol bug), so this path is no
weaker than the full-duplex reader. Probe and idle paths keep the Buffered
variant so they never block. A socket-pair regression test pins the frame
order.

Also in this change:

- GC cleanup unbinds the dropped client's refresh queue from a shared
  invalidate handler (clearRefreshQueue), so a surviving sibling's
  refresh-on-invalidate keeps working instead of feeding a stopped queue.
  A refresh queue is only created inside attachSharedTrackingCSC, which
  also builds the drain handle, so the handle is always present when the
  queue is; a Conn() clone bails before the refresher starts.

- Record usedAt on deadline-free (negative-timeout) reads and writes, so
  a long full-duplex session under ReadTimeout/WriteTimeout=-2 is not
  misjudged as idle-expired by the pool and needlessly reconnected.

* fix(csc): harden miss-coalescing per review

- spill invalidation overflow off the full-duplex reply reader; the worker
  drains it through the same seen/pending dedup, so a burst collapses to one
  delete per key and no cache work runs on the reader
- drop the drain deadline cleanup that re-armed a relaxed timeout
  (WithReaderHardDeadline already restores the deadline)
- cap a coalesced write batch by bytes, not just count
- warn when ClientSideCacheInvalidationBatchWindow exceeds MaxStaleness

Refs #3965

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: sadd amr3e <saddamr3e@gmail.com>
Co-authored-by: Albert Huynh <huynhalb@gmail.com>
Co-authored-by: Vladislav Kotsev <vladygk8@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(csc): address review findings on refresh, batcher, and spill

Refresh-on-invalidate:
- pass the refresher's internal 5s ctx directly to WithReader/WithWriter instead
  of c.context(ctx); c.context strips it to Background when ContextTimeoutEnabled
  is false, so with read/write timeouts disabled a stalled Redis hung the
  refresher and wedged Close (cursor/codex).
- recover() the flush batch and defer its cancel(), so a panic can't kill the
  refresher (silent permanent degradation) or leak the 5s timer.
- skip Reserve's (0,true) 'fetch uncached' targets: they wasted an MGET slot and
  registered server-side tracking for a never-cached key.

Invalidation batcher:
- hold the handler lock across drop()+Flush() and read the batcher/cache fresh
  under it, so a concurrent batcher rebuild can't leave stale-epoch deletes
  applying after the flush and evicting post-flush repopulations (cursor).
- hard-cap the overflow spill and schedule one full Flush past the cap: the
  invalidatable keyset is NOT bounded by cache size (the server tracks keys past
  local LRU eviction), so an unbounded spill could OOM (codex). Adds a test.

Other:
- FTHybridCmd: populate rawVal on the RESP2 path for RawResult parity (Copilot).
- docs: mark ClientSideCacheRefreshOnInvalidate experimental + LocalCache-only
  (Copilot); correct the HasBufferedData 'complete frame' comment; note the
  Effective*Timeout post-expiry side effect; reconcile the refresh header's
  stale-read claim; drop a stale symbol reference.

* fix(csc): follow-up review fixes (spill race, stats, refresh, acquire)

- inval batcher: set flushReq BEFORE releasing stopMu at the spill cap, so a
  concurrent stop() cannot close stopCh and let the worker exit without seeing
  the request (lost flush, stale entries).
- inval batcher: count a dedup-dropped duplicate invalidation as invalidation +
  noop, so InvalidationStats reflects incoming pushes (the duplicate ratio) with
  the window batcher on, matching the inline path.
- csc integration: unbind the refresh queue in the drainer's exit defer, so a
  SELF-DISABLE exit (custom-processor damping / RESP3 downgrade) restores a
  sibling's binding instead of silently breaking the survivor's refresh.
- miss coalescer: derive the connection-acquire budget from a custom
  DialerRetryBackoff (sum the per-attempt delays), so a backoff longer than
  DialerRetryTimeout no longer cancels Get mid-retry-sequence.

Addresses codex re-review findings on #3989.

* fix(csc): address codex re-review (relaxed-timeout race, acquire, spill stats)

- pool relaxed timeout: the counter is a relax/unrelax NOTIFICATION balance, not a
  per-I/O count. getEffective{Read,Write}Timeout no longer decrements it per call;
  a passed deadline triggers a single CAS-coordinated reset (expireRelaxedTimeout),
  so a full-duplex connection's concurrent reader/writer/drain-backstop can't drive
  it negative and wedge later relaxations, and a lost unrelax notification is
  recovered at the deadline. ClearRelaxedTimeout is clamped at zero. Adds a
  concurrent-expiry regression test.
- miss coalescer acquire budget: revert the DialerRetryBackoff summing — calling a
  randomized/stateful backoff out-of-band consumes different jitter samples than
  the pool's actual retries, so the estimate can still be wrong. Use the
  deterministic flat estimate; mc.stop + PoolTimeout bound it for custom backoffs.
- inval batcher: count invalidations superseded by the spill-cap full-Flush
  (enqueue discard + worker pending reset) so CSCRefreshStats doesn't undercount an
  invalidation flood.

Addresses the codex re-review on #3989.

* fix(pool): retire one relaxed holder on expiry

The relaxed-timeout deadline safety net full-cleared the whole window on
expiry. On a full-duplex conn the reader, writer, and drain backstop all
observe the expired deadline concurrently; a CAS on the shared deadline
already gates that to one action, but a full clear also wiped a DIFFERENT
relaxation sharing the conn -- a notification SetRelaxedTimeout outstanding
alongside an expired handoff deadline lost its window (the :566 race).

Retire exactly one holder instead: expireRelaxedTimeout decrements the
counter by one (clamped, clears the timeout values only when the last
holder leaves). To keep overlapping handoffs from leaking a permanently
relaxed conn, SetRelaxedTimeoutWithDeadline now takes a holder slot only
for the FIRST deadline-scoped window and re-arms by replacing the deadline
in place (Swap), so the eventual expiry retires the single holder and the
window clears.

Add deterministic regression tests: a notification window survives an
expired handoff deadline, and overlapping handoffs clear without a leak.

Addresses the codex conn.go:566 finding on #3989.

* fix(csc): split invalidation and deletion stats

InvalidationStats conflated two quantities and undercounted whenever the
window batcher deduped, spilled, or fell back to a full Flush, which drove
a recurring "another uncounted path" review cycle. Split them:

- Invalidations counts keys named in INCOMING pushes, tallied once at the
  handler choke point before dedup/batching.
- Deletions / DeletionsNoop count APPLIED deletes at the two cache choke
  points (DeleteByRedisKey and deleteByRedisKeyCollectingHot), so the count
  is accurate whether refresh-on-invalidate is on or off.

Invalidations - Deletions now measures dedup plus the flood full-Flush
fallback directly, so the batcher no longer needs ad-hoc per-drop counting
(removed addInvalidationStats and its three call sites). Both are exposed
on CSCRefreshStats.

Also snapshot the refresh recency horizon at ENQUEUE into
cscInvalItem.sinceToken and use it in apply, so a batch-window delay cannot
advance the live horizon and chill a key that was hot when the invalidation
arrived (it would degrade to plain eviction instead of a proactive
refresh). Value-passed, no added allocation.

Addresses the cursor csc_inval_batch.go batch-chill finding and the codex
csc_inval_batch.go stats-undercount finding on #3989.

* fix(pool): serialize relaxed-timeout mutations

SetRelaxedTimeoutWithDeadline stored its new timeout values before swapping the
deadline, so an expiry observed concurrently (a full-duplex reader/writer/drain
backstop) could CAS the old deadline to zero and clearRelaxedTimeout the fresh
values between the store and the swap -- leaving a window with a future deadline
but zeroed timeouts (active but ineffective; commands fall back to the shorter
normal timeout during a migration and can fail spuriously). Reordering alone only
moves the race.

Add relaxedMu to serialize all four mutators (Set, SetWithDeadline, Clear,
expire) so the timeout/deadline/counter atomics are always updated or cleared as
one unit. expireRelaxedTimeout re-validates the deadline under the lock (a
generation check): an expiry the caller observed lock-free is dropped if a newer
window replaced it. The read path (getEffective*/HasRelaxedTimeout) stays
lock-free and takes the lock only via expire on the rare observed expiry, so the
per-I/O hot path is unchanged. The former CAS retry loops collapse to plain
load/store under the lock.

Addresses the codex relaxed-timeout rearming-vs-expiry finding on #3989.

* fix(csc): repoint batcher to survivor before stop

The invalidation batcher captured its refresh binding at creation. When the
active refresh owner closed, clearRefreshQueue restored the surviving sibling on
h.refresh and stopped the batcher, but the stop-drain still offered evicted-hot
keys through the closing owner's queue -- whose drainer had already gone, so the
offers were dropped and the survivor never refetched them (they stayed evicted
until a reader missed). setRefreshQueue had the same staleness on a rebind.

Make cscInvalBatcher.refresh an atomic pointer and repoint it at the current
binding (survivor / new owner, nil if none) BEFORE stop() in both
set/clearRefreshQueue, so the stop-drain feeds the live refresher. An in-flight
apply already holding the old pointer feeds its current batch to the old queue --
a bounded, benign residual (offers to a stopped queue are dropped, never panic:
the queue channel is never closed).

Addresses the cursor "batcher drain feeds closed refresher" finding on #3989.

* fix(pool): relaxed timeout as one atomic snapshot

Ofek's review asked for the relaxed-timeout window to be published as one value
so a reader never sees it half-updated. Replace the mutex plus four atomics with
a single atomic.Pointer to an immutable relaxedState snapshot. Set,
SetWithDeadline, Clear, and expire install a new snapshot with a
compare-and-swap; the read path (getEffective* and HasRelaxedTimeout) does one
lock-free Load. This removes the writer-only mutex and the torn read a reader
could see between the counter bump and the timeout store.

It also fixes getEffective* returning the normal timeout on the call that
observes an expired deadline while a notification holder is still active. The
method now re-reads the snapshot after it retires the deadline holder and returns
the surviving relaxed timeout.

Addresses the codex relaxed-timeout thread and Ofek's snapshot request on #3989.

* fix(csc): bound coalesced writes, harden miss retry

Reader-miss coalescing and refresh both write a whole batch before they read the
replies, and the reader waits until the write finishes. A batch larger than the
connection write buffer (default 32 KiB) makes bufio flush in the middle of the
batch, which can deadlock the write and the read on large payloads. The batch
byte cap was 1 MiB. Bound each batch to the write buffer:

- The miss coalescer caps a batch at min(write buffer, 1 MiB). An over-budget
  request is carried in the writer's own state and sent as the first request of
  the next batch, not put back on the shared queue. So a shutdown cannot strand
  it, and it also does not widen a pre-existing reservation-strand window on the
  drop-without-Close path. The first request always goes.
- The refresh chunker applies the same byte bound next to its count cap.

Also route a coalescer session or transport failure through the normal retry
path. settleErr tags such errors as cscSessionError; processCached re-runs the
read with MaxRetries and backoff instead of a raw io.EOF to the caller. A
reply-level result (redis.Nil, WRONGTYPE, LOADING) stays untagged and is returned
as-is.

Addresses the Ofek review threads on #3989: write-before-read batches, the
refresh batch, the byte-cap overshoot, and the coalesced-miss retry path.

* fix(csc): drop ineffectual pending clear

The grabInto carry change made the pending = nil at the top of the writer loop a
dead store: grabInto now reassigns pending (with the next carry) before it is read
again. Drop the clear. No behavior change. Fixes the golangci-lint ineffassign
that CI flagged.

* fix(csc): read coalesced replies while writing the batch

The miss-coalescer wrote the whole batch and only then handed the requests to the
reader, so the reader was idle for the entire write. A batch larger than the
transport send capacity could deadlock: the server blocks writing early replies
that nobody reads while the writer blocks flushing later requests. Bounding the
batch to the write buffer did not fix this — the write buffer does not bound the
socket send buffer (a custom Dialer or a large WriteBufferSize breaks the
assumption), and with per-op timeouts disabled it can wedge.

Hand the batch to the reader BEFORE writing it (as the autopipeline full-duplex
engine already does), so the reader drains reply k while request k+1 is still
being written and relieves the server's send backpressure. The send cannot
deadlock the writer: a batch is at most cscMissBatchMax (128) and inflight is
cscFullDuplexDepth (4096) deep, the writer is sequential so every prior batch was
already flushed, and every reader-exit path either cancels the session context or
follows the writer's own exit. The byte cap remains only as a batch-size bound,
not as the deadlock fix.

Addresses the Ofek and codex/cursor write-before-read findings on #3989.

* fix(csc): stats without refresh, bound refresh write

Two refresh-path fixes from the review:

CSCRefreshStats returned an all-zero struct when the refresh queue was absent, so
with ClientSideCacheRefreshOnInvalidate off (the default) but invalidation
batching on it hid the cache-level Invalidations/Deletions/DeletionsNoop the
handler still records. Read those cache counters independently of the queue; gate
only the queue-specific counters on the queue.

The refresh writes a whole chunk before it reads the replies, so a chunk that the
server cannot drain (transport backpressure) blocks the flush. With per-op write
timeouts disabled the flush is deadline-less and the 5s batch context does NOT
interrupt it, so the refresher goroutine wedges — worse than the review reported.
Bound the write with a positive deadline (shared with the batch context via one
constant) so a stalled flush fails the refresh and the entries degrade to plain
eviction (self-healing). Full read/write concurrency in the refresher is a
possible follow-up, not done here.

Addresses the codex refresh-stats and refresh write-before-read findings on #3989.

* fix(csc): honor recycle/stop on the carry path

grabInto can defer one over-budget request as a carry, held in the writer's
pending slot and written as the first request of the next batch. While pending is
non-nil the writer took the carry branch and skipped the select that watches
recycle and mc.stop, so a producer whose commands are persistently over the byte
cap could keep the held connection busy and starve a maintenance handoff or a
lifetime recycle indefinitely (its OnPut hooks would never run).

Check recycle/stop/session-cancel (non-blocking) between batches even when a carry
is present, and settle the carry on bail — it lives only in the writer's state, so
the teardown drain, which covers inflight, would otherwise leak it and hang its
caller. Settle with the retry-uncached sentinel on recycle/stop so the caller
re-runs on the pool.

Addresses the codex carry-vs-recycle finding on #3989 (a follow-on from the carry
introduced last round).

* fix(csc): bound the refresh read too

Last round bounded the refresh WRITE so a deadline-less flush could not wedge the
refresher. The following WithReader still used the raw ReadTimeout: with per-op
timeouts disabled (options.go maps -2 to -1, which WithReader treats as no
deadline) a stalled reply or push drain parks refreshInvalidatedBatch forever, and
stopCSCRefresher waits on that goroutine, so Client.Close never returns.

Bound the read with a positive deadline (still capped by the 5s batch ctx, shared
via cscRefreshBatchTimeout), symmetric with the write, so a stall fails the refresh
and the entries degrade to plain eviction instead of hanging Close.

Addresses the cursor refresh-read-hangs-Close finding on #3989.

* fix(csc): per-chunk deadline for refresh round trips

The refresh split a window into chunks (by count and by write-buffer bytes) but
ran them all under one cscRefreshBatchTimeout context. When a window held more
than one chunk and their AGGREGATE latency exceeded the budget, later chunks
failed immediately even though each round trip was within the intended per-round-
trip timeout — four 2s chunks under a 5s budget would refresh only the first two
and leave the rest evicted.

Create the bounded context PER refreshInvalidatedBatch call so the documented
per-round-trip timeout applies to each chunk independently. The recover guard
stays at the loop level.

Addresses the codex per-chunk-deadline finding on #3989.

* fix(csc): flush snapshot cache on FLUSH invalidation

The FLUSHDB/FLUSHALL invalidation branch re-read the live h.cache under a second
RLock and flushed that, while per-key deletes use the cache pointer snapshotted at
handler entry. A last-user releaseLocked (drainer teardown / GC cleanup) can nil
h.cache between the entry RUnlock and the flush RLock, so the guarded
'if h.cache != nil' skipped the wipe entirely. On an injected shared cache
(cscOwnsCache == false) the teardown flush is gated off, so post-FLUSH entries
kept serving until TTL / MaxStaleness.

Flush the entry snapshot instead, matching the per-key branches. The second RLock
stays — it guards the batcher epoch against a concurrent rebuild — but only the
batcher is read fresh under it now; the cache uses the snapshot. The nil guard is
dropped: the entry guard already returns on a nil snapshot.

Addresses the cursor 'FLUSH can skip shared cache wipe' finding on #3989.

* fix(csc): drain all pushes before a coalesced/refresh reply

drainPushFrames in blocking mode, for a CUSTOM PushNotificationProcessor, peeked and
processed once. A second invalidation still on the socket ahead of the reply was
then read by the caller's ReadRawReply as the command value — published to the
shared cache under the wrong key and shifting every later reply. Loop peek-then-
process until a non-push (the reply) is next, matching the built-in blocking
discipline; non-blocking probes keep the single-pass behavior. Covers the coalescer
and refresh reply readers.

* fix(csc): bound in-flight coalesced wire by bytes

Miss coalescing serializes each caller's command into req.wire before it can enter
the bounded queue; arbitrarily many callers can block on the send holding a full
wire copy, and large cacheable commands (a big MGET) have no small encoded size, so
a burst could exhaust memory — the queue's item cap does not bound bytes. Track
total in-flight serialized bytes; before serializing, reserve this command's
approximate size and shed over cscMissWireBudgetBytes (8 MiB) to the pooled path
(errCSCRetryUncached), which is bounded by pool turns. Reserving before serialization
means over-budget callers never allocate the wire; a single command larger than the
budget sheds too (the pooled path runs it). TestCSCMissCoalescerWireBudget pins it.

* fix(push): consume a push whose name can't be peeked

The built-in blocking drain, on a frame already confirmed as a push, broke out of
the loop when PeekPushNotificationName failed for a non-'too-long' reason (a
non-string / malformed name). Breaking LEFT the push at the buffer head, so a
reply-expected reader (CSC refresh / miss-coalescer / the single-command pre-read
drain) then read the push as the command value — the intermittent 'redis: nil' and
cross-command reply shift under load. Fall through to ReadReply and CONSUME the
push instead (matching the name-too-long path): a genuine mid-frame read error still
surfaces at ReadReply and retires the connection, and a non-string name is
gracefully ignored (the downstream type assertion fails, so no handler runs).
Buffered probes still stop cleanly on a boundary timeout. Affects every RESP3
client's pre-read drain, not only CSC. TestProcessor_ConsumesUnpeekableNamePush
asserts the following reply is readable (the push was consumed, not left).

* fix(csc): release miss wire budget when the wire is consumed

The miss coalescer reserved wire-budget bytes before serializing a command, then
released them in a defer that fired on every fetch exit — so a caller that abandoned
after enqueue freed the budget while its wire snapshot was still retained in the
session channel / inflight deque. A burst of large abandoned misses could thus drive
the retained wire bytes far past cscMissWireBudgetBytes.

Track the WIRE's lifetime instead of the caller's: store the reservation on the
request (reserved) and release it exactly once, through a new settle helper, when
the request is consumed (applyAndSettle, settleErr, shutdown drain). fetch releases
in its defer only when the request never reached the queue (serialize error, or
Close/ctx before enqueue), guarded by an enqueued flag.

Also honor ContextTimeoutEnabled for the coalescer's enqueue and reply waits: when
it is off the ordinary command path drives socket I/O on context.Background()
(bounded by ReadTimeout, not the caller deadline), so a coalesced miss must not
surface context.DeadlineExceeded on the caller's deadline. Wait on c.context(ctx)
(caller ctx when the policy is on, Background when off); mc.stop stays the
unconditional shutdown signal and returns still report ctx.Err().

Addresses the csc wire-budget and context-timeout review findings on #3989.

* fix(csc): repopulate cache on coalescer-bail retry, order close

Two fixes on the CSC integration path.

Fallback populate: when the miss coalescer bailed (errCSCRetryUncached or a session
error) processCached retried through a nil-capture processWithRetry — it returned
the value but stored nothing, so a connection blip or a wire-budget shed left the
key uncached and every later reader missed it. Re-Reserve and fall through to the
capture path instead, so a successful retry repopulates the cache (Reserve may hand
fetching to another reader, which the capture path already handles).

Close ordering: stopBackgroundDrainer deactivated serving (cscActive=false) before
joining the refresher, but the refresher's stop-path flush re-fetches the in-window
invalidated keys and refreshInvalidatedBatch skips that work once cscActive is false
— so the last collection window was dropped, and on a SHARED (unowned) cache those
hot keys stayed evicted for siblings until a later miss. Stop the refresher FIRST
(serving still on, pool still open), then deactivate before stopping the coalescer,
which still shields a racing clone miss from an ErrClosed on the just-stopped
coalescer.

Addresses the csc fallback-populate and close-refresh-window review findings on #3989.

* fix(csc): cap per-target key bytes in the refresh queue

The refresh queue's depth cap (cscRefreshQueueDepth) bounds it by ITEM count, not
memory: with large key encodings, up to that many queued targets could pin far more
than the cache's own limit. Drop any target whose key bytes (cache key + Redis keys)
exceed cscRefreshTargetMaxBytes before enqueue — like a full-queue drop, it just
means a later reader does a normal miss fetch — bounding retained key memory to
roughly cscRefreshQueueDepth * the cap.

Addresses the csc refresh-queue-bytes review finding on #3989.

* fix(csc): drain queued refresh targets on refresher stop

The refresher's stop branch flushed only the collected window (pending); targets
still buffered in q.ch — offered but not yet collected — were abandoned. Because
stopCSCRefresher rebinds the invalidate handler away from this queue BEFORE
signalling stop, nothing else drains q.ch, so on a shared cache up to
cscRefreshQueueDepth hot entries stayed evicted for the surviving sibling until a
later reader missed them.

Extract drainQueue (the existing non-blocking window-fill, now reused by the
q.ch branch) and, on stop, loop drain+flush until the channel is empty. The
channel is quiescent at that point (no producer), so the loop is finite, and the
pool is still open (closeResources runs after the join), so the final refetch can
complete.

Addresses the codex drain-queued-refresh-targets finding on #3989.

* fix(csc): use live horizon for no-binding invalidations

An invalidation enqueued while the batcher had no refresh binding stamped
sinceToken 0. apply re-reads the refresh binding at apply time, so a refresh
client attaching in between (setRefreshQueue repoints the batcher, then
stop-drains it) applied those items against horizon 0 — treating every valid
entry as hot and offering it for refetch. That is the cold-key refresh loop the
recency horizon exists to prevent, and it re-registers cold keys with the server,
pinning its tracking table.

Stamp cscInvalNoHorizon (-1) when no binding exists at enqueue; apply falls back
to the binding's LIVE horizon (seeded at the attaching client's start), a real
recency test that can only under-refresh, never chase cold keys. LRUClock is a
monotonic sequence >= 0, so -1 cannot collide with a real horizon.
TestInvalBatcherRefreshAttachUsesLiveHorizon pins it (a cold pre-attach entry is
deleted but not offered; a hot one is).

Addresses the cursor refresh-attach-bypasses-recency-gate finding on #3989.

* fix(options): normalize nonpositive DialerRetries

Options.init normalized DialerRetries only when == 0, but the pool treats any
value <= 0 as the default (ConnPool.dialConn). A negative DialerRetries therefore
left the miss-coalescer's acquire deadline (which derives its budget from
opt.DialerRetries) computing a smaller retry count than the pool actually uses.
Normalize <= 0 so both see the same count; document the field accordingly.

Addresses the codex normalize-nonpositive-retry-counts finding on #3989.

* fix(csc): raise push-drain budget under relaxed timeouts

Every push drain ran under a hard 50ms read cap (WithReaderHardDeadline) that
deliberately bypasses maintenance relaxation. Right for a speculative probe on a
healthy conn; wrong mid-failover: relaxation is active exactly because the server
is slow, a push frame can fragment past 50ms, and a mid-frame timeout is a desync
— so the pre-command path retired a HEALTHY tracked connection (evicting its CSC
coverage) and, with MaxRetries=0, failed the command outright. The old pre-PR
drain honored relaxation via WithReader.

Add pushDrainBudget: the hard cap, raised to the connection's relaxed timeout
while relaxation is active (EffectiveReadTimeout; one-directional — a smaller
relaxed window never lowers the cap). Paid only when bytes are actually mid-frame:
an empty or completed drain returns without blocking regardless of the budget.
Pinned by TestPushDrainBudgetHonorsRelaxation.

Addresses the pre-command-drain finding in the #3989 deep review.

* fix(csc): join inval batcher on stop, wait on re-reserve loss

Two review fixes on the invalidation/miss integration paths.

Batcher join: stop() only signalled; the batcher's stop-drain and final flush ran
asynchronously after Close (or a rebuild) returned — a straggler goroutine past
Close, and on a shared injected cache a late apply could evict an entry a
successor client just repopulated. Give the batcher a done channel (closed when
run() exits) and a join(); every stop site (releaseLocked, setInvalBatchWindow,
set/clearRefreshQueue) now stops AND joins, making teardown and rebuilds
synchronous. join is safe under h.mu: run() takes no handler locks. Pinned by
TestInvalBatcherStopJoinIsSynchronous. Also resolves the join-before-tighten
review thread.

Bail-loser wait: after a coalescer session failure, waiters raced the re-Reserve
and every LOSER fell through to an independent pooled processWithRetry — one hot
key turned into N pooled requests exactly mid-recovery. A loser now waits on the
new owner (csc.Get parks on the in-progress entry, same shape as the
first-Reserve loser path) and only falls back to the pooled path after a second
Reserve loss.

Addresses the codex drain-after-close and keep-same-key-waiters findings on #3989.

* fix(csc): consume window demand nudge on refresh flush

A demand signal buffered while a window was being flushed survived the flush:
signalDemand fires only for keys in pendingSet, so the nudge referred to a key
this flush was already refetching, yet it immediately flushed the NEXT window on
its first collected key — defeating the coalescing window and inflating
DemandFlushes. flush() now consumes one buffered nudge after clearing the window
(collect runs only on the refresher goroutine, so no future-window demand can
exist yet).

Addresses the codex discard-stale-demand-signals finding on #3989.

* fix(csc): emit coalesced reply-error metric exactly once

applyAndSettle emitted the native error metric for every non-nil parsed reply
while a caller whose context expired late emitted its cancellation independently
— the same operation recorded twice with conflicting error types whenever the
cancellation raced the reply.

Centralize emission at the caller's req.done receive (emitReplyErr): reply errors
(redis.Nil, WRONGTYPE, ...) are recorded once, where the caller consumes them;
session errors and shed retries stay excluded (their processWithRetry re-run
emits its own outcome); a caller returning ctx.Err() emits only its cancellation.
req.servedBy is safe at the receive (the done receive is the field's
happens-before edge), so the metric now also carries the serving conn on the
paths that previously reported from the reader side.

Addresses the codex double reply+cancel metric finding on #3989.

* fix(csc): staleness warning covers injected LocalCache

The ClientSideCacheInvalidationBatchWindow > MaxStaleness misconfiguration
warning was gated on ClientSideCacheConfig != nil, so a pre-built cache injected
via ClientSideCache was never checked — a 10ms-staleness cache with a 500ms batch
window served pre-invalidation values 50x past its own contract, silently. When
the config is nil, read the injected *LocalCache's effective bound
(effectiveMaxStaleness) and run the same comparison.

Addresses the injected-cache warning gap in the #3989 deep review.

* fix(csc): budget refresh round trips by expected reply bytes

The refresh chunker bounded only the REQUEST side (key bytes vs the write
buffer); the reply side was unbounded. Within one chunk the server streams
replies for the early keys while the client is still writing the late ones, and
against a flow-controlling middlebox (a proxy or tunnel that stops reading when
its buffer toward the client fills — vanilla Redis instead buffers replies in
RAM) enough in-flight reply bytes stall the connection both ways: the classic
pipeline write-read deadlock, surfacing as the per-chunk deadline failing the
whole chunk and refresh silently degrading to eviction for large-value
workloads.

Budget each round trip's EXPECTED reply volume: collectHotAndDelete captures the
dying entry's value size (free, under the same shard lock) on the refetch
target, and the chunk boundary — extracted pure as cscRefreshChunkEnd and
unit-tested — now caps count, request bytes, and expected reply bytes (32 KiB,
comfortably inside typical kernel socket buffering). An estimate, not a
guarantee (a value may have grown since eviction); the per-chunk deadline stays
the backstop, and full read/write concurrency in the refresher remains the
follow-up if the deployment path is confirmed to apply per-connection
backpressure.

Refs the refresh write-before-read review threads on #3989.

* docs(csc): freshness contract, push spec, metrics TODO

Outcomes of the push-consumption review discussion, no behavior change.

- Document the invalidation-freshness contract on ClientSideCacheConfig: applied
  at arrival on full-duplex conns, at the next command on active conns, at the
  next drainer tick on idle conns; MaxStaleness is the hard bound everywhere.
- Add .claude/specs/push.md: the checkpoint consumption model and its physics
  (frame completion, TLS peek opacity, custom-processor gating), the freshness
  guarantee, and the rules that keep the model closed — no new speculative drain
  sites; features needing fresher pushes own a reader (FD-session pattern). Also
  records the planned NotificationDrainer capability follow-up.
- TODO(convergence) at the coalescer-bail re-run: start retry metrics at attempt
  1 via the explicit-start plumbing once the full-duplex branch lands here.

* fix(csc): non-blocking GC teardown, classify attributed refresh replies

The drop-without-Close GC finalizer set cscActive false BEFORE signalling the
refresher, so refreshInvalidatedBatch bailed and the stop-drain flush was a no-op
— in-window hot keys stayed evicted for a surviving sibling. And clearRefreshQueue
/ setRefreshQueue / releaseLocked called the batcher join() while holding h.mu,
blocking the GC finalizer (documented non-blocking) and stalling a sibling in
ensureBatcher on the hot path (miss-coalescer reader).

Detach the batcher under h.mu (detachBatcherLocked, stop-only) and join OUTSIDE
the lock — on the Close / self-disable / attach paths, and NOT AT ALL on the GC
finalizer, which is reordered (clearRefreshQueue -> refresher signalStop ->
deactivate) so its async stop-drain is runnable while serving is still on. The
guaranteed final-flush window stays the Close-path join; the finalizer stays
non-blocking. setInvalBatchWindow gets the same treatment.

Also: the refresher's cacheability check inspected raw[0], but a RESP3
attribute-prefixed error (|... frame) has raw[0]==RespAttr, so an error reply was
published as a false refresh success (next reader must evict + refetch).
Classify the FULL reply (cscRefreshReplyCacheable via classifyCachedReply), like
the miss-coalescer.

Addresses the GC-cleanup teardown findings and the attributed-error refresh
finding on #3989.

* fix(csc): cancel stranded miss on shutdown, charge wire, bound spill

Three bounds/lifecycle fixes on the coalescer and invalidation batcher.

Stranded miss: fetch's mc.stop wait branch cancelled the reservation and
re-drained mc.ch, but the wctx.Done (caller-deadline) branch only claimAbandoned,
leaving a post-drain request IN_PROGRESS so a later Get waited out StaleTimeout.
cancelIfStopping now cancels + re-drains on that branch too — gated on mc.stop
being observed, so a healthy late-cancel does not no-op a live reader's
fulfillCached (token-guarded cache populate).

Wire budget: cmdApproxBytes charges ~8 bytes for variable-size non-key args
(BinaryMarshaler, *string), so the 8 MiB in-flight wire budget under-counted.
reconcileWireBytes charges the delta between the estimate and the actual
len(req.wire) after serialization, keeping req.reserved in lockstep (settle
releases exactly once) so the counter reflects reality and the next reserve sheds.

Invalidation spill: the spill was bounded by item count only; a burst of distinct
large namespaced keys retained GBs below the item cap. Track retained key bytes
(cscInvalSpillMaxBytes = 8 MiB) and take the same full-Flush fallback when either
bound trips (a single oversized key trips it alone).

Addresses the shutdown-strand, wire-size-budget, and invalidation-backlog-bytes
findings on #3989.

* fix(csc): shed a coalesced miss when its real wire exceeds budget

The wire budget is reserved from cmdApproxBytes before serialization, then
reconciled to the actual size after. Round 5 only corrected the counter and kept
an over-budget wire, so N concurrent misses with undercounted variable-size args
(BinaryMarshaler, *string, net.IP) each pass the pre-serialize gate and then
charge their true size, drifting the in-flight total well past
cscMissWireBudgetBytes while every snapshot stays queued.

reconcileWireBytes now returns keep/shed: when charging the actual size would
cross the cap it rolls the delta back, leaves req.reserved at the estimate (fetch's
!enqueued defer releases exactly that), and fetch sheds to the pooled path — so the
over-budget wire is dropped, not retained in mc.ch/inflight. A shrinking delta
always keeps. Pinned by TestCSCMissCoalescerShedsWhenActualOverBudget.

Addresses the recheck-wire-budget-after-serialization finding on #3989.

* fix(csc): bound invalidation channel bytes, not just spill

Round 5 bounded the overflow spill by retained key bytes, but the fast-path
channel admits by item COUNT only: a burst of large keys — or one key larger than
the whole byte cap — retains unbounded bytes in ch before the spill path ever
engages.

Track ch retained key bytes (chBytes): enqueue adds on a successful ch send and
the worker subtracts on consume. A pre-admission gate diverts to the full-Flush
fallback when this key alone exceeds the byte cap, or when the ch bytes plus this
key would cross it, instead of admitting to ch. Pinned by
TestInvalBatcherChByteGateTripsFlush and TestInvalBatcherOversizedKeyTripsFlush.

Addresses the invalidation-byte-cap-before-channel-admission finding on #3989.

* docs(csc): note refresh-stats scope on a shared cache

CSCRefreshStats mixes cache-global counters (Invalidations/Deletions/DeletionsNoop,
read from the shared LocalCache) with this-client's-queue counters
(Enqueued/Refreshed/...). On a shared cache+processor the active refresh binding is
the last-attached client's queue, so an earlier client reports every invalidation
but ~zero refresh work — correct (it is not the refreshing client), but a naive
refresh/invalidation ratio then reads as broken. Document that the ratio is
meaningful only single-client or for the client holding the active binding. No
behavior change (scoping the counters per-client would change what Invalidations
means for existing readers).

Addresses the keep-shared-cache-refresh-statistics-in-one-scope finding on #3989.

* fix(csc): reserve invalidation channel bytes atomically

The ch byte gate was a non-atomic Load then Add-after-send: concurrent producers
(invalidations on multiple tracked conns) could each read a below-cap chBytes, all
send a large key, then all Add — overshooting cscInvalSpillMaxBytes while the
counter briefly read near zero. Reserve atomically before the send (add, and roll
back if it crosses the cap or the ch is full, mirroring reserveWireBytes); an
over-cap reservation diverts to the full-Flush fallback. Pinned by
TestInvalBatcherChByteReservationIsAtomic.

Addresses the reserve-invalidation-channel-bytes-atomically finding on #3989.

* fix(csc): generation-tag refresh demand notifications

A reader that passed signalDemand's pendingSet check just before flush cleared the
window could send its nudge AFTER the one-time drain, and that stale nudge then
flushed the NEXT (unrelated) window early — inflating DemandFlushes and defeating
the coalescing window. Stamp each nudge with the window generation (read before the
membership check); flush bumps the generation on clear; the refresher ignores a
nudge whose generation is retired (demandIsCurrent). Pinned by
TestCSCRefreshDemandGenerationIgnoresStaleSignal.

Addresses the generation-tag-demand-notifications finding on #3989.

* fix(csc): honor caller cancellation while enqueuing a miss

fetch used wctx (Background when ContextTimeoutEnabled is false) in the enqueue
select, so a caller that cancelled could not abort while merely WAITING FOR A QUEUE
SLOT — pre-I/O backpressure — even though the ordinary path passes the caller ctx to
ConnPool.Get. The enqueue-admission select now watches the original ctx; wctx stays
only on the reply wait, which maps to socket I/O and follows the timeout policy.
Pinned by TestCSCMissCoalescerEnqueueHonorsCallerCancel.

Addresses the keep-caller-cancellation-while-waiting-to-enqueue finding on #3989.

* fix(csc): don't re-run a limiter-denied coalesced miss

When the coalescer's session acquisition failed because Options.Limiter.Allow
REJECTED admission, the rejection was tagged cscSessionError like a transport
failure, so processCached re-ran the command through processWithRetry -> getConn ->
Limiter.Allow a SECOND time. A stateful or token-based limiter can permit that
second call, executing the very miss whose original admission was denied.

Split the admission from the dial (getConnLimited: getConn wraps it, byte-identical
for existing callers) so the coalescer can tell an Allow rejection apart from a
dial/transport failure. On a rejection, settle the caller with the error UNTAGGED
(settlePlain/drainQueuePlain) so processCached returns it as-is — one Allow total,
matching the ordinary path (shouldRetry is false for a generic limiter error). A
genuine transport failure keeps its cscSessionError retry behavior. Pinned by
TestGetConnLimitedSurfacesRejection.

Addresses the preserve-limiter-rejections-during-session-acquisition finding on #3989.

* fix(csc): unify teardown ordering across close and self-disable

Three interrelated CSC teardown findings, resolved by one canonical order.

New once-guarded stopCSCRefresherAndCoalescer runs: stop the coalescer (releasing
its held pool connection) -> run the refresher's final flush while cscActive is
still true -> deactivate. Called from both the clean Close path and the drainer
goroutine's self-disable defer (neither is the refresher/coalescer goroutine, so
the joins never self-block).

- Self-disable (disableCSCServing / custom-processor damping / drainer seeing
  cscActive go false) previously exited without stopping the refresher or
  coalescer — the goroutines and a held pool conn leaked for the client's life.
  It now runs the same teardown. Start guards on startCSCRefresher/
  startCSCMissCoalescer close a construction-time disable window.
- Handler-initiated close (a push handler calling Client.Close) no longer
  preemptively deactivates + stops the drainer before the canonical close; it lets
  the async close drive the refresher-first-while-active ordering (mirrors the GC
  path).
- Releasing the coalescer connection before the refresher's flush avoids a Close
  stall (up to ~160s on a small pool: refresh chunks waiting their deadline while
  the coalescer held the only conn). Reversing the old deactivate-first step is
  safe: every coalescer stop path returns errCSCRetryUncached/cscSessionError, so
  processCached re-runs on the still-open pool — no raw ErrClosed reaches callers.

Tests + red-checks in csc_test.go. Deferred: quiescing invalidation producers that
snapshotted the queue before unbind (needs a hot-path lock; the residual is a
bounded teardown-only refetch, not a stale serve) — see the review reply.

Addresses the self-disable-leak, handler-close-ordering, and
release-coalescer-before-refresh findings on #3989.

* fix(csc): preserve invalidation dedup across a size-cap flush

run()'s size-cap flush (len(pending) >= the batch cap) cleared the epoch-scoped
seen map without a full-cache Flush, so a duplicate key straddling the cap was
applied twice; with refresh-on-invalidate the first apply refetched and published
and the second evicted that fresh entry. Split flush into applyPending + clearSeen;
the size-cap path applies WITHOUT clearing seen, so per-key dedup spans the whole
window (timer/stop/full-flush still clear it). Epoch-bump was rejected: epoch is
captured at enqueue and only valid paired with a full Flush, so bumping here would
skip in-flight ch/spill items and lose their deletes. Pinned by
TestInvalBatcherSizeCapPreservesDedup.

Addresses the size-cap-flush-drops-dedup finding on #3989.

* fix(csc): bound the shutdown refresh drain on first failure

On Close with a large backlog against a dead/stalled server, the stop-drain gave
each chunk a fresh 5s deadline and continued after errors — a full queue could
delay Close ~160s (hours if large replies shrink chunks). The stop-drain now bails
after the first failed chunk (those entries are evicted with this client's coverage
on Close anyway). The normal (non-shutdown) refresh path is unchanged. Pinned by
TestCSCRefresherStopDrainBailsOnFirstFailure.

Addresses the cap-the-entire-shutdown-refresh-drain finding on #3989.

* fix(csc): bound concurrent coalesced wire serialization

cmdApproxBytes underestimates variable-size args, so many concurrent misses all
passed the approx reservation and each allocated a full wire snapshot BEFORE
reconcileWireBytes could shed — a concurrent burst transiently coexisted many
multi-MB buffers and could exhaust memory even though none stayed queued. Add a
serialize semaphore (cap = budget/batch) acquired before writeCmd and released
after the reconcile decision, always before the enqueue select, so no slot is held
across a channel send or reply wait (deadlock-free with the single worker); a
caller that can't get a slot sheds to the pooled path. Pinned by
TestCSCMissCoalescerBoundsConcurrentSerialize and the fetch-level shed test.

Addresses the bound-concurrent-serialization finding on #3989.

* fix(csc): share refresh queue into clones for demand flush

clone() (WithTimeout/WithContext) copied the miss coalescer but not
cscRefreshQueue, so a derived client's queue was nil and signalDemand no-opped —
the clone never early-flushed the owner's in-window batch, so a demand-worthy miss
waited the full cscRefreshWindow. Share the queue pointer into the clone (like the
pool hook); lifecycle stays safe — the refresher/drainer handles are owner-only
(nil in clones) so a clone only signals (nil-safe, non-blocking), never stops.
Pinned by TestCloneSharesRefreshQueueForDemand.

Addresses the clones-skip-refresh-demand-flush finding on #3989.

* fix(csc): guard invalidation deletes by fetch-issue order

Round-10 kept the batcher's per-window dedup (seen) across a size-cap flush so a
key straddling the cap could not be applied twice — the second apply would evict
the entry the first apply's refresh had just refetched. That traded a benign extra
miss for two defects: seen grew unbounded under sustained load (the timer that
clears it is deferred by every size-cap t.Reset), and a genuine re-invalidation of
a refreshed key was deduped away and served stale. Worse, keep-seen could not fix
the case where ONE cached command depends on SEVERAL Redis keys (e.g. MGET k1 k2):
a refresh triggered by k1's invalidation republishes the entry, and k2's
invalidation — a different seen entry — then re-evicts it. seen is keyed by Redis
key, so it structurally cannot reason about whether THIS entry was refetched.

Move the correctness out of dedup and into the delete. Stamp each entry with the
global cscFetchSeq value at the moment its fetch is ISSUED (Reserve), carried
through fulfill; snapshot cscFetchSeq when an invalidation is OBSERVED (enqueue).
collectHotAndDelete now keeps a Valid entry whose fetchSeq exceeds the
invalidation's snapshot: that fetch was issued after the write, so its reply
already reflects it. In-progress placeholders are still removed unconditionally.

Stamp at fetch ISSUE, not fulfill COMPLETION: invalidations and replies travel on
different connections with no ordering, so a stale reply can fulfill AFTER the
invalidate is observed (see deleteByRedisKey). A completion-order stamp would then
retain that stale value; an issue-order stamp is sound because a fetch issued after
the observe necessarily reached a server that had applied the write.

With correctness in the guard, dedup lifetime no longer matters: the size-cap path
goes back to flush() (apply + clearSeen), so seen is bounded and a genuine
re-invalidation is re-applied. A straddling stale duplicate is now a no-op via the
guard instead of via dedup.

Tests: TestCollectHotMultiKeyRefreshSurvives (the MGET case dedup can't fix),
TestCollectHotFetchOrderGuard (stale kept, genuine evicts — the case a fulfill-time
stamp fails), and TestInvalBatcherSizeCapGuardKeepsRefreshed (end-to-end, with a
DeletionStats noop delta proving the duplicate was applied and guard-skipped, not
deduped). Existing collectHotAndDelete callers pass a max snapshot to isolate their
recency/stats assertions from the guard.

Addresses the size-cap-flush-dedup regression and the multi-key re-eviction on #3989.

* fix(csc): extend fetch-order guard to in-progress fetches

A delayed invalidation applied while a replacement fetch was still in progress
removed the reservation even when entry.fetchSeq > fetchSnap — a fetch reserved
AFTER the invalidation was observed, which cannot hold the pre-write value.
Canceling it failed its racing Fulfill, woke every waiter as a miss, and could
duplicate requests / defeat miss coalescing. The guard was Valid-only; drop that
restriction so it keeps any entry (Valid or in-progress) issued after the observe.
An entry reserved at or before the observe (fetchSeq <= fetchSnap) may predate the
write and is still removed. Pinned by TestCollectHotGuardKeepsInProgress.

Addresses the preserve-newer-in-progress-fetches finding on #3989.

* fix(push): align idle CSC drain with the push-drain spec

drainPushNotifications diverged from peekAndProcessPushNotifications /
pushDrainWithin and .claude/specs/push.md on two points.

1. It drained under the raw cscDrainHardReadCap. During maintenance relaxation a
push frame can fragment past that small cap, and a mid-frame timeout is a fatal
desync that removes the connection and evicts its CSC coverage in the very
failover/migration window relaxation is meant to tolerate. Route the drain through
pushDrainBudget so the cap is raised to the relaxed timeout while relaxation is
active (unchanged otherwise).

2. It invoked a custom push processor whenever the socket was readable
(CheckForData), including kernel-only readability that over TLS can be a control
record with zero RESP bytes; the custom blocking loop then hit the hard cap and its
error — always fatal for a custom processor — retired a healthy connection. Gate the
custom processor on the buffered state captured BEFORE the probe: a byte the probe
just fetched from the socket/wrapper is not "already buffered", and handing it to a
blocking loop is the exact spurious retire push.md warns of. So a custom processor
drains only when RESP bytes were already in the reader buffer; kernel-only AND
probed bytes both defer to the next pass. Mirrors peekAndProcessPushNotifications
(buffered captured before any drain). The built-in buffered processor is unchanged —
it still proceeds on kernel-only readability (a boundary timeout is benign). Custom
skipping is bounded by the periodic fallback probe and the MaxStaleness backstop.

Pinned by TestDrainPushNotifications_RelaxedBudgetToleratesFragmentedFrame and
TestDrainPushNotifications_CustomProcessorSkipsKernelOnlyData.

Addresses the apply-relaxed-budgets-to-idle-drainer and enforce-custom-processor-gate
findings on #3989.

* fix(csc): carry the latest fetch snapshot when deduplicating invalidations

When two writes to the same key land in one batching window with a cache refetch
between them (an entry near MaxStaleness expiring and being re-fetched), the second
write's invalidation was deduped and DROPPED, leaving the retained item with the first
write's older fetchSnap. At apply the fetch-order guard then preserved the entry
refetched BETWEEN the writes (its fetchSeq exceeded the stale snapshot), serving it
stale until the next invalidation or MaxStaleness.

The dedup now records each key's index in pending and, on a same-epoch duplicate,
carries the LATEST (max) fetchSnap onto the retained item, so the guard evicts an
entry only refetched before the most recent write. seen and pending are always reset
together, so a stored index never outlives its entry. Pinned by
TestInvalBatcherDedupKeepsLatestFetchSnap. Addresses the retain-newest-invalidation-
snapshot finding (e4xHA).

* fix(csc): cancel partial refresh reservations on a CacheSizer panic

refreshInvalidatedBatch armed its reservation-cancelling defer AFTER the Reserve loop.
A user CacheSizer panics inside Reserve; the outer refresher recovery keeps the worker
alive, but the targets reserved so far were left IN_PROGRESS with no defer installed,
so every reader of those keys blocked until StaleTimeout. Arm the defer BEFORE the
loop so a mid-loop panic cancels the partial set during unwind. Pinned by
TestRefreshInvalidatedBatchCancelsReservationsOnSizerPanic. Addresses the cancel-
partial-refresh-reservations finding (e8hzZ).

* fix(push): keep the short first-byte boundary for speculative drains

pushDrainWithin applied the relaxation-aware budget (pushDrainBudget) to the ENTIRE
drain, including the first-byte wait. Under active maintenance relaxation a TLS control
record — socket-readable with zero RESP bytes — made the speculative/pre-command drain
block for the full relaxed timeout (seconds), stalling a ready command and the idle
drainer. Bound the first-byte wait to the short cap: under relaxation, probe one byte
under the short cap first (a timeout there means no frame began, benign) and grant the
relaxed budget only to finish a frame already in flight — mirroring the idle drainer's
probe-then-drain. Pinned by TestPushDrainWithinShortBoundaryUnderRelaxation. Addresses
the keep-short-boundary-timeout finding (e8hzh).

* fix(csc): pair the batcher with the flushed cache on full invalidation

On a FLUSHDB/FLUSHALL (nil-payload) invalidation, HandlePushNotification snapshotted
the cache at entry but read h.batcher FRESH under the flush lock. A last-user release +
rebind to a different cache (A->B) running under h.mu.Lock between the snapshot and the
flush left h.batcher = B's while cache = A: dropping B's batcher bumped B's epoch and
skipped B's already-queued deletes while only A was flushed, so B served stale entries
until MaxStaleness (zero by default).

Extract fullFlush and drop the batcher ONLY when it still belongs to the cache being
flushed (h.cache == cache). An unchanged binding still drops the fresh batcher — a
same-cache batcher rebuild's queued deletes must be superseded — but a rebind no longer
drops the new binding's batcher. Pinned by
TestInvalidateHandlerFullFlushPairsBatcherWithSnapshotCache.

Addresses the P1 keep-batcher-paired-with-the-snapshotted-cache finding (fCCqu).

* fix(csc): attribute a cancelled coalesced miss to its sent connection

When a caller's context expired after the coalescer session had sent the command but
before the reply was claimed, fetch reported the cancellation with a nil connection and
returned nil, so the error/operation-duration metrics dropped the peer attributes for a
command Redis actually processed. servedBy is unsafe to read on that branch (it races
the session's write; the happens-before edge is the req.done receive, which has not
happened yet).

Add sentConn (atomic.Pointer[pool.Conn]), stored by the session when the batch is
assigned to a connection; the cancel branch reads it to attribute the metric and the
returned conn, falling back to nil only for a true pre-send cancellation. Metrics-only,
no behavior change; the exact send-vs-cancel race is not deterministically unit-pinned.

Addresses the record-sent-coalesced-misses-before-cancellation finding (fCCqy).

* fix(internal): drain pooled semaphore timer unconditionally

putSemTimer gated its stale-fire drain on `!t.Stop()` and skipped it when Stop
returned true. Stop's return cannot decide whether a fire is pending: it reports
false for an already-expired timer, but a fire can settle into the channel while
Stop runs, and Stop can report true in that window. A timer holding an
undelivered fire then went back to the pool dirty, and the next Acquire's
Reset-and-select read the stale value as an instant timeout.

Drain unconditionally with a non-blocking select/default after Stop. It is safe
under both timer-channel semantics and never hangs: after Stop the timer is
disarmed, so at most one fire can be pending; the default branch runs when none
is.

Observed in CI on Go 1.24 (oldstable, GODEBUG unset) as a spurious timeout in
TestFIFOSemaphoreNoStaleTimerFire. Not reproduced locally on go1.24.13 or
go1.25.3 across tens of thousands of iterations; the fix holds by construction,
since Stop's return cannot detect a pending fire.

* fix(csc): recover coalescer callback panics; honor pre-I/O cancel; bound reply deadline

Three coalescer robustness fixes from the #3989 review.

Recover a panicking user CacheSizer (via fulfillCached) or custom Cmder reply
parse on the full-duplex reader goroutine: applyAndSettle now defers a recover
that calls settleErr, so the panic neither crashes the process nor leaves the
caller blocked on req.done (the req is already off inflight, so the teardown
drain cannot reach it). fetch now releases the serialize slot and cancels the
cache reservation on a panic in the writeCmd window (folded into the !enqueued
defer + a serializeHeld flag), instead of leaking the slot and pinning the key
IN_PROGRESS until StaleTimeout. These are guardrails for the documented
'callbacks must not panic' contract; they mirror the refresher/invalidation-
batcher recover pattern and are not separately unit-pinned.

Honor caller cancellation while a coalesced miss waits for a session but has not
yet reached a connection (req.sentConn == nil), even when ContextTimeoutEnabled
is false. The ordinary path passes ctx to ConnPool.Get and only drops to the
policy ctx for socket I/O; the reply wait now matches that, then follows the
policy once I/O has begun.

Arm the reply read under batchBudget (write + read), not bare ReadTimeout: the
writer hands a batch to the reader before flushing it (deadlock avoidance), so a
reply read can arm while its request is still being written, and a slow/large
write within WriteTimeout would spuriously time out and tear down the session.
The drain backstop's zero-progress interval is kept strictly above batchBudget so
a legitimately slow single reply is not mistaken for a stall. This changes a
user-visible reply-timeout bound; no test drives a reply slower than ReadTimeout.

Addresses the coalescer-reader-panic (fCCni-689), fetch-panic-leak (532),
post-enqueue-cancellation (492) and reply-deadline-during-write (392) findings.

* fix(csc): safe cache identity, batcher rebind pairing, per-notification fetch order

Three invalidation-handler correctness fixes from the #3989 review.

Compare caches with sameCache() instead of ==: h.cache == cache in fullFlush
panics on every FLUSHDB/FLUSHALL when the cache's dynamic type is non-comparable
(a supported case — sameCache already exists for it), crashing the push
goroutine. Pinned by TestInvalidateHandlerFullFlushNonComparableCacheNoPanic.

Pair the windowed enqueue batcher with the snapshot cache: after a last-user
release + rebind to a different cache (A->B) between the notification snapshot and
ensureBatcher, the returned batcher belonged to B and deleted B's entries for a
push meant for A. Guard with sameCache(b.cache, cache), else fall through to the
inline delete on the snapshot; fullFlush already guarded the same pairing.

Snapshot the fetch-order sequence once per notification, not per key: a live
per-key cscFetchSeq.Load() included a fetch reserved after the push was observed,
so collectHotAndDelete wrongly evicted the fresh value / cancelled its
reservation. Mirrors cscInvalItem.fetchSnap (taken at enqueue).

The rebind and fetch-order fixes are race-dependent and not separately unit-
pinned; the CSC -race suite covers them. Known follow-up: the batcher enqueue
path (csc_inval_batch.go) still snapshots cscFetchSeq per enqueue call rather than
per notification — a deliberate, larger change left for a focused commit.

Addresses the safe-cache-identity (468), enqueue-cache-rebind (409) and
per-notification-fetch-order (434) findings.

* fix(csc): boundary-probe idle drain; document built-in-processor and hook contract

Route the background idle-drainer read through a relaxation-aware boundary probe,
mirroring pushDrainWithin: under active maintenance relaxation, confirm a RESP
byte under the short hard cap before granting the relaxed budget, so a TLS control
record (socket-readable, zero RESP bytes) cannot stall the single-goroutine
drainer for the full relaxed timeout (3068).

Document the contract instead of chasing every corner case: a custom
PushNotificationProcessor is not supported with client-side caching (CSC relies on
the built-in processor consuming kernel-only data and tolerating boundary
timeouts); the client logs a warning at init in that case. Hooks and user
callbacks must call next, must not call Close or other client control methods
(deadlock), must not panic, and must not mutate client/connection state. Behavior
is undefined otherwise. Documented on Options.ClientSideCacheConfig, AddHook,
.claude/specs/push.md, and AGENTS.md.

Addresses the custom-processor-staleness (3057) and custom-processor-gate
(push.md:35) findings by contract, and the relaxed-idle-drain-probe (3068) finding
by fix.

* Revert "fix(internal): drain pooled semaphore timer unconditionally"

This reverts commit e62712061e.

* fix(csc): publish client weak ref before the drainer starts

cscClientWeak was assigned in cscRegisterCleanups, which runs AFTER attachCSC has
already launched the background drainer. The push-handler adapter's canonical close
reads cscClientWeak.Value() on the drainer goroutine, so a push arriving in that
window both raced the assignment (a data race) and could miss the close. Publish the
weak back-reference in NewClient BEFORE attachCSC, and drop the post-attach set in
cscRegisterCleanups (writing it while the drainer is live would re-introduce the
race). Weak so a dropped *Client stays collectible and its GC cleanup still fires.

From the Copilot review of #3989.

* fix(csc): bound the invalidation batcher's pending batch by bytes

The windowed invalidation batcher size-capped its PENDING batch by item count only
(cscInvalBatchMax=4096). ch and spill bound the bytes WAITING to be consumed, but the
worker then holds each key in pending until the window or the item cap flushes, so a
stream of large keys (each under the admission limit) could retain GBs across one
window while the item count stayed far below the cap. Track retained key bytes and
flush the batch early once they cross cscInvalBatchMaxBytes (8 MiB, mirroring the
spill/refresh key-byte caps) — a normal apply, not a full-cache Flush. Pinned by
TestInvalBatcherPendingByteCapFlushes.

From the Copilot review of #3989.

* fix(csc): stop refreshes from renewing a key's read recency

LocalCache.fulfill stamps lastAccessNs unconditionally, and the refresh hot-key guard
keeps entries whose lastAccessNs is past the refresh horizon. So a background refresh
republish looked like a fresh reader access: a key invalidated faster than the horizon
tick kept refreshing itself even after every reader stopped — a self-sustaining
refetch loop contrary to the cold-key guard.

collectHotAndDelete now captures the evicted entry's reader-access token on the
refresh target, and the refresh republish restores it via LocalCache.restoreAccessToken
after fulfill, so only a real reader read (get, which re-stamps) keeps a key eligible
for refresh. A reader racing the restore only UNDER-refreshes that key (the next reader
refetches), never serves stale. Refresh runs only with the built-in *LocalCache, so the
restore is a *LocalCache method reached via a type assertion; the public Cache surface
is unchanged. Pinned by TestRefreshRepublishDoesNotRenewDemand.

From the Copilot review of #3989.

* test(csc): fix disable-mid-miss flake; correct wire-budget note

TestFullDuplexDisabledMidMissRetriesUncached read cscMissCoalescer.Load().fetch after
disableCSCServing, whose drainer teardown asynchronously swaps the coalescer to nil —
a scheduler delay made the Load() nil and the test nil-deref. Capture the coalescer
BEFORE disabling and call the captured pointer (it still honors the serving-off flag).

Correct TestCSCMissCoalescerBoundsConcurrentSerialize's comment: the serialize
semaphore bounds the NUMBER of concurrent snapshots, not their bytes — peak memory is
cap x per-snapshot size, not the wire budget, because a single undercounted argument
still allocates its full snapshot before reconcileWireBytes sheds it. The per-request
byte ceiling is a deferred follow-up; the comment no longer overclaims the bound.

From the Copilot review of #3989.

* fix(csc): snapshot invalidation fetch order once per push

The windowed invalidation batcher snapshotted cscFetchSeq inside enqueue, i.e. once
PER KEY. A push that invalidates several keys therefore stamped a different fetch-order
snapshot on each cscInvalItem, so a fetch reserved after the push was observed but
before the loop reached a later key was not treated as newer for that key: the
fetch-order guard (cscInvalItem.fetchSnap, checked at apply in collectHotAndDelete)
could then evict a fresh value or cancel an in-progress reservation on a straddling
duplicate. The inline (window==0) delete path already snapshotted once per push.

The push handler now takes fetchSnap := cscFetchSeq.Load() once, before the enqueue
loop, and passes it to enqueueAt, which stamps that shared snapshot on every key in
the push. enqueue keeps the per-call snapshot for single-key/test callers. Pinned by
TestInvalBatcherEnqueueAtStampsSuppliedFetchSnap.

From the review of #3989.

* docs(csc): correct the miss-coalescer serialize-bound comment

The cscMissMaxConcurrentSerialize comment claimed the concurrent set keeps peak memory
at O(budget) with at most a few batch-sized snapshots. That overclaims: reserveWireBytes
gates on cmdApproxBytes, which undercounts a variable-size argument, so a single command
whose encoded form far exceeds the estimate (e.g. an encoding.BinaryMarshaler returning
hundreds of MB) is fully serialized before reconcileWireBytes sheds it. The semaphore
bounds the COUNT of concurrent snapshots, not their bytes; the true worst-case transient
peak is cap x (largest single serialized command). A hard per-request byte ceiling
during writeCmd is a follow-up. Comment only.

From the review of #3989.

* fix(pool): keep relaxed timeout active while a holder survives

HasRelaxedTimeout did a single snapshot Load and returned false the moment a deadline
passed, even when a non-deadline (notification) holder was still active — contradicting
the surviving-holder semantics that getEffectiveReadTimeout already implements. Until an
Effective* call happened to run expireRelaxedTimeout, HasRelaxedTimeout reported no
relaxation while the connection was in fact still relaxed.

Retire the expired deadline holder once (a single CAS, never on the hot unexpired path)
and re-read the snapshot before deciding no relaxation remains, mirroring
getEffectiveReadTimeout. Only tests call HasRelaxedTimeout; there is no production
hot-path caller. Pinned by TestHasRelaxedTimeoutExpiresDeadlineAndReadsSurvivor.

This fixes the boolean; the timeout VALUE still comes from the single shared
(readNs, writeNs) pair (last-writer-wins), whose per-holder preservation is a separate
tracked follow-up.

From the review of #3989.

* fix(csc): apply the batcher fetch-order guard without refresh

When invalidation batching was enabled for a *LocalCache but refresh-on-invalidate was
off, the batcher's apply path discarded it.fetchSnap and did an unconditional
DeleteByRedisKey. A delayed invalidation whose fetch snapshot predated a refetch during
the batch window then deleted the newer value (or cancelled its in-progress fetch),
serving a spurious miss and possibly waking waiters into duplicate fetches.

Route the *LocalCache delete through the sequence-aware deleteByRedisKeyCollectingHot
with it.fetchSnap (discarding collected refresh targets, since refresh is off), so
fetchSeq > fetchSnap is honored exactly as on the inline and refresh-on paths. A
non-*LocalCache cache keeps the plain delete (no per-entry fetch sequence). Pinned by
TestInvalBatcherNoRefreshGuardKeepsRefetched.

From the review of #3989.

* fix(csc): abort an in-progress refresh flush when Close begins

A normal (timer/demand/window-full) refresh flush already running when Close arrived kept
its stopping=false snapshot for the whole run, so it ignored the now-closed h.stop and gave
each remaining chunk a fresh cscRefreshBatchTimeout. With reply budgeting producing
one-target chunks over a 512-key window, a stalled server could hold Close for tens of
minutes. The earlier F2 fix only bounded a flush ENTERED from the stop case.

Check h.stop between chunks on the non-stopping path and bail; the stopping==true stop-drain
already bounds itself by bailing after the first failed chunk. Bailed targets stay evicted
(a reader repopulates them; coverage is revoked on Close). Pinned by
TestCSCRefresherNormalFlushAbortsOnClose.

From the review of #3989.

* fix(csc): bound coalescer Close drain, guard inline invalidation

Two findings from the re-review that reached the miss-coalescer files, plus a
documented dispute:

- Close-time drain (csc_miss_coalesce_modes.go): drainBackstop renewed the wait on
  ANY per-interval progress, so a server trickling one reply per interval kept a deep
  in-flight pipeline (up to cscFullDuplexDepth) draining and blocked Close in wg.Wait
  for hours. Arm a total stopping-drain deadline once — cscMissStopDrainIntervals x
  interval — and close+return once spent, even while readsDone advances. Stated
  tradeoff: past the budget, already-accepted in-flight misses may be cancelled rather
  than completed (Close-never-wedges outranks completing the remainder). Non-stopping
  (age-out/handoff) recycle keeps the unbounded progress rule. drainBackstop is
  extracted to drainBackstopRun so the cap is unit-testable (pure move; the only
  in-body change is c.opt -> mc.c.opt).

- Inline invalidation (csc_integration.go): with refresh-on-invalidate OFF the inline
  delete ignored the fetch-order guard, so a delayed invalidation could cancel a miss
  reserved after this push was observed and wake coalesced waiters as duplicate misses.
  Route it through deleteByRedisKeyCollectingHot (cscInvalNoHorizon, fetchSnap), the
  mirror of the batcher's !canRefresh branch.

- batchBudget 5s floor: documented that the floor also anchors the drain stall interval
  (batchBudget + 1s), so it cannot be removed to honor short read timeouts without
  force-closing healthy conns; the read-deadline/stall-interval decouple is a follow-up.

Pinned by TestCSCMissStopDrainCapsTotalOnTrickle and
TestInvalidateHandlerInlineNoRefreshKeepsRefetched (both red-checked).

From the review of #3989.

* fix(csc): arm coalescer drain cap on a recycle-then-Close race

The round-6 stopping-drain cap only armed when drainBackstopRun was entered with
stopping==true. Age-out and handoff recycle enter with stopping==false, and the
normal-timeout main progress-wait did not watch mc.stop (only the deadline-less branch
did). So if Close arrived AFTER a recycle drain had already started, the supervisor
stayed on the unbounded progress path and a server trickling one reply per interval
could hang Close in wg.Wait for hours — the exact hang the cap was meant to prevent.

The main timer select now watches mc.stop on the non-stopping path (a nil channel once
stopping, so a closed mc.stop cannot busy-spin the select): a Close arriving mid-recycle
flips the drain to the stopping path and the next loop arms the total cap.

Pinned by TestCSCMissStopDrainCapsAfterCloseRaceInRecycle (ReadTimeout/WriteTimeout > 0
so the deadline-less branch is skipped and the drain sits in the main select, then
mc.stop is closed mid-drain; red-checked).

From the review of #3989.

* fix(csc): move single-key enqueue helper to tests

cscInvalBatcher.enqueue had no production caller: the push handler uses
enqueueAt with a fetch snapshot taken once per push, and enqueue was only a
single-key convenience for tests. golangci-lint's unused analyzer skips test
files, so it read the method as dead code and failed the lint job. Move it
verbatim into csc_inval_batch_test.go (same package); no behavior change.

* feat(csc): add ClientSideCacheRefreshRecencyWindow, default refresh-all

Options.ClientSideCacheRefreshRecencyWindow bounds
ClientSideCacheRefreshOnInvalidate to keys read within a configurable window
before their invalidation arrived. Default (0) changes to refreshing every
invalidated Valid entry regardless of recency; a positive value restores a
bounded lookback, rounded up to a guaranteed-covered multiple of the existing
200ms cscRefreshRecencyTick via a small ring of per-tick LRUClock()
snapshots (cscRecencyRing).

Tradeoff, documented on the option: refreshing re-registers a key with the
server tracking table, which manufactures the next invalidation - so the
new default turns a write-heavy, rarely-read key into a self-sustaining
refresh loop driven by write traffic until capacity eviction reaps it. The
window option is how to bound that.

cscInvalNoHorizon (-1) is reused as the "refresh everything" sentinel: it
already sits below any real lastAccessNs token, so the unchanged
deleteByRedisKeyCollectingHot comparison marks every entry hot with no
change to that function. Wired through UniversalOptions.Simple() alongside
the sibling CSC knobs.

Tests: TestCollectHotAndDeleteNoHorizonRefreshesEverything,
TestCscRecencyRingOldest, TestCscRefreshWindowTicks,
TestStartCSCRefresherRecencyWindowWiring, extended
TestUniversalOptionsSimpleCopiesCSCCoalesce.

* fix(csc): skip stop-drain refetch, guard worker teardown race

Two codex findings on #3989.

P1 (csc_refresh_on_invalidate.go:655) — Close's refresher stop-drain
used to refetch every buffered invalidated key over the network,
bailing only after the first FAILED chunk (#3965 F2). Against a
healthy server nothing fails, so that bound never triggered: a full
backlog could cost thousands of round trips, one per chunk once the
reply-byte budget shrinks each chunk to a single target. Those round
trips were always wasted — stopCSCRefresherAndCoalescer's teardown
already documented (csc_integration.go:1305-1316, pre-existing) that
invalidateAllCoverage evicts every entry this connection would have
published immediately after this flush returns, on every teardown path
(Close and both self-disable exits share the one deferred cleanup that
runs stopCSCRefresherAndCoalescer before invalidateAllCoverage). This
finishes that conclusion: flush() now skips the refetch entirely when
stopping, abandoning the buffered targets (they stay evicted; a reader
repopulates them, same outcome an errored refetch already produced).
The now-dead per-chunk stopping branches, the flush return value
(never a live signal to any caller once the skip is unconditional),
and the single-pass drain's window-cap-only shortfall are all folded
in as part of the same change: the h.stop case now loops until q.ch is
fully drained rather than leaving up to ~3584 buffered targets pinned
in the channel, since c.cscRefreshQueue is never nilled on stop.
RefreshFailed is intentionally left untouched by the skip: its doc
scopes it to errored round trips, and an abandoned-by-choice target
never attempted one.

Test: TestCSCRefresherStopDrainSkipsRefetchEntirely, replacing
TestCSCRefresherStopDrainBailsOnFirstFailure. Verified against the true
pre-fix baseline (not just the changed line): reverting to the
pre-existing code makes the erroring pooler's Get run once before the
old bail-on-first-failure logic kicks in, which the test catches.

P2 (csc_integration.go:709) — attachSharedTrackingCSC starts the
drainer before startCSCRefresher/startCSCMissCoalescer publish their
queue/handle. In principle the drainer's self-disable tick (reacting
to an async conn init's disableCSCServing) could observe cscActive
false and run stopCSCRefresherAndCoalescer, consuming workersStopOnce,
before either start function publishes — leaving that worker with no
remaining path to be stopped. This interleaving is not reachable
today: cscMinDrainInterval floors the first tick at 1ms against
synchronous, microsecond-scale construction, and nothing external
holds a reference to call Close mid-construction. That is a
timing-dependent invariant, not a structural one, so replace it with
an explicit mechanism instead of only documenting it: cscDrainHandle
gains workersMu/workersTornDown. stopCSCRefresherAndCoalescer's
teardown body sets workersTornDown=true as its first action under the
lock; each start function checks the same flag under the same lock
immediately before publishing and declines to start (launching
nothing) if teardown already ran. startBackgroundDrainer needs no
matching guard: it is what allocates cscDrainHandle, so no teardown
path can observe a non-nil handle to race against until after it
returns.

Rejected fixing this by reordering attachSharedTrackingCSC to start
the workers before the drainer, per codex's literal suggestion: that
would leave the workers unstoppable by Close if anything panicked
between the reordered start calls and the drainer's construction,
since stopBackgroundDrainer no-ops on a nil cscDrainHandle. The
current order's panic-safety property is worth keeping; the lock-based
guard closes the race without touching it.

Tests: TestCSCConstructionRaceDeclinesToStartAfterTeardown (drives
workersTornDown directly, independent of cscActive; verified to fail
if either start function's guard is reverted),
TestCSCConstructionWinsRaceStartsNormally (control),
TestCSCStopCSCRefresherAndCoalescerSetsWorkersTornDown (plumbing).

* fix(csc): widen worker-publish lock, scope limiter denial to the waker

Two cursor Bugbot findings on e2a103e5.

High: "Teardown joins unstarted CSC workers" — the workersTornDown
check in startCSCRefresher/startCSCMissCoalescer only covered the
initial field write (cscRefreshHandle / cscMissCoalescer), not the
rest of publish. startCSCRefresher still had setRefreshQueue's
predecessor-batcher join after the write, and startCSCMissCoalescer
still had its wg.Add loop after the Store. A concurrent
stopCSCRefresherAndCoalescer (the drainer's own self-disable tick,
reacting to an async conn init's disableCSCServing) could read the
just-published handle/coalescer and act on it before either had
actually finished starting: stopCSCRefresher would block on <-h.done
for a runCSCRefresher goroutine not yet launched (bounded by however
long the join took, not a permanent deadlock, but pointless to allow),
and stopCSCMissCoalescer's wg.Wait() could run concurrently with an
in-progress wg.Add — the sync.WaitGroup misuse case, which can panic.

Fix: both start functions now run their entire publish step — the
field write, the queue/batcher attach (refresher) or the wg.Add/go loop
(coalescer), and the goroutine launch — inside the SAME workersMu
critical section that stopCSCRefresherAndCoalescer's teardown body
acquires as its first action. Teardown cannot read a published
handle/coalescer until publish has fully finished (goroutines running),
because it cannot even acquire the lock until then. Holding the lock
across setRefreshQueue's batcher join is safe: that path is
internal/panic-free (lookupInvalidateHandler resolves to the built-in
push processor's plain registry lookup, never user code, since CSC
requires the built-in processor) and never re-enters workersMu.

Test: TestCSCStartRefresherHoldsLockAcrossBatcherJoin installs a
controlled predecessor batcher (no run() goroutine backing it, so its
join blocks until released) and verifies workersMu stays held —
observed via TryLock, which never blocks, so the check can't deadlock
on the lock it's inspecting — for as long as startCSCRefresher is
parked in that join, even with a concurrent teardown racing it.
Verified against e2a103e5 directly: the test correctly fails there,
since that version's lock only spans the field write. Covers the
refresher path only; the coalescer's wg.Add window gets the same fix
but has no separate test — there is no equivalent controllable
blocking hook in its publish sequence.

Medium: "Limiter denial fails queued coalesced misses" — when
Limiter.Allow rejected the miss-coalescer session's getConn, every
request already queued behind the one that triggered the check
(first) was failed with that same raw, non-retryable error
(drainQueuePlain). Those queued requests never called Allow
themselves; only first did. The existing rationale (avoid a second
Allow call that could admit the very operation just denied) is
correct for first but does not extend to requests the limiter was
never asked about.

Fix: first still gets the raw, untagged error (settlePlain, unchanged).
Every other already-queued request now gets errCSCRetryUncached
(drainQueueErr) instead, so processCached re-runs each on the ordinary
per-command path and each calls Allow independently — the same
handling this engine already uses for a session torn down mid-fetch.
drainQueuePlain is now unused and removed.

Test: TestCSCMissLimiterDenialOnlyFailsWakingRequestPlainly drives
runFullDuplexSession directly with a rejecting Limiter and one queued
follow-up request, asserting first gets the raw sentinel and the
queued request gets errCSCRetryUncached, with Allow called exactly
once. Verified to fail against the prior drainQueuePlain behavior.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: sadd amr3e <saddamr3e@gmail.com>
Co-authored-by: Albert Huynh <huynhalb@gmail.com>
Co-authored-by: Vladislav Kotsev <vladygk8@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

.claude/specs/push.md

@@ -0,0 +1,74 @@
+# Push-notification consumption (RESP3)
+
+How the v9 client consumes server pushes (`>`-typed frames: CSC invalidations,
+maintenance notifications), the guarantees it gives, and the rules that keep the
+model closed. Read this before adding any code that reads pushes.
+
+## The model: checkpoints, not owned readers
+
+v9 connections have no owning reader: the goroutine that holds a conn reads it.
+Pushes are therefore consumed at fixed checkpoints:
+
+| Checkpoint | When | Mode |
+|---|---|---|
+| Reply read | after every command write | blocking loop: frame is a push → process, read again; else it is the reply |
+| Pre-command drain | before a command uses a pooled conn | speculative (peek, bounded) |
+| Cache-hit stale-guard | after a CSC cache hit | speculative |
+| Pool put-path check | before a conn returns to the pool | speculative |
+| Idle-conn drainer | background tick over idle pooled conns | speculative |
+| FD session / pubsub | connection with an owning reader | at arrival |
+
+Speculative checkpoints cannot know whether a complete frame is present. They
+gate on two levels — `HasBufferedData` (bufio) and `MaybeHasData` (kernel
+`MSG_PEEK`) — and drain under a hard read cap (`cscDrainHardReadCap`, raised to
+the relaxed timeout while maintenance relaxation is active: pushDrainBudget).
+Physics of the model, not bugs:
+
+- A frame, once begun, must be completed (may block under the cap); a mid-frame
+  timeout is a real desync and retires the conn.
+- Kernel readability over TLS can mean zero RESP bytes (control records), so
+  bare readability is handled only by the built-in processor, which treats a
+  boundary timeout as benign.
+- A custom `PushNotificationProcessor` implements only the blocking loop, so
+  speculative checkpoints hand it work only when real bytes are buffered
+  (spurious-retire risk otherwise) and skip it for kernel-only data (staleness
+  cost). Both bounded; the fix is the follow-up drainer capability below.
+
+## The freshness guarantee (user-facing)
+
+Invalidations apply: at arrival on FD-autopipeline conns; at the next command on
+active conns; at the next drainer tick on idle pooled conns. Hard cap
+everywhere: the cache's MaxStaleness. This is documented on
+`Options.ClientSideCacheConfig`.
+
+The next-drainer-tick bound holds for the built-in processor, which consumes
+kernel-only readability. A custom `PushNotificationProcessor` is **not supported**
+with client-side caching: CSC relies on the built-in processor to consume
+kernel-only data and to treat a boundary read timeout as benign, which a custom
+processor's blocking-loop contract does not guarantee. With a custom processor,
+invalidation freshness and idle-connection health are undefined — use the built-in
+processor with CSC. The client logs a warning at init if CSC is enabled with a
+custom processor.
+
+## Rules (enforced in review)
+
+1. **No new speculative drain call sites.** Each one re-buys the whole edge-case
+   catalog (partial frames, TLS peeks, cap tuning, custom-processor gating). New
+   features consume pushes through the existing checkpoints or not at all.
+2. **A feature that needs fresher pushes than checkpoints give must own a
+   reader** (the FD-session / miss-coalescer / pubsub pattern: one goroutine
+   owns the conn's read side; pushes are consumed at arrival). It must not add
+   checkpoints.
+3. **The mid-frame rule is not negotiable:** never abandon a partially consumed
+   frame; a mid-frame timeout retires the conn.
+
+## Planned follow-up (needs maintainer sign-off)
+
+One-mode convergence: a `NotificationDrainer` capability interface (consume
+complete frames until the buffer is empty at a boundary; finish any begun frame;
+never wait for an unbegun frame; boundary-empty and boundary-timeout = success).
+Speculative checkpoints type-assert it — custom processors that implement it get
+full treatment (kernel-only data included, no spurious retires); the blocking
+method remains for the reply loop and pub/sub and as the legacy fallback. Most
+users should register handlers on the built-in processor instead of replacing
+the processor.

AGENTS.md

@@ -114,6 +114,16 @@ error, it must call `cmd.SetErr(wrappedErr)` so the typed-error helpers
 (`redis.IsLoadingError`, `IsMovedError`, etc. in `error.go`) keep working through
 `errors.As`. The README has a longer pipeline-hook example.
 
+**Hook contract** (behavior is undefined otherwise — the mechanism assumes
+well-behaved hooks rather than defending every corner case): a hook must call
+`next` (unless deliberately terminating the command), must **not** call `Close`
+or other client control methods (it runs on the goroutine `Close` waits for, so
+it deadlocks), must **not** panic, and must **not** mutate client or connection
+state. It may observe and wrap errors only. User callbacks such as `CacheSizer`
+follow the same no-panic rule. Client-side caching additionally requires the
+**built-in** push processor; a custom `PushNotificationProcessor` is unsupported
+with CSC (the client logs a warning at init). See `.claude/specs/push.md`.
+
 ### Connection pool (`internal/pool`)
 
 Owns dialing, idle/active connection bookkeeping, conn state (`conn_state.go`),

csc_coalesce_options_test.go

@@ -0,0 +1,32 @@
+package redis
+
+import (
+	"testing"
+	"time"
+)
+
+// TestUniversalOptionsSimpleCopiesCSCCoalesce guards that the new CSC miss-
+// coalescing / invalidation-batching knobs reach a standalone Client through
+// UniversalOptions.Simple() (they were previously Options-only, so UniversalClient
+// users could not enable them).
+func TestUniversalOptionsSimpleCopiesCSCCoalesce(t *testing.T) {
+	u := &UniversalOptions{
+		ClientSideCacheRefreshOnInvalidate:     true,
+		ClientSideCacheRefreshRecencyWindow:    3 * time.Millisecond,
+		ClientSideCacheCoalesceMisses:          true,
+		ClientSideCacheInvalidationBatchWindow: 7 * time.Millisecond,
+	}
+	o := u.Simple()
+	if !o.ClientSideCacheRefreshOnInvalidate {
+		t.Error("Simple() dropped ClientSideCacheRefreshOnInvalidate")
+	}
+	if o.ClientSideCacheRefreshRecencyWindow != 3*time.Millisecond {
+		t.Errorf("Simple() ClientSideCacheRefreshRecencyWindow = %v, want 3ms", o.ClientSideCacheRefreshRecencyWindow)
+	}
+	if !o.ClientSideCacheCoalesceMisses {
+		t.Error("Simple() dropped ClientSideCacheCoalesceMisses")
+	}
+	if o.ClientSideCacheInvalidationBatchWindow != 7*time.Millisecond {
+		t.Errorf("Simple() ClientSideCacheInvalidationBatchWindow = %v, want 7ms", o.ClientSideCacheInvalidationBatchWindow)
+	}
+}

csc_integration.go

@@ -26,13 +26,78 @@ func cscRegisterCleanups(c *Client) {
 	if h == nil {
 		return
 	}
+	// The weak back-reference (cscClientWeak) is published by the caller BEFORE
+	// attachCSC starts the drainer (see NewClient), so the push-handler adapter's
+	// canonical close reads it race-free. It is deliberately NOT set here: a write
+	// after the drainer is already live would race that read.
 	// Capture cscActive (a standalone *atomic.Bool, not *Client) so the cleanup
-	// also stops clones from serving once the drainer is gone.
+	// also stops clones from serving once the drainer is gone. Capture the miss
+	// coalescer too (set during attach; nil when off): its sessions wait on
+	// their own stop channel, so a client dropped WITHOUT Close would leak them
+	// and everything they retain (cache, pools). stopWorkers is idempotent and
+	// signal-only, keeping the cleanup non-blocking; the drain-cancel then
+	// settles any queued request whose reservation would otherwise stay
+	// IN_PROGRESS and block a cache-sharing client until StaleTimeout (channel
+	// receive gives each request exactly one consumer).
 	active := c.baseClient.cscActive
+	mc := c.baseClient.cscMissCoalescer.Load()
+	// Capture the refresh handle too (set during attach; nil when refresh is off):
+	// runCSCRefresher parks on its ticker/queue and holds *baseClient, so a client
+	// dropped WITHOUT Close would leak the refresher goroutine and everything it
+	// retains (cache, pools) — the coalescer/drainer stops below would not reach it.
+	// signalStop is idempotent and non-blocking, matching the other stops here. The
+	// handle is channels only (no *Client), so capturing it keeps the wrapper
+	// collectible.
+	rh := c.baseClient.cscRefreshHandle
+	// Capture this client's refresh queue too: signalStop stops the refresher
+	// goroutine, but with a SHARED cache+processor the invalidate handler still
+	// holds this queue as (possibly) the active refresh binding. If a sibling
+	// client survives, invalidations would keep feeding a stopped queue and the
+	// shared cache's refresh-on-invalidate would silently degrade to plain
+	// eviction. clearRefreshQueue unbinds it and restores the sibling's binding,
+	// mirroring stopCSCRefresher on the clean Close path. A queue is only ever
+	// created inside attachSharedTrackingCSC, which also builds the drain handle
+	// (startBackgroundDrainer), so h.invalidateHandler is set whenever q is
+	// non-nil; a Conn() clone bails before startCSCRefresher and never has one.
+	q := c.baseClient.cscRefreshQueue
 	runtime.AddCleanup(c, func(h *cscDrainHandle) {
+		// Order mirrors stopBackgroundDrainer (the clean Close path) but stays
+		// fully NON-BLOCKING — a GC finalizer must never wait. Unbind this queue and
+		// signal the refresher stop while serving is STILL ON (active==true), then
+		// deactivate: refreshInvalidatedBatch bails once cscActive is false, so
+		// deactivating first would make the refresher's stop-drain flush a no-op and
+		// drop in-window hot keys for a surviving sibling on a shared cache.
+		// signalStop only closes a channel, so the final flush runs ASYNCHRONOUSLY;
+		// active.Store(false) lands nanoseconds later and the flush may still miss
+		// the window — the guaranteed final-flush is the Close path, which joins the
+		// refresher before deactivating. This reorder is the free, strictly-better
+		// best effort for the drop-without-Close case.
+		//
+		// clearRefreshQueue detaches+signals the batcher but the finalizer DISCARDS
+		// it (no join): the batcher's own run() drains and exits asynchronously. The
+		// clean Close/self-disable paths join it instead.
+		//
+		// This deliberately DIVERGES from stopCSCRefresherAndCoalescer's canonical
+		// order (coalescer-conn-release before the refresher flush): the finalizer is
+		// signal-only and never joins, so it never waits on a refresher flush and the
+		// connection-release ordering is moot here — the pool may already be gone. The
+		// blocking, correctly-ordered teardown is the Close / self-disable path.
+		if q != nil && h.invalidateHandler != nil {
+			h.invalidateHandler.clearRefreshQueue(q)
+		}
+		if rh != nil {
+			rh.signalStop()
+		}
 		if active != nil {
 			active.Store(false)
 		}
+		if mc != nil {
+			mc.stopWorkers()
+			// Retry-uncached, matching every other stop path: a clone still
+			// alive re-runs the read on the (possibly still open) pool instead
+			// of surfacing a spurious ErrClosed.
+			mc.drainQueueErr(errCSCRetryUncached)
+		}
 		h.signalStop()
 	}, h)
 }
@@ -78,6 +143,211 @@ type invalidateHandler struct {
 	cache     Cache
 	keyPrefix string
 	users     int
+
+	// refresh, when set, receives evicted-but-hot entries for immediate refetch.
+	// Feeding it must never block the invalidation-delivery path. It is the TOP
+	// of refreshStack: clients sharing this handler each attach their own queue
+	// and the newest attachment is active; when it clears, the next-newest live
+	// binding is RESTORED (closing the newest owner must not sever an older
+	// sibling's still-running refresher).
+	refresh *cscRefreshQueue
+
+	// refreshStack holds every live refresh binding in attach order (older
+	// first). Guarded by mu. Small: one entry per client sharing the handler.
+	refreshStack []*cscRefreshQueue
+
+	// batcher offloads invalidation cache-deletes to a windowed background
+	// goroutine (Options.ClientSideCacheInvalidationBatchWindow). Lazily started
+	// (ensureBatcher) and nil when disabled; guarded by mu. Stopped and cleared
+	// when the last user releases (releaseLocked), so its goroutine does not live
+	// past the binding re-arming its timer forever; a later re-acquire starts a
+	// fresh one (picking up the successor's window).
+	batcher *cscInvalBatcher
+
+	// invalBatchWindow is the EFFECTIVE coalescing window for the batcher above:
+	// the strictest window folded in across attached clients (see
+	// setInvalBatchWindow — 0/inline strictest, then smaller nonzero). 0
+	// (default) deletes inline. invalBatchWindowSet distinguishes "no client has
+	// attached a window yet" from an explicit 0 (inline) that must win over any
+	// later nonzero window. Read under mu alongside cache/keyPrefix/refresh.
+	invalBatchWindow    time.Duration
+	invalBatchWindowSet bool
+}
+
+// setInvalBatchWindow folds one client's window into the shared handler's
+// effective window at attach time. Clients sharing a handler may configure
+// different windows but the handler runs ONE batcher, so the effective window
+// is the STRICTEST attached: smallest nonzero, with explicit zero (inline
+// deletes) strictest of all — tightening can never violate a client's staleness
+// bound, while taking the latest as-is could loosen an earlier stricter one.
+// Not re-loosened on close (bindings carry no identity; staying stricter costs
+// only efficiency). On a tighten the running batcher (window fixed at creation)
+// is stopped — stop() flushes, so no queued delete is lost — and the next
+// invalidation rebuilds via ensureBatcher.
+func (h *invalidateHandler) setInvalBatchWindow(w time.Duration) {
+	h.mu.Lock()
+	if h.invalBatchWindowSet {
+		cur := h.invalBatchWindow
+		// Strictness: 0 (inline) is strictest; among nonzero, smaller is stricter.
+		stricter := (w == 0 && cur != 0) || (w != 0 && cur != 0 && w < cur)
+		if !stricter {
+			h.mu.Unlock()
+			return
+		}
+	}
+	h.invalBatchWindowSet = true
+	h.invalBatchWindow = w
+	// On a tighten drop the running batcher (window fixed at creation): the next
+	// invalidation rebuilds under the NEW (stricter) window. Detach+stop under the
+	// lock, then join OUTSIDE it. Joining under h.mu would stall a sibling in
+	// ensureBatcher/HandlePushNotification; the join keeps the old worker's
+	// stop-drain from applying after the rebuild — a late apply under the looser
+	// contract could evict an entry the stricter-window batcher just repopulated.
+	b := h.detachBatcherLocked()
+	h.mu.Unlock()
+	if b != nil {
+		b.join()
+	}
+}
+
+// detachBatcherLocked removes the running batcher from the handler and SIGNALS it
+// to stop, under h.mu; returns the detached batcher (nil when none). It never
+// JOINS: join() waits on the batcher's stop-drain, which must not run while h.mu is
+// held (it would stall a sibling on the hot path — ensureBatcher/HandlePushNotification)
+// and must never block the GC finalizer. Callers needing a SYNCHRONOUS teardown
+// (the Close path) join the returned batcher AFTER releasing h.mu; the GC finalizer
+// discards it (signal-only). A repointing caller (set/clearRefreshQueue) stores the
+// new refresh binding on the batcher BEFORE calling this, so the stop-drain offers
+// hot keys to the surviving refresher.
+func (h *invalidateHandler) detachBatcherLocked() *cscInvalBatcher {
+	b := h.batcher
+	if b == nil {
+		return nil
+	}
+	h.batcher = nil
+	b.stop()
+	return b
+}
+
+// ensureBatcher lazily starts the windowed invalidation batcher. The common
+// case (already started) is a shared RLock; only first-start takes the write
+// lock, so the hot invalidation path stays cheap.
+func (h *invalidateHandler) ensureBatcher() *cscInvalBatcher {
+	h.mu.RLock()
+	b := h.batcher
+	h.mu.RUnlock()
+	if b != nil {
+		return b
+	}
+	h.mu.Lock()
+	defer h.mu.Unlock()
+	// Do not start a batcher for a released binding. releaseLocked stops+nils the
+	// batcher under this same lock when users hits 0, so a push racing that last
+	// release must NOT resurrect a goroutine that nothing would ever stop (once
+	// users is 0, release() no longer runs). The caller falls back to the inline
+	// delete path when this returns nil.
+	if h.users == 0 {
+		return nil
+	}
+	// Read the window under this lock (not a caller snapshot a concurrent
+	// tighten could supersede). 0 = batching off: return nil, caller deletes
+	// inline.
+	w := h.invalBatchWindow
+	if w <= 0 {
+		return nil
+	}
+	if h.batcher == nil {
+		h.batcher = &cscInvalBatcher{
+			window: w,
+			// Snapshot the cache for the batcher's lifetime (⊂ the binding's:
+			// releaseLocked stops it before clearing it) so the release-time
+			// stop-drain still applies queued deletes after h.cache is nilled.
+			cache:  h.cache,
+			ch:     make(chan cscInvalItem, 8192),
+			wake:   make(chan struct{}, 1),
+			stopCh: make(chan struct{}),
+			done:   make(chan struct{}),
+		}
+		// refresh is atomic so set/clearRefreshQueue can repoint it before stop;
+		// seed it with the current binding before the worker starts.
+		h.batcher.refresh.Store(h.refresh)
+		go h.batcher.run()
+	}
+	return h.batcher
+}
+
+// clearRefreshQueue clears the handler's refresh binding ONLY while it still
+// points at q: two clients sharing one cache and push processor each attach
+// their own queue (last attach wins), and a closing client must not clobber
+// the binding of a live sibling that re-attached after it — invalidations
+// would keep deleting entries but silently stop feeding the survivor's
+// refresher.
+//
+// It returns the detached batcher (nil when none) so the caller can join() it
+// OUTSIDE h.mu for a synchronous teardown; the GC finalizer discards it (must not
+// block). See detachBatcherLocked.
+func (h *invalidateHandler) clearRefreshQueue(q *cscRefreshQueue) *cscInvalBatcher {
+	if q == nil {
+		return nil
+	}
+	h.mu.Lock()
+	defer h.mu.Unlock()
+	for i, b := range h.refreshStack {
+		if b == q {
+			h.refreshStack = append(h.refreshStack[:i], h.refreshStack[i+1:]...)
+			break
+		}
+	}
+	if h.refresh != q {
+		return nil // a newer sibling owns the active binding; nothing else changes
+	}
+	// Restore the next-newest live binding (nil when none): the surviving
+	// sibling's refresher keeps getting fed after the newest owner closes.
+	h.refresh = nil
+	if n := len(h.refreshStack); n > 0 {
+		h.refresh = h.refreshStack[n-1]
+	}
+	// Repoint the batcher at the surviving binding (nil if none) BEFORE stopping it,
+	// so the stop-drain offers evicted-hot keys to the live survivor's refresher, not
+	// the closing owner's — whose drainer is already gone, so its offers would just be
+	// dropped. An in-flight apply already holding the old pointer feeds its current
+	// batch to the old queue: a bounded, benign residual.
+	if h.batcher != nil {
+		h.batcher.refresh.Store(h.refresh)
+	}
+	return h.detachBatcherLocked()
+}
+
+// setRefreshQueue binds q as the active refresh queue and returns the detached
+// batcher (nil when none) so the caller can join() it OUTSIDE h.mu — the attach
+// stays synchronous without holding the join under the handler lock.
+func (h *invalidateHandler) setRefreshQueue(q *cscRefreshQueue) *cscInvalBatcher {
+	h.mu.Lock()
+	defer h.mu.Unlock()
+	if q != nil {
+		found := false
+		for _, b := range h.refreshStack {
+			if b == q {
+				found = true
+				break
+			}
+		}
+		if !found {
+			h.refreshStack = append(h.refreshStack, q)
+		}
+	}
+	if h.refresh == q {
+		return nil
+	}
+	h.refresh = q
+	// A running batcher was created with the previous binding; drop it so the next
+	// invalidation rebuilds with the new one (stop() flushes, so no queued delete
+	// is lost). Repoint it at the new binding first, so the stop-drain feeds hot
+	// keys to q rather than the superseded refresher.
+	if h.batcher != nil {
+		h.batcher.refresh.Store(h.refresh)
+	}
+	return h.detachBatcherLocked()
 }
 
 // HandlePushNotification decodes ["invalidate", <keys>] notifications. A nil
@@ -86,7 +356,8 @@ func (h *invalidateHandler) HandlePushNotification(
 	_ context.Context, _ push.NotificationHandlerContext, notification []interface{},
 ) error {
 	h.mu.RLock()
-	cache, keyPrefix := h.cache, h.keyPrefix
+	cache, keyPrefix, refresh := h.cache, h.keyPrefix, h.refresh
+	window := h.invalBatchWindow
 	h.mu.RUnlock()
 	if cache == nil || len(notification) < 2 {
 		return nil
@@ -94,8 +365,74 @@ func (h *invalidateHandler) HandlePushNotification(
 
 	switch payload := notification[1].(type) {
 	case nil:
-		cache.Flush()
+		// FLUSHDB/FLUSHALL: supersede the batcher's queued per-key deletes, then wipe
+		// the snapshotted cache. See fullFlush for the drop/flush ordering and the
+		// binding-pairing invariant.
+		h.fullFlush(cache)
 	case []interface{}:
+		// Count incoming invalidations at the choke point: one per key named in the
+		// push, BEFORE batching/dedup/spill. Applied deletes are counted separately
+		// (DeleteByRedisKey / deleteByRedisKeyCollectingHot) and diverge from this
+		// under dedup — that gap is the signal (see CSCRefreshStats).
+		if lc, ok := cache.(*LocalCache); ok {
+			var n uint64
+			for _, k := range payload {
+				switch k.(type) {
+				case string, []byte:
+					n++
+				}
+			}
+			lc.invalidations.Add(n)
+		}
+		// Offload path: enqueue keys to the windowed background batcher instead of
+		// deleting inline, so invalidation work does not steal time from the
+		// coalescer's miss-reply reader (the low-concurrency churn p99 tail).
+		if window > 0 {
+			if _, ok := cache.(*LocalCache); ok {
+				// nil when the binding was just released (users==0) or when a
+				// concurrent window change turned batching off: fall through to
+				// the inline delete path below rather than enqueue on a nil
+				// batcher (which would panic).
+				// Pair the batcher with the SNAPSHOT cache: after a last-user release +
+				// rebind to a different cache (A->B) between the entry snapshot and here,
+				// ensureBatcher returns B's batcher, which would delete B's entries for a
+				// push meant for A (A would keep serving stale). fullFlush guards the same
+				// A->B pairing. sameCache also returns false for a non-comparable cache
+				// type, so such caches fall through to the inline delete path (correct,
+				// just not batched).
+				if b := h.ensureBatcher(); b != nil && sameCache(b.cache, cache) {
+					// Snapshot fetch-order ONCE for the whole push (mirrors the inline delete
+					// path below and cscInvalItem.fetchSnap): a per-key load inside enqueue
+					// would include a fetch reserved AFTER this push was observed but before
+					// the loop reached that key, so apply would not treat it as newer and
+					// could evict the fresh value / cancel its in-progress reservation.
+					fetchSnap := cscFetchSeq.Load()
+					for _, k := range payload {
+						var name string
+						switch v := k.(type) {
+						case string:
+							name = v
+						case []byte:
+							name = string(v)
+						default:
+							continue
+						}
+						b.enqueueAt(cscNamespacedKey(keyPrefix, name), fetchSnap)
+					}
+					return nil
+				}
+			}
+		}
+		var hot []cscRefreshTarget
+		lc, canRefresh := cache.(*LocalCache)
+		canRefresh = canRefresh && refresh != nil
+		// Snapshot the fetch-order sequence ONCE, at notification-observe time, and reuse
+		// it for every key in this push. A live per-key load would include a fetch
+		// reserved AFTER this push was observed but before the loop reached that key, so
+		// collectHotAndDelete would not treat it as newer and would evict the fresh value
+		// / cancel its in-progress reservation (mirrors cscInvalItem.fetchSnap, taken at
+		// enqueue; see cacheEntry.fetchSeq).
+		fetchSnap := cscFetchSeq.Load()
 		for _, k := range payload {
 			var name string
 			switch v := k.(type) {
@@ -106,26 +443,109 @@ func (h *invalidateHandler) HandlePushNotification(
 			default:
 				continue
 			}
-			cache.DeleteByRedisKey(cscNamespacedKey(keyPrefix, name))
+			nsKey := cscNamespacedKey(keyPrefix, name)
+			if !canRefresh {
+				if lc != nil {
+					// *LocalCache with refresh OFF: still honor the fetch-order guard
+					// (mirrors the batcher's !canRefresh branch), or a delayed invalidation
+					// would delete/cancel a miss reserved AFTER this push was observed
+					// (fetchSeq > fetchSnap) and wake coalesced waiters as duplicate misses.
+					// cscInvalNoHorizon = no refresh horizon; discard collected targets.
+					_ = lc.deleteByRedisKeyCollectingHot(nsKey, cscInvalNoHorizon, fetchSnap, nil)
+				} else {
+					// Non-*LocalCache: no per-entry fetch sequence to compare; plain delete.
+					cache.DeleteByRedisKey(nsKey)
+				}
+				continue
+			}
+			// Inline (window==0) path: observe and delete synchronously. An entry with a
+			// fetchSeq greater than the observe-time snapshot was reserved by a refetch
+			// issued after this push and is correctly kept (see cacheEntry.fetchSeq).
+			hot = lc.deleteByRedisKeyCollectingHot(nsKey, refresh.sinceToken.Load(), fetchSnap, hot[:0])
+			for i := range hot {
+				refresh.offer(hot[i])
+			}
 		}
 	}
 	return nil
 }
 
-func (h *invalidateHandler) release() {
-	h.mu.Lock()
-	if h.users > 0 {
-		h.releaseLocked()
+// fullFlush wipes the snapshotted cache for a FLUSHDB/FLUSHALL (nil-payload)
+// invalidation and supersedes the batcher's queued per-key deletes.
+//
+// drop() before Flush() bumps the batcher epoch, so anything already enqueued
+// (pre-flush, redundant by the flush) is skipped at apply, while an invalidation
+// racing in from another tracked connection AFTER this point carries the new epoch
+// and still applies — its post-flush delete must not be lost.
+//
+// Drop the batcher ONLY when it still belongs to the cache being flushed
+// (sameCache(h.cache, cache)). A same-cache batcher rebuild (setInvalBatchWindow /
+// set/clearRefreshQueue, all under h.mu.Lock) must have its FRESH batcher dropped, or
+// the new batcher's queued deletes survive the flush and evict post-flush
+// repopulations. But a last-user release + rebind to a DIFFERENT cache (A->B) between
+// the caller's entry snapshot and this RLock leaves h.batcher = B's while cache = A;
+// dropping B's batcher would bump B's epoch and skip B's queued deletes while only A is
+// flushed, so B would serve stale (#3989). The sameCache guard drops the batcher only
+// when the binding is unchanged. sameCache (not ==) also avoids a panic when the cache's
+// dynamic type is non-comparable; for such a type it returns false, so the batcher is
+// not dropped on flush — a bounded spurious miss, correct versus a crash.
+//
+// Flush the CACHE SNAPSHOT, not the live h.cache: a last-user releaseLocked can nil
+// h.cache, and a guarded `if h.cache != nil` would then silently skip the wipe. RLock
+// (not Lock) suffices — it blocks the write-locked rebuilds — and drop()/Flush() take
+// their own locks, not h.mu, so there is no lock-order cycle.
+func (h *invalidateHandler) fullFlush(cache Cache) {
+	h.mu.RLock()
+	defer h.mu.RUnlock()
+	if sameCache(h.cache, cache) && h.batcher != nil {
+		h.batcher.drop()
 	}
-	h.mu.Unlock()
+	cache.Flush()
 }
 
-func (h *invalidateHandler) releaseLocked() {
-	h.users--
-	if h.users == 0 {
-		h.cache = nil
-		h.keyPrefix = ""
+func (h *invalidateHandler) release() {
+	h.mu.Lock()
+	var b *cscInvalBatcher
+	if h.users > 0 {
+		b = h.releaseLocked()
 	}
+	h.mu.Unlock()
+	// Join OUTSIDE h.mu: the batcher's stop-drain must not run under the handler
+	// lock (it would stall a sibling on the hot path). release() is reached only from
+	// the drainer goroutine's exit (Close or self-disable), never the GC finalizer, so
+	// blocking here is fine and gives Close a synchronous, no-straggler teardown.
+	if b != nil {
+		b.join()
+	}
+}
+
+// releaseLocked drops one handler user and, on the last release, tears the binding
+// down. It DETACHES and signals the batcher but does not join it (see
+// detachBatcherLocked); it returns the detached batcher so release() can join
+// OUTSIDE h.mu. Returns nil while other users remain.
+func (h *invalidateHandler) releaseLocked() *cscInvalBatcher {
+	h.users--
+	if h.users != 0 {
+		return nil
+	}
+	h.cache = nil
+	h.keyPrefix = ""
+	// Stop the windowed batcher so its goroutine does not outlive the binding.
+	// Detached+signalled here; release() joins it outside the lock, so the last user
+	// closing gets a synchronous teardown (on a shared injected cache a late apply
+	// could otherwise evict an entry a successor client just repopulated). A later
+	// re-acquire starts a fresh batcher via ensureBatcher.
+	b := h.detachBatcherLocked()
+	// A fresh binding folds in its own window; do not inherit this one's.
+	h.invalBatchWindow = 0
+	h.invalBatchWindowSet = false
+	// Clear the refresh bindings with the binding itself: a client dropped
+	// without Close never runs clearRefreshQueue, and a successor reusing
+	// this handler must not inherit (or later restore) a dead queue whose
+	// consumer is gone — hot entries offered there would vanish silently.
+	h.refresh = nil
+	h.refreshStack = nil
+	return b
 }
 
 // sameCache compares Cache interface values without panicking when an
@@ -276,9 +696,17 @@ func (c *baseClient) attachSharedTrackingCSC(ctx context.Context, cache Cache) {
 		internal.Logger.Printf(ctx, "csc: failed to register invalidate handler: %v", err)
 		return
 	}
+	// Thread the invalidation-batch window from Options before any push can
+	// arrive, so the batcher (if enabled) sees the configured window on the very
+	// first invalidation rather than a zero default.
+	if ih := lookupInvalidateHandler(c.pushProcessor); ih != nil {
+		ih.setInvalBatchWindow(c.opt.ClientSideCacheInvalidationBatchWindow)
+	}
 	c.csc = cache
 	c.registerConnEvictHook(cache, reg)
 	c.startBackgroundDrainer()
+	c.startCSCRefresher()
+	c.startCSCMissCoalescer()
 }
 
 // cscHook returns the shared evict-on-remove hook, nil when CSC is off.
@@ -489,6 +917,12 @@ type cscFetchCapture struct {
 	raw     []byte
 	connID  uint64
 	initGen uint64
+
+	// key/token: the reservation a background refetch (refresh or miss-coalesce)
+	// must fulfil. Unused by the single-command path, which passes them to
+	// fulfillCached explicitly.
+	key   string
+	token uint64
 }
 
 // cscConnInitGen returns connID's CSC init generation, captured by _process at
@@ -516,32 +950,38 @@ func (c *baseClient) cscForgetConn(connID uint64) {
 // args, and pipelines — are also caught: the guard matches on the command's
 // leading args, not the typed method.
 var errClientTrackingWithCSC = errors.New(
-	"redis: CLIENT TRACKING is not allowed when client-side caching is enabled")
+	"redis: CLIENT TRACKING is not allowed when client-side caching is enabled",
+)
 
 // errSelectWithCSC rejects runtime SELECT on clients with built-in CSC. Cache
 // keys use Options.DB, while SELECT mutates only the chosen pool connection.
 var errSelectWithCSC = errors.New(
-	"redis: SELECT is not allowed when client-side caching is enabled")
+	"redis: SELECT is not allowed when client-side caching is enabled",
+)
 
 // errAuthWithCSC rejects runtime authentication because it can change one
 // connection's ACL identity without changing the client's fixed cache namespace.
 var errAuthWithCSC = errors.New(
-	"redis: AUTH is not allowed when client-side caching is enabled")
+	"redis: AUTH is not allowed when client-side caching is enabled",
+)
 
 // errHelloWithCSC rejects HELLO with arguments because it can switch a tracked
 // connection out of RESP3 (and can also change authentication).
 var errHelloWithCSC = errors.New(
-	"redis: HELLO with arguments is not allowed when client-side caching is enabled")
+	"redis: HELLO with arguments is not allowed when client-side caching is enabled",
+)
 
 // errResetWithCSC rejects RESET because it disables tracking and switches the
 // connection to RESP2.
 var errResetWithCSC = errors.New(
-	"redis: RESET is not allowed when client-side caching is enabled")
+	"redis: RESET is not allowed when client-side caching is enabled",
+)
 
 // errSubscribeWithCSC rejects raw subscriptions on the ordinary pool. The
 // typed Subscribe methods use dedicated PubSub connections and remain allowed.
 var errSubscribeWithCSC = errors.New(
-	"redis: SUBSCRIBE is not allowed on pooled connections when client-side caching is enabled")
+	"redis: SUBSCRIBE is not allowed on pooled connections when client-side caching is enabled",
+)
 
 // cscCommandError rejects commands that can make a pooled connection's state
 // diverge from the assumptions used by CSC.
@@ -576,10 +1016,34 @@ type cscDrainHandle struct {
 	done              chan struct{}
 	stopOnce          sync.Once
 	teardownOnce      sync.Once
+	workersStopOnce   sync.Once
 	handlerCloseOnce  sync.Once
 	closeOnce         sync.Once
 	closeErr          error
 	invalidateHandler *invalidateHandler
+
+	// workersMu/workersTornDown close the race between
+	// stopCSCRefresherAndCoalescer's one-time teardown and
+	// startCSCRefresher/startCSCMissCoalescer's publish step (a bot review
+	// flagged this: async conn init can call disableCSCServing, which the
+	// drainer goroutine observes on ITS OWN tick and reacts to with the same
+	// teardown Close uses — and that tick can in principle land concurrently
+	// with construction). workersStopOnce alone is not enough: it only
+	// guarantees the teardown body runs once, not that it runs AFTER a
+	// worker exists to be stopped. A start function checks workersTornDown
+	// under this lock immediately before publishing its queue/handle; if
+	// teardown already ran, it declines to start instead of creating a
+	// goroutine workersStopOnce can never be Do'd again to stop. See both
+	// start functions and stopCSCRefresherAndCoalescer's body.
+	//
+	// startBackgroundDrainer (attachSharedTrackingCSC's first call, before
+	// either start function) needs no matching guard: it is what allocates
+	// this struct and assigns it to c.cscDrainHandle, so no teardown path can
+	// observe a non-nil handle — let alone race this lock — until after it
+	// returns. Every teardown entry point (Close, the drainer's own
+	// self-disable) reads c.cscDrainHandle first and no-ops on nil.
+	workersMu       sync.Mutex
+	workersTornDown bool
 }
 
 // signalStop closes stop at most once (so Close and the AddCleanup safety net
@@ -595,20 +1059,37 @@ type cscHandlerClient struct {
 	*baseClient
 }
 
+// closeCanonical closes through the canonical *Client wrapper while it is
+// still alive — Client.Close also stops the cached autopipeliners, which
+// baseClient.Close does not, so closing only the baseClient would leave flush
+// goroutines running against closed pools. Falls back to baseClient.Close if
+// the wrapper was already collected (weak ref: see baseClient.cscClientWeak).
+func (c cscHandlerClient) closeCanonical() error {
+	if cl := c.cscClientWeak.Value(); cl != nil {
+		return cl.Close()
+	}
+	return c.baseClient.Close()
+}
+
 func (c cscHandlerClient) Close() error {
 	h := c.cscDrainHandle
 	if h == nil {
-		return c.baseClient.Close()
+		return c.closeCanonical()
 	}
 	h.handlerCloseOnce.Do(func() {
-		// Close has logically started: stop cache hits immediately and let the
-		// drainer exit as soon as this handler returns.
-		if c.cscActive != nil {
-			c.cscActive.Store(false)
-		}
-		h.signalStop()
+		// Do NOT deactivate serving or signal the drainer stop here. This runs on
+		// the drainer goroutine (a custom push handler called Close), so the
+		// canonical close MUST be async — but let IT drive the teardown so
+		// stopBackgroundDrainer stops the coalescer and refresher in the canonical
+		// order with cscActive STILL TRUE, and the refresher's stop-drain flush
+		// re-fetches the in-window invalidated keys. Preemptively storing
+		// cscActive=false here made that flush a no-op (refreshInvalidatedBatch
+		// bails on the inactive check), dropping the last collection window — the
+		// same class the GC finalizer path was reordered to fix. The async close
+		// reaches deactivate itself, after the flush; the drainer keeps running the
+		// harmless idle drain until stopBackgroundDrainer signals its stop.
 		go func() {
-			if err := c.baseClient.Close(); err != nil {
+			if err := c.closeCanonical(); err != nil {
 				internal.Logger.Printf(context.Background(), "csc: deferred client close failed: %v", err)
 			}
 		}()
@@ -669,7 +1150,23 @@ func (c *baseClient) startBackgroundDrainer() {
 	_, builtinProc := c.pushProcessor.(*push.Processor)
 	go func() {
 		defer func() {
-			active.Store(false)
+			// Self-disable exits (custom-processor damping, or a RESP3/tracking
+			// downgrade flipping cscActive that a tick then observes) run the SAME
+			// refresher+coalescer teardown as the clean Close path, so a client that
+			// KEEPS RUNNING after turning CSC off does not leak those goroutines for
+			// its life — the refresher parked on its window, and a coalescer session
+			// still holding a pool connection (F1). stopCSCRefresherAndCoalescer is
+			// idempotent with the Close path (workersStopOnce), and it joins the
+			// refresher and coalescer goroutines — never THIS drainer — so running it
+			// here cannot self-block. It also deactivates serving and UNBINDS this
+			// client's refresh queue (stopCSCRefresher -> clearRefreshQueue), which on
+			// a shared cache+processor restores a surviving sibling's binding, so this
+			// defer no longer needs its own active.Store(false)/clearRefreshQueue.
+			// Both self-disable paths reach here with cscActive already false
+			// (disableCSCServing set it; the damping branch above deactivates before
+			// returning), so the helper's refresher flush no-ops here — the ordering is
+			// kept only because it is free and uniform with the Close path.
+			c.stopCSCRefresherAndCoalescer()
 			if c.cscPoolHook != nil {
 				if reg, ok := c.connPool.(poolHookSupport); ok {
 					reg.RemovePoolHook(c.cscPoolHook)
@@ -719,6 +1216,14 @@ func (c *baseClient) startBackgroundDrainer() {
 						"csc: disabling client-side caching: the custom push notification processor failed %d consecutive drains "+
 							"(each failure removes a connection because the reader may be mid-frame); "+
 							"caching cannot be kept fresh safely with this processor", consecFatal)
+					// Deactivate BEFORE the defer's teardown so nothing else (ordinary reads
+					// via processCached, a normal in-flight flush's refreshInvalidatedBatch
+					// bail) keeps treating a persistently-broken custom processor as still
+					// serving. The refresher's OWN stop-drain flush no longer depends on this
+					// timing either way: it now unconditionally skips its refetch when
+					// stopping (see runCSCRefresher's h.stop case), so the exit stays prompt
+					// regardless. The defer still stops and joins both workers (F1).
+					active.Store(false)
 					return
 				}
 			}
@@ -752,10 +1257,12 @@ func (c *baseClient) stopBackgroundDrainer() {
 		return
 	}
 	h.teardownOnce.Do(func() {
-		// Stop serving cache hits on any clone before the drainer is gone.
-		if c.cscActive != nil {
-			c.cscActive.Store(false)
-		}
+		// Stop the coalescer and refresher in the shared canonical order (coalescer
+		// connection released first, then the refresher's final flush runs while
+		// serving is still active, then deactivate). Idempotent with the drainer's
+		// own self-disable defer via workersStopOnce. The pool is still open here
+		// (closeResources tears it down after this), so the final refetch can run.
+		c.stopCSCRefresherAndCoalescer()
 		h.signalStop()
 		<-h.done
 		// The drainer's exit defer revoked and evicted this pool's coverage
@@ -766,12 +1273,103 @@ func (c *baseClient) stopBackgroundDrainer() {
 	})
 }
 
+// stopCSCRefresherAndCoalescer tears down the reader-miss coalescer and the
+// refresh-on-invalidate goroutine in the one order that satisfies every teardown
+// constraint, exactly once. It is shared by the clean Close path
+// (stopBackgroundDrainer) and the SELF-DISABLE exit (the drainer goroutine's
+// defer), so a client that turns CSC off itself stops these goroutines instead of
+// leaking them for its lifetime (F1). workersStopOnce makes the two callers
+// idempotent and race-free — whichever reaches it first runs the body, the other
+// is a no-op — so cscRefreshHandle/cscMissCoalescer are never torn down twice
+// concurrently. Neither caller is the refresher or a coalescer goroutine, so the
+// joins below never self-block.
+//
+// Order:
+//  1. Stop the coalescer FIRST, while cscActive is still TRUE. stopCSCMissCoalescer
+//     swaps its pointer to nil (new misses take the ordinary pooled path), signals
+//     its sessions, and JOINS them — RELEASING the held pool connection. On a small
+//     pool (PoolSize 1) under continuous miss traffic the coalescer can otherwise
+//     hold the only connection, so the refresher's final flush in step 2 would wait
+//     its full per-chunk deadline for every chunk (~160s Close stall) and lose the
+//     warming (F3). Stopping the coalescer with serving STILL ACTIVE is safe: every
+//     fetch stop path returns errCSCRetryUncached and every session stop path settles
+//     the same (settleErr tags all other failures cscSessionError), so processCached
+//     re-runs each read on the still-open pool — a caller never sees a raw ErrClosed.
+//     A teardown-window miss then takes the ordinary pooled path and contends with
+//     the step-2 flush for the connection, which is strictly better than the coalescer
+//     holding it (bounded by one RTT per miss, not the session's idle/recycle hold).
+//     This is why the earlier "deactivate before stopping the coalescer" step is gone:
+//     the errCSCRetryUncached backstop it relied on already makes deactivate-first
+//     unnecessary, and it conflicted with releasing the connection before the flush.
+//  2. Stop the refresher. Its stop-drain flush no longer re-fetches the in-window
+//     invalidated keys over the network at all (runCSCRefresher's h.stop case):
+//     the drainer defer's invalidateAllCoverage revokes this pool's coverage and
+//     evicts every entry attributed to this client's refresh connection right
+//     after (pre-existing; see TestStopBackgroundDrainerEvictsSharedCacheCoverage),
+//     so a refetch here would only be discarded a moment later — it used to run
+//     anyway and could cost thousands of pointless round trips against a healthy
+//     server (codex #3989 P1). The targets are abandoned instead: counted into
+//     RefreshFailed, left evicted for a reader to repopulate. Because the flush no
+//     longer touches the network, the coalescer-first order in step 1 (freeing the
+//     pool connection) is no longer LOAD-BEARING for this step, but stays: it is
+//     still the one canonical order for Close and the self-disable defer, and
+//     still never serves a stale value.
+//  3. Deactivate serving.
+//
+// STARTUP WINDOW: attachSharedTrackingCSC launches the drainer before it assigns
+// cscRefreshHandle/cscMissCoalescer, so this can in principle run (via the
+// drainer's self-disable defer, reacting to a HELLO/tracking downgrade an
+// async conn init observed) WHILE startCSCRefresher/startCSCMissCoalescer are
+// still constructing. workersStopOnce alone only guarantees body runs once —
+// it does not guarantee a worker exists yet to be stopped by it. The
+// workersMu/workersTornDown flag below (set FIRST, before either stop call)
+// closes that gap: a start function checks the flag under the same lock
+// immediately before publishing, so either it sees teardown hasn't happened
+// yet and publishes (and this body's later stop call correctly joins it), or
+// it sees the flag already set and declines to start at all — never leaving
+// a worker workersStopOnce can no longer reach.
+func (c *baseClient) stopCSCRefresherAndCoalescer() {
+	// Owner-only. Every worker (drainer, refresher, coalescer) is created together
+	// under a cscDrainHandle; a caller without one is a clone (clone() copies neither
+	// the handle nor the workers, see redis.go) or a client where CSC never attached,
+	// so it has nothing of its own to stop. Running body() here would call
+	// stopCSCRefresher, whose read-then-nil of cscRefreshHandle is UNSYNCHRONIZED
+	// (single-owner by design), racing the real owner's teardown.
+	h := c.cscDrainHandle
+	if h == nil {
+		return
+	}
+	body := func() {
+		h.workersMu.Lock()
+		h.workersTornDown = true
+		h.workersMu.Unlock()
+		c.stopCSCMissCoalescer()
+		c.stopCSCRefresher()
+		if c.cscActive != nil {
+			c.cscActive.Store(false)
+		}
+	}
+	h.workersStopOnce.Do(body)
+}
+
 // applyCachedReply populates cmd from a previously captured raw RESP reply by
 // replaying it through the command's own readReply.
 func applyCachedReply(cmd Cmder, raw []byte) error {
 	return cmd.readReply(proto.NewReaderSize(bytes.NewReader(raw), len(raw)+1))
 }
 
+// classifyCachedReply reports the same error applyCachedReply would, without a
+// caller command to populate. The miss coalescer uses it on the abandoned path
+// (the caller returned and owns its Cmder again) to decide cache-vs-cancel: a
+// value or Nil is cacheable, a top-level RESP error is not. It reads the frame
+// generically, so it can only diverge from a concrete cmd's readReply on a
+// well-formed reply of an unexpected shape — which the next reader re-parses and
+// drops (see processCached), so a rare mis-cache self-heals.
+func classifyCachedReply(raw []byte) error {
+	_, err := proto.NewReaderSize(bytes.NewReader(raw), len(raw)+1).ReadReply()
+	return err
+}
+
 // isCacheableReplyResult reports whether a fully read Redis reply can be
 // cached. redis.Nil is a normal negative lookup, not a transport/protocol
 // failure; tracking will invalidate it if the key is later created.
@@ -846,6 +1444,15 @@ func (c *baseClient) processCached(ctx context.Context, cmd Cmder, state *proces
 		c.csc.DeleteByCacheKey(key)
 	}
 
+	// Demand trigger: a miss for a key still in the refresher's collection window
+	// flushes that window now (no-op when refresh/coalescing is off). Scope: this
+	// signals THIS client's own refresh queue. With a shared cache+processor across
+	// clients the active refresh binding is the last-attached client's queue, so a
+	// miss on one client does not early-flush a sibling's window — the sibling's
+	// window timer is the backstop. Correctness is unaffected (the key still
+	// refreshes); only the early-flush latency is client-local.
+	c.cscRefreshQueue.signalDemand(key)
+
 	token, shouldFetch := c.csc.Reserve(key, nsRedisKeys)
 	if !shouldFetch {
 		// Another goroutine is fetching; Get below waits until it completes.
@@ -859,10 +1466,76 @@ func (c *baseClient) processCached(ctx context.Context, cmd Cmder, state *proces
 			c.csc.DeleteByCacheKey(key)
 		}
 		// Original fetcher cancelled or its value was invalidated; try to take
-		// over so later waiters still benefit from the cache.
+		// over so later waiters still benefit from the cache. This is the 2x-RTT
+		// path under churn: we waited a round trip and still must fetch ourselves.
 		token, shouldFetch = c.csc.Reserve(key, nsRedisKeys)
 	}
 
+	// Reader-miss coalescing: hand the reserved miss to the batcher (no-op when
+	// off). Load the pointer ONCE: a concurrent Close swaps it to nil, and a
+	// second load after the nil-check would call fetch on a nil receiver.
+	if mc := c.cscMissCoalescer.Load(); shouldFetch && mc != nil {
+		served, err := mc.fetch(ctx, cmd, key, token)
+		// Re-run on the normal path when the coalescer bailed (errCSCRetryUncached)
+		// or hit a session or transport failure (tagged cscSessionError by
+		// settleErr). Then a coalesced miss gets the same MaxRetries and backoff as
+		// any other command, instead of a raw io.EOF or ErrClosed to the caller (Ofek
+		// review #3989). The coalescer always cancels the reservation on failure, so
+		// the re-run starts clean, and processWithRetry handles a cancelled caller
+		// context itself. A command result is not tagged (for example redis.Nil,
+		// WRONGTYPE, or a retryable reply such as LOADING). It is the command's
+		// answer, already applied to cmd, so it is returned as-is. This keeps the
+		// coalesced path's documented tradeoff: no per-command retry for reply-level
+		// errors.
+		var sessErr cscSessionError
+		if err == errCSCRetryUncached || errors.As(err, &sessErr) {
+			// The coalescer bailed and cancelled its reservation. Re-reserve and FALL
+			// THROUGH to the capture path below so a successful retry repopulates the
+			// cache (a nil-capture processWithRetry returned the value but stored
+			// nothing, so connection blips and wire-budget sheds left the key uncached
+			// and later readers missed).
+			//
+			// TODO(convergence): the re-run starts its retry metrics at attempt 0, so
+			// the coalesced attempt is missing from retry_attempts. Fix by starting the
+			// re-run at attempt 1 via the explicit-start plumbing (processStartingAt)
+			// once the full-duplex autopipeline branch, which introduces it, is on this
+			// branch. See #3989 review thread on coalescer attempt counting.
+			token, shouldFetch = c.csc.Reserve(key, nsRedisKeys)
+			if !shouldFetch {
+				// Another waiter won the re-Reserve race and is fetching. WAIT on it
+				// (Get parks on the in-progress entry) instead of falling through to an
+				// independent pooled request: a session failure wakes every same-key
+				// waiter at once, and each running its own processWithRetry would turn
+				// one hot key into a pool stampede mid-recovery — defeating the
+				// coalescing this path exists for. Same shape (and 2x-RTT churn
+				// tradeoff) as the first-Reserve loser path above.
+				if data, ok := c.csc.Get(ctx, key); ok {
+					if err := ctx.Err(); err != nil {
+						return err
+					}
+					if err := applyCachedReply(cmd, data); isCacheableReplyResult(err) {
+						return err
+					}
+					c.csc.DeleteByCacheKey(key)
+				}
+				// The new owner cancelled or its value was invalidated; try to take
+				// over. A second loss falls through to the plain pooled path — after
+				// two waits, forward progress outranks another round of parking.
+				token, shouldFetch = c.csc.Reserve(key, nsRedisKeys)
+			}
+		} else {
+			// Native-recorder attribution: the coalesced fetch contacted Redis on
+			// the session's held connection — report it as one attempt on that conn
+			// so operation-duration metrics carry a real server.address instead of
+			// zero attempts and a nil connection.
+			if state != nil && served != nil {
+				state.attempts++
+				state.lastConn = served
+			}
+			return err
+		}
+	}
+
 	var fc cscFetchCapture
 	var capture *cscFetchCapture
 	if shouldFetch {

csc_inval_batch.go

@@ -0,0 +1,577 @@
+package redis
+
+import (
+	"context"
+	"sync"
+	"sync/atomic"
+	"time"
+
+	"github.com/redis/go-redis/v9/internal"
+)
+
+// Windowed background invalidation batcher.
+//
+// Normally the invalidate handler deletes cache entries INLINE, on whatever
+// goroutine read the RESP3 "invalidate" push — which, for the coalescer's
+// reply reader, means invalidation work steals time from reading miss replies
+// and inflates the low-concurrency churn p99 tail.
+//
+// When Options.ClientSideCacheInvalidationBatchWindow>0 the handler instead
+// ENQUEUES invalidated keys (cheap, off the read path) and a single background
+// goroutine applies the deletes in batches once per window. Deferring an
+// invalidation's application by <= window means a reader may see the
+// pre-invalidation value for <= window, which is exactly what MaxStaleness=window
+// already licenses; set the window <= MaxStaleness to stay within contract (a
+// nonzero window with MaxStaleness=0 is an explicit strictness relaxation).
+
+const cscInvalBatchMax = 4096 // size-cap flush regardless of the timer
+
+// cscInvalSpillMax hard-caps the overflow spill buffer. The spill is normally
+// self-limiting (duplicate keys collapse in the worker's dedup), but the set of
+// keys a server can invalidate is NOT bounded by the local cache size: Redis
+// keeps CLIENT TRACKING for every key a connection read, even after the local
+// LRU evicted it (no per-key untrack), so a workload that reads far more than
+// MaxEntries and then invalidates them can grow the spill without bound. At this
+// many buffered items the batcher stops spilling and schedules ONE full cache
+// Flush instead — a correctness-preserving, O(1)-memory fallback (a flush can
+// never serve stale) that also keeps the reader unblocked.
+const cscInvalSpillMax = 1 << 16 // 65536
+
+// cscInvalSpillMaxBytes hard-caps the RETAINED KEY BYTES in the overflow spill.
+// The item cap alone bounds the spill by COUNT, not memory: the spill holds
+// namespaced key STRINGS (no values), but a burst of distinct large keys can
+// retain GBs while the item count stays far below cscInvalSpillMax. When the
+// retained bytes would cross this bound the batcher takes the SAME full-Flush
+// fallback the item cap triggers (a single oversized key crosses it alone). 8
+// MiB of key strings is already tens of thousands of large keys — in the spirit
+// of the refresh queue's key-byte cap (cscRefreshTargetMaxBytes).
+const cscInvalSpillMaxBytes = 8 << 20 // 8 MiB
+
+// cscInvalBatchMaxBytes size-caps the worker's PENDING batch by retained KEY
+// bytes, the byte twin of cscInvalBatchMax. ch and spill bound the bytes WAITING
+// to be consumed, but the worker then holds each key in `pending` until the window
+// or the item cap flushes — so a stream of large keys (each under the admission
+// limit) could retain GBs in pending across one window while the item count stays
+// far below cscInvalBatchMax. At this many pending key bytes the worker flushes the
+// batch early (a normal apply, not a full-cache Flush), bounding pending memory. 8
+// MiB mirrors the spill/refresh key-byte caps.
+const cscInvalBatchMaxBytes = 8 << 20 // 8 MiB
+
+// cscInvalItem is one queued invalidation, tagged with the batcher epoch it
+// was enqueued under so a full-cache flush can supersede it (see drop()).
+type cscInvalItem struct {
+	key   string
+	epoch uint64
+	// sinceToken is the refresh "recently-read" horizon snapshotted at ENQUEUE,
+	// used by apply for the hot-entry check instead of a live load. A batch window
+	// >= the recency tick (200ms) would otherwise let the horizon advance during
+	// the wait, so a key that was hot when the invalidation arrived fails the
+	// check by apply time and silently degrades to plain eviction. Free size-wise:
+	// it fits the 8-byte alignment padding the struct already carries.
+	//
+	// cscInvalNoHorizon (-1) when refresh was OFF at enqueue. apply re-reads the
+	// refresh binding at APPLY time, and a refresh-enabled client attaching in
+	// between (setRefreshQueue repoints the batcher, then stop-drains it) would
+	// otherwise see horizon 0 and treat EVERY valid entry as hot — the cold-key
+	// refresh loop the horizon exists to prevent. With the sentinel, apply falls
+	// back to the live horizon (seeded at that client's start), which can only
+	// under-refresh, never chase cold keys. LRUClock is a monotonic sequence >= 0,
+	// so -1 cannot collide with a real horizon.
+	sinceToken int64
+	// fetchSnap is cscFetchSeq observed at ENQUEUE (invalidation OBSERVE time).
+	// apply passes it to deleteByRedisKeyCollectingHot, which keeps any Valid entry
+	// whose fetch was issued after it (entry.fetchSeq > fetchSnap) — that value was
+	// refetched after the write, so evicting it would be a spurious miss and would
+	// undo a fresh refresh (see cacheEntry.fetchSeq). Snapshotting at enqueue, not a
+	// live load at apply, is what makes a queued duplicate that straddled a refetch
+	// harmless.
+	fetchSnap uint64
+}
+
+// cscInvalNoHorizon marks an invalidation enqueued while no refresh binding
+// existed (see cscInvalItem.sinceToken). It doubles as the LIVE horizon a
+// cscRefreshQueue holds when Options.ClientSideCacheRefreshRecencyWindow is
+// unset (refresh-everything mode, see startCSCRefresher): -1 is less than any
+// real lastAccessNs token, so deleteByRedisKeyCollectingHot's `> sinceToken`
+// check marks every Valid entry hot regardless of recency, in either case.
+// The two meanings never conflict because they resolve to the same effect.
+const cscInvalNoHorizon = -1
+
+type cscInvalBatcher struct {
+	window time.Duration
+	// batchMax is the size-cap flush threshold; 0 means cscInvalBatchMax. Per-batcher
+	// (not a mutable global) so a test can lower it without racing other batchers'
+	// worker goroutines reading it under -race; production leaves it 0.
+	batchMax int
+	// batchMaxBytes size-caps the pending batch by retained KEY bytes; 0 means
+	// cscInvalBatchMaxBytes. Per-batcher like batchMax so a test can lower it
+	// race-free; production leaves it 0.
+	batchMaxBytes int
+	// cache is snapshotted at creation (batcher lifetime is inside the binding
+	// lifetime): the release-time stop-drain must still apply queued deletes AFTER
+	// releaseLocked nils h.cache, or a successor reusing the shared cache serves
+	// stale until TTL/MaxStaleness.
+	cache Cache
+	// refresh is the active refresh binding. Atomic (not a plain snapshot) because
+	// set/clearRefreshQueue can REPOINT it — under h.mu, before stopping the
+	// batcher — at the surviving/new binding, so the stop-drain feeds hot keys to
+	// the live refresher instead of the dead one the batcher was created with. Read
+	// by both the producer (enqueue) and the worker (apply).
+	refresh atomic.Pointer[cscRefreshQueue]
+
+	ch       chan cscInvalItem
+	stopCh   chan struct{}
+	stopOnce sync.Once
+	// spill absorbs invalidations that arrive while ch is full, so an overflow
+	// never applies a delete inline on the producer — critically the coalescer's
+	// reply reader, where inline apply (lock-held cache work) stalls miss replies
+	// (head-of-line). The worker drains spill through the SAME seen/pending dedup
+	// pipeline as ch, so a burst of duplicate keys collapses to one delete per
+	// key. The worker normally catches up, but the spill is hard-capped at
+	// cscInvalSpillMax: the invalidatable keyset is NOT bounded by cache capacity
+	// (the server tracks keys past local LRU eviction), so at the cap the batcher
+	// schedules one full Flush (flushReq) instead of growing spill. wake nudges
+	// the worker to drain promptly (cap 1, coalescing: one wake drains the whole
+	// slice).
+	spillMu sync.Mutex
+	spill   []cscInvalItem
+	// spillBytes tracks the retained KEY bytes currently held in spill, guarded by
+	// spillMu alongside spill itself. Bounds spill memory by bytes as well as by
+	// item count (see cscInvalSpillMaxBytes); reset with spill on drain/flush.
+	spillBytes int
+	// chBytes tracks the retained KEY bytes currently sitting in ch. ch is bounded
+	// by item COUNT (its slot count) but not memory, so a burst of large keys — or
+	// one key larger than the whole byte cap — would retain unbounded bytes before
+	// the spill path (and its byte cap) ever engages. enqueue RESERVES atomically
+	// before the send (mirroring reserveWireBytes): it charges keyLen up front and
+	// rolls back if the total crosses cscInvalSpillMaxBytes, so concurrent producers
+	// can no longer each read a stale below-cap Load and all overshoot. A send that
+	// then finds ch full rolls the reservation back too (those bytes are tracked on
+	// the spill path via spillBytes); the worker subtracts on consume. Written by
+	// producers (reserve/rollback) and the worker (consume); read by the gate.
+	chBytes atomic.Int64
+	wake    chan struct{}
+	// flushReq: spill hit cscInvalSpillMax, so the worker must drop() + full-Flush
+	// the cache instead of applying a huge backlog. Set by enqueue, consumed by the
+	// worker (CAS to false).
+	flushReq atomic.Bool
+	// spilled counts overflow events (ch full at send time). Internal; read by
+	// tests and available for a future stat, so the feature's degradation under
+	// invalidation bursts is observable instead of silent.
+	spilled atomic.Uint64
+	// epoch versions enqueued invalidations against full-cache flushes: drop()
+	// bumps it, and apply skips items from an older epoch. applyMu serializes
+	// the two: without it, apply could snapshot the epoch, a concurrent
+	// drop()+Flush() land mid-loop, and the remaining stale items would still
+	// be applied AFTER the flush — evicting post-flush repopulations, the exact
+	// case the epoch exists to prevent. drop() holding applyMu means any
+	// in-flight batch finishes BEFORE the flush (harmless: the flush wipes
+	// everything anyway), and every later apply sees the new epoch.
+	epoch   atomic.Uint64
+	applyMu sync.Mutex
+	// stopMu/stopped interlock enqueue against stop: a handler can hold a stale
+	// batcher pointer across a rebuild. enqueue sends under the read side, so a
+	// sent key lands BEFORE stop closes stopCh (the stop-drain sees it); once
+	// stopped, enqueue applies inline — no delete is ever parked in a channel
+	// nothing drains.
+	stopMu  sync.RWMutex
+	stopped bool
+	// done is closed when run() exits (after its stop-drain and final flush);
+	// join() waits on it so stop+join is a synchronous teardown. Nil only in
+	// test-constructed literals whose worker never starts.
+	done chan struct{}
+}
+
+// stop signals run() to flush and exit; idempotent, never touches h.mu, does
+// not wait — safe under the handler lock. Lock order h.mu -> stopMu, never
+// reversed (enqueue's stopMu section is only the flag check and the send).
+func (b *cscInvalBatcher) stop() {
+	b.stopOnce.Do(func() {
+		b.stopMu.Lock()
+		b.stopped = true
+		b.stopMu.Unlock()
+		close(b.stopCh)
+	})
+}
+
+// join blocks until run() has exited — i.e. the stop-drain and final flush have
+// fully applied. Callers pair it with stop() so a teardown or rebuild is
+// SYNCHRONOUS: without the join, the old worker's drain ran after Close (or
+// after a rebuild installed a new batcher with a new window contract), leaving
+// a straggler goroutine past Close and letting a late apply evict entries a
+// successor just repopulated. Safe under h.mu: run() never takes handler locks
+// (its drain touches only the cache shards, applyMu, and non-blocking refresh
+// offers). No-op for a batcher whose worker was never started (test literals).
+func (b *cscInvalBatcher) join() {
+	if b.done == nil {
+		return
+	}
+	<-b.done
+}
+
+// drop marks everything enqueued so far as superseded by a full cache Flush:
+// bumping the epoch makes apply skip stale-epoch items — both queued and in an
+// in-progress batch — without draining anything. The old drain-based drop
+// could discard a NEWER per-key invalidation racing in from another tracked
+// connection after the flush, losing its delete and leaving a repopulated
+// entry stale; with epochs that item carries the new epoch and survives.
+// Callers bump BEFORE flushing the cache, so a stale epoch always means
+// "enqueued before the flush", where the delete is redundant by the flush.
+func (b *cscInvalBatcher) drop() {
+	// Serialize with apply (see applyMu): after drop returns, no stale-epoch
+	// delete can run — the caller may flush the cache immediately.
+	b.applyMu.Lock()
+	b.epoch.Add(1)
+	b.applyMu.Unlock()
+}
+
+// enqueueAt hands a namespaced key to the batcher without blocking the caller and
+// without ever applying a delete inline on the producer (see the spill field):
+// on a full ch it appends to spill and nudges the worker; only once the batcher
+// is stopped (no worker left to drain, see stopMu) does it apply inline so an
+// invalidation is never dropped.
+//
+// fetchSnap is cscFetchSeq observed at the notification OBSERVE time. The push
+// handler batches MANY keys per notification, so it snapshots ONCE and passes the
+// same value for every key in the push (a per-key load would let a fetch reserved
+// after the push was observed slip in as "not newer" for a later key — the same
+// drift the inline delete path already avoids). See the test-only enqueue helper
+// for the single-key convenience form.
+func (b *cscInvalBatcher) enqueueAt(nsKey string, fetchSnap uint64) {
+	it := cscInvalItem{
+		key:        nsKey,
+		epoch:      b.epoch.Load(),
+		sinceToken: cscInvalNoHorizon,
+		// fetchSnap is cscFetchSeq observed at the notification OBSERVE time (shared by
+		// every key in one push) so a Valid entry refetched after this invalidation is
+		// kept at apply, and a queued duplicate that straddles that refetch cannot undo
+		// it (see cscInvalItem.fetchSnap).
+		fetchSnap: fetchSnap,
+	}
+	// Snapshot the refresh recency horizon NOW (enqueue time), so a batch-window
+	// delay can't advance it out from under apply's hot-entry check (see the field
+	// doc). Cheap: one atomic load, no lock. Stays cscInvalNoHorizon when refresh
+	// is off, so a binding that appears before apply cannot read it as horizon 0.
+	if r := b.refresh.Load(); r != nil {
+		it.sinceToken = r.sinceToken.Load()
+	}
+	keyLen := int64(len(it.key))
+	b.stopMu.RLock()
+	if b.stopped {
+		b.stopMu.RUnlock()
+		b.apply([]cscInvalItem{it})
+		return
+	}
+	// Pre-admission byte gate: ch is bounded by item COUNT, not memory. When this
+	// key alone exceeds the byte cap, or the bytes already retained in ch plus this
+	// key would cross it, skip ch entirely and take the full-Flush fallback (below)
+	// — otherwise a distinct-large-key flood retains unbounded bytes in ch before
+	// the spill path ever engages. RESERVE the bytes atomically before the send
+	// (mirroring reserveWireBytes): a stale Load-then-Add let concurrent producers
+	// each read a below-cap value and all overshoot the cap, so charge keyLen up
+	// front and roll back when it crosses.
+	overBytes := keyLen > cscInvalSpillMaxBytes
+	if !overBytes {
+		if b.chBytes.Add(keyLen) > cscInvalSpillMaxBytes {
+			b.chBytes.Add(-keyLen) // over cap: divert to the full-Flush fallback below
+			overBytes = true
+		} else {
+			select {
+			case b.ch <- it:
+				b.stopMu.RUnlock()
+				return
+			default:
+				// ch full: release the reservation (the spill path tracks these bytes
+				// via spillBytes) and fall through to the spill path below.
+				b.chBytes.Add(-keyLen)
+			}
+		}
+	}
+	{
+		// Not admitted to ch (full, or the byte gate diverted here): park on spill
+		// (off the producer's read path) for the worker to drain, instead of applying
+		// inline. Correctness is unchanged — the item carries its enqueue-time epoch
+		// and the worker applies it under applyMu with the same epoch check as the ch
+		// path.
+		//
+		// At either hard cap, stop growing spill: drop this item, clear the backlog,
+		// and ask the worker to full-Flush the cache. Safe because a Flush drops
+		// everything (nothing stale can survive), and correct because the capped case
+		// is a pathological invalidation flood where the cache is churning wholesale
+		// anyway. Needed because the invalidatable keyset is not bounded by cache size
+		// (the server tracks keys past local LRU eviction).
+		b.spillMu.Lock()
+		// Trip on ANY bound: the byte gate above (overBytes), the spill item count, or
+		// the spill retained-key bytes. Bytes matter because the invalidatable keyset
+		// is not bounded by cache size and keys can be large, so a distinct-large-key
+		// flood grows memory long before the item cap. A single oversized key trips it
+		// alone (overBytes).
+		if overBytes || len(b.spill) >= cscInvalSpillMax ||
+			b.spillBytes+len(it.key) > cscInvalSpillMaxBytes {
+			// The cleared backlog plus this item are superseded by the coming
+			// full-Flush and never reach deleteByRedisKeyCollectingHot. They were
+			// already counted as incoming invalidations at the handler; they simply
+			// won't become deletions — the Flush supersedes them wholesale.
+			b.spill = b.spill[:0]
+			b.spillBytes = 0
+			// Set flushReq BEFORE releasing stopMu. stop() takes stopMu.Lock, so it
+			// cannot close stopCh until this store is visible; the worker's stop-drain
+			// then sees the request via fullFlushIfRequested instead of exiting and
+			// losing the flush (which would leave the flood's entries stale). Setting
+			// it after RUnlock would race stop() and drop the flush.
+			b.flushReq.Store(true)
+		} else {
+			b.spill = append(b.spill, it)
+			b.spillBytes += len(it.key)
+		}
+		b.spillMu.Unlock()
+		b.stopMu.RUnlock()
+		b.spilled.Add(1)
+		select {
+		case b.wake <- struct{}{}:
+		default:
+		}
+	}
+}
+
+// takeSpill swaps out the accumulated spill for the worker to drain. Returns nil
+// when empty. The swapped slice is owned by the caller; b.spill starts fresh so
+// concurrent enqueues never race the drain.
+func (b *cscInvalBatcher) takeSpill() []cscInvalItem {
+	b.spillMu.Lock()
+	s := b.spill
+	b.spill = nil
+	b.spillBytes = 0
+	b.spillMu.Unlock()
+	return s
+}
+
+// apply deletes the namespaced keys and feeds evicted-hot entries to the current
+// refresh binding (b.refresh, which set/clearRefreshQueue may have repointed at a
+// survivor before the stop-drain — see the struct field).
+func (b *cscInvalBatcher) apply(items []cscInvalItem) {
+	cache, refresh := b.cache, b.refresh.Load()
+	if cache == nil {
+		return
+	}
+	// Held for the whole batch so a concurrent drop()+Flush() cannot land
+	// mid-loop and leave stale-epoch deletes running post-flush (see applyMu).
+	b.applyMu.Lock()
+	defer b.applyMu.Unlock()
+	cur := b.epoch.Load()
+	lc, canRefresh := cache.(*LocalCache)
+	canRefresh = canRefresh && refresh != nil
+	var hot []cscRefreshTarget
+	for _, it := range items {
+		// Stale epoch: enqueued before a full cache Flush that superseded it
+		// (see drop()); applying it would only evict a post-flush repopulation.
+		if it.epoch != cur {
+			continue
+		}
+		k := it.key
+		if !canRefresh {
+			if lc != nil {
+				// *LocalCache with refresh OFF: still honor the fetch-order guard, or a
+				// delayed invalidation would delete a value refetched during the batch
+				// window (fetchSeq > fetchSnap) — a spurious miss that can also cancel an
+				// in-progress fetch and wake waiters as duplicates. Use the sequence-aware
+				// deletion path and discard any collected refresh targets (refresh is off).
+				_ = lc.deleteByRedisKeyCollectingHot(k, cscInvalNoHorizon, it.fetchSnap, nil)
+			} else {
+				// Non-*LocalCache: no per-entry fetch sequence to compare; plain delete.
+				cache.DeleteByRedisKey(k)
+			}
+			continue
+		}
+		// Use the horizon snapshotted at ENQUEUE, not a live load: a batch-window
+		// delay advances the live horizon and would chill keys that were hot when
+		// the invalidation arrived (see cscInvalItem.sinceToken). An item enqueued
+		// with no refresh binding carries cscInvalNoHorizon; the binding that
+		// exists NOW (a client attached in between) supplies the live horizon, which
+		// is a real recency test — never horizon 0, which would mark every entry hot.
+		since := it.sinceToken
+		if since == cscInvalNoHorizon {
+			since = refresh.sinceToken.Load()
+		}
+		hot = lc.deleteByRedisKeyCollectingHot(k, since, it.fetchSnap, hot[:0])
+		for i := range hot {
+			refresh.offer(hot[i])
+		}
+	}
+}
+
+func (b *cscInvalBatcher) run() {
+	// Closed LAST (deferred first): join() unblocks only after the stop-drain and
+	// final flush below have fully applied, making a stop+join teardown synchronous.
+	defer close(b.done)
+	batchMax := b.batchMax
+	if batchMax <= 0 {
+		batchMax = cscInvalBatchMax
+	}
+	batchMaxBytes := b.batchMaxBytes
+	if batchMaxBytes <= 0 {
+		batchMaxBytes = cscInvalBatchMaxBytes
+	}
+	t := time.NewTimer(b.window)
+	defer t.Stop()
+	pending := make([]cscInvalItem, 0, 256)
+	// pendingBytes tracks the retained KEY bytes in `pending`, so the size-cap flush
+	// bounds the batch by memory as well as by item count (see cscInvalBatchMaxBytes).
+	// Reset together with pending on every flush.
+	pendingBytes := 0
+	// seen dedups by key WITHIN an epoch: the same key arriving again after a
+	// flush bumped the epoch must be re-appended (the old occurrence will be
+	// skipped by apply), or its post-flush delete would be lost to dedup. idx is the
+	// key's index in pending, so a deduped duplicate can update the retained item's
+	// fetch snapshot (see add). seen and pending are always reset together (every
+	// applyPending is inside flush, which then clearSeen()s; fullFlushIfRequested
+	// resets both), so a stored idx never outlives its pending entry.
+	type seenEntry struct {
+		epoch uint64
+		idx   int
+	}
+	seen := make(map[string]seenEntry, 256)
+	clearSeen := func() {
+		for k := range seen {
+			delete(seen, k)
+		}
+	}
+	// applyPending applies the current batch and resets pending, but does NOT clear
+	// seen — the caller decides. fullFlushIfRequested resets pending without applying
+	// (a full cache Flush supersedes it) and clears seen itself; every other path
+	// applies via flush(), which pairs applyPending with clearSeen. seen is a pure
+	// per-window perf dedup: the fetch-order guard, not dedup lifetime, keeps a
+	// straddling duplicate from evicting a refreshed entry (see add).
+	applyPending := func() {
+		if len(pending) == 0 {
+			return
+		}
+		// Recover so a panic in a cache/refresh call can never kill the worker:
+		// overflow is parked on the unbounded spill buffer, so a dead worker would
+		// let spill grow without bound. Log and keep looping instead.
+		func() {
+			defer func() {
+				if r := recover(); r != nil {
+					internal.Logger.Printf(context.Background(),
+						"redis: csc invalidation batch apply panic: %v", r)
+				}
+			}()
+			b.apply(pending)
+		}()
+		pending = pending[:0]
+		pendingBytes = 0
+	}
+	flush := func() {
+		applyPending()
+		clearSeen()
+	}
+	// add runs one item through the epoch-scoped dedup and size-flushes at the
+	// cap. Shared by the ch, spill, and stop-drain paths so spilled items get the
+	// SAME dedup as fast-path items — a burst of duplicate keys collapses to one
+	// delete per key.
+	add := func(it cscInvalItem) {
+		if e, dup := seen[it.key]; !dup || e.epoch != it.epoch {
+			seen[it.key] = seenEntry{epoch: it.epoch, idx: len(pending)}
+			pending = append(pending, it)
+			pendingBytes += len(it.key)
+		} else if it.fetchSnap > pending[e.idx].fetchSnap {
+			// Same key, same epoch: dropped as a duplicate delete, BUT carry the LATEST
+			// fetch snapshot onto the retained item. A second write to the key in this
+			// window arrives with a newer fetchSnap; keeping only the first would let the
+			// apply-time fetch-order guard use the older snapshot and preserve an entry
+			// refetched BETWEEN the two writes, serving it stale until MaxStaleness
+			// (#3989). The item is still a single delete.
+			pending[e.idx].fetchSnap = it.fetchSnap
+		}
+		// A duplicate key in the same epoch is dropped here: apply() deletes the key
+		// once, so it becomes a single deletion. The incoming push was already
+		// counted at the handler, so the dropped duplicate needs no accounting — it
+		// just won't add a deletion, which is exactly the dedup signal.
+		if len(pending) >= batchMax || pendingBytes >= batchMaxBytes {
+			// Size-cap flush (by item COUNT or retained key BYTES): apply the batch AND
+			// clear seen. The byte cap keeps a burst of large keys from retaining GBs in
+			// pending across a window (each key can be under the per-item admission limit
+			// yet sum far past it). A key straddling the cap
+			// can now be applied in both halves, but that is harmless: the fetch-order
+			// guard (cscInvalItem.fetchSnap) makes the second apply a no-op when the
+			// first apply's refetch already republished the entry — the duplicate's
+			// snapshot predates that refetch. So dedup LIFETIME no longer carries
+			// correctness (it did while keep-seen was the only defense against evicting
+			// a refreshed entry — that also left seen unbounded under sustained load and
+			// dropped a genuine re-invalidation of a refreshed key). seen is back to a
+			// pure per-window perf dedup, and clearing it here keeps it bounded without a
+			// full-cache Flush.
+			flush()
+			t.Reset(b.window)
+		}
+	}
+	drainSpill := func() {
+		for _, it := range b.takeSpill() {
+			add(it)
+		}
+	}
+	// fullFlushIfRequested consumes a flushReq set by enqueue when spill hit its
+	// cap: drop the whole backlog (epoch bump skips any stale-epoch delete still
+	// queued/in-flight) and Flush the cache — the O(1)-memory, correctness-
+	// preserving fallback for an invalidation flood. Recover so a Flush panic can't
+	// kill the worker (its death would let spill grow unbounded again).
+	fullFlushIfRequested := func() {
+		if !b.flushReq.CompareAndSwap(true, false) {
+			return
+		}
+		func() {
+			defer func() {
+				if r := recover(); r != nil {
+					internal.Logger.Printf(context.Background(),
+						"redis: csc invalidation full-flush panic: %v", r)
+				}
+			}()
+			b.drop()
+			if b.cache != nil {
+				b.cache.Flush()
+			}
+		}()
+		// The pending batch is superseded by the Flush and never applied as
+		// individual deletions (the Flush invalidates wholesale). Incoming pushes
+		// were already counted at the handler, so just reset the batch.
+		pending = pending[:0]
+		pendingBytes = 0
+		clearSeen()
+	}
+	for {
+		select {
+		case <-b.stopCh:
+			// Stopped (last release, or a window-change rebuild where queued deletes
+			// are still live and must not be lost): drain spill AND ch into the
+			// batch, flush once, exit. enqueue appends spill under stopMu.RLock, so
+			// anything spilled before stop set stopped=true is visible here.
+			fullFlushIfRequested()
+			drainSpill()
+			for draining := true; draining; {
+				select {
+				case it := <-b.ch:
+					b.chBytes.Add(-int64(len(it.key)))
+					add(it)
+				default:
+					draining = false
+				}
+			}
+			drainSpill() // catch keys spilled during the ch drain
+			flush()
+			return
+		case it := <-b.ch:
+			b.chBytes.Add(-int64(len(it.key)))
+			add(it)
+		case <-b.wake:
+			// An overflow parked keys on spill (or hit the cap and asked for a full
+			// flush); handle the flush request first, then drain what's left.
+			fullFlushIfRequested()
+			drainSpill()
+		case <-t.C:
+			fullFlushIfRequested()
+			drainSpill()
+			flush()
+			t.Reset(b.window)
+		}
+	}
+}

csc_inval_batch_test.go

@@ -0,0 +1,650 @@
+package redis
+
+import (
+	"context"
+	"strconv"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+)
+
+// enqueue is the single-key convenience form of enqueueAt, used only by tests:
+// it snapshots the current fetch order per call. Production code always goes
+// through enqueueAt with a fetchSnap taken once at push-observe time (see the
+// enqueueAt doc), so this wrapper lives here rather than in the package to keep
+// it out of the shipped surface.
+func (b *cscInvalBatcher) enqueue(nsKey string) {
+	b.enqueueAt(nsKey, cscFetchSeq.Load())
+}
+
+// countingCache implements just enough of Cache for the batcher's apply path
+// (refresh==nil + a non-*LocalCache cache => apply calls DeleteByRedisKey once
+// per unique key). Other Cache methods are never reached; the embedded nil Cache
+// would panic if one were, which is the intended tripwire.
+type countingCache struct {
+	Cache
+	deletes atomic.Int64
+	flushes atomic.Int64
+	mu      sync.Mutex
+	keys    []string
+}
+
+func (c *countingCache) DeleteByRedisKey(k string) int {
+	c.deletes.Add(1)
+	c.mu.Lock()
+	c.keys = append(c.keys, k)
+	c.mu.Unlock()
+	return 1
+}
+
+func (c *countingCache) Flush() int { c.flushes.Add(1); return 0 }
+
+func newTestBatcher(cache Cache, chCap int, window time.Duration) *cscInvalBatcher {
+	return &cscInvalBatcher{
+		window: window,
+		cache:  cache,
+		ch:     make(chan cscInvalItem, chCap),
+		wake:   make(chan struct{}, 1),
+		stopCh: make(chan struct{}),
+		done:   make(chan struct{}),
+	}
+}
+
+// TestInvalBatcherStopJoinIsSynchronous pins the stop+join teardown contract:
+// once join returns, the stop-drain has fully applied — every enqueued delete is
+// visible, with no Eventually. Without the join the drain ran asynchronously
+// after Close returned (straggler goroutine; a late apply could evict what a
+// successor client on a shared cache just repopulated).
+func TestInvalBatcherStopJoinIsSynchronous(t *testing.T) {
+	cache := &countingCache{}
+	b := newTestBatcher(cache, 8, time.Hour) // window never fires; only the stop-drain applies
+	go b.run()
+
+	b.enqueue("k1")
+	b.enqueue("k2")
+	b.stop()
+	b.join()
+
+	if got := cache.deletes.Load(); got != 2 {
+		t.Fatalf("deletes after stop+join = %d, want 2 (drain must be synchronous)", got)
+	}
+}
+
+// TestInvalBatcherEnqueueSnapshotsSinceToken pins the #16 horizon snapshot across
+// the atomic.Pointer conversion of the batcher's refresh field: enqueue must read
+// the CURRENT refresh binding's sinceToken and stamp it on the item, so a
+// batch-window delay cannot chill a key that was hot when the invalidation
+// arrived.
+func TestInvalBatcherEnqueueSnapshotsSinceToken(t *testing.T) {
+	b := newTestBatcher(&countingCache{}, 4, time.Hour) // worker not started: item stays on ch
+	q := &cscRefreshQueue{}
+	q.sinceToken.Store(4242)
+	b.refresh.Store(q)
+
+	b.enqueue("k")
+
+	select {
+	case it := <-b.ch:
+		if it.sinceToken != 4242 {
+			t.Fatalf("enqueued item sinceToken = %d, want 4242 (snapshot at enqueue)", it.sinceToken)
+		}
+	default:
+		t.Fatal("enqueue did not land on ch")
+	}
+}
+
+// TestInvalBatcherEnqueueAtStampsSuppliedFetchSnap pins the per-push fetch-order
+// snapshot: enqueueAt must stamp the CALLER's fetchSnap, not a fresh cscFetchSeq
+// load. The push handler snapshots cscFetchSeq once per notification and reuses it
+// for every key (csc_integration.go), so a fetch reserved after the push was
+// observed cannot slip in as "not newer" for a later key in the same push.
+func TestInvalBatcherEnqueueAtStampsSuppliedFetchSnap(t *testing.T) {
+	b := newTestBatcher(&countingCache{}, 4, time.Hour) // worker not started: item stays on ch
+
+	prev := cscFetchSeq.Load()
+	cscFetchSeq.Store(999) // the value a stale per-key load would pick up
+	defer cscFetchSeq.Store(prev)
+
+	b.enqueueAt("k", 4242)
+
+	select {
+	case it := <-b.ch:
+		if it.fetchSnap != 4242 {
+			t.Fatalf("enqueued item fetchSnap = %d, want 4242 (caller snapshot, not a fresh load)", it.fetchSnap)
+		}
+	default:
+		t.Fatal("enqueueAt did not land on ch")
+	}
+}
+
+// TestInvalBatcherRefreshAttachUsesLiveHorizon pins the recency gate across a
+// refresh ATTACH: an item enqueued while the batcher had no refresh binding must
+// not be applied as "horizon 0" once a binding appears before apply (setRefreshQueue
+// repoints the batcher, then stop-drains it). Horizon 0 would mark every valid
+// entry hot and refetch cold keys — the loop the horizon exists to prevent. The
+// item carries cscInvalNoHorizon and apply falls back to the live horizon, so a
+// COLD entry is deleted but NOT offered, while a hot one still is.
+func TestInvalBatcherRefreshAttachUsesLiveHorizon(t *testing.T) {
+	lc := NewLocalCache(CacheConfig{MaxEntries: 64})
+	mkValid := func(cacheKey, redisKey string) {
+		tok, fetch := lc.Reserve(cacheKey, []string{redisKey})
+		if tok == 0 || !fetch {
+			t.Fatalf("Reserve(%q) = token %d, shouldFetch %v", cacheKey, tok, fetch)
+		}
+		if !lc.fulfill(cacheKey, tok, 0, []byte("v")) {
+			t.Fatalf("fulfill(%q) failed", cacheKey)
+		}
+	}
+
+	// Cold entry: populated BEFORE the attaching client's horizon exists.
+	mkValid("ck:cold", "rk:cold")
+
+	// Batcher built with refresh OFF; worker not started so items stay on ch.
+	b := newTestBatcher(lc, 4, time.Hour)
+	b.enqueue("rk:cold")
+	var cold cscInvalItem
+	select {
+	case cold = <-b.ch:
+	default:
+		t.Fatal("enqueue did not land on ch")
+	}
+	if cold.sinceToken != cscInvalNoHorizon {
+		t.Fatalf("item enqueued with refresh off: sinceToken = %d, want cscInvalNoHorizon (%d)",
+			cold.sinceToken, cscInvalNoHorizon)
+	}
+
+	// A refresh-enabled client attaches: its horizon is seeded NOW, after the cold
+	// entry, and the batcher is repointed at its queue (as setRefreshQueue does
+	// before the stop-drain).
+	q := &cscRefreshQueue{ch: make(chan cscRefreshTarget, 8)}
+	q.sinceToken.Store(lc.LRUClock())
+	b.refresh.Store(q)
+
+	// Hot control: read after the horizon, enqueued with a real snapshot.
+	mkValid("ck:hot", "rk:hot")
+	b.enqueue("rk:hot")
+	var hot cscInvalItem
+	select {
+	case hot = <-b.ch:
+	default:
+		t.Fatal("enqueue did not land on ch")
+	}
+
+	// The stop-drain applies both under the NEW binding.
+	b.apply([]cscInvalItem{cold, hot})
+
+	// Both deleted ...
+	ctx := context.Background()
+	if _, ok := lc.Get(ctx, "ck:cold"); ok {
+		t.Fatal("cold entry not deleted")
+	}
+	if _, ok := lc.Get(ctx, "ck:hot"); ok {
+		t.Fatal("hot entry not deleted")
+	}
+	// ... but only the hot one is offered for refetch. Without the sentinel the
+	// cold item would apply with horizon 0 and be offered too (len == 2).
+	if n := len(q.ch); n != 1 {
+		t.Fatalf("refresh targets offered = %d, want 1 (hot only); cold key must not be refetched", n)
+	}
+	if got := <-q.ch; got.cacheKey != "ck:hot" {
+		t.Fatalf("offered target = %q, want ck:hot", got.cacheKey)
+	}
+}
+
+// A full ch must send overflow to spill, never apply a delete inline on the
+// caller (which, on the coalescer's reply reader, would stall miss replies).
+func TestInvalBatcherOverflowSpillsNotInline(t *testing.T) {
+	cache := &countingCache{}
+	// Worker NOT started: ch never drains, so enqueues past its cap must overflow.
+	b := newTestBatcher(cache, 2, time.Hour)
+
+	b.enqueue("k1") // -> ch
+	b.enqueue("k2") // -> ch (now full)
+	b.enqueue("k3") // ch full -> spill
+	b.enqueue("k4") // ch full -> spill
+
+	if got := cache.deletes.Load(); got != 0 {
+		t.Fatalf("overflow applied %d deletes inline on the caller; want 0 (must spill)", got)
+	}
+	if got := b.spilled.Load(); got != 2 {
+		t.Fatalf("spilled counter = %d, want 2", got)
+	}
+	b.spillMu.Lock()
+	n := len(b.spill)
+	b.spillMu.Unlock()
+	if n != 2 {
+		t.Fatalf("spill holds %d items, want 2", n)
+	}
+}
+
+// The worker drains BOTH ch and spill through the shared seen/pending dedup, so a
+// burst of duplicate keys collapses to one delete per unique key.
+func TestInvalBatcherWorkerDrainsSpillDedup(t *testing.T) {
+	cache := &countingCache{}
+	// Large window so only the stop-drain flushes (deterministic single apply).
+	b := newTestBatcher(cache, 2, time.Hour)
+
+	const dups = 200
+	for i := 0; i < dups; i++ {
+		b.enqueue("hot") // fills ch (cap 2) then overflows to spill
+	}
+	b.enqueue("cold")
+
+	if b.spilled.Load() == 0 {
+		t.Fatal("setup: expected overflow to spill")
+	}
+	if got := cache.deletes.Load(); got != 0 {
+		t.Fatalf("setup: %d inline deletes before worker start; want 0", got)
+	}
+
+	go b.run()
+	b.stop() // stop-drain: drains spill + ch, flushes once, exits
+
+	deadline := time.Now().Add(2 * time.Second)
+	for time.Now().Before(deadline) && cache.deletes.Load() < 2 {
+		time.Sleep(time.Millisecond)
+	}
+	if got := cache.deletes.Load(); got != 2 {
+		t.Fatalf("applied %d deletes; want 2 (dedup must collapse %d 'hot' + 1 'cold')", got, dups)
+	}
+}
+
+// Past the spill hard cap, enqueue stops growing spill and asks the worker to
+// full-Flush the cache instead (bounded-memory fallback for an invalidation
+// flood on a keyset larger than the local cache).
+func TestInvalBatcherSpillCapTriggersFlush(t *testing.T) {
+	cache := &countingCache{}
+	b := newTestBatcher(cache, 2, time.Hour)
+
+	// Worker not running: fill ch, then overflow past the cap with distinct keys.
+	for i := 0; i < cscInvalSpillMax+16; i++ {
+		b.enqueue("k" + strconv.Itoa(i))
+	}
+
+	if !b.flushReq.Load() {
+		t.Fatal("spill exceeded the cap but flushReq was not set")
+	}
+	b.spillMu.Lock()
+	n := len(b.spill)
+	b.spillMu.Unlock()
+	if n >= cscInvalSpillMax {
+		t.Fatalf("spill = %d, want < cap %d (cap must clear the backlog)", n, cscInvalSpillMax)
+	}
+
+	// The worker must consume flushReq and Flush the whole cache exactly.
+	go b.run()
+	deadline := time.Now().Add(2 * time.Second)
+	for time.Now().Before(deadline) && cache.flushes.Load() == 0 {
+		time.Sleep(time.Millisecond)
+	}
+	b.stop()
+	if cache.flushes.Load() == 0 {
+		t.Fatal("worker did not full-Flush the cache after the spill cap was hit")
+	}
+}
+
+// Past the spill RETAINED-BYTE bound, enqueue takes the same full-Flush fallback as
+// the item cap — a distinct-large-key flood can pin GBs while the item count stays
+// far below cscInvalSpillMax, so the byte bound must trip first.
+func TestInvalBatcherSpillByteCapTriggersFlush(t *testing.T) {
+	cache := &countingCache{}
+	b := newTestBatcher(cache, 2, time.Hour)
+
+	// 1 MiB per key: a dozen distinct keys past ch cross cscInvalSpillMaxBytes (8 MiB)
+	// while the item count stays a few, far under the 65536 item cap.
+	bigKey := strings.Repeat("x", 1<<20)
+	const n = 2 + 12 // ch cap 2, then 12 overflow to spill
+	for i := 0; i < n; i++ {
+		b.enqueue(bigKey + strconv.Itoa(i))
+	}
+
+	if !b.flushReq.Load() {
+		t.Fatal("large-key spill crossed the byte bound but flushReq was not set")
+	}
+	b.spillMu.Lock()
+	items := len(b.spill)
+	b.spillMu.Unlock()
+	if items >= cscInvalSpillMax {
+		t.Fatalf("spill held %d items (>= item cap %d); the BYTE bound should have tripped first",
+			items, cscInvalSpillMax)
+	}
+
+	// The worker must consume flushReq and full-Flush the cache.
+	go b.run()
+	deadline := time.Now().Add(2 * time.Second)
+	for time.Now().Before(deadline) && cache.flushes.Load() == 0 {
+		time.Sleep(time.Millisecond)
+	}
+	b.stop()
+	if cache.flushes.Load() == 0 {
+		t.Fatal("worker did not full-Flush the cache after the spill byte bound was hit")
+	}
+}
+
+// The worker's PENDING batch is size-capped by retained key BYTES, not only by item
+// count: a stream of large keys (each under the admission limit) that stays far below
+// the 4096-item cap must still flush once pending bytes cross the byte cap, so pending
+// memory cannot grow to GBs across one window (fS2RM). The byte cap applies the batch
+// (per-key deletes), it does not full-Flush.
+func TestInvalBatcherPendingByteCapFlushes(t *testing.T) {
+	cache := &countingCache{}
+	b := newTestBatcher(cache, 64, time.Hour) // window never fires; only the byte cap can flush
+	b.batchMaxBytes = 4096                    // small pending byte cap; item cap stays at 4096
+	go b.run()
+
+	// ~1 KiB keys: a dozen distinct keys cross the 4 KiB byte cap several times while
+	// the item count stays a dozen, far under the 4096-item cap.
+	bigKey := strings.Repeat("y", 1<<10)
+	for i := 0; i < 12; i++ {
+		b.enqueue(bigKey + strconv.Itoa(i))
+	}
+
+	// The byte cap must apply the batch BEFORE any stop-drain and WITHOUT the (1h)
+	// window firing. Capture deletes before stop, then tear down.
+	deadline := time.Now().Add(2 * time.Second)
+	for time.Now().Before(deadline) && cache.deletes.Load() == 0 {
+		time.Sleep(time.Millisecond)
+	}
+	got := cache.deletes.Load()
+	b.stop()
+	b.join()
+	if got == 0 {
+		t.Fatal("pending byte cap did not flush: deletes=0 before stop (only the item cap/window would have)")
+	}
+	if f := cache.flushes.Load(); f != 0 {
+		t.Fatalf("byte cap should apply the batch, not full-Flush; flushes=%d", f)
+	}
+}
+
+// TestClearRefreshQueueDetachesBatcherWithoutJoin pins the Group-A contract: the
+// handler paths that tear a batcher down (here clearRefreshQueue; also
+// setRefreshQueue/releaseLocked/setInvalBatchWindow) must DETACH and SIGNAL the
+// batcher under h.mu but NOT join it there — join waits on the stop-drain, which
+// would stall a sibling on the hot path and must never block the GC finalizer. The
+// batcher's run() is never started, so its done channel never closes: the old code
+// (b.join() under the lock) would hang forever, the fixed code returns the detached
+// batcher for the caller to join OUTSIDE the lock.
+func TestClearRefreshQueueDetachesBatcherWithoutJoin(t *testing.T) {
+	cache := NewLocalCache(CacheConfig{MaxEntries: 8})
+	h := &invalidateHandler{}
+	if err := h.bindTo(cache, "p:"); err != nil {
+		t.Fatalf("bindTo: %v", err)
+	}
+	q := &cscRefreshQueue{}
+	h.setRefreshQueue(q) // active binding (no batcher yet, returns nil)
+
+	// A batcher whose run() was NEVER started: done never closes, so join() blocks.
+	b := newTestBatcher(cache, 1, time.Hour)
+	h.mu.Lock()
+	h.batcher = b
+	h.mu.Unlock()
+
+	var got *cscInvalBatcher
+	done := make(chan struct{})
+	go func() {
+		got = h.clearRefreshQueue(q)
+		close(done)
+	}()
+	select {
+	case <-done:
+	case <-time.After(3 * time.Second):
+		t.Fatal("clearRefreshQueue blocked: it joined the batcher under h.mu " +
+			"(a never-started worker); the fix must detach+signal only")
+	}
+	if got != b {
+		t.Fatalf("clearRefreshQueue returned %p, want the detached batcher %p", got, b)
+	}
+	if !b.stopped {
+		t.Fatal("clearRefreshQueue did not signal the batcher to stop")
+	}
+	if h.batcher != nil {
+		t.Fatal("clearRefreshQueue did not detach the batcher from the handler")
+	}
+}
+
+// TestInvalBatcherChByteGateTripsFlush pins finding #2: the fast-path channel is
+// bounded by item COUNT, not memory, so a burst of large keys must trip the
+// full-Flush fallback via the retained ch-byte gate — not silently retain up to
+// one channel's worth of large keys. Worker NOT started, ch cap large, so items
+// stay in ch and only the byte gate can fire.
+func TestInvalBatcherChByteGateTripsFlush(t *testing.T) {
+	b := newTestBatcher(&countingCache{}, 1024, time.Hour)
+
+	keySize := cscInvalSpillMaxBytes / 4 // four fill the 8 MiB cap exactly
+	bigKey := strings.Repeat("k", keySize)
+	// Four admit to ch (4 * cap/4 == cap, still within bound); the fifth would cross
+	// the byte cap, so it must divert to the full-Flush fallback instead of ch.
+	for i := 0; i < 5; i++ {
+		b.enqueue(bigKey + strconv.Itoa(i)) // distinct keys: no dedup collapse
+	}
+
+	if !b.flushReq.Load() {
+		t.Fatal("ch retained > byte cap but the full-Flush fallback was not requested " +
+			"(ch admits by item count only; the byte gate must trip)")
+	}
+	if got := b.chBytes.Load(); got > int64(cscInvalSpillMaxBytes) {
+		t.Fatalf("chBytes = %d exceeds the cap %d; the gate admitted an over-budget key",
+			got, cscInvalSpillMaxBytes)
+	}
+}
+
+// TestInvalBatcherOversizedKeyTripsFlush: a single key larger than the whole byte
+// cap trips the full-Flush fallback on its own, never admitted to ch.
+func TestInvalBatcherOversizedKeyTripsFlush(t *testing.T) {
+	b := newTestBatcher(&countingCache{}, 1024, time.Hour)
+	b.enqueue(strings.Repeat("x", cscInvalSpillMaxBytes+1))
+	if !b.flushReq.Load() {
+		t.Fatal("an oversized single key was admitted instead of tripping the full-Flush fallback")
+	}
+	if got := b.chBytes.Load(); got != 0 {
+		t.Fatalf("chBytes = %d; the oversized key must not be admitted to ch", got)
+	}
+}
+
+// TestInvalBatcherChByteReservationIsAtomic pins the atomic ch-byte reservation:
+// the gate must RESERVE before the send (add-and-rollback), not Load-then-Add.
+// Concurrent producers each sending a large distinct key could otherwise each read
+// a stale below-cap Load, all send, and all Add — overshooting cscInvalSpillMaxBytes
+// while the counter briefly read ~0. Worker NOT started and ch cap far exceeds the
+// producer count, so ONLY the byte reservation (not the item-count bound or a
+// draining worker) can keep chBytes under the cap.
+func TestInvalBatcherChByteReservationIsAtomic(t *testing.T) {
+	b := newTestBatcher(&countingCache{}, 4096, time.Hour)
+
+	const producers = 64
+	bigKey := strings.Repeat("x", 1<<20) // 1 MiB; ~8 fit the 8 MiB cap, the rest divert
+
+	start := make(chan struct{})
+	var wg sync.WaitGroup
+	for i := 0; i < producers; i++ {
+		wg.Add(1)
+		go func(i int) {
+			defer wg.Done()
+			<-start
+			b.enqueue(bigKey + strconv.Itoa(i)) // distinct keys: no dedup collapse
+		}(i)
+	}
+	close(start) // release all producers at once to maximize the reservation race
+	wg.Wait()
+
+	if got := b.chBytes.Load(); got > int64(cscInvalSpillMaxBytes) {
+		t.Fatalf("chBytes = %d exceeds the cap %d after %d concurrent enqueues; the byte "+
+			"reservation overshot (non-atomic Load-then-Add admits over-cap bytes)",
+			got, cscInvalSpillMaxBytes, producers)
+	}
+}
+
+// TestInvalBatcherNoRefreshGuardKeepsRefetched pins r3936994566: the batcher must apply
+// the fetch-order guard even when refresh-on-invalidate is OFF (a *LocalCache with no
+// refresh binding). A delayed invalidation whose fetchSnap predates a refetch must not
+// evict the newer value (fetchSeq > fetchSnap). Before the fix the !canRefresh branch did
+// an unconditional DeleteByRedisKey and served the refetch as a spurious miss.
+func TestInvalBatcherNoRefreshGuardKeepsRefetched(t *testing.T) {
+	ctx := context.Background()
+	lc := NewLocalCache(CacheConfig{MaxEntries: 1024})
+	mkValid := func(cacheKey, redisKey string) {
+		lc.DeleteByCacheKey(cacheKey)
+		tok, fetch := lc.Reserve(cacheKey, []string{redisKey})
+		if tok == 0 || !fetch {
+			t.Fatalf("Reserve(%q) = (%d, %v)", cacheKey, tok, fetch)
+		}
+		if !lc.fulfill(cacheKey, tok, 0, []byte("v")) {
+			t.Fatalf("fulfill(%q) failed", cacheKey)
+		}
+	}
+
+	// Refresh is OFF: no b.refresh binding -> apply takes the !canRefresh branch.
+	b := newTestBatcher(lc, 64, time.Hour)
+
+	mkValid("ck:hot", "rk") // original entry
+	go b.run()
+
+	// Invalidation snapshotted BEFORE the refetch below (its fetchSnap predates it).
+	b.enqueue("rk")
+	// Refetch after the invalidation was observed: the new entry's fetchSeq exceeds the
+	// duplicate's snapshot, so the guard must keep it.
+	mkValid("ck:hot", "rk")
+
+	b.stop()
+	b.join()
+
+	if _, ok := lc.Get(ctx, "ck:hot"); !ok {
+		t.Fatal("refresh-off apply evicted a value refetched after the invalidation; the " +
+			"fetch-order guard must apply even without refresh-on-invalidate (r3936994566)")
+	}
+}
+
+// TestInvalBatcherSizeCapGuardKeepsRefreshed pins #3965 F1 under the fetch-order
+// guard. The size-cap flush now CLEARS the per-epoch dedup (seen), so a key can be
+// applied on both sides of the cap — but a stale duplicate that straddles a
+// refetch must not evict the fresh entry. The duplicate is enqueued (its fetch
+// snapshot frozen) BEFORE the simulated refetch, so at apply the entry's fetchSeq
+// exceeds the duplicate's snapshot and the delete is skipped. The DeletionStats
+// noop delta proves the second occurrence was actually applied (seen was cleared,
+// not deduped) and the guard — not dedup — is what saved the entry.
+func TestInvalBatcherSizeCapGuardKeepsRefreshed(t *testing.T) {
+	const batchMax = 8 // small cap so the size-cap flush fires deterministically
+
+	ctx := context.Background()
+	lc := NewLocalCache(CacheConfig{MaxEntries: 1024})
+	mkValid := func(cacheKey, redisKey string) {
+		lc.DeleteByCacheKey(cacheKey)
+		tok, fetch := lc.Reserve(cacheKey, []string{redisKey})
+		if tok == 0 || !fetch {
+			t.Fatalf("Reserve(%q) = (%d, %v)", cacheKey, tok, fetch)
+		}
+		if !lc.fulfill(cacheKey, tok, 0, []byte("v")) {
+			t.Fatalf("fulfill(%q) failed", cacheKey)
+		}
+	}
+
+	// window never fires: only the size-cap flush and the stop-drain apply.
+	b := newTestBatcher(lc, batchMax*2, time.Hour)
+	b.batchMax = batchMax // per-batcher, set before the worker starts (no global race)
+	q := &cscRefreshQueue{ch: make(chan cscRefreshTarget, 64)}
+	q.sinceToken.Store(lc.LRUClock()) // seed the horizon so the entry reads as hot
+	b.refresh.Store(q)
+
+	mkValid("ck:hot", "rk:hot") // the target entry, read after the horizon
+
+	go b.run()
+
+	// Enqueue the target FIRST, then fillers so pending reaches the cap and the
+	// size-cap flush applies the target in the first half of the window (deleting it
+	// and clearing seen).
+	b.enqueue("rk:hot")
+	for i := 0; i < batchMax-1; i++ {
+		b.enqueue("rk:filler" + strconv.Itoa(i))
+	}
+
+	// Wait for the size-cap flush to FULLY apply batch #1 — all 8 keys, so the
+	// deletion counters are settled before we snapshot them (polling Get would race
+	// apply mid-loop while fillers still bump the counters).
+	deadline := time.Now().Add(2 * time.Second)
+	for {
+		if del, _ := lc.DeletionStats(); del >= batchMax {
+			break
+		}
+		if time.Now().After(deadline) {
+			t.Fatal("size-cap flush did not apply batch #1")
+		}
+		time.Sleep(time.Millisecond)
+	}
+	if _, ok := lc.Get(ctx, "ck:hot"); ok {
+		t.Fatal("apply #1 did not delete the target")
+	}
+
+	// The SAME key arrives again, post-flush, with its fetch snapshot frozen NOW —
+	// BEFORE the refetch below. This is the stale straddling duplicate: it predates
+	// the refresh republish, so the guard must skip it at apply time.
+	delBefore, noopBefore := lc.DeletionStats()
+	b.enqueue("rk:hot")
+
+	// Simulate the refresh republish AFTER the duplicate was enqueued: the refetch
+	// reserves a new entry whose fetchSeq exceeds the duplicate's snapshot.
+	mkValid("ck:hot", "rk:hot")
+
+	b.stop()
+	b.join()
+
+	if _, ok := lc.Get(ctx, "ck:hot"); !ok {
+		t.Fatal("the stale straddling duplicate evicted the refreshed entry; the " +
+			"fetch-order guard must keep an entry refetched after the invalidation (#3965 F1)")
+	}
+	// The duplicate was applied (seen cleared) but skipped by the guard: applied
+	// deletes advanced by 1 and that delete removed nothing (noop), which is the
+	// guard-skip signature — not a dedup drop (which would advance neither).
+	delAfter, noopAfter := lc.DeletionStats()
+	if delAfter-delBefore != 1 || noopAfter-noopBefore != 1 {
+		t.Fatalf("duplicate apply delta = (deletions %d, noop %d), want (1, 1): the "+
+			"second occurrence must be applied and skipped by the guard, not deduped away",
+			delAfter-delBefore, noopAfter-noopBefore)
+	}
+}
+
+// TestInvalBatcherDedupKeepsLatestFetchSnap pins #3989: when two writes to the same
+// key land in one batching window with a refetch between them, the second write's
+// invalidation is deduped, but its NEWER fetch snapshot must be carried onto the
+// retained item — otherwise the apply-time guard uses the first write's older snapshot
+// and preserves the entry refetched BETWEEN the writes, serving it stale.
+func TestInvalBatcherDedupKeepsLatestFetchSnap(t *testing.T) {
+	ctx := context.Background()
+	lc := NewLocalCache(CacheConfig{MaxEntries: 1024})
+	mkValid := func(cacheKey, redisKey string) {
+		lc.DeleteByCacheKey(cacheKey)
+		tok, fetch := lc.Reserve(cacheKey, []string{redisKey})
+		if tok == 0 || !fetch {
+			t.Fatalf("Reserve(%q) = (%d, %v)", cacheKey, tok, fetch)
+		}
+		if !lc.fulfill(cacheKey, tok, 0, []byte("v")) {
+			t.Fatalf("fulfill(%q) failed", cacheKey)
+		}
+	}
+
+	// window never fires; only the stop-drain applies (which dedups ch in order).
+	b := newTestBatcher(lc, 64, time.Hour)
+	q := &cscRefreshQueue{ch: make(chan cscRefreshTarget, 64)}
+	q.sinceToken.Store(lc.LRUClock())
+	b.refresh.Store(q)
+
+	mkValid("ck:hot", "rk") // original entry
+	go b.run()
+
+	// Write 1's invalidation: snapshot BEFORE the refetch below.
+	b.enqueue("rk")
+	// Refetch between the two writes: the entry's fetchSeq now exceeds occ1's snapshot.
+	mkValid("ck:hot", "rk")
+	// Write 2's invalidation: same key, same window -> deduped, but carries a NEWER
+	// snapshot that must overwrite occ1's so the guard evicts the between-writes entry.
+	b.enqueue("rk")
+
+	b.stop()
+	b.join()
+
+	if _, ok := lc.Get(ctx, "ck:hot"); ok {
+		t.Fatal("entry refetched between two same-window writes was preserved; dedup must " +
+			"keep the LATEST fetch snapshot so the guard evicts it (#3989)")
+	}
+}

csc_miss_coalesce.go

@@ -0,0 +1,934 @@
+package redis
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"fmt"
+	"sync"
+	"sync/atomic"
+	"time"
+
+	"github.com/redis/go-redis/v9/internal"
+	"github.com/redis/go-redis/v9/internal/pool"
+	"github.com/redis/go-redis/v9/internal/proto"
+)
+
+// errCSCRetryUncached settles a coalesced miss when the coalescer bows out for a
+// reason unrelated to the command itself: CSC serving was disabled after the
+// miss was queued (a RESP3 downgrade, or CLIENT TRACKING rejected during a
+// connection re-init), so this session would fetch on an untracked conn or none
+// at all. It never reaches the caller — processCached catches it and re-runs the
+// command uncached, so a valid cacheable read is not failed with a spurious
+// pool.ErrClosed. The reservation is always cancelled before this is settled, so
+// no waiter is left IN_PROGRESS.
+var errCSCRetryUncached = errors.New("redis: csc miss-coalescer disabled mid-fetch; retry uncached")
+
+// Reader-miss coalescing, behind Options.ClientSideCacheCoalesceMisses.
+//
+// The two-client CSC shape serves cache HITS locally (no connection) but every
+// MISS fetches on the caching client's MAIN pool, one command per connection.
+// At a small pool that is the wall: N concurrent misses contend one GET at a
+// time (measured: at CSC pool 8 the reader misses alone saturate the pool, p99
+// ~420ms at 64 workers, and no refresh policy helps — the readers are the
+// bottleneck).
+//
+// This streams each miss's ORIGINAL command onto a held tracked (main-pool)
+// full-duplex connection (see csc_miss_coalesce_modes.go), reads the replies
+// back in order, and applies each to its caller's command AND to the shared
+// cache. Individual pipelined commands (not MGET), so it is cluster-safe and
+// works for any read shape, not just string GET. Misses are CALLER-BLOCKING, so
+// the engine is latency-first: a lone miss is written immediately (batching only
+// packs what is already queued, never waits for more), and new misses stream out
+// while earlier replies are still in flight (~1 RTT per miss, no batch
+// phase-lock, no pool Get on the hot path).
+//
+// Being cache-aware is why it cannot just ride the ordinary autopipeliner: it
+// must Reserve/fulfilCached each key so the entry is tracked and
+// single-flighted. The caller Reserves in processCached (only shouldFetch==true
+// misses reach here); the engine owns the token from that point and settles it —
+// fulfilled or cancelled — before waking the caller, or a dangling IN_PROGRESS
+// entry blocks every reader of that key for StaleTimeout.
+//
+// Tradeoff, same as the autopipeline path: a coalesced miss loses per-command
+// MaxRetries/backoff/LOADING handling.
+
+const (
+	cscMissBatchMax   = 128
+	cscMissQueueDepth = 4096
+	// cscFullDuplexDepth caps in-flight commands on the single-connection
+	// full-duplex engine (flow control for the writer).
+	cscFullDuplexDepth = 4096
+	// cscMissBatchBytes is the top limit on the serialized size of a coalesced write
+	// batch. It bounds the buffered write even when the configured write buffer is
+	// very large. The effective cap is the smaller of this value and the write
+	// buffer (see cscMissWriteBatchBytes). A batch larger than the write buffer makes
+	// bufio flush in the middle of the batch, which can deadlock the write and the
+	// read. The first miss always goes, because a single large command must not
+	// stall. So the cap bounds only the extra already-queued misses that share the
+	// write.
+	cscMissBatchBytes = 1 << 20 // 1 MiB
+	// cscMissWireBudgetBytes bounds the TOTAL serialized bytes of in-flight coalesced
+	// misses. The queue's item cap (cscMissQueueDepth) does not bound memory: every
+	// caller serializes and holds a full command copy (req.wire) while blocked on the
+	// send, and cacheable commands (e.g. a large MGET) have no small encoded size, so
+	// a burst of large misses could exhaust the process. Over this budget a miss sheds
+	// to the ordinary pooled path (errCSCRetryUncached) — which is itself bounded by
+	// pool turns — instead of coalescing. 8x one batch lets several full batches be in
+	// flight before shedding.
+	cscMissWireBudgetBytes = 8 * cscMissBatchBytes // 8 MiB
+	// cscMissMaxConcurrentSerialize caps how many fetch calls may hold a wire
+	// snapshot mid-serialization at once. reserveWireBytes gates on cmdApproxBytes,
+	// which UNDERCOUNTS a variable-size arg, so a concurrent burst of large misses
+	// can all pass the byte reservation and each allocate its full wireBuf BEFORE
+	// reconcileWireBytes sheds — a transient set of multi-MiB snapshots the budget
+	// never sees, which can exhaust memory even though none stay queued. Bounding the
+	// concurrent set to budget/batch caps the COUNT of coexisting snapshots, not their
+	// bytes: when Args() sizes its encoded form accurately each snapshot is about
+	// batch-sized, so the peak is ~O(budget). A single command whose encoded form far
+	// exceeds cmdApproxBytes' estimate (e.g. an encoding.BinaryMarshaler returning
+	// hundreds of MB) makes that one snapshot arbitrarily large, so the true worst-case
+	// transient peak is cap x (largest single serialized command), NOT O(budget) — a
+	// hard per-request byte ceiling enforced during writeCmd is a focused follow-up. A
+	// caller that cannot get a slot promptly sheds to the pooled path rather than block.
+	// Serialization is CPU-only and brief, so normal small-command bursts pass without
+	// shedding.
+	cscMissMaxConcurrentSerialize = cscMissWireBudgetBytes / cscMissBatchBytes // 8
+)
+
+// cscMissSerializeWait is how long acquireSerialize waits for a serialization slot
+// before shedding to the pooled path. Serialization never blocks on I/O (it writes
+// to an in-memory buffer), so a slot frees in microseconds; the wait only absorbs
+// brief scheduling contention. A var so a test can set it to 0 (shed immediately)
+// for a deterministic admission assertion.
+var cscMissSerializeWait = 5 * time.Millisecond
+
+// cscMissWriteBatchBytes returns the effective per-batch size cap. It is the
+// connection's write buffer, limited to at most cscMissBatchBytes. When a batch
+// fits the write buffer, bufio flushes it once, at the end of WithWriter, not in
+// the middle. So the reader, which waits until the whole batch is written, starts
+// to drain replies before any request that could block the writer is on the wire.
+// This closes the write and read deadlock window for large payloads.
+func cscMissWriteBatchBytes(opt *Options) int {
+	wb := 0
+	if opt != nil {
+		wb = opt.WriteBufferSize
+	}
+	if wb <= 0 {
+		wb = proto.DefaultBufferSize
+	}
+	if wb > cscMissBatchBytes {
+		wb = cscMissBatchBytes
+	}
+	return wb
+}
+
+// cscCoalesceMissesEnabled is read once, at coalescer construction (per client).
+func cscCoalesceMissesEnabled(opt *Options) bool {
+	return opt != nil && opt.ClientSideCacheCoalesceMisses
+}
+
+// cscReq* are the states of cscMissReq.apply, the single-word interlock over who
+// owns req.cmd: the fetching caller or the reader applying the reply. Exactly
+// one side wins the pending->X CAS, so the reader never writes cmd after the
+// caller has returned and may be reusing it.
+const (
+	cscReqPending   uint32 = iota // no one has claimed cmd yet
+	cscReqAbandoned               // caller's ctx (or Close) fired first: cmd is the caller's again
+	cscReqApplying                // reader claimed cmd first: it will write and settle
+)
+
+// cscMissReq is one caller waiting for its missed key. done carries the fetch
+// result back (the reply is already applied to cmd by the time done fires).
+type cscMissReq struct {
+	cmd      Cmder
+	cacheKey string
+	token    uint64
+	done     chan error
+	// servedBy is the session connection that served (or failed) this request,
+	// set by the engine BEFORE settling done (the channel receive is the
+	// happens-before edge for the caller's read). Feeds the native OTel
+	// recorder: processCached stamps it into processState so a coalesced miss
+	// reports its serving connection and an attempt, like any other command.
+	// Nil when the request never reached a session (enqueue reject, abandon).
+	servedBy *pool.Conn
+	// sentConn is servedBy published atomically: the session stores it when the batch
+	// is assigned to a connection (at send), so a caller that abandons on ctx-cancel
+	// BEFORE the req.done happens-before edge can still attribute the metric to the
+	// serving conn instead of reporting nil for a command Redis actually received
+	// (#3989). Plain servedBy is unsafe to read on that branch (it races the session's
+	// write); this atomic is the safe view for it. Nil until the batch reaches a conn.
+	sentConn atomic.Pointer[pool.Conn]
+	// wire is cmd's RESP encoding, snapshotted at enqueue while the caller
+	// still owned cmd. The session writer writes ONLY these engine-owned bytes
+	// — it never reads cmd — so an abandoning caller may reuse mutable args
+	// (e.g. a []byte key) the moment fetch returns, and a post-abandon
+	// mutation can neither reach the wire nor publish under the original
+	// cache key. Only the reply side touches cmd, gated by the apply interlock.
+	wire []byte
+	// reserved is the wire-budget byte reservation for this request (see
+	// reserveWireBytes). It is charged in fetch before serialization and released
+	// exactly once — by the settle helper when the request is consumed (its wire
+	// leaves mc.ch/inflight), or by fetch itself if the request never reaches the
+	// queue. Tracking the WIRE's lifetime (not the caller's) keeps a cancelled but
+	// still-queued snapshot counted so the budget cannot be reused while thousands
+	// of copies remain retained.
+	reserved int64
+	// apply interlocks ownership of cmd between the fetching caller and the
+	// reader. The caller abandons (CAS pending->abandoned) if its context is
+	// cancelled — or Close races its enqueue — before a reply is applied; the
+	// reader claims (CAS pending->applying) before it writes the reply into cmd. A
+	// plain flag would still race: the reader could read "not abandoned", begin
+	// writing cmd, and the caller cancel mid-write. Either way the reader still
+	// settles the token and publishes to the shared cache (the fetch is never
+	// wasted) — only the cmd write is gated.
+	apply atomic.Uint32
+}
+
+// claimAbandon is the caller's side of the cmd interlock: it succeeds only if no
+// reply is being applied, in which case the reader skips the cmd write.
+func (r *cscMissReq) claimAbandon() bool {
+	return r.apply.CompareAndSwap(cscReqPending, cscReqAbandoned)
+}
+
+// claimApply is the reader's side: it succeeds only if the caller has not
+// abandoned, in which case it is safe to write the caller's cmd.
+func (r *cscMissReq) claimApply() bool {
+	return r.apply.CompareAndSwap(cscReqPending, cscReqApplying)
+}
+
+type cscMissCoalescer struct {
+	c        *baseClient
+	ch       chan *cscMissReq
+	stop     chan struct{}
+	stopOnce sync.Once // guards close(stop): Close and the GC cleanup can both stop
+	wg       sync.WaitGroup
+
+	// stopDrainBudget overrides the Close-time drain cap. Zero in all production paths
+	// (never set outside tests) — cscMissStopDrainIntervals × interval applies. Set only
+	// by tests, which cannot otherwise reach the cap in bounded time: interval is floored
+	// at batchBudget()+1s >= 6s, so the default cap first fires ~48s in.
+	stopDrainBudget time.Duration
+
+	// maxBatchBytes bounds the serialized size of one coalesced write batch to the
+	// connection's write buffer (see cscMissWriteBatchBytes). This keeps bufio from
+	// a flush in the middle of a batch. The reason: the reader waits until the whole
+	// batch is written, because the writer fills inflight only after the write. A
+	// mid-batch flush would put request bytes on the wire and let the server reply
+	// into buffers that nobody drains yet. On large payloads this can deadlock the
+	// write and the read. Set once at construction.
+	maxBatchBytes int
+
+	// wireBytes is the total serialized size of in-flight coalesced misses, bounded
+	// by cscMissWireBudgetBytes. A fetch adds its approximate encoded size before
+	// serializing and subtracts it when done; over budget it sheds to the pooled path.
+	wireBytes atomic.Int64
+
+	// serializeSem bounds how many fetch calls hold a wire snapshot mid-serialization
+	// at once (see cscMissMaxConcurrentSerialize). Buffered to the cap; a slot is held
+	// ONLY across writeCmd+reconcile and released before the enqueue send, so it
+	// bounds the transient allocation set without coupling to queue backpressure — no
+	// slot is ever held across a channel send or a reply wait, so it cannot deadlock
+	// the single-worker session. Nil in test-constructed literals: acquire/release
+	// treat a nil sem as unbounded and skip the serializing/maxSerializing counters
+	// too (so they stay paired), so those keep working; production always sizes it.
+	serializeSem chan struct{}
+	// serializing/maxSerializing track the current and peak concurrent serializations
+	// (updated under a held slot), so a test can assert the bound holds. Internal,
+	// like abandonedApplies.
+	serializing    atomic.Int64
+	maxSerializing atomic.Uint64
+
+	batched    atomic.Uint64 // reqs that went through a batch
+	batches    atomic.Uint64 // batches flushed
+	failed     atomic.Uint64 // reqs settled as errors (conn failure)
+	maxBatchSz atomic.Uint64
+	// abandonedApplies counts replies whose caller had already abandoned the req,
+	// so the cmd write was skipped. Read only by the -race abandoned-path test, to
+	// assert it hit the window it guards.
+	abandonedApplies atomic.Uint64
+}
+
+// startCSCMissCoalescer launches the coalescer sessions. No-op unless
+// Options.ClientSideCacheCoalesceMisses is set and CSC is active, and — see the
+// option's GoDoc — unless the cache is the built-in *LocalCache: the publish path
+// (fulfillCached capture, refresh integration, hot-entry collection) is
+// LocalCache-specific, so a custom Cache falls back to per-caller fetches.
+func (c *baseClient) startCSCMissCoalescer() {
+	if !cscCoalesceMissesEnabled(c.opt) || c.cscMissCoalescer.Load() != nil {
+		return
+	}
+	if _, ok := c.csc.(*LocalCache); !ok {
+		return
+	}
+	// Cheap fast path: if CSC serving is already known inactive, skip building
+	// anything. Not the safety mechanism on its own — see the workersMu check
+	// below, which is what actually closes the construction/teardown race.
+	if a := c.cscActive; a != nil && !a.Load() {
+		return
+	}
+	mc := &cscMissCoalescer{
+		c:             c,
+		ch:            make(chan *cscMissReq, cscMissQueueDepth),
+		stop:          make(chan struct{}),
+		maxBatchBytes: cscMissWriteBatchBytes(c.opt),
+		serializeSem:  make(chan struct{}, cscMissMaxConcurrentSerialize),
+	}
+
+	// publish stores the coalescer and launches its N sessions. Extracted so
+	// both the guarded and the no-drain-handle path below run the exact same
+	// sequence.
+	publish := func() {
+		c.cscMissCoalescer.Store(mc)
+		// N independent full-duplex sessions, each holding its own connection and
+		// pulling misses from the shared queue. Order-free: coalesced misses are
+		// standalone per-key fetches with no cross-request contract, and each
+		// session's per-conn reader still matches replies to its own requests.
+		for i := 0; i < cscFullDuplexConnsDefault; i++ {
+			mc.wg.Add(1)
+			go mc.fullDuplexLoop()
+		}
+	}
+
+	// Run publish under the drain handle's lock, checked against
+	// workersTornDown (see cscDrainHandle, stopCSCRefresherAndCoalescer, and
+	// the mirrored check in startCSCRefresher): a concurrent teardown may
+	// already have consumed workersStopOnce — reachable via the drainer's own
+	// self-disable tick reacting to an async conn init's disableCSCServing,
+	// racing this construction. That teardown must not be handed a coalescer
+	// it will never get another chance to stop (it would then hold a pool
+	// connection for the client's remaining life), so decline to start at all
+	// if it already ran. The WHOLE of publish runs inside the lock, not just
+	// the Store: teardown's body sets workersTornDown as its own first action
+	// under this same lock, so holding it across the wg.Add/go loop guarantees
+	// teardown cannot call mc.wg.Wait() (in stopCSCMissCoalescer) concurrently
+	// with an in-progress mc.wg.Add — sync.WaitGroup's documented misuse case,
+	// which can panic — because teardown cannot even read the published
+	// pointer until every Add has happened (a second bot review caught that
+	// the Store-only version left this window open). dh is nil only for tests
+	// that call this directly without a drain handle; there is nothing to race
+	// against in that case.
+	if dh := c.cscDrainHandle; dh != nil {
+		dh.workersMu.Lock()
+		if !dh.workersTornDown {
+			publish()
+		}
+		dh.workersMu.Unlock()
+		return
+	}
+	publish()
+}
+
+func (c *baseClient) stopCSCMissCoalescer() {
+	// Swap, not load-then-nil: exactly one caller wins even if Close races the
+	// GC cleanup path, so mc.stop is closed once.
+	mc := c.cscMissCoalescer.Swap(nil)
+	if mc == nil {
+		return
+	}
+	mc.stopWorkers()
+	mc.wg.Wait()
+	// Any request still queued after stop is drained and settled so no caller
+	// hangs on its done channel. Retry-uncached, matching fetch's stop paths:
+	// teardown deactivates serving before stopping the coalescer, so a caller
+	// woken here re-runs its read on the (possibly still open) pool instead of
+	// surfacing a spurious ErrClosed mid-window; on a truly closed client the
+	// uncached re-run fails with the real error.
+	for {
+		select {
+		case req := <-mc.ch:
+			mc.c.csc.Cancel(req.cacheKey, req.token)
+			mc.settle(req, errCSCRetryUncached)
+		default:
+			return
+		}
+	}
+}
+
+// stopWorkers signals every coalescer goroutine to exit; idempotent, does not
+// wait. Called by stopCSCMissCoalescer (Close) and by the GC cleanup for a client
+// dropped without Close — the sessions retain the base client (cache, pools), so
+// leaving them running leaks all of it per forgotten client.
+func (mc *cscMissCoalescer) stopWorkers() {
+	mc.stopOnce.Do(func() { close(mc.stop) })
+}
+
+// fetch hands a reserved miss to the coalescer and waits for the result. The
+// caller has already Reserved (shouldFetch==true); the coalescer owns the token
+// from here. Honors the caller's context: if it cancels while waiting, the
+// session still settles the token and populates the cache (the fetch is not
+// wasted), the caller just returns early.
+//
+// served is the session connection that produced the settle (nil when the
+// request never reached one); processCached stamps it into processState so the
+// native OTel recorder attributes the miss like any other command.
+// reserveWireBytes reserves n bytes of the in-flight wire budget. It returns false
+// (reserving nothing) when the reservation would exceed cscMissWireBudgetBytes, so
+// the caller sheds to the pooled path instead of holding a wire copy while blocked
+// on the send. releaseWireBytes returns a prior reservation.
+func (mc *cscMissCoalescer) reserveWireBytes(n int64) bool {
+	if mc.wireBytes.Add(n) > cscMissWireBudgetBytes {
+		mc.wireBytes.Add(-n)
+		return false
+	}
+	return true
+}
+
+func (mc *cscMissCoalescer) releaseWireBytes(n int64) { mc.wireBytes.Add(-n) }
+
+// acquireSerialize takes a serialization slot (see serializeSem/
+// cscMissMaxConcurrentSerialize) and reports whether it succeeded. It tries once
+// without blocking, then waits up to cscMissSerializeWait — a slot frees in
+// microseconds because serialization never blocks on I/O — and sheds (returns
+// false) on the wait or on Close rather than blocking unbounded. A nil sem (test
+// literal) is unbounded. Every true return is paired with exactly one
+// releaseSerialize.
+func (mc *cscMissCoalescer) acquireSerialize() bool {
+	if mc.serializeSem == nil {
+		return true
+	}
+	select {
+	case mc.serializeSem <- struct{}{}:
+		mc.noteSerializing()
+		return true
+	default:
+	}
+	if cscMissSerializeWait <= 0 {
+		return false
+	}
+	t := time.NewTimer(cscMissSerializeWait)
+	defer t.Stop()
+	select {
+	case mc.serializeSem <- struct{}{}:
+		mc.noteSerializing()
+		return true
+	case <-t.C:
+		return false
+	case <-mc.stop:
+		return false
+	}
+}
+
+// releaseSerialize returns a slot taken by a successful acquireSerialize.
+func (mc *cscMissCoalescer) releaseSerialize() {
+	if mc.serializeSem == nil {
+		return
+	}
+	mc.serializing.Add(-1)
+	<-mc.serializeSem
+}
+
+// noteSerializing records the peak concurrent serialization count for the
+// admission-bound test; called only under a freshly taken slot.
+func (mc *cscMissCoalescer) noteSerializing() {
+	n := uint64(mc.serializing.Add(1))
+	for {
+		m := mc.maxSerializing.Load()
+		if n <= m || mc.maxSerializing.CompareAndSwap(m, n) {
+			return
+		}
+	}
+}
+
+// reconcileWireBytes corrects req's reservation to the ACTUAL serialized size once
+// req.wire is built, and reports whether the request still fits the budget. The
+// reservation is charged from cmdApproxBytes BEFORE serialization (so an
+// over-budget miss never allocates the wire), but that estimate charges only ~8
+// bytes for a variable-size non-string/[]byte arg (a BinaryMarshaler, net.IP,
+// *string, ...), which serializes its full bytes. Without a recheck, N concurrent
+// such misses each reserve a few bytes, all pass the pre-serialize gate, and then
+// charging their true size drifts the in-flight total far past
+// cscMissWireBudgetBytes while every snapshot stays queued.
+//
+// Returns true (keep) when the actual size fits: the delta is charged and
+// req.reserved advanced to actual, so settle() releases exactly once. Returns
+// false (shed) when charging the actual size would exceed the budget: the delta is
+// rolled back, req.reserved is LEFT at the estimate (fetch's !enqueued defer
+// releases that), and the caller sheds to the pooled path — so the over-budget
+// wire is dropped, not retained in mc.ch/inflight. A shrinking delta (actual <
+// estimate) always keeps.
+func (mc *cscMissCoalescer) reconcileWireBytes(req *cscMissReq) bool {
+	actual := int64(len(req.wire))
+	delta := actual - req.reserved
+	if delta <= 0 {
+		if delta != 0 {
+			mc.wireBytes.Add(delta)
+			req.reserved = actual
+		}
+		return true
+	}
+	if mc.wireBytes.Add(delta) > cscMissWireBudgetBytes {
+		mc.wireBytes.Add(-delta) // roll back; reserved stays the estimate for the defer release
+		return false
+	}
+	req.reserved = actual
+	return true
+}
+
+// settle releases req's wire-budget reservation and wakes its caller, exactly
+// once. Every path that finishes a QUEUED request (applyAndSettle, settleErr, and
+// the shutdown drain) funnels through here, so the budget a request holds while its
+// wire sits in mc.ch/inflight is returned precisely when that wire is done. Requests
+// that never reach the queue release their reservation in fetch instead.
+func (mc *cscMissCoalescer) settle(req *cscMissReq, err error) {
+	mc.releaseWireBytes(req.reserved)
+	req.done <- err
+}
+
+// emitReplyErr records the native error metric for a settled REPLY error (e.g.
+// redis.Nil, WRONGTYPE) at the point the caller consumes it — the single
+// emission site for coalesced reply errors, giving parity with processWithRetry
+// on the uncoalesced path while staying exactly-once per operation. Excluded:
+// session errors and shed retries (processCached re-runs those through
+// processWithRetry, which emits its own outcome) and the cancellation branch (a
+// caller returning ctx.Err() already emitted its cancellation; emitting the late
+// reply too recorded two conflicting error types for one operation). Reading
+// req.servedBy is safe here: the caller's req.done receive is the
+// happens-before edge for that field.
+func (mc *cscMissCoalescer) emitReplyErr(ctx context.Context, req *cscMissReq, err error) {
+	if err == nil || err == errCSCRetryUncached {
+		return
+	}
+	var sessErr cscSessionError
+	if errors.As(err, &sessErr) {
+		return
+	}
+	errorCallback := pool.GetMetricErrorCallback()
+	if errorCallback == nil {
+		return
+	}
+	errorType, statusCode, isInternal := classifyCommandError(err)
+	errorCallback(ctx, errorType, req.servedBy, statusCode, isInternal, 0)
+}
+
+func (mc *cscMissCoalescer) fetch(ctx context.Context, cmd Cmder, cacheKey string, token uint64) (served *pool.Conn, err error) {
+	req := &cscMissReq{cmd: cmd, cacheKey: cacheKey, token: token, done: make(chan error, 1)}
+	// Bound total in-flight serialized bytes before allocating this command's wire
+	// snapshot: reserve its approximate encoded size and, if that would exceed
+	// cscMissWireBudgetBytes, shed to the ordinary pooled path (errCSCRetryUncached)
+	// rather than serialize-and-block — so a burst of large misses cannot exhaust
+	// memory with wire copies. Reserving BEFORE serialization means over-budget
+	// callers never allocate the wire. A single command larger than the whole budget
+	// sheds too (the pooled path runs it, so there is no progress hazard).
+	approxBytes := cmdApproxBytes(cmd)
+	if !mc.reserveWireBytes(approxBytes) {
+		mc.c.csc.Cancel(cacheKey, token)
+		return nil, errCSCRetryUncached
+	}
+	req.reserved = approxBytes
+	// Release the reservation HERE only if the request never gets queued (serialize
+	// error, or Close/ctx before enqueue). Once enqueued, ownership transfers to the
+	// request and settle() releases it when its wire actually leaves mc.ch/inflight —
+	// so a caller cancelling after enqueue does not free the budget while its wire
+	// snapshot is still retained (the OOM vector a caller-lifetime release allowed).
+	enqueued := false
+	defer func() {
+		if !enqueued {
+			mc.releaseWireBytes(req.reserved)
+			// Release the cache reservation too when the request never got queued,
+			// including the panic path (writeCmd serializing a user arg that panics),
+			// which skips the explicit Cancel on the error returns and would otherwise
+			// leave the key IN_PROGRESS until StaleTimeout, blocking later readers.
+			// Cancel is token-guarded, so this is a no-op next to the explicit Cancels.
+			mc.c.csc.Cancel(cacheKey, token)
+		}
+	}()
+	// Honor ContextTimeoutEnabled for the REPLY wait below (an enqueued request's
+	// socket I/O): when it is false the ordinary command path drives socket I/O on
+	// context.Background() (bounded by ReadTimeout, not the caller deadline), so a
+	// coalesced miss must not surface context.DeadlineExceeded on the caller's
+	// deadline. c.context returns the caller ctx when the policy is on and Background
+	// when off. The ENQUEUE-admission wait instead honors the original ctx directly
+	// (pre-I/O backpressure aborts on caller cancellation regardless of the policy —
+	// see that select). mc.stop stays the unconditional shutdown signal.
+	wctx := mc.c.context(ctx)
+	// Bound the number of callers serializing a wire snapshot AT ONCE before
+	// allocating this one: reserveWireBytes above gates on cmdApproxBytes, which
+	// undercounts variable-size args, so a concurrent burst can all pass it and each
+	// allocate a full multi-MiB wireBuf before reconcileWireBytes sheds — a transient
+	// set the byte budget never sees (#3965 F3). The slot is held ONLY across
+	// writeCmd+reconcile and released before the enqueue send below, so it never
+	// couples to queue backpressure. A caller that cannot get a slot sheds to the
+	// pooled path (the !enqueued defer releases the byte reservation).
+	if !mc.acquireSerialize() {
+		mc.c.csc.Cancel(cacheKey, token)
+		return nil, errCSCRetryUncached
+	}
+	// Release the slot even if writeCmd panics while serializing a user arg (a
+	// panicking Marshaler): without this the slot leaks and counts against
+	// cscMissMaxConcurrentSerialize forever, shedding later misses to the pooled
+	// path. Cleared after each explicit release below, so the normal path still
+	// releases before the enqueue select (the defer then no-ops).
+	serializeHeld := true
+	defer func() {
+		if serializeHeld {
+			mc.releaseSerialize()
+		}
+	}()
+	// Snapshot the wire form NOW, while the caller still owns cmd: the session
+	// writer writes these engine-owned bytes and never reads cmd again, so an
+	// abandoning caller can immediately reuse mutable args (e.g. a []byte key)
+	// without racing arg serialization, a mutated key can never be sent or
+	// published under the original cache key, and the fetch always completes —
+	// the reservation always settles. The reply side still claims cmd (apply
+	// interlock) before writing the result into it.
+	var wireBuf bytes.Buffer
+	if err := writeCmd(proto.NewWriter(&wireBuf), cmd); err != nil {
+		mc.releaseSerialize()
+		serializeHeld = false
+		mc.c.csc.Cancel(cacheKey, token)
+		return nil, err
+	}
+	req.wire = wireBuf.Bytes()
+	// Reconcile the reservation to the real serialized size now that the wire exists:
+	// cmdApproxBytes can undercount a large variable-size arg (see reconcileWireBytes).
+	// If the true size would push the in-flight total over budget, shed to the pooled
+	// path instead of enqueueing an over-budget snapshot (the !enqueued defer releases
+	// the estimate). Without this, concurrent undercounted misses all pass the
+	// pre-serialize gate and then blow the budget while every wire stays queued.
+	fits := mc.reconcileWireBytes(req)
+	// Release the serialization slot BEFORE the enqueue select: the snapshot exists
+	// and is reconciled, so the transient-allocation concern the slot bounds is over.
+	mc.releaseSerialize()
+	serializeHeld = false
+	if !fits {
+		mc.c.csc.Cancel(cacheKey, token)
+		return nil, errCSCRetryUncached
+	}
+	// Reject early if the coalescer is already stopping, so a send does not win the
+	// select race against a closed mc.stop and land in mc.ch after the shutdown
+	// drain, where nothing would pick it up. This narrows but cannot close that
+	// race; the mc.stop case in the wait select below is what guarantees the caller
+	// never hangs on a post-drain req.
+	select {
+	case <-mc.stop:
+		// Retry-uncached, not ErrClosed: the coalescer stopping does not mean
+		// the CLIENT is closed (teardown deactivates serving before stopping
+		// the coalescer, and a clone can race that window) — the uncached
+		// re-run either succeeds on the open pool or surfaces the real error.
+		mc.c.csc.Cancel(cacheKey, token)
+		return nil, errCSCRetryUncached
+	default:
+	}
+	select {
+	case mc.ch <- req:
+		enqueued = true
+	case <-mc.stop:
+		mc.c.csc.Cancel(cacheKey, token)
+		return nil, errCSCRetryUncached
+	case <-ctx.Done():
+		// Enqueue-admission backpressure is PRE-I/O (nothing written yet), so honor
+		// the caller's cancellation directly on ctx — NOT wctx. wctx follows
+		// ContextTimeoutEnabled because it gates SOCKET I/O (the reply-wait select
+		// below); a caller cancelling while merely waiting for a queue slot must abort
+		// regardless of that policy, matching the ordinary path passing ctx to
+		// ConnPool.Get.
+		mc.c.csc.Cancel(cacheKey, token)
+		return nil, ctx.Err()
+	}
+	// The reply wait honors caller cancellation in two phases. PRE-I/O (the batch has
+	// not reached a connection, req.sentConn == nil) a caller cancel aborts on the RAW
+	// ctx regardless of ContextTimeoutEnabled — matching the ordinary path, which passes
+	// ctx to ConnPool.Get and only drops to the policy ctx for socket I/O. Once I/O has
+	// begun (sentConn set) cancellation follows wctx (the policy): ctxDone is disabled so
+	// an enqueued command Redis is already serving is not abandoned early on a deadline
+	// the ordinary socket-I/O path would not enforce.
+	abandon := func() (*pool.Conn, error) {
+		// Report the cancellation metric here, like processWithRetry: the reader may
+		// still complete the background fetch (success via applyAndSettle, or its own
+		// error via settleErr), so neither records THIS caller's cancellation — without
+		// this it is undercounted whenever coalescing is on. Attribute to the SENT conn
+		// via req.sentConn (an atomic view safe on this pre-req.done branch, unlike
+		// servedBy, which would race the session's write): if the command was already
+		// sent, Redis received it, so report that conn; nil only for a true pre-send cancel.
+		sentConn := req.sentConn.Load()
+		if errorCallback := pool.GetMetricErrorCallback(); errorCallback != nil {
+			errorType, statusCode, isInternal := classifyCommandError(ctx.Err())
+			errorCallback(ctx, errorType, sentConn, statusCode, isInternal, 0)
+		}
+		// Lost the interlock: the reader claimed cmd and is mid-apply — wait (the receive
+		// is a happens-before edge) rather than race by reusing cmd, and return its
+		// result. Won it: the reader skips the cmd write; in the LIVE case it still
+		// publishes and settles the token, so leave it — except when stopping, where the
+		// req may be stranded past the shutdown drain (cancelIfStopping). Return ctx.Err()
+		// either way, matching processWithRetry.
+		if !req.claimAbandon() {
+			<-req.done
+			return req.servedBy, ctx.Err()
+		}
+		mc.cancelIfStopping(cacheKey, token)
+		return sentConn, ctx.Err()
+	}
+	ctxDone := ctx.Done()
+	for {
+		select {
+		case err := <-req.done:
+			mc.emitReplyErr(ctx, req, err)
+			return req.servedBy, err
+		case <-mc.stop:
+			// Close raced our enqueue: the req may have landed in mc.ch after the
+			// shutdown drain, so nothing will settle req.done. Winning the interlock
+			// means no reply is being applied — cancel the reservation and return instead
+			// of hanging (a duplicate Cancel, if the drain also got this req, is a no-op
+			// on a settled token).
+			if req.claimAbandon() {
+				mc.c.csc.Cancel(cacheKey, token)
+				// The req may be sitting in mc.ch AFTER the shutdown drain ran, where
+				// no worker will ever dequeue it — a live WithTimeout clone's pointer
+				// would then retain it (and its Cmder) indefinitely. Re-run the
+				// non-blocking drain to empty the queue; settling our own abandoned
+				// req is harmless (done is buffered, the duplicate Cancel is a no-op).
+				mc.drainQueueErr(errCSCRetryUncached)
+				return nil, errCSCRetryUncached
+			}
+			// The reader claimed cmd and is mid-apply: wait so we do not read/reuse
+			// cmd concurrently (the receive is a happens-before edge), then return
+			// the settle result itself — the reply may have been applied successfully.
+			e := <-req.done
+			mc.emitReplyErr(ctx, req, e)
+			return req.servedBy, e
+		case <-wctx.Done():
+			// Caller stopped waiting under ContextTimeoutEnabled (otherwise wctx is
+			// Background and never fires — matching the ordinary path's I/O timeout policy).
+			return abandon()
+		case <-ctxDone:
+			// Raw caller cancellation. Honor it only PRE-I/O (sentConn == nil); once the
+			// batch reached a conn, disable this case and fall through to the wctx policy,
+			// so a command Redis is already serving is not abandoned early when
+			// ContextTimeoutEnabled is off (wctx == Background there).
+			if req.sentConn.Load() != nil {
+				ctxDone = nil
+				continue
+			}
+			return abandon()
+		}
+	}
+}
+
+// cancelIfStopping releases an abandoned request's reservation, but ONLY when the
+// coalescer is stopping. On a caller-deadline abandon (fetch's wctx.Done branch,
+// after the caller won the cmd interlock) the live reader still dequeues the req
+// and publishes to the cache, so the token is left to it. Once mc.stop is closed
+// though, the req may be stranded in mc.ch past the shutdown drain with nothing to
+// settle it: its reservation would stay IN_PROGRESS and block a later reader of the
+// key until StaleTimeout (worse on a shared cache not flushed on close). So mirror
+// the mc.stop wait branch — cancel the token and re-drain the queue. The
+// non-blocking select keeps the live path's publish intact; Cancel is token-guarded,
+// so a later reader's fresh reservation is unaffected.
+func (mc *cscMissCoalescer) cancelIfStopping(cacheKey string, token uint64) {
+	select {
+	case <-mc.stop:
+		mc.c.csc.Cancel(cacheKey, token)
+		mc.drainQueueErr(errCSCRetryUncached)
+	default:
+	}
+}
+
+// applyAndSettle applies raw to the caller's command, publishes to the cache when
+// cacheable (under the reading connection's tracking generation), and wakes the
+// caller. Sends req.done exactly once.
+//
+// It claims the cmd interlock first: an abandoned caller (context cancelled, or
+// Close raced its enqueue) has its Cmder back and may be reading or reusing it,
+// so the reply must NOT be written there. The fetch is still not wasted — the
+// reply is classified from a throwaway parse and, when cacheable, published for
+// the next reader; only the caller's cmd write is skipped.
+func (mc *cscMissCoalescer) applyAndSettle(req *cscMissReq, raw []byte, connID, capturedGen uint64) {
+	// A user CacheSizer (via fulfillCached) or a custom Cmder's reply parse (via
+	// applyCachedReply) runs on the full-duplex reader goroutine, which has no other
+	// recover. A panic here would crash the process AND leave this caller blocked on
+	// req.done forever (the req is already off inflight, so the teardown drain can
+	// never reach it). Recover, then settleErr: it cancels the reservation, tags a
+	// cscSessionError so the caller re-runs on the pooled path, and wakes the caller.
+	// The normal settle below is the terminal statement, so a panic never double-settles.
+	defer func() {
+		if r := recover(); r != nil {
+			internal.Logger.Printf(context.Background(),
+				"csc: miss-coalesce apply/publish panic (recovered): %v", r)
+			mc.settleErr(req, fmt.Errorf("csc: apply panic: %v", r))
+		}
+	}()
+	c := mc.c
+	var applyErr error
+	if req.claimApply() {
+		applyErr = applyCachedReply(req.cmd, raw)
+	} else {
+		// Abandoned: classify without touching the caller's cmd.
+		applyErr = classifyCachedReply(raw)
+		mc.abandonedApplies.Add(1)
+	}
+	if isCacheableReplyResult(applyErr) {
+		fc := &cscFetchCapture{
+			raw:     raw,
+			connID:  connID,
+			initGen: capturedGen,
+			key:     req.cacheKey,
+			token:   req.token,
+		}
+		c.fulfillCached(req.cacheKey, req.token, fc)
+	} else {
+		// WRONGTYPE / NOPERM / ...: returned to the caller, not cached.
+		c.csc.Cancel(req.cacheKey, req.token)
+	}
+	// No error-metric emission here: the CALLER emits at its req.done receive
+	// (emitReplyErr), which makes emission exactly-once per operation. Emitting on
+	// this side too double-counted an op whose caller cancelled late — the caller
+	// recorded its context cancellation while this reply error was recorded here,
+	// two conflicting error types for one operation. An abandoned caller's op is
+	// accounted by its cancellation alone.
+	mc.settle(req, applyErr)
+}
+
+// cscMissStopDrainIntervals caps the whole Close-time (stopping) drain at N
+// stall-detection intervals. Without a total cap the per-interval progress check
+// renews forever, so a server trickling one reply per interval keeps a deep in-flight
+// pipeline (up to cscFullDuplexDepth) draining and blocks Close in wg.Wait for hours.
+// This is an explicit TRADEOFF against the unbounded-progress rule the non-stopping
+// drain keeps (see drainBackstop): a deep, legitimately slow-but-progressing drain
+// CAN exceed N intervals, and past that budget Close-never-wedges outranks completing
+// the remainder — some accepted in-flight replies may be cut. N × interval scales the
+// budget with the configured (and maintenance-relaxed) timeouts instead of a fixed
+// wall-clock; N=8 keeps a normal Close prompt while bounding the trickle case to tens
+// of seconds. Only the stopping path is capped; an age-out/handoff recycle stays
+// unbounded (the client is alive, only this conn's recycle waits).
+const cscMissStopDrainIntervals = 8
+
+// batchBudget bounds one coalesced batch's write + reads (connection acquisition
+// is bounded separately, by acquireCtx). It follows the client's configured
+// timeouts instead of a fixed cap: a client that deliberately sets
+// ReadTimeout/WriteTimeout high for slow cacheable reads must not see only its
+// coalesced misses cut off early (WithReader clamps the read to min(ctx deadline,
+// ReadTimeout), so a shorter ctx would win). The 5s floor covers scheduling
+// overhead when the configured timeouts are tiny or disabled, and also keeps the
+// drainBackstop stall interval (batchBudget + 1s) safely above the reader's per-read
+// bound so a single slow reply is never mistaken for a stall.
+func (mc *cscMissCoalescer) batchBudget() time.Duration {
+	opt := mc.c.opt
+	var budget time.Duration
+	if opt.WriteTimeout > 0 {
+		budget += opt.WriteTimeout
+	}
+	if opt.ReadTimeout > 0 {
+		budget += opt.ReadTimeout
+	}
+	if budget < 5*time.Second {
+		budget = 5 * time.Second
+	}
+	return budget
+}
+
+// countBatch records batch-size accounting. The max is a CAS loop: with
+// concurrent sessions a load-then-store could commit a smaller value over a
+// larger one that landed in between, permanently underreporting the maximum.
+func (mc *cscMissCoalescer) countBatch(batch []*cscMissReq) {
+	mc.batches.Add(1)
+	mc.batched.Add(uint64(len(batch)))
+	n := uint64(len(batch))
+	for {
+		cur := mc.maxBatchSz.Load()
+		if n <= cur || mc.maxBatchSz.CompareAndSwap(cur, n) {
+			return
+		}
+	}
+}
+
+// cscSessionError tags an error that comes from a coalescer session or transport
+// failure (settleErr), not from the command's own reply (applyAndSettle).
+// processCached uses this tag to tell the two apart. It re-runs a session failure
+// on the normal path, which applies MaxRetries and backoff. It returns a
+// reply-level result as-is (for example redis.Nil, WRONGTYPE, or a retryable reply
+// such as LOADING), because that result is the answer. cscSessionError unwraps, so
+// errors.Is and errors.As still see the cause.
+type cscSessionError struct{ err error }
+
+func (e cscSessionError) Error() string { return e.err.Error() }
+func (e cscSessionError) Unwrap() error { return e.err }
+
+// settleErr cancels the reservation and fails one waiting caller. It tags the
+// error as cscSessionError, except the retry-uncached sentinel, which
+// processCached matches by identity. The tag makes the caller re-run the read on
+// the normal, fully instrumented path instead of surfacing a raw transport
+// failure. That re-run emits the native error metric, so settleErr does not. To
+// emit it here as well would double-count a re-run that fails, and would wrongly
+// flag a re-run that succeeds. The retry-uncached path already used this rule.
+// Every session failure now re-runs, so the rule applies to all of them.
+func (mc *cscMissCoalescer) settleErr(req *cscMissReq, err error) {
+	mc.c.csc.Cancel(req.cacheKey, req.token)
+	if err != errCSCRetryUncached {
+		err = cscSessionError{err}
+	}
+	mc.settle(req, err)
+	mc.failed.Add(1)
+}
+
+// settleAllErr fails every request in batch from index `from` onward.
+func (mc *cscMissCoalescer) settleAllErr(batch []*cscMissReq, from int, err error) {
+	for i := from; i < len(batch); i++ {
+		mc.settleErr(batch[i], err)
+	}
+}
+
+// drainQueueErr fails every request currently queued (non-blocking). Used when a
+// connection cannot be acquired, so a caller on a context without a deadline is
+// not blocked forever and no reservation is left IN_PROGRESS.
+func (mc *cscMissCoalescer) drainQueueErr(err error) {
+	for {
+		select {
+		case req := <-mc.ch:
+			mc.settleErr(req, err)
+		default:
+			return
+		}
+	}
+}
+
+// settlePlain cancels the reservation and fails one waiting caller with err
+// UNTAGGED — no cscSessionError, no retry-uncached sentinel — so processCached
+// returns it as-is instead of re-running on the pooled path. Used for a
+// Limiter.Allow rejection at session acquire: a re-run would call Limiter.Allow a
+// SECOND time and a stateful/token limiter could admit the very operation it just
+// denied. The caller's req.done receive still emits the error metric once
+// (emitReplyErr), matching the ordinary path's single emission for a rejection.
+func (mc *cscMissCoalescer) settlePlain(req *cscMissReq, err error) {
+	mc.c.csc.Cancel(req.cacheKey, req.token)
+	mc.settle(req, err)
+	mc.failed.Add(1)
+}
+
+// grabInto appends first, plus any misses already queued, into dst. It does not
+// block. It stops at cscMissBatchMax commands or at mc.maxBatchBytes of serialized
+// payload. It reuses dst's backing array. first always goes, whatever its size,
+// because a single large command must not stall. The byte cap bounds only the
+// extra already-queued misses that share the write.
+//
+// grabInto returns the batch and a carry. The carry is the one request it pulled
+// that would push the batch past the byte cap. To pack that request on would
+// exceed the write buffer and risk the mid-batch-flush deadlock. grabInto does NOT
+// put the carry back on mc.ch; it returns it to the writer, which sends it as the
+// first request of the next batch. This keeps the request in the writer's own
+// state (never stranded on mc.ch during shutdown) and guarantees progress: a lone
+// large request ships next as its own batch. The carry is nil when nothing was
+// deferred.
+func (mc *cscMissCoalescer) grabInto(dst []*cscMissReq, first *cscMissReq) ([]*cscMissReq, *cscMissReq) {
+	dst = append(dst, first)
+	nbytes := len(first.wire)
+	for len(dst) < cscMissBatchMax {
+		select {
+		case more := <-mc.ch:
+			if nbytes+len(more.wire) > mc.maxBatchBytes {
+				return dst, more // defer: carry it to the next batch
+			}
+			dst = append(dst, more)
+			nbytes += len(more.wire)
+		default:
+			return dst, nil
+		}
+	}
+	return dst, nil
+}
+
+// Coalescer observability is the client's normal telemetry (the otel operation
+// and error callbacks fire for coalesced misses like any other command path).
+// The engine's internal counters (batched/batches/failed/maxBatchSz on
+// cscMissCoalescer) exist for in-package tests, which read them directly off
+// c.cscMissCoalescer — deliberately NOT exported as a stats API.

csc_miss_coalesce_modes.go

@@ -0,0 +1,741 @@
+package redis
+
+import (
+	"context"
+	"sync"
+	"sync/atomic"
+	"time"
+
+	"github.com/redis/go-redis/v9/internal"
+	"github.com/redis/go-redis/v9/internal/pool"
+	"github.com/redis/go-redis/v9/internal/proto"
+	"github.com/redis/go-redis/v9/push"
+)
+
+// The full-duplex miss-coalescer engine — THE engine behind
+// Options.ClientSideCacheCoalesceMisses (misses are caller-blocking, so the
+// latency-first streaming engine is the only one; see csc_miss_coalesce.go for
+// the removed alternatives).
+//
+// fullDuplexLoop runs sessions holding one tracked connection with concurrent
+// writer + reader goroutines: commands stream out while replies stream back, so
+// many commands are in flight on a single socket (the rueidis pipelining
+// model) — hides RTT with one connection instead of N.
+//
+// A session holds its connection outside the normal pool Get/Put cycle and
+// redials on error. Correctness note: a held connection spans reconnects, so on
+// ANY connection error every in-flight request is failed (token cancelled,
+// caller woken) rather than matched to a reply across tracking generations, and
+// a published reply is gated on the reading connection's id+generation still
+// matching the one captured when the command was written.
+//
+// (An earlier "pinned" engine — one held conn, serial half-duplex batches, NO
+// idle invalidation drain — was a benchmark-only prototype and has been
+// removed: it could serve stale after an invalidation while idle.)
+
+const cscModeBackoff = 5 * time.Millisecond
+
+func opCtx() (context.Context, context.CancelFunc) {
+	return context.WithTimeout(context.Background(), 5*time.Second)
+}
+
+// acquireCtx bounds a session connection acquisition. Unlike opCtx's fixed 5s
+// it honors the larger of the configured PoolTimeout (a saturated pool may
+// legitimately make Get wait that long) and the pool's full dial budget
+// (DialerRetries attempts of DialTimeout plus backoffs — a shorter outer
+// deadline would cancel Get mid-dial-sequence and forfeit configured retries).
+// Cancelled when the coalescer stops, so Close does not stall behind a blocked
+// Get.
+func (mc *cscMissCoalescer) acquireCtx() (context.Context, context.CancelFunc) {
+	opt := mc.c.opt
+	d := 5 * time.Second
+	if pt := opt.PoolTimeout; pt > d {
+		d = pt
+	}
+	// Options.init resolves the dial knobs to nonzero defaults. Use a DETERMINISTIC
+	// estimate that never invokes DialerRetryBackoff: calling a randomized/stateful
+	// backoff here (e.g. DialRetryBackoffExponential) consumes different jitter
+	// samples than the pool's actual retries, so the estimate can still be wrong —
+	// summing it out-of-band is not sound. A custom backoff whose real delays
+	// exceed this flat estimate should raise PoolTimeout; this deadline is only a
+	// floor anyway, since mc.stop also cancels the Get on Close.
+	if db := time.Duration(opt.DialerRetries)*(opt.DialTimeout+opt.DialerRetryTimeout) + opt.DialTimeout; db > d {
+		d = db
+	}
+	ctx, cancel := context.WithTimeout(context.Background(), d)
+	done := make(chan struct{})
+	go func() {
+		select {
+		case <-mc.stop:
+			cancel()
+		case <-done:
+		}
+	}()
+	return ctx, func() { close(done); cancel() }
+}
+
+// ---- full-duplex engine (feature-complete) ----
+
+// cscFullDuplexConnsDefault is how many independent full-duplex sessions (each
+// its own held connection) the fullduplex engine runs. 1 keeps the footprint
+// minimal (optimal at high concurrency). More would shard the miss +
+// invalidation-drain load across connections, cutting the low-concurrency p99
+// tail. Order-free: coalesced misses are independent per-key fetches.
+const cscFullDuplexConnsDefault = 1
+
+// cscFullDuplexIdleProbe is how often the reader drains server-initiated pushes
+// on the held connection while no reply is pending. A var (not const) so a test
+// can disable the idle drain (set it huge) as a negative control.
+var cscFullDuplexIdleProbe = 5 * time.Millisecond
+
+// cscFullDuplexSessionIdle: a session with no miss for this long ends and
+// returns its held connection (at PoolSize:1 an idle hold would block
+// non-cacheable commands until PoolTimeout). The next miss re-acquires; under
+// steady load the timer never fires. Var for tests.
+var cscFullDuplexSessionIdle = time.Second
+
+// fullDuplexRecycleAge bounds how long a session holds cn so OnPut pool hooks
+// (metrics, health, queued maintenance handoff) run periodically. Capped by the
+// conn's REMAINING lifetime (ExpiresAt, jitter included): an old conn must not
+// be held past the expiry the pool's reaper enforces.
+func (c *baseClient) fullDuplexRecycleAge(cn *pool.Conn) time.Duration {
+	age := 30 * time.Second
+	// Lifetime is honored via the conn's ACTUAL absolute expiry only —
+	// ExpiresAt already includes ConnMaxLifetimeJitter. Capping at the raw
+	// ConnMaxLifetime first would collapse every positive-jitter connection
+	// back to the unjittered lifetime, re-synchronizing recycles across
+	// clients started together (the herd the jitter option exists to prevent).
+	if exp := cn.ExpiresAt(); !exp.IsZero() {
+		if remain := time.Until(exp); remain < age {
+			age = remain
+		}
+	}
+	// A conn at (or past) expiry still serves the session it was just leased
+	// for — recycle promptly rather than with a zero/negative timer.
+	if age < 10*time.Millisecond {
+		age = 10 * time.Millisecond
+	}
+	return age
+}
+
+// fullDuplexLoop runs full-duplex sessions back to back: each session holds one
+// tracked connection until it errors, is recycled (handoff/close/age), or stop
+// is requested. Exits on stop.
+func (mc *cscMissCoalescer) fullDuplexLoop() {
+	defer mc.wg.Done()
+	for {
+		select {
+		case <-mc.stop:
+			return
+		default:
+		}
+		stopped, errored := mc.runFullDuplexSession()
+		if stopped {
+			return
+		}
+		// Back off only after an ERROR end, so a persistent dial failure does not
+		// hot-spin; a clean idle/recycle end continues immediately — the next
+		// session blocks on the miss queue anyway, and sleeping here would add a
+		// flat cscModeBackoff of latency to a miss already waiting in mc.ch.
+		if !errored {
+			continue
+		}
+		select {
+		case <-mc.stop:
+			return
+		case <-time.After(cscModeBackoff):
+		}
+	}
+}
+
+// runFullDuplexSession acquires one tracked connection and pipelines misses on
+// it with concurrent writer and reader goroutines. It ends when:
+//   - a socket read/write errors (fail path: fail in-flight, Close + Remove);
+//   - the connection is marked for handoff/close, the session ages out, or stop
+//     is requested (graceful path: drain in-flight, Put so OnPut runs pool hooks
+//     and any queued maintenance handoff).
+//
+// Returns true iff stop was requested (the loop should exit).
+func (mc *cscMissCoalescer) runFullDuplexSession() (stopped, backoff bool) {
+	c := mc.c
+
+	// Do not hold a pool connection while idle: wait for the first miss BEFORE
+	// acquiring. An eagerly-held session connection would, at a small pool
+	// (PoolSize:1), starve non-cacheable commands (PING/SET/uncached reads) until
+	// PoolTimeout while the session sat waiting for work. The pulled miss is
+	// written first by the writer below.
+	var first *cscMissReq
+	select {
+	case <-mc.stop:
+		return true, false
+	case first = <-mc.ch:
+	}
+
+	getCtx, getCancel := mc.acquireCtx()
+	cn, limited, err := c.getConnLimited(getCtx)
+	getCancel()
+	if err != nil {
+		// Distinguish a teardown cancellation from a genuine acquire failure by
+		// checking mc.stop DIRECTLY, not by err's value: getConn runs the dialer,
+		// credentials callbacks, and init hooks, any of which can return
+		// context.Canceled from their OWN context while the coalescer is NOT
+		// stopping. mc.stop is closed only by stopWorkers, and close(mc.stop) is
+		// ordered before acquireCtx cancels its context (which is ordered before
+		// getConn returns), so a real teardown is always visible here.
+		select {
+		case <-mc.stop:
+			// Tearing down: settle retry-uncached — like the other stop paths (fetch,
+			// the serving-disabled branch below, stopCSCMissCoalescer) — so processCached
+			// re-runs each read on the still-open pool instead of surfacing a spurious
+			// cancellation, and report stopped so the loop exits without backing off.
+			mc.settleErr(first, errCSCRetryUncached)
+			mc.drainQueueErr(errCSCRetryUncached)
+			return true, false // stop requested: exit the loop
+		default:
+		}
+		if limited {
+			// Limiter.Allow REJECTED admission for the session connection (before any
+			// dial): first was the ONE request that woke the session and triggered that
+			// Allow call. Surface the rejection to IT unchanged — settle without the
+			// cscSessionError tag so processCached returns it as-is instead of
+			// re-running on the pooled path, whose getConn would call Limiter.Allow a
+			// SECOND time and could admit (defeat) the denial the limiter just issued.
+			//
+			// The requests already queued behind first never called Allow themselves —
+			// they were just waiting for the session to write their command. Failing
+			// them with first's raw, non-retryable error would deny operations the
+			// limiter was never asked about. Settle them retry-uncached instead: each
+			// re-runs on the ordinary per-command path and calls Allow independently,
+			// which is what would have happened had they never been queued (cursor
+			// review #3989). Still an error end for the session loop (backoff also
+			// damps Allow frequency under sustained rejection).
+			mc.settlePlain(first, err)
+			mc.drainQueueErr(errCSCRetryUncached)
+			return false, true
+		}
+		// Genuine acquire failure (dial error, pool exhaustion, an unrelated
+		// context.Canceled from a custom dialer/creds/init hook). Fail the first miss
+		// plus everything queued so a caller on a deadline-less context is not blocked
+		// forever and no reservation leaks IN_PROGRESS, then back off and retry so the
+		// sole worker stays alive — requests arriving during backoff are failed on the
+		// next attempt's drain.
+		mc.settleErr(first, err)
+		mc.drainQueueErr(err)
+		return false, true // error end: the loop backs off before re-acquiring
+	}
+	// CSC serving may have been disabled after the miss was queued (e.g. a HELLO 3
+	// downgrade or CLIENT TRACKING rejected during initConn). Holding the conn to
+	// wait for more misses would be pointless — none get routed here once serving
+	// is off — and could tie up a small pool. Release it cleanly and fail pending.
+	if a := c.cscActive; a != nil && !a.Load() {
+		relCtx, relCancel := opCtx()
+		c.releaseConn(relCtx, cn, nil)
+		relCancel()
+		// CSC is off now, but the commands are fine — settle with the retry-uncached
+		// sentinel so processCached re-runs each on the normal path instead of
+		// surfacing a spurious pool.ErrClosed for a valid cacheable read.
+		mc.settleErr(first, errCSCRetryUncached)
+		mc.drainQueueErr(errCSCRetryUncached)
+		return false, false // clean end: no new misses route here while serving is off
+	}
+	connID := cn.GetID()
+	gen := c.cscConnInitGen(connID)
+
+	inflight := make(chan *cscMissReq, cscFullDuplexDepth)
+	// readsDone counts replies the reader has fully consumed. The drain
+	// backstop samples it as its progress signal: unlike len(inflight), it sees
+	// the ACTIVE read too (the reader pops a request before reading its reply,
+	// so a lone deep read under a maintenance-relaxed timeout would otherwise
+	// look like a zero-progress stall and get its socket closed mid-reply).
+	var readsDone atomic.Uint64
+	sctx, scancel := context.WithCancel(context.Background())
+	defer scancel()
+
+	var sessErr atomic.Value // error, set only on I/O failure
+	var failOnce sync.Once
+	fail := func(e error) {
+		failOnce.Do(func() {
+			if e == nil {
+				e = pool.ErrClosed
+			}
+			sessErr.Store(e)
+			scancel()
+		})
+	}
+	errored := func() bool { _, ok := sessErr.Load().(error); return ok }
+	reasonErr := func() error {
+		if e, ok := sessErr.Load().(error); ok {
+			return e
+		}
+		return pool.ErrClosed
+	}
+
+	recycle := make(chan struct{})
+	var recycleOnce sync.Once
+	doRecycle := func() { recycleOnce.Do(func() { close(recycle) }) }
+	var stopFlag atomic.Bool
+
+	// handoffWanted reports whether the connection should be returned to the pool
+	// so the OnPut hooks (including a queued maintenance handoff) can run.
+	handoffWanted := func() bool {
+		return !cn.IsUsable() || cn.ShouldHandoff() || cn.CloseOnPutReason() != ""
+	}
+
+	var swg sync.WaitGroup
+	swg.Add(2)
+
+	// Writer: pull a request (plus whatever else is queued), write the batch,
+	// then enqueue each written request to the reader in wire order. The writer
+	// is the SOLE closer of inflight and closes it as its last act on every exit
+	// path, which is how the reader learns the session drained.
+	go func() {
+		defer swg.Done()
+		defer close(inflight)
+		idleT := time.NewTimer(cscFullDuplexSessionIdle)
+		defer idleT.Stop()
+		buf := make([]*cscMissReq, 0, cscMissBatchMax)
+		pending := first // the miss that woke the session; write it before blocking
+		for {
+			var req *cscMissReq
+			if pending != nil {
+				// A carry from the previous batch. Still honor recycle/stop/cancel
+				// between batches even under a sustained carry stream, or a
+				// persistently over-budget producer would starve a maintenance handoff
+				// or lifetime recycle (the OnPut hooks would never run). Settle the
+				// carry on bail: it is the writer's own state, so the teardown drain
+				// (which covers only inflight) would otherwise leak it and hang its
+				// caller.
+				select {
+				case <-recycle:
+					mc.settleErr(pending, errCSCRetryUncached)
+					return
+				case <-mc.stop:
+					stopFlag.Store(true)
+					doRecycle()
+					mc.settleErr(pending, errCSCRetryUncached)
+					return
+				case <-sctx.Done():
+					mc.settleErr(pending, reasonErr())
+					return
+				default:
+				}
+				// grabInto below overwrites pending with the next carry, so do not
+				// clear it here (that store would be dead).
+				req = pending
+			} else {
+				if !idleT.Stop() {
+					select {
+					case <-idleT.C:
+					default:
+					}
+				}
+				idleT.Reset(cscFullDuplexSessionIdle)
+				select {
+				case <-recycle:
+					return
+				case <-mc.stop:
+					stopFlag.Store(true)
+					doRecycle()
+					return
+				case <-sctx.Done():
+					return
+				case <-idleT.C:
+					// No miss for the grace period: end the session cleanly so the
+					// held connection goes back to the pool instead of blocking
+					// non-cacheable traffic at a small pool until the recycle age.
+					// The reader drains anything still in flight (close(inflight)
+					// is the graceful no-more signal); the next miss starts a fresh
+					// session, which acquires only when work is in hand.
+					return
+				case req = <-mc.ch:
+				}
+			}
+			// grabInto may defer one over-budget request; carry it in pending so the
+			// next iteration writes it as the first request of its own batch. Held in
+			// the writer's own state, never put back on mc.ch, so a shutdown cannot
+			// strand it there. The error paths below settle it if this batch fails.
+			buf, pending = mc.grabInto(buf[:0], req)
+			mc.countBatch(buf)
+
+			// Attribute every request in this batch to the session connection up
+			// front. A write-side failure settles the batch before the reply path
+			// runs, so without this the error metric and processCached would report a
+			// nil connection and zero attempts for commands that did reach cn. buf is
+			// []*cscMissReq, so this persists to the reply path (which re-sets it).
+			for _, r := range buf {
+				r.servedBy = cn
+				r.sentConn.Store(cn) // atomic view for a caller abandoning before req.done
+			}
+
+			// Hand the batch to the reader BEFORE writing it, so the reader drains
+			// reply k while the writer is still writing and flushing k+1. If the
+			// reader started only after the whole batch was written (the old order),
+			// a batch larger than the transport send capacity could deadlock: the
+			// server blocks writing early replies that nobody reads while the writer
+			// blocks flushing later requests. With the reader draining concurrently,
+			// those early replies are consumed and the server keeps accepting the
+			// rest of the write.
+			//
+			// This send cannot deadlock the writer. It blocks only when inflight is
+			// full (cscFullDuplexDepth), and a batch is at most cscMissBatchMax
+			// (128 << 4096). The writer is one goroutine and runs sequentially, so
+			// every PRIOR batch was already flushed by its own WithWriter before this
+			// iteration. A full channel therefore holds at least
+			// cscFullDuplexDepth-cscMissBatchMax already-flushed requests, and the
+			// reader consumes FIFO, so its front is always a flushed request whose
+			// reply is on the wire: the reader advances, frees a slot, and the send
+			// proceeds. (Mirrors the autopipeline full-duplex engine, which likewise
+			// enqueues each batch before it writes.)
+			//
+			// A stalled or exited reader cannot strand the writer here either: every
+			// reader-exit path either cancels sctx (readOne/tick failure via fail, or
+			// the reader's own sctx.Done case) — so the select below escapes — or
+			// happens only after the writer already exited and closed inflight. Keep
+			// that true: a reader return added after doRecycle without cancelling sctx
+			// would break this escape.
+			sent := 0
+			for _, r := range buf {
+				select {
+				case inflight <- r:
+					sent++
+				case <-sctx.Done():
+					// Session cancelled mid-handoff. The reader and the teardown settle
+					// whatever reached inflight; the writer settles the rest and the
+					// carry (never in inflight).
+					mc.settleAllErr(buf, sent, reasonErr())
+					if pending != nil {
+						mc.settleErr(pending, reasonErr())
+					}
+					return
+				}
+			}
+
+			// sctx directly, NOT c.context(sctx): c.context() strips the engine's
+			// cancellable session ctx under ContextTimeoutEnabled=false, and with
+			// per-op timeouts disabled a blocked read/write could then never be
+			// interrupted (the supervisor's conn-close is the backstop).
+			werr := cn.WithWriter(sctx, c.opt.WriteTimeout, func(wr *proto.Writer) error {
+				for _, r := range buf {
+					// r.wire was snapshotted at enqueue while the caller owned
+					// cmd (see cscMissReq.wire): the writer never reads cmd, so
+					// an abandoned caller's args are never touched here, and an
+					// abandoned fetch still completes — its reply publishes to
+					// the cache and settles the reservation.
+					if _, e := wr.Write(r.wire); e != nil {
+						return e
+					}
+				}
+				return nil
+			})
+			if werr != nil {
+				fail(werr)
+				// The batch is already in inflight, so the reader (and the teardown
+				// drain) settle it. The writer only owns the carry, which never
+				// entered inflight.
+				if pending != nil {
+					mc.settleErr(pending, werr)
+				}
+				return
+			}
+		}
+	}()
+
+	// Reader: settle replies in the order the writer enqueued them (== wire
+	// order), and drain server-initiated pushes (invalidations, maintenance)
+	// even while idle. Publishes only while the connection's id+generation still
+	// match the session's.
+	go func() {
+		defer swg.Done()
+		ticker := time.NewTicker(cscFullDuplexIdleProbe)
+		defer ticker.Stop()
+
+		readOne := func(req *cscMissReq) bool { // false => fatal, reader must exit
+			// batchBudget (write + read), not bare ReadTimeout: the writer hands the batch
+			// to this reader BEFORE flushing it (deadlock avoidance), so a reply read can
+			// arm while its request is still being written — an oversized first command
+			// (allowed past the batch byte cap) or a slow/WAN link then charges write-flush
+			// time against a bare ReadTimeout and spuriously times out (and tears down) a
+			// session whose write is within WriteTimeout (#3989). batchBudget is exactly the
+			// batch's write+read bound; relaxation still raises it further via EffectiveReadTimeout.
+			rerr := cn.WithReader(sctx, mc.batchBudget(), func(rd *proto.Reader) error {
+				// Push handling with the nonblocking Close adapter: this reader
+				// is part of mc.wg, so a custom push handler calling Close() on
+				// the raw client would self-deadlock (Close waits on mc.wg while
+				// the reader is parked inside the handler). The adapter's Close
+				// signals and defers the blocking teardown to a goroutine.
+				// Drain server pushes before reading this request's reply, through the
+				// shared helper in BLOCKING mode: block on the socket and skip pushes
+				// until the reply is the next frame, so a second invalidation still on
+				// the socket ahead of the reply is not read by ReadRawReply as this
+				// request's reply. (The Buffered variant stopped at buffer-empty and let
+				// such a socket-pending push through, shifting the reply stream by one
+				// frame.) A custom processor is handed only a confirmed push frame (peek
+				// first). FATAL, not log-and-continue: a drain error means bytes may have
+				// been consumed mid-frame, so continuing into ReadRawReply on a
+				// desynchronized stream could apply a push fragment as this request's
+				// reply and publish it to the cache under the wrong key. Fail the session
+				// so the connection is removed.
+				if e := c.drainPushFrames(sctx, cn, rd, true); e != nil {
+					internal.Logger.Printf(sctx, "csc: miss-coalesce push drain: %v", e)
+					return e
+				}
+				raw, e := rd.ReadRawReply()
+				if e != nil {
+					return e
+				}
+				// Deliver the reply the caller is waiting for and let applyAndSettle
+				// gate only the CACHE PUBLISH on the connection's id/generation
+				// (fulfillCached checks the captured gen). Previously FD failed the
+				// caller with ErrClosed on a mid-flight id/gen change even though the
+				// reply was read fine, losing a good result (#3965).
+				req.servedBy = cn // before settle: done is the happens-before edge
+				mc.applyAndSettle(req, raw, connID, gen)
+				return nil
+			})
+			if rerr != nil {
+				fail(rerr)
+				req.servedBy = cn // attribute the failed read to the session conn
+				mc.settleErr(req, rerr)
+				return false
+			}
+			readsDone.Add(1)
+			return true
+		}
+
+		// handoff/close is a rare, non-latency-critical event, so it is checked off
+		// the per-reply hot path: on every idle tick, and once every readsPerHandoffCheck
+		// replies (~a few ms of traffic at any throughput).
+		const readsPerHandoffCheck = 128
+		reads := 0
+		for {
+			select {
+			case req, ok := <-inflight:
+				if !ok {
+					return // writer closed: session drained clean
+				}
+				if !readOne(req) {
+					return
+				}
+				reads++
+				if reads%readsPerHandoffCheck == 0 && handoffWanted() {
+					doRecycle()
+				}
+			case <-ticker.C:
+				// No reply was ready this tick: drain any server-initiated pushes
+				// (invalidations, MOVING, ...) unconditionally — the held connection
+				// is out of the pool, so the background drainer never visits it. Safe
+				// even if a reply is on the wire: the push processor peeks the reply
+				// type and consumes only push frames, leaving the reply for the next
+				// inflight read.
+				if e := c.peekAndProcessPushNotifications(sctx, cn); e != nil {
+					fail(e)
+					return
+				}
+				// Opaque-transport fallback, HELD-CONN ONLY: where readiness cannot
+				// be inspected (Windows; wrappers exposing neither syscall.Conn nor
+				// NetConn) the gated peek never fires and nothing else drains this
+				// conn. Throttled timed drain (no-op on inspectable transports).
+				// Deliberately not in peekAndProcessPushNotifications: pooled paths
+				// have drainer coverage, and a blanket fallback perturbs sequenced
+				// push handling.
+				// Built-in processor ONLY: this probe reads speculatively (no
+				// readiness signal exists on an opaque transport), and the
+				// built-in processor swallows the no-data boundary timeout. A
+				// CUSTOM processor's contract documents being invoked when
+				// notifications are KNOWN present — a natural implementation
+				// surfaces the empty-probe timeout, which would fail (and
+				// redial) a healthy session on every idle probe. Custom +
+				// opaque conns forgo the speculative drain; their invalidations
+				// land on session end/recycle at the latest.
+				if _, builtin := c.pushProcessor.(*push.Processor); builtin {
+					if cn.TakeCscPeriodicReadPending(cscFallbackProbeInterval) {
+						if e := c.timedPushDrain(sctx, cn); e != nil {
+							fail(e)
+							return
+						}
+					}
+				}
+				if handoffWanted() {
+					doRecycle()
+				}
+			case <-sctx.Done():
+				return
+			}
+		}
+	}()
+
+	// Supervisor: request a graceful recycle on stop or when the session ages
+	// out. A handoff/close request comes from the reader via doRecycle().
+	recycleTimer := time.NewTimer(c.fullDuplexRecycleAge(cn))
+	defer recycleTimer.Stop()
+	superDone := make(chan struct{})
+	supDead := make(chan struct{}) // closed when the supervisor exits (joined before release)
+	// drainBackstop bounds a graceful drain by PROGRESS, not wall clock: no
+	// context can interrupt a deadline-less socket read, so a stalled drain must
+	// force the I/O out with a conn close (else Close wedges in wg.Wait) — but a
+	// HEALTHY drain of a deep in-flight pipeline (up to cscFullDuplexDepth
+	// replies, ~in-flight × RTT, possibly under a maintenance-relaxed timeout)
+	// may legitimately exceed any fixed budget. Progress is COMPLETED READS
+	// (readsDone), not queue length: the reader pops a request before reading
+	// its reply, so len(inflight) is blind to the active read and would call a
+	// lone deep read a stall. Each budget interval that completed at least one
+	// reply extends the wait; only a zero-progress interval closes the conn.
+	drainBackstop := func(stopping bool) { mc.drainBackstopRun(cn, &readsDone, superDone, stopping) }
+	go func() {
+		defer close(supDead)
+		select {
+		case <-mc.stop:
+			stopFlag.Store(true)
+			doRecycle()
+			drainBackstop(true)
+		case <-recycleTimer.C:
+			doRecycle()
+			drainBackstop(false)
+		case <-recycle:
+			// Reader-triggered recycle (handoffWanted: unusable / handoff-marked /
+			// close-on-put). The writer exits, but replies may remain in flight —
+			// without a backstop a stalled read would hold the conn (and postpone
+			// the requested handoff) until Close. Same bounded drain as the timer
+			// path; doRecycle already ran (this channel IS the recycle signal).
+			drainBackstop(false)
+		case <-sctx.Done(): // I/O error: unblock a reader/writer parked on the socket
+			_ = cn.Close()
+		case <-superDone:
+		}
+	}()
+
+	swg.Wait()
+	close(superDone)
+	// JOIN the supervisor before releasing cn: the deferred scancel makes
+	// sctx.Done() ready, and a still-parked supervisor could pick it over the
+	// equally-ready superDone and close a healthy conn already back in the pool.
+	<-supDead
+
+	// Teardown. Error path: the socket was (or is being) closed; Remove it.
+	// Graceful path: the connection is clean (every written reply was read), so
+	// Put it — OnPut runs the pool hooks and any queued maintenance handoff, and
+	// the handoff/reinit path evicts this connection's now-uncovered CSC entries.
+	relCtx, relCancel := opCtx()
+	if errored() {
+		_ = cn.Close()
+		c.releaseConn(relCtx, cn, reasonErr())
+	} else {
+		c.releaseConn(relCtx, cn, nil)
+	}
+	relCancel()
+
+	// Fail anything the writer enqueued that the reader never consumed (only
+	// possible on the error path; the graceful path drains inflight fully). The
+	// writer always closes inflight on exit, so a receive here can report the
+	// channel closed (ok=false) — do not settle a nil request.
+	for {
+		select {
+		case r, ok := <-inflight:
+			if !ok {
+				return stopFlag.Load(), errored()
+			}
+			mc.settleErr(r, reasonErr())
+		default:
+			return stopFlag.Load(), errored()
+		}
+	}
+}
+
+// drainBackstop bounds a graceful drain of the full-duplex miss-coalescer session
+// by PROGRESS (completed reads, readsDone), not wall clock — see the block comment at
+// the call site and cscMissStopDrainIntervals for the stopping-path total cap.
+// Extracted from runFullDuplexSession so the stopping-drain budget is unit-testable.
+func (mc *cscMissCoalescer) drainBackstopRun(cn *pool.Conn, readsDone *atomic.Uint64, superDone <-chan struct{}, stopping bool) {
+	prev := readsDone.Load()
+	// On the stopping (Close) path the progress branch below must not renew forever:
+	// a server trickling one reply per interval keeps readsDone advancing, so with up
+	// to cscFullDuplexDepth in-flight, Close would block in wg.Wait for hours. Cap the
+	// whole stopping drain with a total deadline, armed ONCE (the first stopping loop),
+	// at cscMissStopDrainIntervals × interval. This is a TRADEOFF, not a free win: a
+	// deep, legitimately slow-but-progressing drain CAN exceed that budget (see the
+	// unbounded-progress rationale below), and past it Close-never-wedges outranks
+	// completing the remainder — some accepted in-flight replies may be cut. Non-stopping
+	// (age-out / handoff recycle) keeps the unbounded rule: the client stays alive, only
+	// this conn's recycle waits.
+	var stopDeadline time.Time
+	for {
+		// One blocked read may legitimately wait out its full reply deadline
+		// (readOne arms batchBudget, and a maintenance-relaxed timeout can exceed
+		// even that), so the zero-progress verdict must not fire before the reader's
+		// own deadline. Keep the interval strictly ABOVE batchBudget — the reader's
+		// per-read bound — so a single legitimately-slow reply is never mistaken for a
+		// stall and its connection force-closed. Recomputed per interval: an expired
+		// relaxation shrinks it back.
+		interval := mc.batchBudget() + time.Second
+		rt := cn.EffectiveReadTimeout(mc.c.opt.ReadTimeout)
+		wt := cn.EffectiveWriteTimeout(mc.c.opt.WriteTimeout)
+		if rt+time.Second > interval {
+			interval = rt + time.Second
+		}
+		if wt+time.Second > interval {
+			interval = wt + time.Second
+		}
+		// Arm the total stopping-drain deadline once (see cscMissStopDrainIntervals):
+		// N intervals from the first stopping loop, so a trickling server cannot renew
+		// the drain past a bounded budget on Close. mc.stopDrainBudget overrides the cap
+		// (tests only; zero in production).
+		if stopping && stopDeadline.IsZero() {
+			budget := cscMissStopDrainIntervals * interval
+			if mc.stopDrainBudget > 0 {
+				budget = mc.stopDrainBudget
+			}
+			stopDeadline = time.Now().Add(budget)
+		}
+		// Deadline-less I/O (no active relaxation): the user explicitly
+		// disabled the read and/or write deadline, so the RECYCLE path must
+		// not cut a session whose reader is in a legitimately long reply OR
+		// whose writer is blocked flushing a large request — readsDone
+		// cannot show progress before a reply exists, so a deadline-free
+		// blocked WRITE looks exactly like a stall. Wait for the session
+		// (or Close) instead. Close still terminates: once stop fires, the
+		// bounded zero-progress close below applies, which is the shutdown
+		// contract (Close never wedges). NOTE the <= 0: Options.init
+		// normalizes -1 (indefinite) to 0 and -2 (no deadline calls) to
+		// -1, so both deadline-less modes land at <= 0 here.
+		if !stopping && (rt <= 0 || wt <= 0) {
+			select {
+			case <-superDone:
+				return
+			case <-mc.stop:
+				stopping = true
+			}
+			continue
+		}
+		// Watch mc.stop on the non-stopping (recycle) path too: if Close arrives AFTER an
+		// age-out/handoff recycle drain has already started here, flip to the stopping path
+		// so the next loop arms the total cap. Without this, a recycle-then-Close race stays
+		// on this unbounded progress wait and a trickling server hangs Close in wg.Wait —
+		// the exact hang cscMissStopDrainIntervals was added to prevent. nil once stopping:
+		// mc.stop is then closed and would busy-spin the select.
+		var stopCh <-chan struct{}
+		if !stopping {
+			stopCh = mc.stop
+		}
+		select {
+		case <-superDone:
+			return
+		case <-stopCh:
+			stopping = true
+			continue
+		case <-time.After(interval):
+			cur := readsDone.Load()
+			if cur != prev {
+				prev = cur // still consuming replies; give it another interval
+				// ...unless the total stopping-drain budget is spent: Close-never-wedges
+				// outranks completing the remaining in-flight against a trickling server.
+				if !stopDeadline.IsZero() && !time.Now().Before(stopDeadline) {
+					_ = cn.Close()
+					return
+				}
+				continue
+			}
+			_ = cn.Close()
+			return
+		}
+	}
+}

csc_miss_coalesce_modes_test.go

@@ -0,0 +1,626 @@
+package redis
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"io"
+	"strconv"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/redis/go-redis/v9/internal/pool"
+	"github.com/redis/go-redis/v9/internal/proto"
+)
+
+// TestFullDuplexIdleInvalidation pins the feature-complete full-duplex engine's
+// idle push drain, WITH a negative control.
+//
+// The engine holds one connection out of the pool, so the background CSC drainer
+// never visits it; the session's reader must drain server-initiated pushes even
+// when no reply is pending, or an invalidation arriving while the coalescer is
+// idle is never processed and the entry serves stale until TTL.
+//
+//   - idle_drain_on_converges: with the drain at its default 5ms cadence, a key
+//     mutated from a second client while the coalescer is idle must converge —
+//     only possible if the idle reader processed the invalidation (a cache hit is
+//     served locally without touching the socket).
+//   - idle_drain_off_stays_stale: the NEGATIVE CONTROL. With the idle drain
+//     effectively disabled (probe set to ~1h), the same sequence must NOT
+//     converge. If it does, some other path drains the held connection and the
+//     positive case would prove nothing.
+func TestFullDuplexIdleInvalidation(t *testing.T) {
+	probe := NewClient(&Options{Addr: internalTestRedisAddr(), Protocol: 3})
+	if err := probe.Ping(context.Background()).Err(); err != nil {
+		probe.Close()
+		t.Skipf("no redis: %v", err)
+	}
+	probe.Close()
+
+	t.Run("idle_drain_on_converges", func(t *testing.T) {
+		if !fdIdleConverges(t, "fdidle:on") {
+			t.Fatal("with the idle drain ON, an invalidation on the held connection " +
+				"was never processed: the entry served stale")
+		}
+	})
+
+	t.Run("idle_drain_off_stays_stale", func(t *testing.T) {
+		old := cscFullDuplexIdleProbe
+		cscFullDuplexIdleProbe = time.Hour // idle drain effectively disabled
+		t.Cleanup(func() { cscFullDuplexIdleProbe = old })
+		if fdIdleConverges(t, "fdidle:off") {
+			t.Fatal("NEGATIVE CONTROL FAILED: with the idle drain disabled the key still " +
+				"converged, so something other than the idle reader drains the held " +
+				"connection — the positive case does not prove the idle-drain fix")
+		}
+	})
+}
+
+// fdIdleConverges warms a key through the full-duplex coalescer, lets the
+// coalescer go idle, mutates the key from a second client, and reports whether
+// the caching client converges to the new value within ~0.8s.
+func fdIdleConverges(t *testing.T, key string) bool {
+	t.Helper()
+	ctx := context.Background()
+
+	cached := NewClient(&Options{
+		Addr:                          internalTestRedisAddr(),
+		Protocol:                      3,
+		PoolSize:                      4,
+		ClientSideCacheConfig:         &ClientSideCacheConfig{MaxEntries: 1000},
+		ClientSideCacheCoalesceMisses: true,
+	})
+	seeder := NewClient(&Options{Addr: internalTestRedisAddr(), Protocol: 3})
+	defer cached.Close()
+	defer seeder.Close()
+	if err := cached.Ping(ctx).Err(); err != nil {
+		t.Fatalf("ping: %v", err)
+	}
+	if cached.cscMissCoalescer.Load() == nil {
+		t.Fatal("coalescer not active (config not honored?)")
+	}
+	t.Cleanup(func() { seeder.Del(context.Background(), key) })
+
+	if err := seeder.Set(ctx, key, "v1", 0).Err(); err != nil {
+		t.Fatal(err)
+	}
+	// Warm through the coalescer: the miss is fetched on the held connection and
+	// publishes the entry tracked on that connection.
+	if got, err := cached.Get(ctx, key).Result(); err != nil || got != "v1" {
+		t.Fatalf("warm read = %q, %v; want v1", got, err)
+	}
+	if got, err := cached.Get(ctx, key).Result(); err != nil || got != "v1" {
+		t.Fatalf("second read = %q, %v; want v1 (should be a local hit)", got, err)
+	}
+	if mcv := cached.cscMissCoalescer.Load(); mcv == nil || mcv.batches.Load() == 0 {
+		t.Fatal("coalescer never batched; the warm read did not route through it")
+	}
+
+	// Go idle, then invalidate from the second client. The push arrives on the
+	// held connection; only the session's reader can process it.
+	time.Sleep(60 * time.Millisecond)
+	if err := seeder.Set(ctx, key, "v2", 0).Err(); err != nil {
+		t.Fatal(err)
+	}
+
+	deadline := time.Now().Add(800 * time.Millisecond)
+	for time.Now().Before(deadline) {
+		if got, _ := cached.Get(ctx, key).Result(); got == "v2" {
+			return true
+		}
+		time.Sleep(10 * time.Millisecond)
+	}
+	return false
+}
+
+// TestFullDuplexSessionReleasesIdleConn pins #3965: a full-duplex session must
+// return its held connection after the idle grace instead of sitting on it until
+// the 30s recycle age — at PoolSize:1 a non-cacheable command (PING) would
+// otherwise block behind the idle session until PoolTimeout and fail.
+func TestFullDuplexSessionReleasesIdleConn(t *testing.T) {
+	ctx := context.Background()
+	probe := NewClient(&Options{Addr: internalTestRedisAddr(), Protocol: 3})
+	if err := probe.Ping(ctx).Err(); err != nil {
+		probe.Close()
+		t.Skipf("no redis: %v", err)
+	}
+	probe.Close()
+
+	old := cscFullDuplexSessionIdle
+	cscFullDuplexSessionIdle = 50 * time.Millisecond
+	t.Cleanup(func() { cscFullDuplexSessionIdle = old })
+
+	cached := NewClient(&Options{
+		Addr:                          internalTestRedisAddr(),
+		Protocol:                      3,
+		PoolSize:                      1, // the FD session's held conn is the ONLY conn
+		PoolTimeout:                   2 * time.Second,
+		ClientSideCacheConfig:         &ClientSideCacheConfig{MaxEntries: 100},
+		ClientSideCacheCoalesceMisses: true,
+	})
+	seeder := NewClient(&Options{Addr: internalTestRedisAddr(), Protocol: 3})
+	defer cached.Close()
+	defer seeder.Close()
+
+	key := "fdidlerel:k"
+	if err := seeder.Set(ctx, key, "v", 0).Err(); err != nil {
+		t.Fatal(err)
+	}
+	t.Cleanup(func() { seeder.Del(context.Background(), key) })
+	// Route a miss through the FD session so it acquires the pool's only conn.
+	if got, err := cached.Get(ctx, key).Result(); err != nil || got != "v" {
+		t.Fatalf("warm read = %q, %v", got, err)
+	}
+
+	// Past the grace the session must have released the conn; a non-cacheable
+	// PING then completes promptly. Without the release it blocks the full
+	// PoolTimeout and fails (the session held the conn toward the 30s recycle).
+	time.Sleep(200 * time.Millisecond)
+	start := time.Now()
+	if err := cached.Ping(ctx).Err(); err != nil {
+		t.Fatalf("PING after idle grace failed: %v — the idle FD session is still holding the pool's only conn", err)
+	}
+	if d := time.Since(start); d > time.Second {
+		t.Fatalf("PING took %v; want prompt (idle session should have released the conn)", d)
+	}
+}
+
+// TestClassifyCachedReply pins the abandoned-path reply classifier: it must agree
+// with isCacheableReplyResult on the error-vs-value axis (value and nil are
+// cacheable, a top-level RESP error is not) WITHOUT a caller command to populate.
+// This is what lets applyAndSettle still publish an abandoned request's reply to
+// the shared cache while skipping the caller's now-returned Cmder (#3965 :57).
+func TestClassifyCachedReply(t *testing.T) {
+	cases := []struct {
+		name      string
+		raw       string
+		cacheable bool
+	}{
+		{"bulk string", "$2\r\nOK\r\n", true},
+		{"integer", ":5\r\n", true},
+		{"resp2 nil", "$-1\r\n", true},
+		{"resp3 null", "_\r\n", true},
+		{"array", "*2\r\n$1\r\na\r\n$1\r\nb\r\n", true},
+		{"redis error", "-WRONGTYPE Operation against a key holding the wrong kind of value\r\n", false},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			got := isCacheableReplyResult(classifyCachedReply([]byte(tc.raw)))
+			if got != tc.cacheable {
+				t.Fatalf("classifyCachedReply(%q) cacheable = %v; want %v", tc.raw, got, tc.cacheable)
+			}
+		})
+	}
+}
+
+// TestCSCMissReqClaimInterlock pins the cmd-ownership CAS used by the abandoned
+// path (#3965 :57): exactly one of the caller (claimAbandon) and the
+// worker/reader (claimApply) may win from the pending state, so the worker never
+// writes the caller's Cmder after the caller has returned and may be reusing it.
+func TestCSCMissReqClaimInterlock(t *testing.T) {
+	// Caller wins: a later apply claim must lose, so the worker skips the write.
+	r := &cscMissReq{}
+	if !r.claimAbandon() {
+		t.Fatal("first claimAbandon must win from pending")
+	}
+	if r.claimApply() {
+		t.Fatal("claimApply must lose after the caller abandoned")
+	}
+	if r.claimAbandon() {
+		t.Fatal("claimAbandon must be idempotent-losing on repeat")
+	}
+
+	// Worker wins: a later abandon claim must lose, so the caller waits.
+	w := &cscMissReq{}
+	if !w.claimApply() {
+		t.Fatal("first claimApply must win from pending")
+	}
+	if w.claimAbandon() {
+		t.Fatal("claimAbandon must lose after the worker started applying")
+	}
+
+	// Concurrent: exactly one side wins.
+	for i := 0; i < 2000; i++ {
+		rr := &cscMissReq{}
+		var abandon, apply bool
+		var wg sync.WaitGroup
+		wg.Add(2)
+		go func() { defer wg.Done(); abandon = rr.claimAbandon() }()
+		go func() { defer wg.Done(); apply = rr.claimApply() }()
+		wg.Wait()
+		if abandon == apply {
+			t.Fatalf("iter %d: exactly one claim must win, got abandon=%v apply=%v", i, abandon, apply)
+		}
+	}
+}
+
+// TestCSCMissWireSnapshotImmuneToArgMutation pins the fix for the abandoned-
+// args race: the wire form is snapshotted at enqueue while the caller owns
+// cmd, so a caller that abandons and then mutates a []byte arg can neither
+// change what goes on the wire nor cause a reply for the mutated key to be
+// published under the original cache key.
+func TestCSCMissWireSnapshotImmuneToArgMutation(t *testing.T) {
+	key := []byte("orig-key")
+	cmd := NewStringCmd(context.Background(), "get", key)
+
+	var wireBuf bytes.Buffer
+	if err := writeCmd(proto.NewWriter(&wireBuf), cmd); err != nil {
+		t.Fatalf("snapshot encode: %v", err)
+	}
+	wire := wireBuf.Bytes()
+	if !bytes.Contains(wire, []byte("orig-key")) {
+		t.Fatalf("snapshot must contain the original key, got %q", wire)
+	}
+
+	copy(key, "MUTATED!") // caller reuses its buffer after abandoning
+
+	if !bytes.Contains(wire, []byte("orig-key")) || bytes.Contains(wire, []byte("MUTATED!")) {
+		t.Fatalf("wire snapshot changed after caller mutated its arg buffer: %q", wire)
+	}
+}
+
+// TestCSCMissCoalesceAbandonedFetchNoRace is a -race guard for #3965 :57: a caller
+// whose context cancels mid-fetch returns and reads its Cmder while the coalescer
+// may still be settling the reply. The claim interlock must keep the worker from
+// writing the caller's Cmder after it returned. Probabilistic (timing-dependent),
+// so its value is under `go test -race`; a regression that drops the interlock
+// trips the detector.
+func TestCSCMissCoalesceAbandonedFetchNoRace(t *testing.T) {
+	probe := NewClient(&Options{Addr: internalTestRedisAddr(), Protocol: 3})
+	if err := probe.Ping(context.Background()).Err(); err != nil {
+		probe.Close()
+		t.Skipf("no redis: %v", err)
+	}
+	probe.Close()
+
+	cached := NewClient(&Options{
+		Addr:                          internalTestRedisAddr(),
+		Protocol:                      3,
+		PoolSize:                      4,
+		ClientSideCacheConfig:         &ClientSideCacheConfig{MaxEntries: 100000},
+		ClientSideCacheCoalesceMisses: true,
+	})
+	seeder := NewClient(&Options{Addr: internalTestRedisAddr(), Protocol: 3})
+	defer cached.Close()
+	defer seeder.Close()
+
+	const N = 400
+	keys := make([]string, N)
+	for i := range keys {
+		keys[i] = "cscabandon:" + strconv.Itoa(i)
+		if err := seeder.Set(context.Background(), keys[i], "v", 0).Err(); err != nil {
+			t.Fatalf("seed: %v", err)
+		}
+	}
+	t.Cleanup(func() {
+		for _, k := range keys {
+			seeder.Del(context.Background(), k)
+		}
+	})
+
+	var wg sync.WaitGroup
+	for i, k := range keys {
+		wg.Add(1)
+		go func(k string, i int) {
+			defer wg.Done()
+			// Non-zero so ctx.Err() is nil at entry (the request reaches the
+			// coalescer) but small enough that many cancel while the fetch is in
+			// flight — the window the interlock guards.
+			d := time.Duration(40+(i%24)*20) * time.Microsecond
+			ctx, cancel := context.WithTimeout(context.Background(), d)
+			defer cancel()
+			cmd := cached.Get(ctx, k)
+			// Touch the cmd right after return; a worker writing it concurrently
+			// would be a data race.
+			_, _ = cmd.Result()
+		}(k, i)
+	}
+	wg.Wait()
+
+	// Guard against a silently-empty run: if no fetch was abandoned mid-flight the
+	// interlock window was never exercised, so a green -race proves nothing. Skip
+	// (don't pass) so the gap is visible. When it does land, this is the neutralize
+	// anchor — reverting claimApply to an unconditional write makes -race fire here.
+	if n := cached.cscMissCoalescer.Load().abandonedApplies.Load(); n == 0 {
+		t.Skip("timing window never landed: no fetch was abandoned mid-flight; " +
+			"interlock not exercised this run")
+	}
+}
+
+// TestFullDuplexDisabledMidMissRetriesUncached pins #3965 :188: when CSC serving
+// is disabled after a miss is reserved (a RESP3 downgrade / CLIENT TRACKING
+// rejection), the full-duplex session must NOT surface pool.ErrClosed for a valid
+// cacheable read. It settles the reserved miss with the retry-uncached sentinel
+// (so processCached re-runs it on the normal path) and releases the reservation.
+func TestFullDuplexDisabledMidMissRetriesUncached(t *testing.T) {
+	ctx := context.Background()
+	probe := NewClient(&Options{Addr: internalTestRedisAddr(), Protocol: 3})
+	if err := probe.Ping(ctx).Err(); err != nil {
+		probe.Close()
+		t.Skipf("no redis: %v", err)
+	}
+	probe.Close()
+
+	cached := NewClient(&Options{
+		Addr:                          internalTestRedisAddr(),
+		Protocol:                      3,
+		PoolSize:                      4,
+		ClientSideCacheConfig:         &ClientSideCacheConfig{MaxEntries: 1000},
+		ClientSideCacheCoalesceMisses: true,
+	})
+	seeder := NewClient(&Options{Addr: internalTestRedisAddr(), Protocol: 3})
+	defer cached.Close()
+	defer seeder.Close()
+	if err := cached.Ping(ctx).Err(); err != nil {
+		t.Skipf("no redis: %v", err)
+	}
+	if cached.cscMissCoalescer.Load() == nil {
+		t.Fatal("coalescer not active")
+	}
+
+	key := "fdmiss:disabled"
+	if err := seeder.Set(ctx, key, "hello", 0).Err(); err != nil {
+		t.Fatal(err)
+	}
+	t.Cleanup(func() { seeder.Del(context.Background(), key) })
+
+	// Reserve the miss exactly as processCached would (namespaced), then disable CSC
+	// serving before handing it to the session — so the session's pre-start check
+	// sees serving off.
+	nsKey := cscNamespacedKey(cached.cscKeyPrefix, key)
+	token, shouldFetch := cached.csc.Reserve(nsKey, []string{nsKey})
+	if !shouldFetch {
+		t.Fatal("expected to own the reservation")
+	}
+	// Capture the coalescer BEFORE disabling: disableCSCServing signals the drainer,
+	// whose teardown asynchronously swaps cscMissCoalescer to nil. A scheduler delay
+	// between the two would make a live cscMissCoalescer.Load() below return nil and
+	// nil-deref. The captured pointer still honors the serving-off flag fetch checks.
+	mc := cached.cscMissCoalescer.Load()
+	if mc == nil {
+		t.Fatal("coalescer not active")
+	}
+	cached.disableCSCServing(ctx, "test: force retry-uncached path")
+
+	cmd := NewStringCmd(ctx, "get", key)
+	if _, err := mc.fetch(ctx, cmd, nsKey, token); err != errCSCRetryUncached {
+		t.Fatalf("fetch after CSC disabled = %v; want errCSCRetryUncached (should not surface ErrClosed)", err)
+	}
+	// The reservation must be released, or later readers block IN_PROGRESS until
+	// StaleTimeout: a fresh Reserve must again own the fetch.
+	if _, again := cached.csc.Reserve(nsKey, []string{nsKey}); !again {
+		t.Fatal("reservation left IN_PROGRESS after retry-uncached settle")
+	}
+}
+
+// TestGrabIntoBoundsBatchToWriteBuffer pins the write-buffer bound on coalesced
+// batches (Ofek review #3989). A request that would push the batch past
+// maxBatchBytes is returned as the carry, not packed on. So bufio does not flush
+// in the middle of a batch while the reader waits, which could deadlock the write
+// and the read on large payloads. The first request always goes. The carry stays
+// in the writer's own state, not on mc.ch, so a shutdown cannot strand it.
+func TestGrabIntoBoundsBatchToWriteBuffer(t *testing.T) {
+	mc := &cscMissCoalescer{
+		ch:            make(chan *cscMissReq, 8),
+		maxBatchBytes: 100,
+	}
+	first := &cscMissReq{wire: make([]byte, 40)}
+	small := &cscMissReq{wire: make([]byte, 40)} // 40+40=80 <= 100: fits
+	big := &cscMissReq{wire: make([]byte, 80)}   // 80 more would overshoot 100: carried
+	mc.ch <- small
+	mc.ch <- big
+
+	batch, carry := mc.grabInto(nil, first)
+	if len(batch) != 2 || batch[0] != first || batch[1] != small {
+		t.Fatalf("batch = %d reqs, want [first, small] within the byte cap", len(batch))
+	}
+	if carry != big {
+		t.Fatalf("carry = %p, want the overshooting request %p", carry, big)
+	}
+	// The carry must not be put back on mc.ch.
+	select {
+	case <-mc.ch:
+		t.Fatal("overshooting request was put back on mc.ch instead of carried")
+	default:
+	}
+}
+
+// TestSettleErrTagsTransportFailureForRetry checks the retry path (Ofek review
+// #3989). settleErr tags a coalescer session or transport failure as
+// cscSessionError, so processCached re-runs it on the normal path with MaxRetries
+// instead of a raw io.EOF. The retry-uncached sentinel stays matchable by
+// identity. A command reply stays untagged and is returned as-is.
+func TestSettleErrTagsTransportFailureForRetry(t *testing.T) {
+	mc := &cscMissCoalescer{c: &baseClient{csc: NewLocalCache(CacheConfig{MaxEntries: 4})}}
+
+	// A transport failure is tagged and still unwraps to the cause.
+	req := &cscMissReq{cacheKey: "k", done: make(chan error, 1)}
+	mc.settleErr(req, io.EOF)
+	got := <-req.done
+	var se cscSessionError
+	if !errors.As(got, &se) {
+		t.Fatalf("settleErr(io.EOF) = %T, want cscSessionError so the caller re-runs", got)
+	}
+	if !errors.Is(got, io.EOF) {
+		t.Fatal("cscSessionError must unwrap to the underlying cause")
+	}
+
+	// The retry-uncached sentinel is matched by identity, not tagged.
+	req2 := &cscMissReq{cacheKey: "k", done: make(chan error, 1)}
+	mc.settleErr(req2, errCSCRetryUncached)
+	if got := <-req2.done; got != errCSCRetryUncached {
+		t.Fatalf("settleErr(sentinel) = %v, want errCSCRetryUncached unwrapped", got)
+	}
+
+	// A command reply (redis.Nil) applied by applyAndSettle is NOT a session error:
+	// processCached returns it as-is, keeping the no-per-command-retry tradeoff.
+	if errors.As(error(Nil), &se) {
+		t.Fatal("redis.Nil must not classify as a session error")
+	}
+}
+
+// TestGrabIntoAlwaysIncludesFirst: a lone first larger than the cap still goes
+// (latency-first), and anything queued behind it is carried.
+func TestGrabIntoAlwaysIncludesFirst(t *testing.T) {
+	mc := &cscMissCoalescer{
+		ch:            make(chan *cscMissReq, 8),
+		maxBatchBytes: 100,
+	}
+	first := &cscMissReq{wire: make([]byte, 200)} // alone exceeds the cap
+	more := &cscMissReq{wire: make([]byte, 10)}
+	mc.ch <- more
+
+	batch, carry := mc.grabInto(nil, first)
+	if len(batch) != 1 || batch[0] != first {
+		t.Fatalf("batch = %d reqs, want just the oversized first", len(batch))
+	}
+	if carry != more {
+		t.Fatalf("carry = %p, want the deferred request %p", carry, more)
+	}
+}
+
+// recordCloseConn records whether Close was called (drainBackstopRun force-closes the
+// conn when the stopping-drain budget is spent).
+type recordCloseConn struct {
+	mockNetConn
+	closed atomic.Bool
+}
+
+func (c *recordCloseConn) Close() error { c.closed.Store(true); return nil }
+
+// TestCSCMissStopDrainCapsTotalOnTrickle pins the P1 (csc_miss_coalesce_modes.go): on the
+// stopping (Close) path, drainBackstopRun must not renew the drain forever when the server
+// trickles one reply per interval. Without a total cap the per-interval progress check
+// renews indefinitely — with up to cscFullDuplexDepth in-flight, Close blocks in wg.Wait
+// for hours. With the cap it force-closes the conn and returns once the stopping budget is
+// spent, EVEN WHILE readsDone keeps advancing.
+//
+// stopDrainBudget is set below one interval so the FIRST timer (interval is floored at
+// batchBudget()+1s >= 6s) already sees the budget spent — exercising the cap in bounded
+// time instead of the ~48s (8 × interval) production default.
+//
+// Red-check: remove ONLY the stopDeadline ARMING in drainBackstopRun (leave the comparison)
+// — the trickle keeps progress alive, the drain never returns, and the watchdog fires.
+func TestCSCMissStopDrainCapsTotalOnTrickle(t *testing.T) {
+	nc := &recordCloseConn{mockNetConn: mockNetConn{addr: ":6379"}}
+	cn := pool.NewConn(nc)
+	mc := &cscMissCoalescer{
+		c:               &baseClient{opt: &Options{}}, // Read/WriteTimeout 0 -> batchBudget floor 5s -> interval 6s
+		stop:            make(chan struct{}),
+		stopDrainBudget: 50 * time.Millisecond, // below one interval: the first timer sees it spent
+	}
+
+	var readsDone atomic.Uint64
+	superDone := make(chan struct{}) // never closed: force the timer path, not a clean finish
+	trickleStop := make(chan struct{})
+	var wg sync.WaitGroup
+	wg.Add(1)
+	go func() {
+		defer wg.Done()
+		tk := time.NewTicker(200 * time.Millisecond)
+		defer tk.Stop()
+		for {
+			select {
+			case <-trickleStop:
+				return
+			case <-tk.C:
+				readsDone.Add(1) // steady progress that OUTLIVES the first interval
+			}
+		}
+	}()
+
+	done := make(chan struct{})
+	go func() {
+		mc.drainBackstopRun(cn, &readsDone, superDone, true)
+		close(done)
+	}()
+
+	select {
+	case <-done:
+	case <-time.After(20 * time.Second):
+		close(trickleStop)
+		wg.Wait()
+		t.Fatal("drainBackstopRun did not return: the stopping drain renewed forever despite the total cap (arming missing?)")
+	}
+	close(trickleStop)
+	wg.Wait()
+
+	if !nc.closed.Load() {
+		t.Fatal("connection was not force-closed when the stopping-drain budget was spent")
+	}
+	// Prove the CAP fired (progress branch), not the zero-progress branch: the trickle
+	// bumped across the first ~6s interval, so readsDone advanced before the drain returned.
+	// The zero-progress branch requires an interval with NO bump, which the trickle prevents.
+	if got := readsDone.Load(); got < 2 {
+		t.Fatalf("readsDone = %d; want >= 2 (the trickle must outlive the first interval so the CAP fires, not zero-progress)", got)
+	}
+}
+
+// TestCSCMissStopDrainCapsAfterCloseRaceInRecycle pins the cursor High follow-up
+// (r...fiab4): the cap must also bound a drain that STARTED as a non-stopping recycle
+// (age-out / handoff) and only then saw Close. Such a drain enters drainBackstopRun with
+// stopping=false; the main progress-wait must watch mc.stop, flip to stopping, and arm the
+// cap — otherwise a recycle-then-Close race stays on the unbounded path and a trickling
+// server hangs Close in wg.Wait.
+//
+// ReadTimeout/WriteTimeout are > 0 so the deadline-less branch (which already watched
+// mc.stop) is SKIPPED and the drain sits in the main timer select — the branch that lacked
+// the mc.stop watch. Red-check: remove the `case <-stopCh:` from the main select — the
+// close below is never observed, stopping never flips, the cap never arms, and the watchdog
+// fires.
+func TestCSCMissStopDrainCapsAfterCloseRaceInRecycle(t *testing.T) {
+	nc := &recordCloseConn{mockNetConn: mockNetConn{addr: ":6379"}}
+	cn := pool.NewConn(nc)
+	mc := &cscMissCoalescer{
+		// rt>0 && wt>0 (tiny) -> batchBudget floors to 5s (interval 6s) but the deadline-less
+		// branch is skipped, so the drain reaches the main timer select with stopping=false.
+		c:               &baseClient{opt: &Options{ReadTimeout: time.Millisecond, WriteTimeout: time.Millisecond}},
+		stop:            make(chan struct{}),
+		stopDrainBudget: 50 * time.Millisecond,
+	}
+
+	var readsDone atomic.Uint64
+	superDone := make(chan struct{})
+	trickleStop := make(chan struct{})
+	var wg sync.WaitGroup
+	wg.Add(1)
+	go func() {
+		defer wg.Done()
+		tk := time.NewTicker(200 * time.Millisecond)
+		defer tk.Stop()
+		for {
+			select {
+			case <-trickleStop:
+				return
+			case <-tk.C:
+				readsDone.Add(1)
+			}
+		}
+	}()
+
+	done := make(chan struct{})
+	go func() {
+		mc.drainBackstopRun(cn, &readsDone, superDone, false) // STARTS as a recycle drain
+		close(done)
+	}()
+
+	// Let the recycle drain settle into the main timer select, THEN Close races in.
+	time.Sleep(100 * time.Millisecond)
+	close(mc.stop)
+
+	select {
+	case <-done:
+	case <-time.After(20 * time.Second):
+		close(trickleStop)
+		wg.Wait()
+		t.Fatal("drainBackstopRun did not return after a recycle-then-Close race: the main progress wait did not observe mc.stop, so the cap never armed")
+	}
+	close(trickleStop)
+	wg.Wait()
+
+	if !nc.closed.Load() {
+		t.Fatal("connection was not force-closed after the recycle-then-Close race")
+	}
+}

csc_refresh_on_invalidate.go

@@ -0,0 +1,953 @@
+package redis
+
+import (
+	"context"
+	"sync"
+	"sync/atomic"
+	"time"
+
+	"github.com/redis/go-redis/v9/internal"
+	"github.com/redis/go-redis/v9/internal/pool"
+	"github.com/redis/go-redis/v9/internal/proto"
+)
+
+// Refresh-on-invalidate: re-fetch a key as soon as its invalidation arrives,
+// instead of dropping it and waiting for a reader to discover the miss.
+//
+// Why this is the lever worth pulling. The connection requirement of a CSC client
+// is roughly `miss_rate x throughput x RTT`, so the hit rate is the only quantity
+// that moves throughput and connection count in the same direction. Every other
+// item measured in this investigation trades one against the other. Under churn
+// the misses are exactly the invalidated keys, and they are known the moment the
+// push arrives — a whole round trip before any reader asks.
+//
+// # Why it re-inserts rather than refreshes in place
+//
+// The time-based sweeper (csc_revalidate.go) can use RefreshValue, which replaces
+// the value of a LIVE entry. This path cannot: the entry must be DELETED the
+// instant the invalidation lands, because leaving a known-stale value readable
+// while a refresh is in flight is the exact bug this whole line of work exists to
+// remove. So the entry is dropped first and re-inserted through the ordinary
+// Reserve/Fulfil path.
+//
+// Residual (not the same bug): because this republishes a value nobody asked for,
+// a SECOND write whose invalidation is still on the refresh connection's socket
+// behind the reply can be applied after the republish, so a reader briefly gets a
+// stale HIT where, without refresh, they'd have taken a fresh MISS. It self-heals
+// — the republish is on a CLIENT TRACKING conn, so that pending invalidation is
+// delivered and deletes the entry — and is bounded by the drain interval /
+// MaxStaleness. This is a narrower window than the "stale value readable for the
+// whole refresh RTT" case above, not a reintroduction of it.
+//
+// That has a useful consequence: Reserve single-flights. If a reader misses the
+// key while a background refresh is already in flight, it waits on that
+// reservation instead of issuing a second fetch — so the key is fetched once, by
+// one connection, no matter how many parties want it.
+//
+// # On fetching from the "owner" connection
+//
+// A reasonable worry is that refreshing on an arbitrary connection would register
+// the key on several connections at once, so one write would produce several
+// invalidations. It does not, because default (non-BCAST) tracking is ONE-SHOT:
+// when the server sends `invalidate k` it removes k from its tracking table, so
+// at the moment this path runs no connection is registered for k. The refresh
+// creates exactly one registration, whichever connection performs it, and Reserve
+// guarantees only one refresh runs.
+//
+// What connection choice does still affect is eviction blast radius: an entry is
+// attributed to the connection that fetched it (see EvictByConn), so refreshing
+// many entries onto one connection concentrates them there. Pinning the refresh to
+// the original owner would fix that, but the pool has no "give me connection N"
+// operation — withPipelineConn takes whichever is free. Noted as a follow-up; it
+// is a robustness question, not a correctness or invalidation-count one.
+
+// cscNoRefreshOnInvalidate force-disables the feature. The gate is
+// Options.ClientSideCacheRefreshOnInvalidate; this stays constant false.
+const cscNoRefreshOnInvalidate = false
+
+const (
+	// cscRefreshQueueDepth bounds the pending-refresh backlog. Bounded, and it
+	// DROPS rather than blocks: the producer is the invalidation drainer, and
+	// stalling that would delay the invalidations themselves — trading a hit-rate
+	// optimization for a correctness-critical path. A drop just means the key is
+	// refetched by whichever reader wants it next, which is today's behaviour.
+	cscRefreshQueueDepth = 4096
+
+	// cscRefreshTargetMaxBytes caps the key bytes (cacheKey + Redis keys) a single
+	// queued refresh target may retain. The depth cap alone bounds the queue by ITEM
+	// count, not memory: with large key encodings, up to cscRefreshQueueDepth targets
+	// could pin far more than the cache's own limit. A target over this cap is dropped
+	// rather than queued — like a full-queue drop, it just means a later reader does a
+	// normal miss fetch. Bounds the queue's retained key memory to roughly
+	// cscRefreshQueueDepth * this.
+	cscRefreshTargetMaxBytes = 4 << 10 // 4 KiB
+
+	// cscRefreshBatchMax caps how many keys ride one round trip.
+	cscRefreshBatchMax = 128
+
+	// cscRefreshReplyBudgetBytes caps one refresh round trip's EXPECTED REPLY
+	// volume (sum of the evicted values' sizes, cscRefreshTarget.valBytes). The
+	// key-byte budget bounds only what we write; replies for a chunk's early keys
+	// stream back while its later keys are still being written, and against a
+	// flow-controlling middlebox (a proxy or tunnel that stops reading when its
+	// buffer to us fills — vanilla Redis instead buffers replies in RAM) enough
+	// in-flight reply bytes stall the connection both ways: the classic pipeline
+	// write-read deadlock, which here surfaces as the per-chunk deadline failing
+	// the whole chunk. 32 KiB stays comfortably inside typical kernel socket
+	// buffering (~64K+64K), so expected replies alone cannot jam the path; a
+	// value that grew far past its evicted size since invalidation can still
+	// overshoot — the estimate narrows the window, the deadline stays the
+	// backstop. The first target of a chunk always goes regardless, so an
+	// oversized single value cannot stall the loop.
+	cscRefreshReplyBudgetBytes = 32 << 10
+
+	// cscRefreshBatchTimeout bounds one refresh round trip. It caps the batch ctx
+	// AND, when the user disabled per-op write timeouts, the batch write deadline —
+	// so a stalled deadline-less flush cannot wedge the refresher (the ctx does not
+	// interrupt a deadline-less socket write). The two uses must agree, so they
+	// share this constant.
+	cscRefreshBatchTimeout = 5 * time.Second
+
+	// cscRefreshRecencyTick is how often the "recently read" horizon advances.
+	cscRefreshRecencyTick = 200 * time.Millisecond
+
+	cscRefreshWarnEvery = 30 * time.Second
+)
+
+// cscRefreshWindow is the coalescing window. Invalidations are COLLECTED for this
+// long before the batch is refetched, instead of firing a round trip as each
+// trickle of pushes arrives. The window starts when the first key is collected
+// and does NOT slide on later arrivals — under continuous churn a sliding reset
+// would never fire and the design would degenerate to the size cap alone. The
+// window is flushed early by demand (a reader touching a collected key) or by the
+// size cap; whichever comes first. Longer window = fewer, larger round trips (the
+// point: it lets refresh work at a small pipeline pool) at the cost of staleness
+// bounded by the window for keys nobody reads. A var (not const) so a test can
+// take the timer out of the picture.
+var cscRefreshWindow = 500 * time.Millisecond
+
+// cscRefreshWindowMaxKeys flushes the window early once this many distinct keys
+// have collected, so a burst cannot outrun the window ahead of the queue's own
+// bounded backlog. A multiple of the per-round-trip cap.
+const cscRefreshWindowMaxKeys = 4 * cscRefreshBatchMax
+
+// cscDemandRefresh gates the demand trigger. On (default): a read for a key still
+// sitting in the window flushes the whole window immediately, so an actively-read
+// batch refreshes in ~one RTT instead of waiting out the window, while an idle
+// batch waits the full window (and nobody is reading it, so the wait is free).
+// Off: window + size cap only. On by default.
+const cscDemandRefresh = true
+
+// cscRefreshCooldown optionally suppresses a refresh for an entry published within
+// this window. DEFAULT 0, meaning off — measurement said to leave it off.
+//
+// The motivation was real. The same key read on several connections is registered
+// by the server once per connection, so ONE write produces one invalidation push
+// per registration, and that happens routinely: when a batched read finds another
+// fetch already in flight it declines the reservation and goes to the wire anyway
+// rather than stall the batch (the miss-coalescer path), registering the key a
+// second time. Measured: 34-38% of incoming invalidation pushes match no entry at
+// all, which is the signature of exactly this.
+//
+// But a time window is the wrong instrument, for two reasons.
+//
+// First, Reserve already dedups the part that costs anything. A duplicate
+// invalidation that lands while the refresh is still in flight finds the key
+// reserved, is declined, and is dropped for one map operation. Only a duplicate
+// arriving AFTER the refresh completed can start a second fetch — which needs the
+// refresh to be faster than the gap between duplicates, i.e. loopback, not WAN.
+//
+// Second, the window cannot tell a duplicate of the write it already handled from a
+// genuinely NEW write arriving a few milliseconds later, so it suppresses both. The
+// A/B showed the cost of that: at 20k invalidations/s, enabling it cut refresh work
+// 36% (289,822 -> 184,780) but LOST throughput (1,347,964 -> 1,326,234) and hit rate
+// (93.6% -> 93.0%). Doing the extra work was cheaper than skipping some of the
+// useful part.
+//
+// Left defaulted to 0 (off); measurement said to leave it off.
+//
+//nolint:unused // deliberately-off, kept as a documented measured knob (see above) for the follow-up that wires cooldown; referencing it now would misrepresent the shipped behavior.
+var cscRefreshCooldown time.Duration
+
+// cscRefreshTarget is one entry to re-fetch: the cache key doubles as the wire
+// form of the command that produced it, and redisKeys are already namespaced, so
+// both can be handed straight back to Reserve.
+type cscRefreshTarget struct {
+	cacheKey  string
+	redisKeys []string
+	token     uint64
+	// valBytes approximates the refetch reply size: the length of the value the
+	// invalidation just evicted (captured under the shard lock in
+	// collectHotAndDelete). The refresher budgets each round trip's EXPECTED
+	// REPLY bytes with it — the request side is tiny (keys), but replies for the
+	// chunk's early keys stream back while later keys are still being written,
+	// and unbounded reply volume is what jams a flow-controlling middlebox
+	// (proxy/tunnel) into the classic pipeline write-read deadlock. An estimate,
+	// not a guarantee: the value may have grown since eviction (often why it was
+	// invalidated), so this narrows the jam window rather than closing it.
+	valBytes int
+	// accessNs is the evicted entry's reader-access token (lastAccessNs), captured
+	// under the shard lock in collectHotAndDelete. The refresh republish restores it
+	// after fulfill so a background refresh does NOT count as a fresh reader access:
+	// fulfill stamps a new token, which would keep the key above the refresh horizon
+	// forever and make each invalidation refresh it again even after all readers stop
+	// (a self-sustaining refetch loop, contrary to the cold-key guard). Restoring the
+	// original token means only a real reader read (get, which re-stamps) keeps a key
+	// eligible for refresh. See restoreAccessToken.
+	accessNs int64
+}
+
+// cscRefreshQueue carries invalidated-but-hot keys from the drainer to the
+// refresher.
+type cscRefreshQueue struct {
+	ch chan cscRefreshTarget
+
+	// pendingSet mirrors the cache keys currently sitting in the refresher's
+	// collection window, so the read path (processCached, miss branch) can test
+	// membership without a lock and signal demand. Written and cleared only by
+	// the refresher goroutine; read concurrently by every reader — hence sync.Map.
+	pendingSet sync.Map
+	// demandCh carries a single "flush now" nudge from a reader that touched a
+	// pending key, STAMPED with the window generation it belongs to (see demandGen).
+	// Buffered depth 1 and sent non-blocking: one pending nudge is all the refresher
+	// needs, and the window timer is the backstop if it is dropped.
+	demandCh chan uint64
+	// demandGen tags the current collection window. The refresher bumps it every
+	// time it clears a window (in flush); signalDemand stamps the value it reads
+	// BEFORE the pendingSet membership check onto its nudge. A reader that passed the
+	// check just before a clear then sends AFTER it: its nudge carries the retired
+	// generation, so the refresher compares it unequal to the current one and ignores
+	// it — a stale nudge can no longer flush the NEXT window early, which would defeat
+	// the coalescing window and inflate DemandFlushes.
+	demandGen atomic.Uint64
+
+	// sinceToken is the "recently read" horizon: only entries whose last read is
+	// newer than this are worth refetching. Without it the refresher would chase
+	// cold keys, and because a refresh RE-REGISTERS the key with the server, that
+	// would both pin the server's tracking table and manufacture the next
+	// invalidation — a feedback loop that generates its own work. Same guard the
+	// time-based sweeper uses for the same reason.
+	//
+	// Holds cscInvalNoHorizon (-1) forever when
+	// Options.ClientSideCacheRefreshRecencyWindow is unset (the default): every
+	// Valid entry is then "hot" by construction, and the feedback-loop tradeoff
+	// above is an accepted, documented default (see the option's doc comment) —
+	// bound it with the option to restore this guard's original intent.
+	sinceToken atomic.Int64
+	// recency holds the multi-tick history backing sinceToken when
+	// ClientSideCacheRefreshRecencyWindow is set; nil in the default
+	// refresh-everything mode, where sinceToken is fixed and this is unused.
+	recency *cscRecencyRing
+
+	enqueued      atomic.Uint64
+	dropped       atomic.Uint64
+	refreshed     atomic.Uint64
+	refreshFailed atomic.Uint64
+	demandFlushes atomic.Uint64
+}
+
+// signalDemand nudges the refresher to flush its collection window now, but only
+// if cacheKey is actually sitting in that window — i.e. this read is a miss for a
+// key whose invalidation we already collected. Called on the read miss path, so
+// it must stay cheap: one sync.Map load and, on the rare hit, one non-blocking
+// send. The read is already paying a server round trip for its own key; the
+// nudge just warms the rest of the co-invalidated batch a window earlier.
+func (q *cscRefreshQueue) signalDemand(cacheKey string) {
+	if q == nil || !cscDemandRefresh {
+		return
+	}
+	// Read the generation BEFORE the membership check, so a window cleared between
+	// here and the send retires this stamp: the refresher then sees a stale
+	// generation and ignores the nudge (see demandGen). Reading it AFTER the check
+	// would stamp the NEW window and flush it early — the bug this guards.
+	gen := q.demandGen.Load()
+	if _, ok := q.pendingSet.Load(cacheKey); !ok {
+		return
+	}
+	select {
+	case q.demandCh <- gen:
+	default:
+	}
+}
+
+// demandIsCurrent reports whether a demand nudge stamped with gen still refers to
+// the window the refresher is collecting now. A nudge from a retired generation
+// (its window was already flushed) is ignored, so it cannot flush the next window
+// early. Extracted so the generation check is unit-tested (like cscRefreshChunkEnd).
+func (q *cscRefreshQueue) demandIsCurrent(gen uint64) bool {
+	return gen == q.demandGen.Load()
+}
+
+// offer enqueues without blocking, counting what it had to drop. A target whose
+// key bytes exceed cscRefreshTargetMaxBytes is dropped too, so the item-bounded
+// queue cannot pin unbounded key memory (a dropped target is refetched by the next
+// reader that wants it).
+func (q *cscRefreshQueue) offer(t cscRefreshTarget) {
+	if cscRefreshTargetBytes(t) > cscRefreshTargetMaxBytes {
+		q.dropped.Add(1)
+		return
+	}
+	select {
+	case q.ch <- t:
+		q.enqueued.Add(1)
+	default:
+		q.dropped.Add(1)
+	}
+}
+
+// cscRefreshTargetBytes approximates the key memory a queued target retains: its
+// cache key plus every Redis key string.
+func cscRefreshTargetBytes(t cscRefreshTarget) int {
+	n := len(t.cacheKey)
+	for _, k := range t.redisKeys {
+		n += len(k)
+	}
+	return n
+}
+
+// cscRefreshChunkEnd returns the exclusive end index of the next refresh round
+// trip starting at `start`: at most cscRefreshBatchMax targets, request bytes
+// (cache key sans prefix = wire form) within writeBudget, and expected reply
+// bytes (valBytes of the evicted values) within cscRefreshReplyBudgetBytes. The
+// first target always goes, so a single oversized key or value cannot stall the
+// loop. Pure; unit-tested like fdBatchEnd.
+func cscRefreshChunkEnd(targets []cscRefreshTarget, start, prefixLen, writeBudget int) int {
+	end, reqBytes, replyBytes := start, 0, 0
+	for end < len(targets) && end-start < cscRefreshBatchMax {
+		w := len(targets[end].cacheKey) - prefixLen
+		r := targets[end].valBytes
+		if end > start && (reqBytes+w > writeBudget || replyBytes+r > cscRefreshReplyBudgetBytes) {
+			break
+		}
+		reqBytes += w
+		replyBytes += r
+		end++
+	}
+	return end
+}
+
+// CSCRefreshStats reports refresh-on-invalidate activity: keys queued, keys
+// dropped because the backlog was full, and values actually republished.
+//
+// Experimental: this API may change in a minor release.
+type CSCRefreshStats struct {
+	Enqueued  uint64
+	Dropped   uint64
+	Refreshed uint64
+
+	// RefreshFailed counts refresh round trips that errored (a connection or
+	// protocol failure during the batch). Those keys stay evicted and a later read
+	// repopulates them, so a rising count means refresh is degrading to plain
+	// eviction. Counted per errored batch, not per key.
+	RefreshFailed uint64
+
+	// DemandFlushes counts collection windows flushed early because a reader
+	// missed a key still sitting in the window (vs flushed by the window timer or
+	// the size cap). High relative to total flushes means the demand trigger is
+	// doing its job — actively-read batches refresh in ~one RTT instead of
+	// waiting out the window.
+	DemandFlushes uint64
+
+	// Invalidations counts keys named in INCOMING invalidation pushes, tallied at
+	// the handler before dedup/batching. Deletions counts keys the cache actually
+	// processed for removal; under duplicate pushes Deletions < Invalidations
+	// because the batcher dedups (and, under an invalidation flood, the spill-cap
+	// full-Flush supersedes queued deletes wholesale — those count as invalidations
+	// but not deletions). So Invalidations - Deletions measures dedup + flood
+	// fallback, not dedup alone. DeletionsNoop counts applied deletes that matched
+	// no live entry, the direct duplicate-invalidation signature.
+	Invalidations uint64
+	Deletions     uint64
+	DeletionsNoop uint64
+}
+
+// CSCRefreshStats returns the client's refresh-on-invalidate counters.
+//
+// SCOPE on a SHARED cache: Invalidations/Deletions/DeletionsNoop are cache-global
+// (read from the shared LocalCache, so every client attached to it reports the
+// same totals), while Enqueued/Dropped/Refreshed/RefreshFailed/DemandFlushes come
+// from THIS client's refresh queue. With a shared cache+processor the active
+// refresh binding is the last-attached client's queue, so an earlier client can
+// report every invalidation but ~zero refresh work — not a defect, that client
+// genuinely is not the one refreshing. Treat a refresh/invalidation RATIO as
+// meaningful only for a single-client cache or for the client holding the active
+// binding; across a shared cache the two groups are different scopes.
+//
+// Experimental: this API may change in a minor release.
+func (c *Client) CSCRefreshStats() CSCRefreshStats {
+	var st CSCRefreshStats
+	// Cache-level counters (Invalidations, Deletions, DeletionsNoop) are recorded
+	// by the invalidation handler whenever CSC is on, even with
+	// ClientSideCacheRefreshOnInvalidate off (the default) and only batching
+	// enabled. Read them independently of the refresh queue, or a batch-only setup
+	// would always report zero for activity it clearly performs.
+	if lc, ok := c.baseClient.csc.(*LocalCache); ok {
+		st.Invalidations = lc.InvalidationStats()
+		st.Deletions, st.DeletionsNoop = lc.DeletionStats()
+	}
+	// The remaining counters exist only when the refresh queue is running.
+	if q := c.baseClient.cscRefreshQueue; q != nil {
+		st.Enqueued = q.enqueued.Load()
+		st.Dropped = q.dropped.Load()
+		st.Refreshed = q.refreshed.Load()
+		st.RefreshFailed = q.refreshFailed.Load()
+		st.DemandFlushes = q.demandFlushes.Load()
+	}
+	return st
+}
+
+// cscRefreshOnInvalidateEnabled reports whether this client should re-fetch
+// invalidated keys in the background.
+func (c *baseClient) cscRefreshOnInvalidateEnabled() bool {
+	if cscNoRefreshOnInvalidate || c.csc == nil || c.opt == nil {
+		return false
+	}
+	if !c.opt.ClientSideCacheRefreshOnInvalidate {
+		return false
+	}
+	// The collect step needs the concrete cache: a Cache implementation that only
+	// satisfies the interface cannot report which entries it removed.
+	_, ok := c.csc.(*LocalCache)
+	return ok
+}
+
+// startCSCRefresher launches the refresher goroutine and hands the drainer the
+// queue to feed.
+func (c *baseClient) startCSCRefresher() {
+	if !c.cscRefreshOnInvalidateEnabled() || c.cscRefreshQueue != nil {
+		return
+	}
+	// Cheap fast path: if CSC serving is already known inactive, skip building
+	// anything. Not the safety mechanism on its own — see the workersMu check
+	// below, which is what actually closes the construction/teardown race.
+	if a := c.cscActive; a != nil && !a.Load() {
+		return
+	}
+	lc := c.csc.(*LocalCache)
+	q := &cscRefreshQueue{
+		ch:       make(chan cscRefreshTarget, cscRefreshQueueDepth),
+		demandCh: make(chan uint64, 1),
+	}
+	// Default (window <= 0): refresh every invalidated Valid entry regardless of
+	// recency. cscInvalNoHorizon (-1) is less than any real lastAccessNs token,
+	// so it marks everything hot; sinceToken then never changes (see the
+	// recency-tick case in runCSCRefresher, which no-ops when q.recency is nil).
+	q.sinceToken.Store(cscInvalNoHorizon)
+	if w := c.opt.ClientSideCacheRefreshRecencyWindow; w > 0 {
+		q.recency = newCscRecencyRing(cscRefreshWindowTicks(w))
+		q.recency.push(lc.LRUClock())
+		q.sinceToken.Store(q.recency.oldest())
+	}
+
+	h := &cscRevalidateHandle{stop: make(chan struct{}), done: make(chan struct{})}
+
+	// publish attaches the queue to the invalidate handler (so pushes start
+	// feeding it, joining any detached predecessor batcher's stop-drain into it
+	// first) and launches the goroutine. Extracted so both the guarded and the
+	// no-drain-handle path below run the exact same sequence.
+	publish := func() {
+		c.cscRefreshQueue = q
+		c.cscRefreshHandle = h
+		// The invalidate handler is what sees the pushes, so it owns the producer
+		// end. Join the detached batcher (setRefreshQueue only detaches + signals)
+		// so the attach is synchronous — the drain has fully applied before the
+		// goroutine launches below.
+		if ih := lookupInvalidateHandler(c.opt.PushNotificationProcessor); ih != nil {
+			if b := ih.setRefreshQueue(q); b != nil {
+				b.join()
+			}
+		}
+		go c.runCSCRefresher(h, lc, q)
+	}
+
+	// Run publish under the drain handle's lock, checked against
+	// workersTornDown (see cscDrainHandle, stopCSCRefresherAndCoalescer): a
+	// concurrent teardown may already have consumed workersStopOnce —
+	// reachable via the drainer's own self-disable tick reacting to an async
+	// conn init's disableCSCServing, racing this construction (a bot review
+	// flagged this). That teardown must not be handed a handle it will never
+	// get another chance to stop, so decline to publish (and start nothing) if
+	// it already ran. The WHOLE of publish runs inside the lock, not just the
+	// field writes: teardown's body sets workersTornDown as its own first
+	// action under this same lock, so holding it across setRefreshQueue's
+	// batcher join and the goroutine launch guarantees teardown cannot observe
+	// a published handle/coalescer whose goroutine isn't fully running yet — a
+	// second bot review caught that the field-writes-only version still let
+	// stopCSCRefresher block on a <-h.done that nothing had started yet
+	// (bounded by however long the join took, not a permanent deadlock, but
+	// unnecessary and worth just closing). dh is nil only for tests that call
+	// this directly without a drain handle; there is nothing to race against in
+	// that case.
+	//
+	// Safety of holding workersMu this long: publish only calls internal,
+	// panic-free code (setRefreshQueue/join, a launch of our own
+	// runCSCRefresher) — lookupInvalidateHandler's GetHandler is a plain
+	// registry map lookup, never user code, because CSC requires the built-in
+	// push processor (a custom one is rejected at init). A panic here would
+	// leave workersMu held forever and deadlock every later teardown, so this
+	// only holds because publish cannot panic. Lock order while held:
+	// workersMu -> (invalidateHandler.mu acquired and released entirely inside
+	// setRefreshQueue) -> block on the detached batcher's done. Nothing on
+	// that path re-enters workersMu; keep it that way if either lock's scope
+	// grows.
+	if dh := c.cscDrainHandle; dh != nil {
+		dh.workersMu.Lock()
+		if !dh.workersTornDown {
+			publish()
+		}
+		dh.workersMu.Unlock()
+		return
+	}
+	publish()
+}
+
+func (c *baseClient) runCSCRefresher(h *cscRevalidateHandle, lc *LocalCache, q *cscRefreshQueue) {
+	defer close(h.done)
+
+	recency := time.NewTicker(cscRefreshRecencyTick)
+	defer recency.Stop()
+
+	// The collection window. It starts (timer armed) when the first key of a
+	// batch is collected and is NOT re-armed on later arrivals — a sliding reset
+	// would never fire under continuous churn, leaving only the size cap. Flushed
+	// by the timer, by demand, or by the size cap; whichever fires first.
+	window := time.NewTimer(cscRefreshWindow)
+	defer window.Stop()
+	if !window.Stop() {
+		<-window.C
+	}
+	windowArmed := false
+
+	// pending is keyed by cache key so a key invalidated twice within one window
+	// is refetched once. Owned solely by this goroutine.
+	pending := make(map[string]cscRefreshTarget, cscRefreshWindowMaxKeys)
+	var lastWarn time.Time
+
+	collect := func(t cscRefreshTarget) {
+		if _, dup := pending[t.cacheKey]; dup {
+			return
+		}
+		pending[t.cacheKey] = t
+		q.pendingSet.Store(t.cacheKey, struct{}{})
+		if !windowArmed {
+			windowArmed = true
+			window.Reset(cscRefreshWindow)
+		}
+	}
+
+	// flush applies the collected window. When stopping is true (Close's
+	// stop-drain) it skips the refetch below entirely instead of running it:
+	// stopCSCRefresherAndCoalescer's teardown (step 2 doc, invalidateAllCoverage,
+	// csc_integration.go:1305-1316) evicts every entry attributed to this
+	// client's refresh connection right after this call returns, regardless of
+	// whether the refetch would have succeeded — so running it first only spends
+	// network round trips on a result that is discarded a moment later. Against a
+	// healthy server that cost used to go unbounded: nothing here ever fails, so
+	// the previous "bail after the first failed chunk" bound (#3965 F2) never
+	// triggered, and a full backlog could cost thousands of round trips — one per
+	// chunk once the reply-byte budget shrinks each chunk to a single target
+	// (codex #3989 P1). The abandoned targets stay evicted; a reader repopulates
+	// them normally, the same outcome a failed refetch already produced. The
+	// normal (non-stopping) path is unchanged: it refetches every chunk, only
+	// aborting if Close begins while it's still running (see the h.stop check
+	// below).
+	flush := func(demand, stopping bool) {
+		if windowArmed {
+			windowArmed = false
+			if !window.Stop() {
+				// Drain a possibly-already-fired timer so the next Reset is clean.
+				select {
+				case <-window.C:
+				default:
+				}
+			}
+		}
+		if len(pending) == 0 {
+			return
+		}
+		if demand {
+			q.demandFlushes.Add(1)
+		}
+		// Snapshot to a slice and clear the collection state before any network
+		// I/O, so keys invalidated during the refetch start a fresh window rather
+		// than being lost or double-counted.
+		targets := make([]cscRefreshTarget, 0, len(pending))
+		for k, t := range pending {
+			targets = append(targets, t)
+			q.pendingSet.Delete(k)
+			delete(pending, k)
+		}
+		// Retire this window's generation: a demand nudge stamped with it (a reader
+		// that passed signalDemand's pendingSet check just before the clear above,
+		// then sends AFTER it) now compares unequal to the current generation and is
+		// ignored by the demandCh case — a stale nudge can no longer flush the NEXT
+		// window early, defeating the coalescing window and inflating DemandFlushes.
+		q.demandGen.Add(1)
+		// Then drop any nudge already buffered for THIS just-retired window: collect
+		// runs only on this goroutine, so nothing can have stamped the new generation
+		// yet, making a buffered value provably stale. Draining keeps the depth-1
+		// buffer clear for the next window's first legit nudge (the generation check
+		// above is the correctness guard; this stays buffer hygiene).
+		select {
+		case <-q.demandCh:
+		default:
+		}
+		if stopping {
+			// Abandoned by choice, not by network failure: leave RefreshFailed (one
+			// count per errored ROUND TRIP, per its doc) alone. These targets simply
+			// stay evicted, same as an errored refetch's targets already do.
+			return
+		}
+		// A window can hold more than one round trip's worth; chunk it. Keys that
+		// self-healed during the window (a reader missed and repopulated them) are
+		// now Valid, so Reserve inside refreshInvalidatedBatch declines them for
+		// free — no MGET slot is spent rewriting fresh data.
+		// Recover inside a closure so a panic in refreshInvalidatedBatch (cache/RESP/
+		// network path) does not kill the refresher goroutine — which would silently
+		// and permanently degrade refresh-on-invalidate to plain eviction for the
+		// client's lifetime. Mirrors the invalidation batcher's flush guard. The
+		// per-round-trip deadline is created PER CHUNK below, not once for the whole
+		// window, so a slow early chunk cannot expire later chunks' own budget.
+		func() {
+			defer func() {
+				if r := recover(); r != nil {
+					q.refreshFailed.Add(1)
+					internal.Logger.Printf(context.Background(),
+						"csc: refresh-on-invalidate batch panic (recovered): %v", r)
+				}
+			}()
+			// Chunk by count (cscRefreshBatchMax), by serialized REQUEST bytes (the
+			// write buffer), and by expected REPLY bytes (cscRefreshReplyBudgetBytes —
+			// see its doc: the reply side, not the request side, is what jams a
+			// flow-controlled path). Like the miss-coalescer, the refresh writes the
+			// whole chunk before it reads the replies. The first target in a chunk
+			// always goes, even if it alone exceeds a budget, so the loop always makes
+			// progress. Boundary logic extracted pure (cscRefreshChunkEnd) and
+			// unit-tested.
+			writeBudget := cscMissWriteBatchBytes(c.opt)
+			prefixLen := len(c.cscKeyPrefix)
+			for start := 0; start < len(targets); {
+				// Abort this (always non-stopping — see the stopping check above) flush
+				// if Close begins while it's still running: continuing to give each
+				// remaining chunk a fresh cscRefreshBatchTimeout against a dead/stalled
+				// server would delay Close by minutes (with reply budgeting producing
+				// one-target chunks, ~43 min for a full window). This covers a flush that
+				// was ENTERED before Close and is still running when h.stop closes (#3965
+				// F2 follow-up); the stop-drain's OWN flush call never reaches here at all
+				// now. Bailed targets stay evicted — a reader repopulates them and this
+				// client's coverage is revoked on Close anyway.
+				select {
+				case <-h.stop:
+					return
+				default:
+				}
+				end := cscRefreshChunkEnd(targets, start, prefixLen, writeBudget)
+				// Per-chunk deadline: each round trip gets its own cscRefreshBatchTimeout,
+				// so aggregate latency across chunks cannot expire the later chunks (#3989).
+				rctx, rcancel := context.WithTimeout(context.Background(), cscRefreshBatchTimeout)
+				n, err := c.refreshInvalidatedBatch(rctx, targets[start:end])
+				rcancel()
+				q.refreshed.Add(uint64(n))
+				if err != nil {
+					// One count per errored round trip: those keys were not refreshed and
+					// stay evicted (a reader repopulates them). This is the signal that
+					// refresh is degrading to plain eviction.
+					q.refreshFailed.Add(1)
+					if time.Since(lastWarn) > cscRefreshWarnEvery {
+						lastWarn = time.Now()
+						internal.Logger.Printf(context.Background(),
+							"csc: refresh-on-invalidate batch failed: %v", err)
+					}
+				}
+				start = end
+			}
+		}()
+	}
+
+	// drainQueue moves targets already waiting in q.ch into the window without
+	// blocking, up to the window cap. Returns true when q.ch was empty (nothing
+	// more to take), false when it stopped at the cap with the channel possibly
+	// still holding more.
+	drainQueue := func() (empty bool) {
+		for len(pending) < cscRefreshWindowMaxKeys {
+			select {
+			case more := <-q.ch:
+				collect(more)
+			default:
+				return true
+			}
+		}
+		return false
+	}
+
+	for {
+		select {
+		case <-h.stop:
+			// Pull everything buffered in q.ch (offered but not yet collected) and
+			// hand it to flush, which (stopping==true) abandons it without a refetch
+			// — see flush's doc: a refetch here would only be discarded by the
+			// teardown that runs right after this join anyway. stopCSCRefresher
+			// rebinds the handler away from this queue BEFORE signalling stop, so
+			// nothing new arrives here, making this loop finite. Loop until q.ch is
+			// fully drained rather than stopping after one window-cap-sized batch:
+			// c.cscRefreshQueue is never nilled on stop (see stopCSCRefresher), so a
+			// caller that keeps a reference to the closed *Client would otherwise
+			// keep every undrained target's key bytes reachable through it — up to
+			// cscRefreshQueueDepth entries at cscRefreshTargetMaxBytes each. Cheap
+			// now that flush does no network I/O: at most
+			// cscRefreshQueueDepth/cscRefreshWindowMaxKeys rounds.
+			for {
+				empty := drainQueue()
+				flush(false, true)
+				if empty {
+					return
+				}
+			}
+
+		case <-recency.C:
+			// Advance the horizon: entries not read within the configured window
+			// stop being worth refetching. Default (q.recency nil, window
+			// unset): sinceToken is fixed at cscInvalNoHorizon — refresh
+			// everything, forever — so there is nothing to advance.
+			if q.recency != nil {
+				q.recency.push(lc.LRUClock())
+				q.sinceToken.Store(q.recency.oldest())
+			}
+
+		case t := <-q.ch:
+			collect(t)
+			// Opportunistically take whatever else is already waiting so one wake
+			// drains the backlog into the window.
+			drainQueue()
+			if len(pending) >= cscRefreshWindowMaxKeys {
+				flush(false, false)
+			}
+
+		case g := <-q.demandCh:
+			// A reader missed a key still in the window: the batch is being used, so
+			// refetch it now instead of waiting out the window. Ignore a nudge stamped
+			// with a retired generation — its window was already flushed, and acting on
+			// it would flush the current (unrelated) window early (see signalDemand).
+			if q.demandIsCurrent(g) {
+				flush(true, false)
+			}
+
+		case <-window.C:
+			flush(false, false)
+		}
+	}
+}
+
+// stopCSCRefresher joins the refresher goroutine.
+func (c *baseClient) stopCSCRefresher() {
+	h := c.cscRefreshHandle
+	if h == nil {
+		return
+	}
+	c.cscRefreshHandle = nil
+	// Clear only OUR binding: a sibling client sharing this cache/processor may
+	// have re-attached its own queue after ours, and an unconditional nil here
+	// would sever the survivor's refresher (see clearRefreshQueue).
+	if ih := lookupInvalidateHandler(c.opt.PushNotificationProcessor); ih != nil {
+		// Join the detached batcher OUTSIDE h.mu (clearRefreshQueue only detaches +
+		// signals): the Close path needs a synchronous, no-straggler teardown, and
+		// joining under the handler lock would stall a sibling on the hot path.
+		if b := ih.clearRefreshQueue(c.cscRefreshQueue); b != nil {
+			b.join()
+		}
+	}
+	h.signalStop()
+	<-h.done
+}
+
+// cscRefreshReplyCacheable reports whether a refetched raw reply may be published
+// as a fresh cache entry. It classifies the FULL reply (like the coalescer's
+// classifyCachedReply), not raw[0]: a RESP3 attribute-prefixed error leads with
+// RespAttr ('|'), so a first-byte error check would treat an attributed error as
+// cacheable and publish it as a false success (bumping Refreshed; the next reader
+// would then have to evict and refetch). A nil reply IS cacheable (a negative
+// lookup, or a since-deleted key caching as missing); an empty reply never is.
+// Extracted pure so the classification is unit-tested (like cscRefreshChunkEnd).
+func cscRefreshReplyCacheable(raw []byte) bool {
+	return len(raw) > 0 && isCacheableReplyResult(classifyCachedReply(raw))
+}
+
+// refreshInvalidatedBatch re-fetches one chunk of invalidated keys in a single
+// round trip and publishes each reply as a fresh entry. Returns how many entries
+// it published.
+//
+// Every reservation this takes is either published or released before returning.
+// An abandoned reservation is worse than no refresh at all: LocalCache.Get WAITS
+// on an in-progress entry, so one orphan blocks every reader of that key until
+// StaleTimeout. (Measured elsewhere in this work: that mistake cost 30x and 8500x
+// throughput in the intra-batch rewrites.)
+func (c *baseClient) refreshInvalidatedBatch(ctx context.Context, targets []cscRefreshTarget) (int, error) {
+	prefix := c.cscKeyPrefix
+	if prefix == "" {
+		return 0, nil
+	}
+	if a := c.cscActive; a != nil && !a.Load() {
+		return 0, nil
+	}
+
+	// Reserve before touching the network, so only keys this batch owns are sent.
+	// A declined reservation means a reader is already fetching it — leave it to
+	// them rather than duplicating the work.
+	kept := make([]cscRefreshTarget, 0, len(targets))
+	// Arm the cancellation defer BEFORE the reservation loop. A user CacheSizer can
+	// panic inside Reserve; the outer refresher recovery keeps the worker alive, so
+	// without the defer already installed the targets reserved so far would stay
+	// IN_PROGRESS and block every reader of those keys until StaleTimeout (#3989). Any
+	// reservation still holding a token when this returns OR unwinds was never settled;
+	// release it rather than leave a placeholder readers would block on.
+	defer func() {
+		for i := range kept {
+			if kept[i].token != 0 {
+				c.csc.Cancel(kept[i].cacheKey, kept[i].token)
+			}
+		}
+	}()
+	for _, t := range targets {
+		token, shouldFetch := c.csc.Reserve(t.cacheKey, t.redisKeys)
+		// token==0 with shouldFetch==true is Reserve's "fetch uncached" signal
+		// (oversized entry / over-capacity / lost race), not an owned reservation.
+		// Skipping it avoids spending an MGET slot on a reply we'd discard and, more
+		// importantly, registering server-side tracking for a key we never cache
+		// (which would manufacture one spurious future invalidation).
+		if !shouldFetch || token == 0 {
+			continue
+		}
+		t.token = token
+		kept = append(kept, t)
+	}
+	if len(kept) == 0 {
+		return 0, nil
+	}
+
+	published := 0
+	// Publish on the MAIN pool (tracked). On this base the dedicated pipeline pool
+	// is deliberately EXCLUDED from CLIENT TRACKING (PR #3959), so a refetch
+	// published via withPipelineConn would be un-invalidatable and serve stale
+	// until TTL. withConn lands on a CLIENT TRACKING ON connection, same as the
+	// miss-coalescer.
+	err := c.withConn(ctx, func(ctx context.Context, cn *pool.Conn) error {
+		connID := cn.GetID()
+		// Coverage generation captured BEFORE the reads: if this conn loses
+		// tracking mid-batch, every reply is discarded rather than published, the
+		// same discipline fulfillCached and revalidateBatch use.
+		capturedGen := c.cscConnInitGen(connID)
+
+		// Pass the refresher's internally-bounded ctx (5s) DIRECTLY, not
+		// c.context(ctx): c.context applies the user's ContextTimeoutEnabled policy,
+		// which — when disabled (the default) — swaps our ctx for context.Background,
+		// stripping our own deadline. With ReadTimeout/WriteTimeout also disabled
+		// (-1/-2) a stalled Redis would then hang this reader forever, and Close
+		// (stopCSCRefresher waits on the refresher goroutine) would wedge. The
+		// miss-coalescer passes its session ctx directly for the same reason.
+		// Bound the write even when per-op timeouts are disabled. The refresh writes
+		// the whole chunk before it reads, so a deadline-less flush the server cannot
+		// drain (transport backpressure) would wedge the refresher: the 5s ctx does
+		// not interrupt a deadline-less socket write. A positive timeout makes
+		// WithWriter set a write deadline (still capped by the 5s ctx), so a stalled
+		// flush fails the refresh and the entries degrade to plain eviction
+		// (self-healing) instead of hanging the goroutine.
+		writeTimeout := c.opt.WriteTimeout
+		if writeTimeout <= 0 {
+			writeTimeout = cscRefreshBatchTimeout
+		}
+		if err := cn.WithWriter(ctx, writeTimeout, func(wr *proto.Writer) error {
+			for i := range kept {
+				// The cache key is the namespaced RESP encoding of the command that
+				// produced the entry; strip the namespace and it is already wire form.
+				if _, err := wr.Write([]byte(kept[i].cacheKey[len(prefix):])); err != nil {
+					return err
+				}
+			}
+			return nil
+		}); err != nil {
+			return err
+		}
+
+		// Bound the read the same way as the write above: with per-op timeouts
+		// disabled WithReader skips SetReadDeadline (options.go maps -2 to -1, which
+		// WithReader treats as no deadline), so a stalled reply or push drain would
+		// park refreshInvalidatedBatch forever — and stopCSCRefresher waits on this
+		// goroutine, so Client.Close would never return. A positive deadline (still
+		// capped by the 5s ctx) turns that into a timeout that fails the refresh.
+		readTimeout := c.opt.ReadTimeout
+		if readTimeout <= 0 {
+			readTimeout = cscRefreshBatchTimeout
+		}
+		return cn.WithReader(ctx, readTimeout, func(rd *proto.Reader) error {
+			for i := range kept {
+				// Invalidation pushes share this connection with replies, so drain
+				// them first or a push frame would be read as a value and cached.
+				// Use the nonblocking Close adapter (like the miss-coalescer and
+				// background drainer): this reader is part of the refresher's
+				// waitgroup, so a custom push handler calling Close() on the raw
+				// client would self-deadlock (Close waits on the very goroutine
+				// parked in the handler). And PROPAGATE a processor error instead of
+				// logging and continuing: a surfaced error means bytes may have been
+				// consumed mid-frame, so reading the next reply on a desynced stream
+				// could publish a push fragment under the wrong cache key — abort so
+				// withConn retires the connection.
+				if c.opt.Protocol == 3 && c.pushProcessor != nil {
+					// Route through the shared helper in BLOCKING mode: block on the
+					// socket and skip pushes until the refetch reply is the next frame, so
+					// a second invalidation still on the socket ahead of the reply is not
+					// read by ReadRawReply below and published under the wrong cache key.
+					// (The Buffered variant stopped at buffer-empty and let such a
+					// socket-pending push through.) A custom processor is handed only a
+					// confirmed push frame (peek first); PeekReplyType is attribute-aware,
+					// so a fragmented attribute prefix is handled without a separate
+					// buffered scan. The helper sets no read deadline, so this reader's
+					// ReadRawReply is unaffected.
+					if err := c.drainPushFrames(ctx, cn, rd, true); err != nil {
+						internal.Logger.Printf(ctx, "csc: refresh push drain: %v", err)
+						return err
+					}
+				}
+				raw, err := rd.ReadRawReply()
+				if err != nil {
+					// The reply stream is now out of step with kept; abort so the
+					// connection is retired rather than reused mid-batch. The deferred
+					// release cancels every reservation still outstanding.
+					return err
+				}
+				if !cscRefreshReplyCacheable(raw) {
+					// Not cacheable (WRONGTYPE after a type change, NOPERM, ...). A nil
+					// reply is NOT an error: a negative lookup is cacheable, and a key
+					// that has since been deleted should cache as missing. See
+					// cscRefreshReplyCacheable: it classifies the FULL frame, so a RESP3
+					// attribute-prefixed error is not mis-read as cacheable.
+					c.csc.Cancel(kept[i].cacheKey, kept[i].token)
+					kept[i].token = 0
+					continue
+				}
+				fc := &cscFetchCapture{
+					raw:     raw,
+					connID:  connID,
+					initGen: capturedGen,
+					key:     kept[i].cacheKey,
+					token:   kept[i].token,
+				}
+				if c.fulfillCached(kept[i].cacheKey, kept[i].token, fc) {
+					published++
+					// Undo the fresh access token fulfill stamped: a background refresh is
+					// not a reader read, so it must not renew the key's refresh eligibility,
+					// or each invalidation would keep refreshing it after all readers stop
+					// (see restoreAccessToken). Refresh runs only with the built-in
+					// *LocalCache; the hook path fulfills that same cache.
+					if lc, ok := c.csc.(*LocalCache); ok {
+						lc.restoreAccessToken(kept[i].cacheKey, kept[i].accessNs)
+					}
+				}
+				// fulfillCached cancels on its own failure paths, so the token is
+				// settled either way.
+				kept[i].token = 0
+			}
+			return nil
+		})
+	})
+	return published, err
+}

csc_refresh_on_invalidate_test.go

@@ -0,0 +1,546 @@
+package redis
+
+import (
+	"context"
+	"errors"
+	"strconv"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/redis/go-redis/v9/internal/pool"
+)
+
+// erroringPooler fails every Get, so refreshInvalidatedBatch (via withConn)
+// returns an error deterministically without a live server. gets counts calls,
+// so a test can assert the stop-drain never even attempts a round trip.
+type erroringPooler struct {
+	pool.Pooler
+	gets atomic.Int64
+}
+
+func (p *erroringPooler) Get(context.Context) (*pool.Conn, error) {
+	p.gets.Add(1)
+	return nil, errors.New("csc test: pool get always fails")
+}
+
+// TestCSCRefresherStopDrainSkipsRefetchEntirely pins #3989 P1: the Close
+// stop-drain must not spend ANY network round trip on the buffered backlog — a
+// refetch here is always discarded a moment later by the teardown that follows
+// (invalidateAllCoverage evicts every entry this connection would publish), so
+// attempting it only delays Close. This supersedes #3965 F2's weaker "bail
+// after the first failed chunk" bound, which never triggered against a HEALTHY
+// server: with several chunks queued and Get never even called, the drain must
+// abandon every target without touching the pool.
+func TestCSCRefresherStopDrainSkipsRefetchEntirely(t *testing.T) {
+	// No cscRefreshWindow mutation: runCSCRefresher runs synchronously below with
+	// stop already signalled, so the drain completes in microseconds against the
+	// failing pool — the default window/recency timers never fire, and mutating the
+	// shared global would risk a -race read against another test's live refresher.
+	lc := NewLocalCache(CacheConfig{MaxEntries: 10000})
+	pooler := &erroringPooler{}
+	c := &baseClient{
+		opt:          &Options{},
+		csc:          lc,
+		cscKeyPrefix: "p:",
+		connPool:     pooler,
+	}
+	q := &cscRefreshQueue{
+		ch:       make(chan cscRefreshTarget, 1024),
+		demandCh: make(chan uint64, 1),
+	}
+	q.sinceToken.Store(lc.LRUClock())
+	h := &cscRevalidateHandle{stop: make(chan struct{}), done: make(chan struct{})}
+
+	// More than one chunk's worth (cscRefreshBatchMax) but within one drainQueue
+	// round (< cscRefreshWindowMaxKeys) so the whole backlog is flushed at once; a
+	// regression that still refetches would then attempt multiple chunks.
+	const chunks = 3
+	const targets = chunks * cscRefreshBatchMax
+	for i := 0; i < targets; i++ {
+		key := "p:" + strconv.Itoa(i)
+		q.ch <- cscRefreshTarget{cacheKey: key, redisKeys: []string{key}}
+	}
+
+	// Signal stop up front, then run the refresher synchronously: it takes the
+	// stop-drain path, abandons the backlog, and returns.
+	close(h.stop)
+	c.runCSCRefresher(h, lc, q)
+
+	if got := pooler.gets.Load(); got != 0 {
+		t.Fatalf("pool.Get calls = %d; want 0 — the stop-drain must not attempt any "+
+			"refetch at all, not even the first chunk (#3989 P1)", got)
+	}
+	if got := q.refreshed.Load(); got != 0 {
+		t.Fatalf("refreshed = %d; want 0 — no refetch ran", got)
+	}
+	if got := q.refreshFailed.Load(); got != 0 {
+		t.Fatalf("refreshFailed = %d; want 0 — these targets were abandoned by choice, "+
+			"not by a failed round trip, so the round-trip-failure metric must stay "+
+			"untouched (its doc: \"counts refresh round trips that errored\")", got)
+	}
+}
+
+// stopClosingPooler closes stop on its first Get, then fails every Get, counting calls.
+// It drives a NORMAL (non-stopping) refresh flush that is interrupted by Close mid-loop.
+type stopClosingPooler struct {
+	pool.Pooler
+	stop chan struct{}
+	gets int
+}
+
+func (p *stopClosingPooler) Get(context.Context) (*pool.Conn, error) {
+	p.gets++
+	if p.gets == 1 {
+		close(p.stop) // Close arrives while this normal flush is already running
+	}
+	return nil, errors.New("csc test: pool get always fails")
+}
+
+// TestCSCRefresherNormalFlushAbortsOnClose pins r3937722481: a normal (non-stopping)
+// refresh flush already running when Close begins must abort BETWEEN chunks instead of
+// giving each remaining chunk a fresh cscRefreshBatchTimeout against a dead/stalled
+// server (which, with reply budgeting producing one-target chunks, could hold Close for
+// tens of minutes). A full window is 4 chunks; the first chunk's Get closes h.stop, so
+// the between-chunks check must bail the rest — exactly ONE chunk is attempted.
+//
+// Red-check: drop the `if !stopping { select <-h.stop }` guard in flush's chunk loop —
+// the normal flush ignores the now-closed stop and attempts all 4 chunks (gets == 4).
+func TestCSCRefresherNormalFlushAbortsOnClose(t *testing.T) {
+	lc := NewLocalCache(CacheConfig{MaxEntries: 100000})
+	h := &cscRevalidateHandle{stop: make(chan struct{}), done: make(chan struct{})}
+	p := &stopClosingPooler{stop: h.stop}
+	c := &baseClient{opt: &Options{}, csc: lc, cscKeyPrefix: "p:", connPool: p}
+	q := &cscRefreshQueue{
+		ch:       make(chan cscRefreshTarget, cscRefreshWindowMaxKeys+16),
+		demandCh: make(chan uint64, 1),
+	}
+	q.sinceToken.Store(lc.LRUClock())
+
+	// A full window (4 * cscRefreshBatchMax) triggers a NORMAL flush(false,false) from the
+	// q.ch case — this is NOT the stop-drain path (h.stop is not pre-closed).
+	for i := 0; i < cscRefreshWindowMaxKeys; i++ {
+		key := "p:" + strconv.Itoa(i)
+		q.ch <- cscRefreshTarget{cacheKey: key, redisKeys: []string{key}}
+	}
+
+	done := make(chan struct{})
+	go func() { defer close(done); c.runCSCRefresher(h, lc, q) }()
+
+	select {
+	case <-done:
+	case <-time.After(5 * time.Second):
+		t.Fatal("refresher did not converge — a normal flush kept refetching after Close began")
+	}
+
+	if p.gets != 1 {
+		t.Fatalf("pool Gets = %d; want 1 — a normal flush must abort mid-loop once Close begins "+
+			"(r3937722481), not attempt every chunk", p.gets)
+	}
+}
+
+// TestStartCSCRefresherRecencyWindowWiring pins the construction-time wiring
+// between Options.ClientSideCacheRefreshRecencyWindow and the live
+// cscRefreshQueue: default (window unset) must leave q.recency nil and pin
+// sinceToken at the unbounded sentinel (refresh everything); a configured
+// window must build a ring sized by cscRefreshWindowTicks and seed
+// sinceToken from it (a real token, not the sentinel). The ring's own
+// push/oldest arithmetic and the window->ticks rounding are separately
+// pinned (TestCscRecencyRingOldest, TestCscRefreshWindowTicks); this only
+// covers startCSCRefresher actually wiring them together.
+func TestStartCSCRefresherRecencyWindowWiring(t *testing.T) {
+	t.Run("default_is_unbounded", func(t *testing.T) {
+		lc := NewLocalCache(CacheConfig{MaxEntries: 1000})
+		c := &baseClient{
+			opt:          &Options{ClientSideCacheRefreshOnInvalidate: true},
+			csc:          lc,
+			cscKeyPrefix: "p:",
+		}
+		c.startCSCRefresher()
+		defer c.stopCSCRefresher()
+
+		q := c.cscRefreshQueue
+		if q == nil {
+			t.Fatal("startCSCRefresher did not build a refresh queue")
+		}
+		if q.recency != nil {
+			t.Fatal("default (window unset) built a recency ring; want nil (unbounded mode)")
+		}
+		if got := q.sinceToken.Load(); got != cscInvalNoHorizon {
+			t.Fatalf("default sinceToken = %d, want cscInvalNoHorizon (%d)", got, cscInvalNoHorizon)
+		}
+	})
+
+	t.Run("configured_window_builds_ring", func(t *testing.T) {
+		lc := NewLocalCache(CacheConfig{MaxEntries: 1000})
+		c := &baseClient{
+			opt: &Options{
+				ClientSideCacheRefreshOnInvalidate:  true,
+				ClientSideCacheRefreshRecencyWindow: 2 * cscRefreshRecencyTick,
+			},
+			csc:          lc,
+			cscKeyPrefix: "p:",
+		}
+		c.startCSCRefresher()
+		defer c.stopCSCRefresher()
+
+		q := c.cscRefreshQueue
+		if q == nil {
+			t.Fatal("startCSCRefresher did not build a refresh queue")
+		}
+		if q.recency == nil {
+			t.Fatal("configured window left q.recency nil; want a ring")
+		}
+		if got, want := len(q.recency.buf), cscRefreshWindowTicks(2*cscRefreshRecencyTick); got != want {
+			t.Fatalf("ring size = %d, want %d (cscRefreshWindowTicks(2*tick))", got, want)
+		}
+		if got := q.sinceToken.Load(); got == cscInvalNoHorizon {
+			t.Fatal("configured window initialized sinceToken to the unbounded sentinel")
+		}
+	})
+}
+
+// TestCloneSharesRefreshQueueForDemand pins #3965 F4: clone() (WithTimeout/
+// WithContext) must SHARE the owner's refresh queue so a derived client's
+// processCached can signal demand on it. Without the share the clone's field is
+// nil and signalDemand no-ops, so a clone's miss waits the full window instead of
+// nudging the owner's in-window batch. Lifecycle stays owner-only: the clone only
+// signals (nil-safe, non-blocking) and never stops the shared refresher.
+func TestCloneSharesRefreshQueueForDemand(t *testing.T) {
+	parent := &baseClient{
+		opt:             &Options{},
+		cscRefreshQueue: &cscRefreshQueue{demandCh: make(chan uint64, 1)},
+	}
+	// A key sitting in the refresher's window: signalDemand nudges only for these.
+	parent.cscRefreshQueue.pendingSet.Store("ck:x", struct{}{})
+
+	clone := parent.clone()
+	if clone.cscRefreshQueue != parent.cscRefreshQueue {
+		t.Fatal("clone did not share the owner's refresh queue; a clone's signalDemand " +
+			"would no-op on a nil queue (#3965 F4)")
+	}
+
+	// A demand nudge through the clone's pointer must reach the SHARED queue.
+	clone.cscRefreshQueue.signalDemand("ck:x")
+	select {
+	case <-parent.cscRefreshQueue.demandCh:
+		// The clone's signal landed on the owner's window.
+	default:
+		t.Fatal("clone signalDemand did not reach the shared queue's demand channel (#3965 F4)")
+	}
+}
+
+func TestCSCRefreshOnInvalidateTurnsAMissIntoAHit(t *testing.T) {
+	for _, enabled := range []bool{true, false} {
+		name := "enabled_next_read_is_a_hit"
+		if !enabled {
+			name = "disabled_next_read_is_a_miss"
+		}
+		t.Run(name, func(t *testing.T) {
+			ctx := context.Background()
+			cached := NewClient(&Options{
+				Addr:     internalTestRedisAddr(),
+				Protocol: 3,
+				// Default DrainInterval: the invalidation must actually arrive.
+				ClientSideCacheConfig:              &ClientSideCacheConfig{MaxEntries: 1000},
+				ClientSideCacheRefreshOnInvalidate: enabled,
+			})
+			seeder := NewClient(&Options{Addr: internalTestRedisAddr(), Protocol: 3})
+			if err := cached.Ping(ctx).Err(); err != nil {
+				cached.Close()
+				seeder.Close()
+				t.Skipf("no redis: %v", err)
+			}
+			defer cached.Close()
+			defer seeder.Close()
+
+			key := "cscrefresh:" + name
+			t.Cleanup(func() { seeder.Del(context.Background(), key) })
+
+			// Retry loop: the "recently read" horizon advances on a ticker, so a read
+			// can fall just the wrong side of it. Each attempt re-reads immediately
+			// before the write to put the entry firmly inside the window.
+			var observed bool
+			for attempt := 0; attempt < 12 && !observed; attempt++ {
+				if err := seeder.Set(ctx, key, "v1", 0).Err(); err != nil {
+					t.Fatal(err)
+				}
+				// Warm it, and mark it recently read.
+				if got, err := cached.Get(ctx, key).Result(); err != nil || got != "v1" {
+					t.Fatalf("warm read = %q, %v; want v1", got, err)
+				}
+
+				beforeRefresh := cached.CSCRefreshStats().Refreshed
+				if err := seeder.Set(ctx, key, "v2", 0).Err(); err != nil {
+					t.Fatal(err)
+				}
+
+				if !enabled {
+					// Give the drainer time to apply the invalidation, then confirm the
+					// read is a MISS: this is the behaviour the feature improves on.
+					time.Sleep(150 * time.Millisecond)
+					missesBefore := cached.CSCStats().Misses
+					if got, err := cached.Get(ctx, key).Result(); err != nil || got != "v2" {
+						t.Fatalf("read after invalidation = %q, %v; want v2", got, err)
+					}
+					if cached.CSCStats().Misses == missesBefore {
+						t.Fatal("with refresh-on-invalidate DISABLED the read after an " +
+							"invalidation was a cache HIT; this case cannot detect the " +
+							"feature working")
+					}
+					observed = true
+					break
+				}
+
+				// Enabled: wait for the background refresh to republish the entry.
+				deadline := time.Now().Add(2 * time.Second)
+				for time.Now().Before(deadline) {
+					if cached.CSCRefreshStats().Refreshed > beforeRefresh {
+						observed = true
+						break
+					}
+					time.Sleep(5 * time.Millisecond)
+				}
+				if !observed {
+					continue // the entry fell outside the recency window; try again
+				}
+
+				// The refreshed entry must serve the NEW value, with no miss.
+				missesBefore := cached.CSCStats().Misses
+				got, err := cached.Get(ctx, key).Result()
+				if err != nil {
+					t.Fatal(err)
+				}
+				if got != "v2" {
+					t.Fatalf("read after refresh = %q; want v2 — the refresh republished "+
+						"a stale value", got)
+				}
+				if cached.CSCStats().Misses != missesBefore {
+					t.Fatalf("read after refresh was a MISS; the whole point is that the "+
+						"background refresh had already put the new value in place "+
+						"(refreshed=%d)", cached.CSCRefreshStats().Refreshed)
+				}
+			}
+			if !observed {
+				t.Fatal("no attempt produced the behaviour under test in 12 tries")
+			}
+		})
+	}
+}
+
+func TestCSCRefreshDemandTriggerFlushesWindowEarly(t *testing.T) {
+	origWindow := cscRefreshWindow
+	cscRefreshWindow = 5 * time.Second // timer must not be the cause
+	defer func() { cscRefreshWindow = origWindow }()
+
+	ctx := context.Background()
+	cached := NewClient(&Options{
+		Addr:                               internalTestRedisAddr(),
+		Protocol:                           3,
+		ClientSideCacheConfig:              &ClientSideCacheConfig{MaxEntries: 1000},
+		ClientSideCacheRefreshOnInvalidate: true,
+	})
+	seeder := NewClient(&Options{Addr: internalTestRedisAddr(), Protocol: 3})
+	if err := cached.Ping(ctx).Err(); err != nil {
+		cached.Close()
+		seeder.Close()
+		t.Skipf("no redis: %v", err)
+	}
+	defer cached.Close()
+	defer seeder.Close()
+
+	const demandKey = "cscrefresh:demand:trigger"
+	const siblingKey = "cscrefresh:demand:sibling"
+	t.Cleanup(func() { seeder.Del(context.Background(), demandKey, siblingKey) })
+
+	var observed bool
+	for attempt := 0; attempt < 12 && !observed; attempt++ {
+		if err := seeder.MSet(ctx, demandKey, "d1", siblingKey, "s1").Err(); err != nil {
+			t.Fatal(err)
+		}
+		// Warm both and mark them recently read, so both are collected (the
+		// refresher ignores cold keys).
+		if _, err := cached.Get(ctx, demandKey).Result(); err != nil {
+			t.Fatal(err)
+		}
+		if got, err := cached.Get(ctx, siblingKey).Result(); err != nil || got != "s1" {
+			t.Fatalf("warm sibling = %q, %v; want s1", got, err)
+		}
+
+		beforeDemand := cached.CSCRefreshStats().DemandFlushes
+		beforeRefresh := cached.CSCRefreshStats().Refreshed
+
+		// Invalidate BOTH: each push deletes its entry and collects it into the
+		// window (both are recently read).
+		if err := seeder.MSet(ctx, demandKey, "d2", siblingKey, "s2").Err(); err != nil {
+			t.Fatal(err)
+		}
+		// Let the drainer apply both invalidations and the refresher collect them.
+		time.Sleep(120 * time.Millisecond)
+
+		// Read the demand key: it is a miss (just invalidated), which fires the
+		// demand signal and must flush the whole window — refetching the sibling.
+		if got, err := cached.Get(ctx, demandKey).Result(); err != nil || got != "d2" {
+			t.Fatalf("read demand key = %q, %v; want d2", got, err)
+		}
+
+		// The sibling must now refresh WELL within the 5s window. If it does, the
+		// timer cannot be responsible — only the demand flush.
+		deadline := time.Now().Add(1500 * time.Millisecond)
+		for time.Now().Before(deadline) {
+			if cached.CSCRefreshStats().Refreshed > beforeRefresh {
+				observed = true
+				break
+			}
+			time.Sleep(5 * time.Millisecond)
+		}
+		if !observed {
+			continue // entries fell outside the recency horizon; retry
+		}
+
+		if got := cached.CSCRefreshStats().DemandFlushes; got <= beforeDemand {
+			t.Fatalf("DemandFlushes did not advance (%d -> %d): the sibling refreshed "+
+				"but not via the demand trigger", beforeDemand, got)
+		}
+		// The sibling read must now be a HIT serving the new value: the demand
+		// flush refetched it a window early.
+		missesBefore := cached.CSCStats().Misses
+		if got, err := cached.Get(ctx, siblingKey).Result(); err != nil || got != "s2" {
+			t.Fatalf("sibling after demand flush = %q, %v; want s2", got, err)
+		}
+		if cached.CSCStats().Misses != missesBefore {
+			t.Fatal("sibling read after the demand flush was a MISS; the demand trigger " +
+				"should have refetched it when its neighbor was read")
+		}
+	}
+	if !observed {
+		t.Fatal("no attempt produced a demand-triggered refresh in 12 tries")
+	}
+}
+
+// TestCSCRefreshRepublishedEntryIsInvalidatable pins the invariant that the
+// background refresh path publishes its entry on a CSC-TRACKED connection.
+// Refresh reads run on a background pipelined connection (withPipelineConn),
+// which on this base falls back to / draws from a pool whose connections all
+// run CLIENT TRACKING while CSC is active. If a refetch ever published on an
+// untracked connection, the republished entry could never be invalidated and
+// would serve stale until TTL — the exact trap the coalescer's main-pool
+// choice avoids. This test would catch that regression: it refreshes an entry,
+// then mutates the key again from a second client and requires the caching
+// client to converge to the new value. With no TTL set, convergence is only
+// possible if the second invalidation was actually delivered, i.e. the
+// republished entry was tracked.
+func TestCSCRefreshRepublishedEntryIsInvalidatable(t *testing.T) {
+	ctx := context.Background()
+	cached := NewClient(&Options{
+		Addr:                               internalTestRedisAddr(),
+		Protocol:                           3,
+		ClientSideCacheConfig:              &ClientSideCacheConfig{MaxEntries: 1000},
+		ClientSideCacheRefreshOnInvalidate: true,
+	})
+	seeder := NewClient(&Options{Addr: internalTestRedisAddr(), Protocol: 3})
+	if err := cached.Ping(ctx).Err(); err != nil {
+		cached.Close()
+		seeder.Close()
+		t.Skipf("no redis: %v", err)
+	}
+	defer cached.Close()
+	defer seeder.Close()
+
+	key := "cscrefresh:republished-invalidatable"
+	t.Cleanup(func() { seeder.Del(context.Background(), key) })
+
+	var observed bool
+	for attempt := 0; attempt < 12 && !observed; attempt++ {
+		if err := seeder.Set(ctx, key, "v1", 0).Err(); err != nil {
+			t.Fatal(err)
+		}
+		// Warm it and mark it recently read so the refresher will collect it.
+		if got, err := cached.Get(ctx, key).Result(); err != nil || got != "v1" {
+			t.Fatalf("warm read = %q, %v; want v1", got, err)
+		}
+
+		beforeRefresh := cached.CSCRefreshStats().Refreshed
+		if err := seeder.Set(ctx, key, "v2", 0).Err(); err != nil {
+			t.Fatal(err)
+		}
+		// Wait for the background refresh to REPUBLISH the entry.
+		deadline := time.Now().Add(2 * time.Second)
+		for time.Now().Before(deadline) {
+			if cached.CSCRefreshStats().Refreshed > beforeRefresh {
+				break
+			}
+			time.Sleep(5 * time.Millisecond)
+		}
+		if cached.CSCRefreshStats().Refreshed == beforeRefresh {
+			continue // the entry fell outside the recency window; retry
+		}
+		if got, err := cached.Get(ctx, key).Result(); err != nil || got != "v2" {
+			t.Fatalf("read after refresh = %q, %v; want v2", got, err)
+		}
+
+		// Discriminating check: mutate the key again from the second client. The
+		// republished entry must be invalidated and the client must converge to
+		// v3. No TTL is set, so if the republished entry were published on an
+		// untracked connection no invalidation would arrive and the client would
+		// serve v2 forever — this poll would then time out.
+		if err := seeder.Set(ctx, key, "v3", 0).Err(); err != nil {
+			t.Fatal(err)
+		}
+		converge := time.Now().Add(2 * time.Second)
+		for time.Now().Before(converge) {
+			if got, _ := cached.Get(ctx, key).Result(); got == "v3" {
+				observed = true
+				break
+			}
+			time.Sleep(10 * time.Millisecond)
+		}
+		if !observed {
+			t.Fatal("after a background refresh republished the entry, a later " +
+				"invalidation never reached the client: the republished entry was " +
+				"NOT server-side tracked and would serve stale until TTL")
+		}
+	}
+	if !observed {
+		t.Fatal("no attempt produced a background refresh to test in 12 tries")
+	}
+}
+
+// TestRefreshInvalidatedBatchCancelsReservationsOnSizerPanic pins #3989: if a user
+// CacheSizer panics inside Reserve mid reservation loop, the targets reserved so far
+// must be cancelled during unwind (the cancellation defer is armed BEFORE the loop),
+// not left IN_PROGRESS for readers to block on until StaleTimeout.
+func TestRefreshInvalidatedBatchCancelsReservationsOnSizerPanic(t *testing.T) {
+	calls := 0
+	sizer := func(_ string, _ []string, value []byte) int64 {
+		calls++
+		if calls == 2 {
+			panic("boom: CacheSizer panic mid reservation loop")
+		}
+		return int64(len(value))
+	}
+	lc := NewLocalCache(CacheConfig{MaxEntries: 64, Sizer: sizer})
+	c := &baseClient{csc: lc, cscKeyPrefix: "p"}
+
+	targets := []cscRefreshTarget{
+		{cacheKey: "ck:1", redisKeys: []string{"rk:1"}},
+		{cacheKey: "ck:2", redisKeys: []string{"rk:2"}},
+	}
+
+	func() {
+		defer func() {
+			if r := recover(); r == nil {
+				t.Fatal("expected the sizer panic to propagate (the outer refresher recovers it)")
+			}
+		}()
+		_, _ = c.refreshInvalidatedBatch(context.Background(), targets)
+	}()
+
+	// ck:1 was reserved before the panic on ck:2; the defer must have cancelled it, so a
+	// fresh Reserve sees no lingering IN_PROGRESS placeholder (shouldFetch, non-zero tok).
+	tok, shouldFetch := lc.Reserve("ck:1", []string{"rk:1"})
+	if !shouldFetch || tok == 0 {
+		t.Fatalf("ck:1 reservation not cancelled after the panic (Reserve = %d, %v); a reader "+
+			"would block on the orphaned placeholder until StaleTimeout (#3989)", tok, shouldFetch)
+	}
+}

csc_refresh_support.go

@@ -0,0 +1,189 @@
+package redis
+
+import (
+	"sync"
+	"time"
+)
+
+// Support shims for refresh-on-invalidate and reader-miss coalescing, kept in
+// one file so the feature is a clean addition over the CSC base.
+
+// cscRevalidateHandle is the stop/join handle for a background CSC goroutine.
+type cscRevalidateHandle struct {
+	stop     chan struct{}
+	done     chan struct{}
+	stopOnce sync.Once
+}
+
+// signalStop closes stop at most once (so stopCSCRefresher and the AddCleanup
+// safety net cannot double-close) and does not join — a GC cleanup must not block.
+func (h *cscRevalidateHandle) signalStop() {
+	h.stopOnce.Do(func() { close(h.stop) })
+}
+
+// LRUClock returns the current global recency token; the refresher uses it as
+// the "recently read" horizon.
+func (c *LocalCache) LRUClock() int64 { return lruSequence.Load() }
+
+// cscRecencyRing holds the last N per-tick LRUClock() snapshots, giving
+// startCSCRefresher's sinceToken horizon a bounded multi-tick lookback
+// instead of the single most-recent tick, so
+// Options.ClientSideCacheRefreshRecencyWindow can span more than one
+// cscRefreshRecencyTick. oldest() is the horizon: once the ring has filled,
+// an entry read since oldest() was captured is guaranteed to fall within the
+// configured window, regardless of where an invalidation lands relative to
+// the tick phase (see the ring-sizing comment in startCSCRefresher). Owned
+// solely by the refresher goroutine — push/oldest are not called
+// concurrently, so no lock.
+type cscRecencyRing struct {
+	buf []int64
+	pos int
+	n   int
+}
+
+// cscRecencyRingMaxSize caps the ring allocation newCscRecencyRing builds. At
+// the 200ms tick this is ~58 hours of coverage — a pathological
+// ClientSideCacheRefreshRecencyWindow (a duration typo, or math.MaxInt64)
+// degrades to that cap instead of an oversized or panicking make([]int64, N).
+const cscRecencyRingMaxSize = 1 << 20
+
+// newCscRecencyRing builds a ring of the given size, clamped to
+// [1, cscRecencyRingMaxSize].
+func newCscRecencyRing(size int) *cscRecencyRing {
+	if size < 1 {
+		size = 1
+	}
+	if size > cscRecencyRingMaxSize {
+		size = cscRecencyRingMaxSize
+	}
+	return &cscRecencyRing{buf: make([]int64, size)}
+}
+
+// cscRefreshWindowTicks converts a requested
+// Options.ClientSideCacheRefreshRecencyWindow into a cscRecencyRing size.
+// Ring size N gives a GUARANTEED-minimum lookback of (N-1)*cscRefreshRecencyTick
+// once the ring has filled (oldest() is then N-1 ticks behind the latest
+// push) — the tick phase relative to an arbitrary invalidation can cost up to
+// one full tick of coverage, so N-1 must itself already be >= ceil(w/tick)
+// for the enforced window to never fall short of w. Hence N = ceil(w/tick)+1,
+// giving an enforced window in [w, w+tick). Extracted pure so the rounding is
+// unit-tested; w<=0 (unbounded mode) never reaches this function.
+func cscRefreshWindowTicks(w time.Duration) int {
+	ticks := int(w / cscRefreshRecencyTick)
+	if w%cscRefreshRecencyTick != 0 {
+		ticks++
+	}
+	return ticks + 1
+}
+
+// push records the latest per-tick snapshot, evicting the oldest once full.
+func (r *cscRecencyRing) push(v int64) {
+	r.buf[r.pos] = v
+	r.pos = (r.pos + 1) % len(r.buf)
+	if r.n < len(r.buf) {
+		r.n++
+	}
+}
+
+// oldest returns the least-recent snapshot currently held. Before the ring
+// fills (just after the refresher starts), it returns the earliest snapshot
+// taken so far, so every access since the refresher started counts as hot
+// until the configured window has had time to mature. Once full, r.pos
+// always points at the slot about to be overwritten next — the oldest value
+// held — a standard circular-buffer property.
+func (r *cscRecencyRing) oldest() int64 {
+	if r.n < len(r.buf) {
+		return r.buf[0]
+	}
+	return r.buf[r.pos]
+}
+
+// InvalidationStats reports INCOMING invalidation pushes: the count of keys named
+// in server invalidation messages, tallied at the handler before dedup/batching.
+func (c *LocalCache) InvalidationStats() (invalidations uint64) {
+	return c.invalidations.Load()
+}
+
+// DeletionStats reports APPLIED invalidations. deletions counts keys the cache
+// actually processed for removal (post-dedup, so <= InvalidationStats under
+// duplicate pushes); noop counts those that removed no live entry — a key that
+// matched nothing, or one whose only match was skipped by the fetch-order guard
+// because it was refetched after the invalidation (see collectHotAndDelete).
+// Both are the duplicate/stale-invalidation signature.
+func (c *LocalCache) DeletionStats() (deletions, noop uint64) {
+	return c.deletions.Load(), c.deletionsNoop.Load()
+}
+
+// deleteByRedisKeyCollectingHot deletes every cache entry tracked under redisKey
+// and returns those that were VALID and read since sinceToken, as refetch
+// targets. Delete and collect happen under one shard lock: the entry's cache key
+// and recency are known only there, and it is about to be removed. fetchSnap is
+// the cscFetchSeq value observed when this invalidation arrived; a Valid entry
+// whose fetch was ISSUED after that (entry.fetchSeq > fetchSnap) is kept — it was
+// refetched after the write, so evicting it would be a spurious miss (see
+// cacheEntry.fetchSeq).
+func (c *LocalCache) deleteByRedisKeyCollectingHot(redisKey string, sinceToken int64, fetchSnap uint64, dst []cscRefreshTarget) []cscRefreshTarget {
+	removed := 0
+	for i := range c.shards {
+		var n int
+		dst, n = c.shards[i].collectHotAndDelete(redisKey, sinceToken, fetchSnap, dst)
+		removed += n
+	}
+	// Applied-delete accounting (refresh-on path). Twin of DeleteByRedisKey; the
+	// incoming push was already counted at the handler.
+	c.deletions.Add(1)
+	if removed == 0 {
+		c.deletionsNoop.Add(1)
+	}
+	return dst
+}
+
+func (s *cacheShard) collectHotAndDelete(redisKey string, sinceToken int64, fetchSnap uint64, dst []cscRefreshTarget) ([]cscRefreshTarget, int) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+
+	cacheKeys, ok := s.byRedisKey[redisKey]
+	if !ok {
+		return dst, 0
+	}
+	toRemove := make([]string, 0, len(cacheKeys))
+	for cacheKey := range cacheKeys {
+		toRemove = append(toRemove, cacheKey)
+	}
+	removed := 0
+	for _, cacheKey := range toRemove {
+		entry, exists := s.entries[cacheKey]
+		// Fetch-order guard: an entry whose fetch was ISSUED after this invalidation
+		// was observed (fetchSeq > fetchSnap) cannot hold the pre-write value, so keep
+		// it — whether already Valid or still in-progress. Evicting a Valid one would
+		// be a spurious miss (and a stale queued duplicate straddling a refetch must
+		// not undo the fresh value, #3965); canceling an in-progress one would fail its
+		// racing Fulfill, wake every waiter as a miss, and defeat miss coalescing. An
+		// entry reserved at or before the observe (fetchSeq <= fetchSnap) may predate
+		// the write — possibly fetched on a different stream (see deleteByRedisKey) —
+		// so it is removed, and any racing Fulfill for it correctly fails.
+		if exists && entry.fetchSeq > fetchSnap {
+			continue
+		}
+		if exists && entry.state == cacheEntryValid && entry.lastAccessNs.Load() > sinceToken {
+			keys := make([]string, len(entry.redisKeys))
+			copy(keys, entry.redisKeys)
+			dst = append(dst, cscRefreshTarget{
+				cacheKey:  cacheKey,
+				redisKeys: keys,
+				// Preserve the reader-access token so the refresh republish can restore it
+				// and not renew demand (see cscRefreshTarget.accessNs / restoreAccessToken).
+				// Re-load is safe: the shard lock is held, so no writer intervenes.
+				accessNs: entry.lastAccessNs.Load(),
+				// The dying entry's payload size approximates the refetch REPLY size —
+				// the refresher chunks round trips by expected reply bytes with it (see
+				// cscRefreshChunkEnd). Known for free here, under the same shard lock.
+				valBytes: len(entry.value),
+			})
+		}
+		if s.removeEntryLocked(cacheKey) {
+			removed++
+		}
+	}
+	return dst, removed
+}

csc_refresh_support_test.go

@@ -0,0 +1,432 @@
+package redis
+
+import (
+	"context"
+	"testing"
+	"time"
+)
+
+// TestCollectHotAndDeleteRecencyFilter pins the recency gate that decides which
+// invalidated entries the refresher re-fetches. Every invalidated entry is
+// deleted, but only those that were VALID and read since the horizon are
+// returned as refetch targets. Getting the comparison or the horizon wrong
+// would silently refetch every invalidated key — extra round trips, no test
+// failure — so the cold case is asserted explicitly alongside the hot one.
+func TestCollectHotAndDeleteRecencyFilter(t *testing.T) {
+	ctx := context.Background()
+	lc := NewLocalCache(CacheConfig{MaxEntries: 64})
+
+	mkValid := func(cacheKey, redisKey string) {
+		tok, fetch := lc.Reserve(cacheKey, []string{redisKey})
+		if tok == 0 || !fetch {
+			t.Fatalf("Reserve(%q) = token %d, shouldFetch %v; want a fresh reservation", cacheKey, tok, fetch)
+		}
+		if !lc.fulfill(cacheKey, tok, 0, []byte("v")) {
+			t.Fatalf("fulfill(%q) failed", cacheKey)
+		}
+	}
+
+	// Cold entry is created first; the horizon is snapshotted after it, so its
+	// recency token is at or below the horizon.
+	mkValid("ck:cold", "rk:cold")
+	horizon := lruSequence.Load()
+
+	// Hot entry is created after the horizon snapshot, so its recency token is
+	// strictly above the horizon regardless of any concurrent LRU activity.
+	mkValid("ck:hot", "rk:hot")
+
+	// Cold: NOT collected (read no more recently than the horizon), but still deleted.
+	// snap = max: the fetch-order guard never skips, so this isolates the hot/cold
+	// recency behavior (guard interplay is covered separately).
+	if got := lc.deleteByRedisKeyCollectingHot("rk:cold", horizon, ^uint64(0), nil); len(got) != 0 {
+		t.Fatalf("cold entry collected as a refetch target: %v", got)
+	}
+	if _, ok := lc.Get(ctx, "ck:cold"); ok {
+		t.Fatal("cold entry was not deleted by the invalidation")
+	}
+
+	// Hot: collected (read after the horizon) and deleted.
+	got := lc.deleteByRedisKeyCollectingHot("rk:hot", horizon, ^uint64(0), nil)
+	if len(got) != 1 || got[0].cacheKey != "ck:hot" {
+		t.Fatalf("hot entry not collected as the sole refetch target: %v", got)
+	}
+	if _, ok := lc.Get(ctx, "ck:hot"); ok {
+		t.Fatal("hot entry was not deleted by the invalidation")
+	}
+
+	// Unknown redis key: nothing collected, no panic on the missing index.
+	if got := lc.deleteByRedisKeyCollectingHot("rk:nonexistent", 0, ^uint64(0), nil); len(got) != 0 {
+		t.Fatalf("unknown redis key produced refetch targets: %v", got)
+	}
+}
+
+// TestCollectHotAndDeleteNoHorizonRefreshesEverything pins the default
+// refresh-everything mode (Options.ClientSideCacheRefreshRecencyWindow
+// unset): cscInvalNoHorizon passed as sinceToken must collect a Valid entry
+// as a refetch target REGARDLESS of how long ago it was last read — the
+// mirror image of TestCollectHotAndDeleteRecencyFilter's cold case, which is
+// what a real (non-sentinel) horizon excludes.
+func TestCollectHotAndDeleteNoHorizonRefreshesEverything(t *testing.T) {
+	ctx := context.Background()
+	lc := NewLocalCache(CacheConfig{MaxEntries: 64})
+
+	tok, fetch := lc.Reserve("ck:old", []string{"rk:old"})
+	if tok == 0 || !fetch {
+		t.Fatalf("Reserve(ck:old) = (%d, %v)", tok, fetch)
+	}
+	if !lc.fulfill("ck:old", tok, 0, []byte("v")) {
+		t.Fatal("fulfill(ck:old) failed")
+	}
+
+	// A horizon taken long after the entry's last read would normally exclude
+	// it (see TestCollectHotAndDeleteRecencyFilter). cscInvalNoHorizon must
+	// override that and collect it anyway.
+	for i := 0; i < 100; i++ {
+		lruSequence.Add(1)
+	}
+
+	got := lc.deleteByRedisKeyCollectingHot("rk:old", cscInvalNoHorizon, ^uint64(0), nil)
+	if len(got) != 1 || got[0].cacheKey != "ck:old" {
+		t.Fatalf("cscInvalNoHorizon did not collect the entry as a refetch target: %v", got)
+	}
+	if _, ok := lc.Get(ctx, "ck:old"); ok {
+		t.Fatal("entry was not deleted by the invalidation")
+	}
+}
+
+// TestCscRecencyRingOldest pins the ring's circular-buffer arithmetic:
+// oldest() must return the earliest-still-held snapshot both before the ring
+// fills (ramp-up) and after it wraps repeatedly.
+func TestCscRecencyRingOldest(t *testing.T) {
+	r := newCscRecencyRing(3)
+
+	// Ramp-up: not yet full, oldest is always the first push.
+	r.push(10)
+	if got := r.oldest(); got != 10 {
+		t.Fatalf("after 1 push: oldest = %d, want 10", got)
+	}
+	r.push(20)
+	if got := r.oldest(); got != 10 {
+		t.Fatalf("after 2 pushes: oldest = %d, want 10 (ring not full yet)", got)
+	}
+
+	// Full: oldest is the least-recent of the last 3.
+	r.push(30)
+	if got := r.oldest(); got != 10 {
+		t.Fatalf("after 3 pushes (just filled): oldest = %d, want 10", got)
+	}
+
+	// Wraps: each push evicts the true oldest.
+	r.push(40) // holds {20,30,40}
+	if got := r.oldest(); got != 20 {
+		t.Fatalf("after 4 pushes: oldest = %d, want 20", got)
+	}
+	r.push(50) // holds {30,40,50}
+	if got := r.oldest(); got != 30 {
+		t.Fatalf("after 5 pushes: oldest = %d, want 30", got)
+	}
+
+	// Size 1 degenerates to "just the latest push" — today's pre-option behavior.
+	one := newCscRecencyRing(1)
+	one.push(7)
+	if got := one.oldest(); got != 7 {
+		t.Fatalf("size-1 ring: oldest = %d, want 7", got)
+	}
+	one.push(9)
+	if got := one.oldest(); got != 9 {
+		t.Fatalf("size-1 ring after second push: oldest = %d, want 9", got)
+	}
+
+	// size < 1 clamps to 1 rather than panicking on a zero-length buffer.
+	zero := newCscRecencyRing(0)
+	zero.push(5)
+	if got := zero.oldest(); got != 5 {
+		t.Fatalf("size-0 ring (clamped to 1): oldest = %d, want 5", got)
+	}
+}
+
+// TestCscRefreshWindowTicks pins the window->ring-size rounding: the enforced
+// window (ticks-1)*tick must never fall short of the requested duration,
+// regardless of how it divides the tick, and must match exactly on an exact
+// multiple (no off-by-one slack beyond the documented [w, w+tick) range).
+func TestCscRefreshWindowTicks(t *testing.T) {
+	tick := cscRefreshRecencyTick
+	cases := []struct {
+		name string
+		w    time.Duration
+		want int
+	}{
+		{"exact one tick", tick, 2},
+		{"exact two ticks", 2 * tick, 3},
+		{"sub-tick rounds up to one tick's guarantee", tick / 4, 2},
+		{"just over one tick", tick + time.Millisecond, 3},
+		{"just under two ticks", 2*tick - time.Millisecond, 3},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			if got := cscRefreshWindowTicks(tc.w); got != tc.want {
+				t.Fatalf("cscRefreshWindowTicks(%v) = %d, want %d", tc.w, got, tc.want)
+			}
+			// The guarantee itself: enforced minimum lookback (ticks-1)*tick must
+			// cover the requested window.
+			enforced := time.Duration(tc.want-1) * tick
+			if enforced < tc.w {
+				t.Fatalf("cscRefreshWindowTicks(%v): enforced minimum %v < requested %v", tc.w, enforced, tc.w)
+			}
+		})
+	}
+}
+
+// TestCollectHotFetchOrderGuard is the discriminating test for the fetch-order
+// guard (#3965). A stale invalidation snapshots the fetch clock at OBSERVE time; an
+// entry refetched AFTER that (fetchSeq > snapshot) already reflects the write and
+// must survive — evicting it would be a spurious miss and would undo a refresh. A
+// GENUINE later invalidation, observed after the refetch, must still evict. This is
+// the case that a fulfill-time stamp would get wrong: the invalidation and the
+// reply race on different connections, so a stale reply can fulfill after the
+// invalidate is observed; only stamping at fetch ISSUE (Reserve) is sound.
+func TestCollectHotFetchOrderGuard(t *testing.T) {
+	ctx := context.Background()
+	lc := NewLocalCache(CacheConfig{MaxEntries: 64})
+	mkValid := func(cacheKey string, redisKeys ...string) {
+		lc.DeleteByCacheKey(cacheKey)
+		tok, fetch := lc.Reserve(cacheKey, redisKeys)
+		if tok == 0 || !fetch {
+			t.Fatalf("Reserve(%q) = (%d, %v)", cacheKey, tok, fetch)
+		}
+		if !lc.fulfill(cacheKey, tok, 0, []byte("v")) {
+			t.Fatalf("fulfill(%q) failed", cacheKey)
+		}
+	}
+
+	// Entry fetched, then an invalidation observed: snapshot the fetch clock now.
+	mkValid("ck", "rk")
+	snapStale := cscFetchSeq.Load()
+
+	// The entry is refetched AFTER the invalidation was observed (a refresh or a
+	// concurrent read miss): its fetchSeq now exceeds the stale snapshot.
+	mkValid("ck", "rk")
+
+	// Stale invalidation applied: guard keeps the refetched entry and does not offer
+	// it for refetch.
+	if got := lc.deleteByRedisKeyCollectingHot("rk", 0, snapStale, nil); len(got) != 0 {
+		t.Fatalf("refetched entry offered as a refetch target: %v", got)
+	}
+	if _, ok := lc.Get(ctx, "ck"); !ok {
+		t.Fatal("stale invalidation evicted an entry refetched after it (fetch-order guard failed)")
+	}
+
+	// A GENUINE later invalidation (observed now, after the refetch) must evict.
+	snapFresh := cscFetchSeq.Load()
+	lc.deleteByRedisKeyCollectingHot("rk", 0, snapFresh, nil)
+	if _, ok := lc.Get(ctx, "ck"); ok {
+		t.Fatal("genuine post-refetch invalidation did not evict the entry")
+	}
+}
+
+// TestCollectHotMultiKeyRefreshSurvives pins the multi-key variant of #3965 that
+// dedup (seen) structurally cannot fix: one cached command depends on several
+// Redis keys, so a refresh triggered by one key's invalidation republishes the
+// entry, and another key's (stale) invalidation must not re-evict it. seen is keyed
+// by Redis key, so k1 and k2 are different dedup entries — only the per-entry
+// fetch-order guard can reason about "was THIS entry refetched after me".
+func TestCollectHotMultiKeyRefreshSurvives(t *testing.T) {
+	ctx := context.Background()
+	lc := NewLocalCache(CacheConfig{MaxEntries: 64})
+	mkValid := func(cacheKey string, redisKeys ...string) {
+		lc.DeleteByCacheKey(cacheKey)
+		tok, fetch := lc.Reserve(cacheKey, redisKeys)
+		if tok == 0 || !fetch {
+			t.Fatalf("Reserve(%q) = (%d, %v)", cacheKey, tok, fetch)
+		}
+		if !lc.fulfill(cacheKey, tok, 0, []byte("v")) {
+			t.Fatalf("fulfill(%q) failed", cacheKey)
+		}
+	}
+
+	// Two cached commands sharing k2: C1 = MGET k1 k2, C2 = GET k2.
+	mkValid("mget:k1k2", "k1", "k2")
+	mkValid("get:k2", "k2")
+
+	// Both k1 and k2 change; the client observes both invalidations now (same push,
+	// so one snapshot covers both items).
+	snap := cscFetchSeq.Load()
+
+	// k1's invalidation applies first: it deletes C1 (depends on k1).
+	if _, ok := lc.Get(ctx, "mget:k1k2"); !ok {
+		t.Fatal("precondition: C1 should be present")
+	}
+	lc.deleteByRedisKeyCollectingHot("k1", 0, snap, nil)
+	if _, ok := lc.Get(ctx, "mget:k1k2"); ok {
+		t.Fatal("k1 invalidation should have deleted C1")
+	}
+
+	// The refresh refetches C1 (MGET k1 k2) — issued after both writes, so the reply
+	// already reflects the new k2. fetchSeq now exceeds snap.
+	mkValid("mget:k1k2", "k1", "k2")
+
+	// k2's invalidation applies second, fanning out to BOTH C1 and C2 via byRedisKey.
+	// C1 was refetched after snap -> guard keeps it; C2 predates snap -> evicted.
+	lc.deleteByRedisKeyCollectingHot("k2", 0, snap, nil)
+
+	if _, ok := lc.Get(ctx, "mget:k1k2"); !ok {
+		t.Fatal("k2's stale invalidation evicted the refetched multi-key entry C1 " +
+			"(guard must keep an MGET result refetched after the write)")
+	}
+	if _, ok := lc.Get(ctx, "get:k2"); ok {
+		t.Fatal("k2's invalidation should have evicted C2 (GET k2 was not refetched)")
+	}
+}
+
+// TestCSCRefreshReplyCacheable pins the refresh cacheability decision on the FULL
+// reply, not raw[0]. A RESP3 attribute-prefixed error leads with RespAttr ('|'),
+// so a first-byte error check would publish an attributed error as a fresh entry
+// (a false success that bumps Refreshed and forces the next reader to evict and
+// refetch). A nil reply IS cacheable; an empty reply never is.
+func TestCSCRefreshReplyCacheable(t *testing.T) {
+	cases := []struct {
+		name      string
+		raw       string
+		cacheable bool
+	}{
+		{"plain value", "$2\r\nhi\r\n", true},
+		{"resp3 nil", "_\r\n", true},
+		{"resp2 nil", "$-1\r\n", true},
+		{"top-level error", "-WRONGTYPE nope\r\n", false},
+		{"blob error", "!11\r\nERR bad arg\r\n", false},
+		// The bug: the error frame is prefixed by a RESP3 attribute, so raw[0]=='|'.
+		{"attribute-prefixed error", "|1\r\n$3\r\nttl\r\n:100\r\n-WRONGTYPE nope\r\n", false},
+		{"attribute-prefixed value", "|1\r\n$3\r\nttl\r\n:100\r\n$2\r\nhi\r\n", true},
+		{"empty", "", false},
+	}
+	for _, tc := range cases {
+		if got := cscRefreshReplyCacheable([]byte(tc.raw)); got != tc.cacheable {
+			t.Errorf("%s: cscRefreshReplyCacheable = %v, want %v", tc.name, got, tc.cacheable)
+		}
+	}
+}
+
+// TestCSCRefreshChunkEnd pins the refresh round-trip boundaries: count cap,
+// request-byte budget (key wire form), and the expected-REPLY-byte budget —
+// the reply side is what jams a flow-controlled path, so large evicted values
+// must shrink the chunk. The first target always goes.
+func TestCSCRefreshChunkEnd(t *testing.T) {
+	mk := func(key string, val int) cscRefreshTarget {
+		return cscRefreshTarget{cacheKey: key, valBytes: val}
+	}
+
+	// Reply budget splits the chunk: three 15KiB values fit two per chunk
+	// under the 32KiB reply budget (request bytes tiny).
+	big := []cscRefreshTarget{mk("get:a", 15<<10), mk("get:b", 15<<10), mk("get:c", 15<<10)}
+	if got := cscRefreshChunkEnd(big, 0, 0, 1<<20); got != 2 {
+		t.Fatalf("reply budget: end = %d, want 2 (two 15KiB replies fit, third exceeds 32KiB)", got)
+	}
+	if got := cscRefreshChunkEnd(big, 2, 0, 1<<20); got != 3 {
+		t.Fatalf("tail: end = %d, want 3", got)
+	}
+
+	// A single oversized value still goes (progress guarantee).
+	huge := []cscRefreshTarget{mk("get:x", 1<<20), mk("get:y", 1)}
+	if got := cscRefreshChunkEnd(huge, 0, 0, 1<<20); got != 1 {
+		t.Fatalf("oversized first: end = %d, want 1 (always include the first target)", got)
+	}
+
+	// Request-byte budget still enforced: long keys, zero-size values.
+	longKey := make([]byte, 100)
+	for i := range longKey {
+		longKey[i] = 'k'
+	}
+	keys := []cscRefreshTarget{mk(string(longKey), 0), mk(string(longKey), 0), mk(string(longKey), 0)}
+	if got := cscRefreshChunkEnd(keys, 0, 0, 150); got != 1 {
+		t.Fatalf("request budget: end = %d, want 1 (second 100B key exceeds 150B budget)", got)
+	}
+
+	// Count cap: many tiny targets stop at cscRefreshBatchMax.
+	many := make([]cscRefreshTarget, cscRefreshBatchMax+10)
+	for i := range many {
+		many[i] = mk("get:k", 1)
+	}
+	if got := cscRefreshChunkEnd(many, 0, 0, 1<<20); got != cscRefreshBatchMax {
+		t.Fatalf("count cap: end = %d, want %d", got, cscRefreshBatchMax)
+	}
+}
+
+// TestCSCRefreshDemandGenerationIgnoresStaleSignal pins the demand generation gate:
+// a demand nudge is stamped with the window generation it belongs to, and the
+// refresher ignores a nudge whose generation was retired — so a nudge for an
+// already-flushed window cannot flush the NEXT one early (which would defeat the
+// coalescing window and inflate DemandFlushes). Pins signalDemand's stamping and the
+// demandIsCurrent truth table (the goroutine race lives inside runCSCRefresher).
+func TestCSCRefreshDemandGenerationIgnoresStaleSignal(t *testing.T) {
+	if !cscDemandRefresh {
+		t.Skip("demand refresh disabled")
+	}
+	q := &cscRefreshQueue{demandCh: make(chan uint64, 1)}
+
+	q.pendingSet.Store("ck", struct{}{})
+	q.signalDemand("ck")
+	var stale uint64
+	select {
+	case stale = <-q.demandCh:
+	default:
+		t.Fatal("signalDemand did not stamp a nudge for a pending key")
+	}
+	if !q.demandIsCurrent(stale) {
+		t.Fatal("a nudge for the CURRENT window must be honored")
+	}
+
+	q.pendingSet.Delete("ck")
+	q.demandGen.Add(1) // refresher clears window N and retires its generation
+
+	if q.demandIsCurrent(stale) {
+		t.Fatal("a nudge from a retired generation must be ignored")
+	}
+
+	q.pendingSet.Store("ck2", struct{}{})
+	q.signalDemand("ck2")
+	select {
+	case g := <-q.demandCh:
+		if !q.demandIsCurrent(g) {
+			t.Fatal("a nudge for the new current window must be honored")
+		}
+	default:
+		t.Fatal("signalDemand did not stamp a nudge for the new window's key")
+	}
+}
+
+// TestCollectHotGuardKeepsInProgress pins that the fetch-order guard covers
+// in-progress reservations, not only Valid entries (#3989). A reservation issued
+// AFTER the invalidation was observed (fetchSeq > snapshot) cannot hold the
+// pre-write value, so it must survive — removing it would fail its racing Fulfill
+// and wake every waiter as a miss. A reservation issued at or before the observe is
+// still removed (it may predate the write).
+func TestCollectHotGuardKeepsInProgress(t *testing.T) {
+	ctx := context.Background()
+	lc := NewLocalCache(CacheConfig{MaxEntries: 64})
+
+	// In-progress reservation issued BEFORE the invalidation is observed.
+	tokOld, fetchOld := lc.Reserve("ck:old", []string{"rk"})
+	if tokOld == 0 || !fetchOld {
+		t.Fatalf("Reserve(ck:old) = (%d, %v)", tokOld, fetchOld)
+	}
+	snap := cscFetchSeq.Load()
+	// In-progress reservation issued AFTER the observe.
+	tokNew, fetchNew := lc.Reserve("ck:new", []string{"rk"})
+	if tokNew == 0 || !fetchNew {
+		t.Fatalf("Reserve(ck:new) = (%d, %v)", tokNew, fetchNew)
+	}
+
+	lc.deleteByRedisKeyCollectingHot("rk", 0, snap, nil)
+
+	// Pre-observe reservation removed: its racing Fulfill now fails.
+	if lc.fulfill("ck:old", tokOld, 0, []byte("v")) {
+		t.Fatal("pre-observe in-progress reservation should have been removed (Fulfill must fail)")
+	}
+	// Post-observe reservation kept: its Fulfill still publishes.
+	if !lc.fulfill("ck:new", tokNew, 0, []byte("v")) {
+		t.Fatal("post-observe in-progress reservation was wrongly removed (Fulfill failed)")
+	}
+	if _, ok := lc.Get(ctx, "ck:new"); !ok {
+		t.Fatal("post-observe in-progress entry did not survive the invalidation")
+	}
+}

csc_wirebudget_test.go

@@ -0,0 +1,278 @@
+package redis
+
+import (
+	"bytes"
+	"context"
+	"testing"
+	"time"
+
+	"github.com/redis/go-redis/v9/internal/proto"
+)
+
+// TestCSCMissCoalescerWireBudget pins the in-flight wire-byte budget that bounds a
+// burst of large coalesced misses: over budget, a miss sheds to the pooled path
+// (reserveWireBytes returns false) instead of holding a wire copy while blocked.
+func TestCSCMissCoalescerWireBudget(t *testing.T) {
+	mc := &cscMissCoalescer{}
+	// Reserve up to the budget: succeeds.
+	if !mc.reserveWireBytes(cscMissWireBudgetBytes) {
+		t.Fatal("reserve up to budget should succeed")
+	}
+	if got := mc.wireBytes.Load(); got != cscMissWireBudgetBytes {
+		t.Fatalf("wireBytes = %d, want %d", got, int64(cscMissWireBudgetBytes))
+	}
+	// Over budget: sheds and reserves nothing (counter unchanged).
+	if mc.reserveWireBytes(1) {
+		t.Fatal("reserve over budget should shed (return false)")
+	}
+	if got := mc.wireBytes.Load(); got != cscMissWireBudgetBytes {
+		t.Fatalf("shed must not change the counter; wireBytes = %d, want %d", got, int64(cscMissWireBudgetBytes))
+	}
+	// Release frees the budget again.
+	mc.releaseWireBytes(cscMissWireBudgetBytes)
+	if got := mc.wireBytes.Load(); got != 0 {
+		t.Fatalf("wireBytes after release = %d, want 0", got)
+	}
+	// A single command larger than the whole budget sheds too (the pooled path runs
+	// it, so there is no progress hazard) and leaves the counter at 0.
+	if mc.reserveWireBytes(cscMissWireBudgetBytes + 1) {
+		t.Fatal("a single over-budget command should shed")
+	}
+	if got := mc.wireBytes.Load(); got != 0 {
+		t.Fatalf("wireBytes after over-budget shed = %d, want 0", got)
+	}
+}
+
+// bigBinArg is a command argument whose serialized size is large but which
+// cmdApproxBytes charges only ~8 bytes for (the BinaryMarshaler default branch).
+type bigBinArg struct{ n int }
+
+func (a bigBinArg) MarshalBinary() ([]byte, error) { return make([]byte, a.n), nil }
+
+// TestCSCMissCoalescerReconcilesWireBudget pins the fix for the under-counted wire
+// budget: cmdApproxBytes charges ~8 bytes for a variable-size non-string/[]byte arg
+// (BinaryMarshaler, net.IP, *string, ...), so the pre-serialization reserve can
+// undercount a large miss and let the in-flight budget drift past
+// cscMissWireBudgetBytes. reconcileWireBytes corrects the counter to the ACTUAL
+// serialized size and keeps req.reserved in lockstep so settle() releases exactly
+// once.
+func TestCSCMissCoalescerReconcilesWireBudget(t *testing.T) {
+	ctx := context.Background()
+	cmd := NewCmd(ctx, "set", "k", bigBinArg{4096})
+
+	estimate := cmdApproxBytes(cmd)
+	var buf bytes.Buffer
+	if err := writeCmd(proto.NewWriter(&buf), cmd); err != nil {
+		t.Fatalf("writeCmd: %v", err)
+	}
+	wire := buf.Bytes()
+	actual := int64(len(wire))
+	// Precondition: the estimate really does undercount, or the test proves nothing.
+	if estimate >= actual {
+		t.Fatalf("estimate %d >= actual %d; cmdApproxBytes did not undercount, test is vacuous",
+			estimate, actual)
+	}
+
+	mc := &cscMissCoalescer{}
+	// Reserve the estimate (as fetch does BEFORE serialization), then reconcile once
+	// the wire exists.
+	if !mc.reserveWireBytes(estimate) {
+		t.Fatal("reserve estimate failed")
+	}
+	req := &cscMissReq{reserved: estimate, wire: wire}
+	mc.reconcileWireBytes(req)
+
+	if got := mc.wireBytes.Load(); got != actual {
+		t.Fatalf("after reconcile wireBytes = %d, want the actual wire size %d "+
+			"(undercount not corrected)", got, actual)
+	}
+	if req.reserved != actual {
+		t.Fatalf("req.reserved = %d, want %d (settle must release the reconciled amount)",
+			req.reserved, actual)
+	}
+	// settle() releases exactly req.reserved; the counter must return to zero.
+	mc.releaseWireBytes(req.reserved)
+	if got := mc.wireBytes.Load(); got != 0 {
+		t.Fatalf("after release wireBytes = %d, want 0 (release is not exactly-once)", got)
+	}
+}
+
+// TestCSCMissCoalescerCancelIfStopping pins the shutdown-strand fix (B2). On a
+// caller-deadline abandon the LIVE reader still dequeues the req and publishes, so
+// the token is left to it. But once the coalescer is stopping, a req stranded in
+// mc.ch past the shutdown drain would leave its key IN_PROGRESS and block a later
+// reader until StaleTimeout — so cancelIfStopping cancels the reservation, but only
+// when stop is observed.
+func TestCSCMissCoalescerCancelIfStopping(t *testing.T) {
+	cache := NewLocalCache(CacheConfig{MaxEntries: 8})
+	mc := &cscMissCoalescer{
+		c:    &baseClient{csc: cache},
+		ch:   make(chan *cscMissReq, 4),
+		stop: make(chan struct{}),
+	}
+
+	token, fetch := cache.Reserve("ck", []string{"rk"})
+	if token == 0 || !fetch {
+		t.Fatalf("Reserve = (%d, %v); want a fresh in-progress reservation", token, fetch)
+	}
+
+	// Live coalescer: no-op. The reservation stays IN_PROGRESS (a second Reserve
+	// declines) so the live reader can still publish.
+	mc.cancelIfStopping("ck", token)
+	if _, sf := cache.Reserve("ck", []string{"rk"}); sf {
+		t.Fatal("live cancelIfStopping released the reservation; the reader must still publish")
+	}
+
+	// Stopping: the reservation MUST be released so a later reader misses and
+	// refetches instead of blocking to StaleTimeout.
+	close(mc.stop)
+	mc.cancelIfStopping("ck", token)
+	if tok2, sf := cache.Reserve("ck", []string{"rk"}); !sf || tok2 == 0 {
+		t.Fatalf("after stopping cancelIfStopping: Reserve = (%d, %v); want a fresh "+
+			"reservation (the IN_PROGRESS token was not cancelled)", tok2, sf)
+	}
+}
+
+// TestCSCMissCoalescerShedsWhenActualOverBudget pins finding #1's second half:
+// reconcileWireBytes must SHED (return false) when the true serialized size would
+// push the in-flight total past the budget, not just correct the counter and keep
+// an over-budget wire — else N concurrent undercounted misses each pass the
+// pre-serialize gate and then blow the budget while every wire stays queued.
+func TestCSCMissCoalescerShedsWhenActualOverBudget(t *testing.T) {
+	ctx := context.Background()
+	cmd := NewCmd(ctx, "set", "k", bigBinArg{4096})
+	estimate := cmdApproxBytes(cmd)
+	var buf bytes.Buffer
+	if err := writeCmd(proto.NewWriter(&buf), cmd); err != nil {
+		t.Fatalf("writeCmd: %v", err)
+	}
+	wire := buf.Bytes()
+	actual := int64(len(wire))
+	if estimate >= actual {
+		t.Fatalf("estimate %d >= actual %d; test is vacuous", estimate, actual)
+	}
+
+	mc := &cscMissCoalescer{}
+	// Preload the budget so the estimate still fits but reconciling to the ACTUAL
+	// size crosses the cap.
+	preload := cscMissWireBudgetBytes - estimate - actual/2
+	mc.wireBytes.Store(preload)
+	if !mc.reserveWireBytes(estimate) {
+		t.Fatal("reserve estimate should fit (only the reconciled actual exceeds the cap)")
+	}
+	req := &cscMissReq{reserved: estimate, wire: wire}
+
+	if mc.reconcileWireBytes(req) {
+		t.Fatal("reconcileWireBytes kept an over-budget wire; want shed (false)")
+	}
+	// Rolled back to the estimate reservation (reserved unchanged), so fetch's
+	// !enqueued defer releases exactly the estimate and the counter returns to preload.
+	if req.reserved != estimate {
+		t.Fatalf("req.reserved = %d, want the estimate %d (roll back on shed)", req.reserved, estimate)
+	}
+	if got := mc.wireBytes.Load(); got != preload+estimate {
+		t.Fatalf("wireBytes = %d, want preload+estimate %d (delta must be rolled back)",
+			got, preload+estimate)
+	}
+	mc.releaseWireBytes(req.reserved)
+	if got := mc.wireBytes.Load(); got != preload {
+		t.Fatalf("after release wireBytes = %d, want preload %d", got, preload)
+	}
+}
+
+// TestCSCMissCoalescerBoundsConcurrentSerialize pins #3965 F3: the serialization
+// admission caps HOW MANY callers hold a wire snapshot at once
+// (cscMissMaxConcurrentSerialize), so a concurrent burst cannot coexist an unbounded
+// NUMBER of snapshots. It bounds peak snapshot memory to cap x per-snapshot size, NOT
+// to the wire budget: a single undercounted argument (e.g. a BinaryMarshaler returning
+// hundreds of MiB) still allocates its full snapshot before reconcileWireBytes sheds
+// it, so up to cap such snapshots can coexist. A per-request byte ceiling (bounding the
+// serializer itself, and capping buildCacheKey which serializes uncapped upstream) is a
+// deferred follow-up. Single-threaded and deterministic: it fills the semaphore, then
+// confirms further acquisitions shed (never exceed the cap) and that a release frees
+// exactly one slot.
+func TestCSCMissCoalescerBoundsConcurrentSerialize(t *testing.T) {
+	// Shed immediately when the sem is full, so the over-cap attempts are
+	// deterministic (no timing dependence on the wait).
+	origWait := cscMissSerializeWait
+	cscMissSerializeWait = 0
+	defer func() { cscMissSerializeWait = origWait }()
+
+	const cap = cscMissMaxConcurrentSerialize
+	mc := &cscMissCoalescer{
+		serializeSem: make(chan struct{}, cap),
+		stop:         make(chan struct{}),
+	}
+
+	// Fill every slot: the sem has exactly cap slots, so all cap acquisitions win.
+	for i := 0; i < cap; i++ {
+		if !mc.acquireSerialize() {
+			t.Fatalf("acquire %d/%d failed with the sem not yet full", i+1, cap)
+		}
+	}
+	if got := int(mc.maxSerializing.Load()); got != cap {
+		t.Fatalf("maxSerializing = %d after filling; want %d", got, cap)
+	}
+
+	// With every slot held, further acquisitions must SHED — never past the cap.
+	for i := 0; i < cap*3; i++ {
+		if mc.acquireSerialize() {
+			t.Fatal("acquireSerialize admitted a caller past the cap (#3965 F3)")
+		}
+	}
+	if got := int(mc.maxSerializing.Load()); got != cap {
+		t.Fatalf("maxSerializing = %d after over-cap attempts; want %d (the cap must bind)", got, cap)
+	}
+
+	// A release frees exactly one slot.
+	mc.releaseSerialize()
+	if !mc.acquireSerialize() {
+		t.Fatal("acquireSerialize failed after a release freed a slot")
+	}
+}
+
+// TestCSCMissCoalescerFetchShedsWhenSerializeSaturated pins that fetch actually
+// CONSULTS the serialization admission (#3965 F3): with every slot held and a zero
+// wait, a fetch must shed to the pooled path (errCSCRetryUncached) before it
+// serializes or enqueues. This is the fetch-level red-check the admission-only test
+// lacks — remove the acquireSerialize call from fetch and this hangs on the
+// un-settled reply wait (timeout failure).
+func TestCSCMissCoalescerFetchShedsWhenSerializeSaturated(t *testing.T) {
+	origWait := cscMissSerializeWait
+	cscMissSerializeWait = 0 // shed immediately when the sem is full
+	defer func() { cscMissSerializeWait = origWait }()
+
+	cache := NewLocalCache(CacheConfig{MaxEntries: 8})
+	mc := &cscMissCoalescer{
+		c:            &baseClient{opt: &Options{}, csc: cache},
+		ch:           make(chan *cscMissReq, 1),
+		stop:         make(chan struct{}),
+		serializeSem: make(chan struct{}, cscMissMaxConcurrentSerialize),
+	}
+	// Saturate every serialization slot so fetch's acquireSerialize must shed.
+	for i := 0; i < cscMissMaxConcurrentSerialize; i++ {
+		mc.serializeSem <- struct{}{}
+	}
+
+	token, fetch := cache.Reserve("ck", []string{"rk"})
+	if token == 0 || !fetch {
+		t.Fatalf("Reserve = (%d, %v); want a fresh reservation", token, fetch)
+	}
+
+	errCh := make(chan error, 1)
+	go func() {
+		_, err := mc.fetch(context.Background(), makeCmd("get", "rk"), "ck", token)
+		errCh <- err
+	}()
+
+	select {
+	case err := <-errCh:
+		if err != errCSCRetryUncached {
+			t.Fatalf("fetch returned %v; want errCSCRetryUncached (a saturated serialization "+
+				"admission must shed before serializing/enqueueing)", err)
+		}
+	case <-time.After(2 * time.Second):
+		t.Fatal("fetch did not shed on a saturated serialization admission; it must consult " +
+			"acquireSerialize before writeCmd (#3965 F3)")
+	}
+}

internal/pool/conn.go

@@ -11,10 +11,11 @@ import (
 	"sync/atomic"
 	"time"
 
+	uberatomic "go.uber.org/atomic"
+
 	"github.com/redis/go-redis/v9/internal"
 	"github.com/redis/go-redis/v9/internal/maintnotifications/logs"
 	"github.com/redis/go-redis/v9/internal/proto"
-	uberatomic "go.uber.org/atomic"
 )
 
 var noDeadline = time.Time{}
@@ -76,6 +77,27 @@ func generateConnID() uint64 {
 	return connIDCounter.Add(1)
 }
 
+// relaxedState is a snapshot of one connection's relaxed-timeout window. Conn
+// publishes it through the relaxed atomic pointer. Do not change a stored
+// relaxedState. Each mutator copies the current snapshot, changes the copy, and
+// installs the copy with a compare-and-swap. A reader that holds an older pointer
+// still sees a consistent window.
+//
+// Two sources use this state (see the relaxed-timeout methods):
+//   - Maintenance notifications. SetRelaxedTimeout adds a holder.
+//     ClearRelaxedTimeout removes it. There is no deadline.
+//   - Handoff. SetRelaxedTimeoutWithDeadline sets a deadline that expires the
+//     window. Handoff never calls Clear.
+//
+// count is the number of current holders. A deadline window holds one slot only,
+// even after re-arms, because there is one deadlineNs.
+type relaxedState struct {
+	readNs     int64 // relaxed read timeout, nanoseconds
+	writeNs    int64 // relaxed write timeout, nanoseconds
+	deadlineNs int64 // auto-expiry, unix nanos; 0 = no deadline
+	count      int32 // number of holders; the window clears when count reaches 0
+}
+
 type Conn struct {
 	// Connection identifier for unique tracking
 	id uint64
@@ -141,17 +163,17 @@ type Conn struct {
 	// by ConnPool.Put.
 	closeOnPutReason uberatomic.String
 
-	// maintenanceNotifications upgrade support: relaxed timeouts during migrations/failovers
-
-	// Using atomic operations for lock-free access to avoid mutex contention
-	relaxedReadTimeoutNs  atomic.Int64 // time.Duration as nanoseconds
-	relaxedWriteTimeoutNs atomic.Int64 // time.Duration as nanoseconds
-	relaxedDeadlineNs     atomic.Int64 // time.Time as nanoseconds since epoch
-
-	// Counter to track multiple relaxed timeout setters if we have nested calls
-	// will be decremented when ClearRelaxedTimeout is called or deadline is reached
-	// if counter reaches 0, we clear the relaxed timeouts
-	relaxedCounter atomic.Int32
+	// relaxed holds the relaxed-timeout window for maintenance notifications
+	// (migrations and failovers). One atomic pointer publishes the whole window: the
+	// read timeout, the write timeout, the optional deadline, and the holder count.
+	// A reader always gets a consistent snapshot. A reader never sees a half-updated
+	// window, such as a new deadline with old timeouts. nil means no relaxation. The
+	// mutators (SetRelaxedTimeout, SetRelaxedTimeoutWithDeadline,
+	// ClearRelaxedTimeout, expireRelaxedTimeout) install a new relaxedState with a
+	// compare-and-swap. The read path (getEffective* and HasRelaxedTimeout) does one
+	// lock-free Load. Each mutation allocates one relaxedState. This cost is small,
+	// because mutations occur per maintenance notification, not per I/O.
+	relaxed atomic.Pointer[relaxedState]
 
 	// onClose is read and cleared by Close while initConn (running inside
 	// SetNetConnAndInitConn under the INITIALIZING state) installs it via
@@ -228,6 +250,7 @@ func NewConnWithBufferSize(netConn net.Conn, readBufSize, writeBufSize int) *Con
 func (cn *Conn) UsedAt() time.Time {
 	return time.Unix(0, cn.usedAt.Load())
 }
+
 func (cn *Conn) SetUsedAt(tm time.Time) {
 	cn.usedAt.Store(tm.UnixNano())
 }
@@ -235,6 +258,7 @@ func (cn *Conn) SetUsedAt(tm time.Time) {
 func (cn *Conn) UsedAtNs() int64 {
 	return cn.usedAt.Load()
 }
+
 func (cn *Conn) SetUsedAtNs(ns int64) {
 	cn.usedAt.Store(ns)
 }
@@ -242,6 +266,7 @@ func (cn *Conn) SetUsedAtNs(ns int64) {
 func (cn *Conn) LastPutAtNs() int64 {
 	return cn.lastPutAt.Load()
 }
+
 func (cn *Conn) SetLastPutAtNs(ns int64) {
 	cn.lastPutAt.Store(ns)
 }
@@ -506,145 +531,210 @@ func (cn *Conn) IsPubSub() bool {
 	return cn.pubsub
 }
 
-// SetRelaxedTimeout sets relaxed timeouts for this connection during maintenanceNotifications upgrades.
-// These timeouts will be used for all subsequent commands until the deadline expires.
-// Uses atomic operations for lock-free access.
-// Note: Metrics should be recorded by the caller (notification handler) which has context about
-// the notification type and pool name.
+// SetRelaxedTimeout sets the relaxed timeouts for this connection during a
+// maintenance-notification upgrade. They apply to every later command until an
+// equal number of ClearRelaxedTimeout calls remove this holder. This method is
+// lock-free. It installs a new snapshot with a compare-and-swap and keeps any
+// deadline that is already in effect.
+// Note: the caller (the notification handler) records the metrics, because it
+// knows the notification type and the pool name.
 func (cn *Conn) SetRelaxedTimeout(readTimeout, writeTimeout time.Duration) {
-	cn.relaxedCounter.Add(1)
-	cn.relaxedReadTimeoutNs.Store(int64(readTimeout))
-	cn.relaxedWriteTimeoutNs.Store(int64(writeTimeout))
-}
-
-// SetRelaxedTimeoutWithDeadline sets relaxed timeouts with an expiration deadline.
-// After the deadline, timeouts automatically revert to normal values.
-// Uses atomic operations for lock-free access.
-func (cn *Conn) SetRelaxedTimeoutWithDeadline(readTimeout, writeTimeout time.Duration, deadline time.Time) {
-	cn.SetRelaxedTimeout(readTimeout, writeTimeout)
-	cn.relaxedDeadlineNs.Store(deadline.UnixNano())
-}
-
-// ClearRelaxedTimeout removes relaxed timeouts, returning to normal timeout behavior.
-// Uses atomic operations for lock-free access.
-func (cn *Conn) ClearRelaxedTimeout() {
-	// Atomically decrement counter and check if we should clear
-	newCount := cn.relaxedCounter.Add(-1)
-	deadlineNs := cn.relaxedDeadlineNs.Load()
-	if newCount <= 0 && (deadlineNs == 0 || time.Now().UnixNano() >= deadlineNs) {
-		// Use atomic load to get current value for CAS to avoid stale value race
-		current := cn.relaxedCounter.Load()
-		if current <= 0 && cn.relaxedCounter.CompareAndSwap(current, 0) {
-			cn.clearRelaxedTimeout()
+	for {
+		cur := cn.relaxed.Load()
+		var next relaxedState
+		if cur != nil {
+			next = *cur
+		}
+		next.readNs = int64(readTimeout)
+		next.writeNs = int64(writeTimeout)
+		next.count++
+		if cn.relaxed.CompareAndSwap(cur, &next) {
+			return
 		}
 	}
 }
 
-func (cn *Conn) clearRelaxedTimeout() {
-	cn.relaxedReadTimeoutNs.Store(0)
-	cn.relaxedWriteTimeoutNs.Store(0)
-	cn.relaxedDeadlineNs.Store(0)
-	cn.relaxedCounter.Store(0)
-
-	// Note: Metrics for timeout unrelaxing are not recorded here because we don't have
-	// context about which notification type or pool triggered the relaxation.
-	// In practice, relaxed timeouts expire automatically via deadline, so explicit
-	// unrelaxing metrics are less critical than the initial relaxation metrics.
+// SetRelaxedTimeoutWithDeadline sets the relaxed timeouts and a deadline. After
+// the deadline the window reverts on its own, because handoff never calls Clear.
+// Only the first deadline window takes a holder slot. An overlapping handoff
+// re-arms the window: it replaces the deadline in place and does not add a holder.
+// So the later expiry removes exactly one holder and the window clears. The
+// connection does not stay relaxed forever. This method is lock-free.
+func (cn *Conn) SetRelaxedTimeoutWithDeadline(readTimeout, writeTimeout time.Duration, deadline time.Time) {
+	deadlineNs := deadline.UnixNano()
+	for {
+		cur := cn.relaxed.Load()
+		var next relaxedState
+		if cur != nil {
+			next = *cur
+		}
+		next.readNs = int64(readTimeout)
+		next.writeNs = int64(writeTimeout)
+		if next.deadlineNs == 0 {
+			// There is no deadline holder yet. Take one. A re-arm already has a
+			// deadline set, so it replaces the deadline and does not add a holder.
+			next.count++
+		}
+		next.deadlineNs = deadlineNs
+		if cn.relaxed.CompareAndSwap(cur, &next) {
+			return
+		}
+	}
 }
 
-// HasRelaxedTimeout returns true if relaxed timeouts are currently active on this connection.
-// This checks both the timeout values and the deadline (if set).
-// Uses atomic operations for lock-free access.
+// ClearRelaxedTimeout removes one holder that SetRelaxedTimeout added. When the
+// last holder is gone and no unexpired deadline remains, it drops the whole
+// window. The count has a lower bound of zero. So a second clear, or a clear after
+// a deadline expiry already emptied the window, does nothing and cannot block a
+// later relaxation. This method is lock-free.
+func (cn *Conn) ClearRelaxedTimeout() {
+	for {
+		cur := cn.relaxed.Load()
+		if cur == nil || cur.count <= 0 {
+			return // already cleared (for example, by a deadline expiry)
+		}
+		next := *cur
+		next.count--
+		// Keep the deadline gate. After an explicit clear the window can still hold
+		// the relaxed timeouts until the safety deadline. Drop the window only when
+		// the last holder leaves and the deadline is unset or already past.
+		newPtr := &next
+		if next.count <= 0 && (next.deadlineNs == 0 || time.Now().UnixNano() >= next.deadlineNs) {
+			newPtr = nil
+		}
+		if cn.relaxed.CompareAndSwap(cur, newPtr) {
+			return
+		}
+	}
+}
+
+// expireRelaxedTimeout removes the deadline holder after the deadline passes.
+// getEffective* calls it when a read finds the deadline expired. The
+// compare-and-swap checks that the current snapshot still holds THIS deadline. A
+// concurrent SetRelaxedTimeout* can install a newer window; then this expiry is
+// stale and does nothing. It removes only the deadline holder (count minus one).
+// A notification holder on the same connection stays active. The window clears in
+// full only when the deadline holder was the last holder. One snapshot publishes
+// the whole window, so a reader never sees a half-cleared state.
+func (cn *Conn) expireRelaxedTimeout(deadlineNs int64) {
+	for {
+		cur := cn.relaxed.Load()
+		if cur == nil || cur.deadlineNs != deadlineNs {
+			// Already cleared, or replaced by a newer window: stale expiry.
+			return
+		}
+		next := *cur
+		next.deadlineNs = 0
+		if next.count > 0 {
+			next.count--
+		}
+		newPtr := &next
+		if next.count <= 0 {
+			newPtr = nil
+		}
+		if cn.relaxed.CompareAndSwap(cur, newPtr) {
+			internal.Logger.Printf(context.Background(), logs.UnrelaxedTimeoutAfterDeadline(cn.GetID()))
+			return
+		}
+	}
+}
+
+// HasRelaxedTimeout returns true when the relaxed timeouts are active on this
+// connection. Active means a holder is present, a timeout is set, and any deadline
+// is still in the future. When the deadline has passed it removes the deadline holder
+// once (like getEffective*) and re-reads the snapshot: a surviving non-deadline holder
+// keeps the window active. Before this, an expired deadline made it report false even
+// while another holder was still active, contradicting the surviving-holder semantics.
+// This reports the boolean only; the timeout VALUE it would relax to may still be the
+// expired holder's, since all holders share one (readNs, writeNs) pair — see the
+// per-holder timeout follow-up.
 func (cn *Conn) HasRelaxedTimeout() bool {
-	// Fast path: no relaxed timeouts are set
-	if cn.relaxedCounter.Load() <= 0 {
+	cur := cn.relaxed.Load()
+	if cur == nil || cur.count <= 0 || (cur.readNs <= 0 && cur.writeNs <= 0) {
 		return false
 	}
-
-	readTimeoutNs := cn.relaxedReadTimeoutNs.Load()
-	writeTimeoutNs := cn.relaxedWriteTimeoutNs.Load()
-
-	// If no relaxed timeouts are set, return false
-	if readTimeoutNs <= 0 && writeTimeoutNs <= 0 {
-		return false
-	}
-
-	deadlineNs := cn.relaxedDeadlineNs.Load()
-	// If no deadline is set, relaxed timeouts are active
-	if deadlineNs == 0 {
+	if cur.deadlineNs == 0 || time.Now().UnixNano() < cur.deadlineNs {
 		return true
 	}
-
-	// If deadline is set, check if it's still in the future
-	return time.Now().UnixNano() < deadlineNs
+	// The deadline passed. Remove the deadline holder, then re-read: a notification
+	// holder can still be active with no deadline (surviving-holder semantics), matching
+	// getEffectiveReadTimeout.
+	cn.expireRelaxedTimeout(cur.deadlineNs)
+	s := cn.relaxed.Load()
+	if s == nil || s.count <= 0 || (s.readNs <= 0 && s.writeNs <= 0) {
+		return false
+	}
+	return s.deadlineNs == 0 || time.Now().UnixNano() < s.deadlineNs
 }
 
-// getEffectiveReadTimeout returns the timeout to use for read operations.
-// If relaxed timeout is set and not expired, it takes precedence over the provided timeout.
-// This method automatically clears expired relaxed timeouts using atomic operations.
+// EffectiveReadTimeout reports the read timeout that a read on this connection
+// uses now. It returns the active relaxed timeout when the window is set and
+// unexpired. Otherwise it returns normalTimeout. A caller that bounds how long a
+// blocked read may take (for example, the CSC full-duplex drain backstop) must use
+// this method, not the configured ReadTimeout. With ReadTimeout, a relaxed read
+// ends too soon.
+//
+// It is safe to call from several goroutines at once (the reader, the writer, and
+// the drain backstop of a full-duplex connection). It reads one atomic snapshot.
+// The only change it makes is the deadline safety net: when the deadline has
+// passed, it removes the deadline holder once (see expireRelaxedTimeout). It never
+// removes an active, unexpired window. If a notification holder survives that
+// expiry, it returns that holder's relaxed timeout, not normalTimeout.
+func (cn *Conn) EffectiveReadTimeout(normalTimeout time.Duration) time.Duration {
+	return cn.getEffectiveReadTimeout(normalTimeout)
+}
+
+// EffectiveWriteTimeout is the write side of EffectiveReadTimeout. Use it to bound
+// how long a blocked write may take. It has the same snapshot behavior.
+func (cn *Conn) EffectiveWriteTimeout(normalTimeout time.Duration) time.Duration {
+	return cn.getEffectiveWriteTimeout(normalTimeout)
+}
+
+// getEffectiveReadTimeout returns the timeout for read operations. It returns the
+// relaxed read timeout while the window is set and unexpired. Otherwise it returns
+// normalTimeout. When the deadline has passed, it removes the deadline holder and
+// then reads the window again. A surviving notification holder's relaxed timeout
+// still takes priority over normalTimeout.
 func (cn *Conn) getEffectiveReadTimeout(normalTimeout time.Duration) time.Duration {
-	readTimeoutNs := cn.relaxedReadTimeoutNs.Load()
-
-	// Fast path: no relaxed timeout set
-	if readTimeoutNs <= 0 {
+	cur := cn.relaxed.Load()
+	if cur == nil || cur.readNs <= 0 {
 		return normalTimeout
 	}
-
-	deadlineNs := cn.relaxedDeadlineNs.Load()
-	// If no deadline is set, use relaxed timeout
-	if deadlineNs == 0 {
-		return time.Duration(readTimeoutNs)
+	if cur.deadlineNs == 0 {
+		return time.Duration(cur.readNs)
 	}
-
-	// Use cached time to avoid expensive syscall (max 50ms staleness is acceptable for timeout checks)
-	nowNs := getCachedTimeNs()
-	// Check if deadline has passed
-	if nowNs < deadlineNs {
-		// Deadline is in the future, use relaxed timeout
-		return time.Duration(readTimeoutNs)
-	} else {
-		// Deadline has passed, clear relaxed timeouts atomically and use normal timeout
-		newCount := cn.relaxedCounter.Add(-1)
-		if newCount <= 0 {
-			internal.Logger.Printf(context.Background(), logs.UnrelaxedTimeoutAfterDeadline(cn.GetID()))
-			cn.clearRelaxedTimeout()
-		}
-		return normalTimeout
+	// Use the cached time to avoid an expensive system call. Up to 50 ms of
+	// staleness is acceptable.
+	if getCachedTimeNs() < cur.deadlineNs {
+		return time.Duration(cur.readNs)
 	}
+	// The deadline passed. Remove the deadline holder, then read the window again.
+	// A notification holder can still be active with no deadline. Use its relaxed
+	// timeout. Do not end this call early with normalTimeout.
+	cn.expireRelaxedTimeout(cur.deadlineNs)
+	if s := cn.relaxed.Load(); s != nil && s.readNs > 0 && (s.deadlineNs == 0 || getCachedTimeNs() < s.deadlineNs) {
+		return time.Duration(s.readNs)
+	}
+	return normalTimeout
 }
 
-// getEffectiveWriteTimeout returns the timeout to use for write operations.
-// If relaxed timeout is set and not expired, it takes precedence over the provided timeout.
-// This method automatically clears expired relaxed timeouts using atomic operations.
+// getEffectiveWriteTimeout is the write side of getEffectiveReadTimeout.
 func (cn *Conn) getEffectiveWriteTimeout(normalTimeout time.Duration) time.Duration {
-	writeTimeoutNs := cn.relaxedWriteTimeoutNs.Load()
-
-	// Fast path: no relaxed timeout set
-	if writeTimeoutNs <= 0 {
+	cur := cn.relaxed.Load()
+	if cur == nil || cur.writeNs <= 0 {
 		return normalTimeout
 	}
-
-	deadlineNs := cn.relaxedDeadlineNs.Load()
-	// If no deadline is set, use relaxed timeout
-	if deadlineNs == 0 {
-		return time.Duration(writeTimeoutNs)
+	if cur.deadlineNs == 0 {
+		return time.Duration(cur.writeNs)
 	}
-
-	// Use cached time to avoid expensive syscall (max 50ms staleness is acceptable for timeout checks)
-	nowNs := getCachedTimeNs()
-	// Check if deadline has passed
-	if nowNs < deadlineNs {
-		// Deadline is in the future, use relaxed timeout
-		return time.Duration(writeTimeoutNs)
-	} else {
-		// Deadline has passed, clear relaxed timeouts atomically and use normal timeout
-		newCount := cn.relaxedCounter.Add(-1)
-		if newCount <= 0 {
-			internal.Logger.Printf(context.Background(), logs.UnrelaxedTimeoutAfterDeadline(cn.GetID()))
-			cn.clearRelaxedTimeout()
-		}
-		return normalTimeout
+	if getCachedTimeNs() < cur.deadlineNs {
+		return time.Duration(cur.writeNs)
 	}
+	cn.expireRelaxedTimeout(cur.deadlineNs)
+	if s := cn.relaxed.Load(); s != nil && s.writeNs > 0 && (s.deadlineNs == 0 || getCachedTimeNs() < s.deadlineNs) {
+		return time.Duration(s.writeNs)
+	}
+	return normalTimeout
 }
 
 // SetOnClose installs fn as the callback invoked exactly once when this
@@ -1005,6 +1095,13 @@ func (cn *Conn) ClearHandoffState() {
 	cn.stateMachine.Transition(StateIdle)
 }
 
+// ExpiresAt returns the connection's absolute lifetime expiry (zero when no
+// ConnMaxLifetime applies; jitter included). Set once at dial, so a plain read
+// is safe. Long-holding callers bound their hold by the REMAINING lifetime.
+func (cn *Conn) ExpiresAt() time.Time {
+	return cn.expiresAt
+}
+
 // HasBufferedData safely checks if the connection has buffered data.
 // This method is used to avoid data races when checking for push notifications.
 func (cn *Conn) HasBufferedData() bool {
@@ -1071,6 +1168,14 @@ func (cn *Conn) WithReader(
 		if err := netConn.SetReadDeadline(cn.deadline(ctx, effectiveTimeout)); err != nil {
 			return err
 		}
+	} else {
+		// A negative timeout skips SetReadDeadline, and thus deadline(), which is
+		// the only per-I/O usedAt update. Record usage anyway so a long
+		// deadline-free hold (e.g. a full-duplex CSC session under ReadTimeout=-2)
+		// is not misjudged as idle-expired by the pool on the next Get and
+		// needlessly closed + redialed. Cheap: one atomic store, only on the rare
+		// deadline-free path; harmless on a nil netConn.
+		cn.SetUsedAtNs(getCachedTimeNs())
 	}
 	return fn(cn.rd)
 }
@@ -1114,6 +1219,11 @@ func (cn *Conn) WithWriter(
 			// Connection is not available - return preallocated error
 			return errConnNotAvailableForWrite
 		}
+	} else {
+		// See WithReader: keep usedAt fresh on the deadline-free write path too so
+		// a long-held conn (ReadTimeout/WriteTimeout=-2) is not misjudged as
+		// idle-expired by the pool on the next Get.
+		cn.SetUsedAtNs(getCachedTimeNs())
 	}
 
 	// Reset the buffered writer if needed, should not happen

internal/pool/conn_check.go

@@ -89,9 +89,20 @@ func maybeHasData(conn net.Conn) bool {
 }
 
 func checkForData(conn net.Conn) (bool, error) {
-	// Unlike the general health check, CSC only uses this as a readiness hint
-	// before a bounded read, so unwrapping TLS is safe and avoids blocking.
-	_ = conn.SetDeadline(time.Time{})
+	// Clear any residual READ deadline first: a prior command read (WithReader)
+	// leaves its deadline armed, and once it expires rawConn.Read fails fast
+	// with "raw-read ... i/o timeout" BEFORE the non-blocking peek runs — the
+	// caller would misread an idle-but-healthy conn as dead (the CSC drainer
+	// then removes it and evicts its cache coverage; with the full-duplex
+	// coalescer concentrating a cache's coverage on one held conn, that one
+	// spurious removal wipes the whole cache). Read-only on purpose, unlike the
+	// SetDeadline reset this replaced: checkForData runs CONCURRENTLY with
+	// command WRITES on a held full-duplex connection, and a full SetDeadline
+	// would strip an armed write deadline mid-write. No concurrent READ can
+	// race this: every caller either owns the conn's read side (FD session
+	// reader between reads) or holds the conn exclusively (drainer borrow,
+	// pool health check).
+	_ = conn.SetReadDeadline(time.Time{})
 	sysConn, ok := underlyingSyscallConn(conn)
 	if !ok {
 		return false, nil

internal/pool/conn_check_test.go

@@ -144,4 +144,16 @@ var _ = Describe("tests conn_check with real conns", func() {
 		Expect(connCheck(conn)).NotTo(HaveOccurred())
 		Expect(conn.Close()).NotTo(HaveOccurred())
 	})
+
+	It("checkForData survives an expired residual read deadline", func() {
+		// A prior WithReader leaves its read deadline armed; once expired,
+		// rawConn.Read would fail fast with i/o timeout before the peek runs.
+		// checkForData must clear it and report the idle conn healthy — the CSC
+		// drainer removes the conn (evicting its cache coverage) on any error.
+		Expect(conn.SetReadDeadline(time.Now().Add(-time.Second))).NotTo(HaveOccurred())
+		hasData, err := checkForData(conn)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(hasData).To(BeFalse())
+		Expect(conn.Close()).NotTo(HaveOccurred())
+	})
 })

internal/pool/conn_relaxed_timeout_test.go

@@ -7,6 +7,36 @@ import (
 	"time"
 )
 
+// Snapshot accessors for tests: the relaxed-timeout window is published as one
+// atomic pointer (nil == no relaxation), so read its fields through the snapshot.
+func (cn *Conn) relaxedHolderCount() int32 {
+	if s := cn.relaxed.Load(); s != nil {
+		return s.count
+	}
+	return 0
+}
+
+func (cn *Conn) relaxedReadNs() int64 {
+	if s := cn.relaxed.Load(); s != nil {
+		return s.readNs
+	}
+	return 0
+}
+
+func (cn *Conn) relaxedWriteNs() int64 {
+	if s := cn.relaxed.Load(); s != nil {
+		return s.writeNs
+	}
+	return 0
+}
+
+func (cn *Conn) relaxedDeadline() int64 {
+	if s := cn.relaxed.Load(); s != nil {
+		return s.deadlineNs
+	}
+	return 0
+}
+
 // TestConcurrentRelaxedTimeoutClearing tests the race condition fix in ClearRelaxedTimeout
 func TestConcurrentRelaxedTimeoutClearing(t *testing.T) {
 	// Create a dummy connection for testing
@@ -20,7 +50,7 @@ func TestConcurrentRelaxedTimeoutClearing(t *testing.T) {
 	cn.SetRelaxedTimeout(time.Second, time.Second)
 
 	// Verify counter is 3
-	if count := cn.relaxedCounter.Load(); count != 3 {
+	if count := cn.relaxedHolderCount(); count != 3 {
 		t.Errorf("Expected relaxed counter to be 3, got %d", count)
 	}
 
@@ -36,18 +66,229 @@ func TestConcurrentRelaxedTimeoutClearing(t *testing.T) {
 	wg.Wait()
 
 	// Verify counter is 0 and timeouts are cleared
-	if count := cn.relaxedCounter.Load(); count != 0 {
+	if count := cn.relaxedHolderCount(); count != 0 {
 		t.Errorf("Expected relaxed counter to be 0 after clearing, got %d", count)
 	}
-	if timeout := cn.relaxedReadTimeoutNs.Load(); timeout != 0 {
+	if timeout := cn.relaxedReadNs(); timeout != 0 {
 		t.Errorf("Expected relaxed read timeout to be 0, got %d", timeout)
 	}
-	if timeout := cn.relaxedWriteTimeoutNs.Load(); timeout != 0 {
+	if timeout := cn.relaxedWriteNs(); timeout != 0 {
 		t.Errorf("Expected relaxed write timeout to be 0, got %d", timeout)
 	}
 }
 
 // TestRelaxedTimeoutCounterRaceCondition tests the specific race condition scenario
+// TestRelaxedTimeoutExpiryConcurrentNoNegative reproduces the full-duplex race:
+// a reader, a writer, and the drain backstop all call Effective*Timeout on one
+// connection whose relaxed deadline has passed. The old expiry path decremented
+// the counter per call, so a single expired deadline was decremented many times
+// and left the counter negative — after which a later SetRelaxedTimeout could not
+// restore relaxation. The deadline-triggered CAS reset must fire exactly once.
+func TestRelaxedTimeoutExpiryConcurrentNoNegative(t *testing.T) {
+	netConn := &net.TCPConn{}
+	cn := NewConn(netConn)
+	defer cn.Close()
+
+	// One relax holder whose deadline is already in the past. Use a far-past
+	// deadline (getEffective* reads a cached clock with up to 50ms staleness, so a
+	// near-past deadline could read as unexpired and flake).
+	cn.SetRelaxedTimeoutWithDeadline(time.Second, time.Second, time.Now().Add(-time.Minute))
+
+	var wg sync.WaitGroup
+	for i := 0; i < 64; i++ {
+		wg.Add(3)
+		go func() { defer wg.Done(); cn.EffectiveReadTimeout(time.Second) }()  // reader
+		go func() { defer wg.Done(); cn.EffectiveWriteTimeout(time.Second) }() // writer
+		go func() { defer wg.Done(); cn.EffectiveReadTimeout(time.Second) }()  // drain backstop
+	}
+	wg.Wait()
+
+	// Reset fired once: counter is 0, never negative.
+	if count := cn.relaxedHolderCount(); count != 0 {
+		t.Fatalf("relaxed counter = %d after concurrent expiry, want 0 (never negative)", count)
+	}
+	// A fresh relaxation must take effect — a wedged negative counter would make
+	// HasRelaxedTimeout return false and the effective timeout stay normal.
+	cn.SetRelaxedTimeout(2*time.Second, 2*time.Second)
+	if !cn.HasRelaxedTimeout() {
+		t.Fatal("relaxation not restored after an expiry storm — counter was wedged")
+	}
+	if got := cn.EffectiveReadTimeout(time.Millisecond); got != 2*time.Second {
+		t.Fatalf("effective read timeout = %v, want restored relaxed 2s", got)
+	}
+}
+
+// TestRelaxedTimeoutExpiryKeepsNotificationWindow reproduces the notification-vs-
+// handoff clobber: a notification relaxation is outstanding on the SAME conn as an
+// expired handoff deadline. The old expiry path full-cleared the window, wiping
+// the notification's relaxation. The fixed path retires only the deadline holder,
+// so the notification window survives — and, because the whole window is one
+// snapshot, the very call that triggers the expiry re-reads it and returns the
+// surviving relaxed timeout rather than falling back to normal. Deterministic: no
+// goroutines, and a far-past deadline the cached clock always reads expired.
+func TestRelaxedTimeoutExpiryKeepsNotificationWindow(t *testing.T) {
+	netConn := &net.TCPConn{}
+	cn := NewConn(netConn)
+	defer cn.Close()
+
+	// Handoff relaxation with an already-expired deadline: counter=1, deadline past.
+	cn.SetRelaxedTimeoutWithDeadline(time.Second, time.Second, time.Now().Add(-time.Minute))
+	// Notification relaxation arrives on the same conn (no deadline): counter=2,
+	// timeouts overwritten to 2s, deadline still the expired handoff one.
+	cn.SetRelaxedTimeout(2*time.Second, 2*time.Second)
+	if count := cn.relaxedHolderCount(); count != 2 {
+		t.Fatalf("relaxed counter = %d after handoff+notification, want 2", count)
+	}
+
+	// The first Effective* observes the expired deadline, retires the handoff
+	// holder, then re-reads the surviving notification window: it returns the
+	// notification's 2s, NOT normal (a relaxed window is still active).
+	if got := cn.EffectiveReadTimeout(time.Millisecond); got != 2*time.Second {
+		t.Fatalf("first effective read = %v, want surviving notification relaxed 2s", got)
+	}
+
+	// Expiry retired only the handoff holder: the notification window must remain.
+	if count := cn.relaxedHolderCount(); count != 1 {
+		t.Fatalf("relaxed counter = %d after expiry, want 1 (notification holder kept)", count)
+	}
+	if !cn.HasRelaxedTimeout() {
+		t.Fatal("notification relaxation was clobbered by the handoff deadline expiry")
+	}
+	if got := cn.EffectiveReadTimeout(time.Millisecond); got != 2*time.Second {
+		t.Fatalf("effective read after expiry = %v, want notification relaxed 2s", got)
+	}
+	if got := cn.EffectiveWriteTimeout(time.Millisecond); got != 2*time.Second {
+		t.Fatalf("effective write after expiry = %v, want notification relaxed 2s", got)
+	}
+
+	// The surviving holder still clears cleanly on its explicit unrelax.
+	cn.ClearRelaxedTimeout()
+	if count := cn.relaxedHolderCount(); count != 0 {
+		t.Fatalf("relaxed counter = %d after clearing the notification holder, want 0", count)
+	}
+	if cn.HasRelaxedTimeout() {
+		t.Fatal("relaxation still active after the last holder cleared")
+	}
+}
+
+// TestHasRelaxedTimeoutExpiresDeadlineAndReadsSurvivor pins r3934148728: called on a
+// conn whose handoff DEADLINE holder has expired while a notification holder survives,
+// HasRelaxedTimeout must retire the deadline holder and re-read, reporting true — not
+// return false on the stale snapshot just because no Effective* call has triggered the
+// expiry yet. (TestRelaxedTimeoutExpiryKeepsNotificationWindow only reaches
+// HasRelaxedTimeout AFTER an Effective* already expired the deadline.)
+func TestHasRelaxedTimeoutExpiresDeadlineAndReadsSurvivor(t *testing.T) {
+	netConn := &net.TCPConn{}
+	cn := NewConn(netConn)
+	defer cn.Close()
+
+	// Expired handoff deadline holder + a surviving notification holder (no deadline).
+	cn.SetRelaxedTimeoutWithDeadline(time.Second, time.Second, time.Now().Add(-time.Minute))
+	cn.SetRelaxedTimeout(2*time.Second, 2*time.Second)
+	if count := cn.relaxedHolderCount(); count != 2 {
+		t.Fatalf("relaxed counter = %d, want 2", count)
+	}
+
+	// Call HasRelaxedTimeout FIRST (before any Effective*): it must expire the deadline
+	// holder and re-read, seeing the surviving notification holder -> true.
+	if !cn.HasRelaxedTimeout() {
+		t.Fatal("HasRelaxedTimeout returned false while a notification holder survives the expired deadline (r3934148728)")
+	}
+	// The expiry retired only the deadline holder.
+	if count := cn.relaxedHolderCount(); count != 1 {
+		t.Fatalf("relaxed counter = %d after HasRelaxedTimeout expiry, want 1 (notification holder kept)", count)
+	}
+}
+
+// TestRelaxedTimeoutOverlappingHandoffsClear guards the other direction: two
+// overlapping deadline-scoped handoffs on one conn must NOT leave it permanently
+// relaxed. The holder guard means the re-arm replaces the deadline in place
+// without stacking a holder, so the surviving deadline's expiry clears the window.
+func TestRelaxedTimeoutOverlappingHandoffsClear(t *testing.T) {
+	netConn := &net.TCPConn{}
+	cn := NewConn(netConn)
+	defer cn.Close()
+
+	// Handoff A, then handoff B re-arms before A's deadline. One holder, deadline B.
+	cn.SetRelaxedTimeoutWithDeadline(time.Second, time.Second, time.Now().Add(time.Hour))
+	cn.SetRelaxedTimeoutWithDeadline(2*time.Second, 2*time.Second, time.Now().Add(-time.Minute))
+	if count := cn.relaxedHolderCount(); count != 1 {
+		t.Fatalf("relaxed counter = %d after overlapping handoffs, want 1 (re-arm reuses holder)", count)
+	}
+
+	// B's deadline is expired: the next Effective* retires the only holder and the
+	// window clears — the conn does not stay relaxed for the rest of its life.
+	if got := cn.EffectiveReadTimeout(time.Millisecond); got != time.Millisecond {
+		t.Fatalf("effective read = %v, want normal 1ms after both handoffs expired", got)
+	}
+	if count := cn.relaxedHolderCount(); count != 0 {
+		t.Fatalf("relaxed counter = %d after expiry, want 0 (no leaked holder)", count)
+	}
+	if cn.HasRelaxedTimeout() {
+		t.Fatal("conn left permanently relaxed after overlapping handoffs expired")
+	}
+}
+
+// TestRelaxedTimeoutConcurrentSetAndRead schedules the race the snapshot design
+// must handle. One setter re-installs a deadline window many times, with
+// alternating past and future deadlines. At the same time 64 readers call
+// Effective*, which can observe an expiry and call expireRelaxedTimeout. Under
+// -race this must stay clean, because the reads and the compare-and-swaps operate
+// on one atomic pointer. The count must never go negative and must not drift above
+// one holder. A relaxation set after the storm must still take effect.
+func TestRelaxedTimeoutConcurrentSetAndRead(t *testing.T) {
+	netConn := &net.TCPConn{}
+	cn := NewConn(netConn)
+	defer cn.Close()
+
+	const iters = 500
+	var wg sync.WaitGroup
+
+	wg.Add(1)
+	go func() {
+		defer wg.Done()
+		for i := 0; i < iters; i++ {
+			if i%2 == 0 {
+				cn.SetRelaxedTimeoutWithDeadline(time.Second, time.Second, time.Now().Add(-time.Minute))
+			} else {
+				cn.SetRelaxedTimeoutWithDeadline(time.Second, time.Second, time.Now().Add(time.Hour))
+			}
+		}
+	}()
+	for r := 0; r < 64; r++ {
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			for i := 0; i < iters; i++ {
+				cn.EffectiveReadTimeout(time.Millisecond)
+				cn.EffectiveWriteTimeout(time.Millisecond)
+				if c := cn.relaxedHolderCount(); c < 0 {
+					t.Errorf("relaxed counter went negative (%d) under concurrent set/read", c)
+					return
+				}
+			}
+		}()
+	}
+	wg.Wait()
+
+	// One setter never stacks holders (the deadline slot is reused), so the counter
+	// is always 0 or 1 — never negative, never drifting up.
+	if c := cn.relaxedHolderCount(); c < 0 || c > 1 {
+		t.Fatalf("relaxed counter = %d after storm, want 0 or 1", c)
+	}
+	// Drain any residual holder, then prove a fresh relaxation still takes effect.
+	for cn.relaxedHolderCount() > 0 {
+		cn.ClearRelaxedTimeout()
+	}
+	cn.SetRelaxedTimeout(2*time.Second, 2*time.Second)
+	if !cn.HasRelaxedTimeout() {
+		t.Fatal("relaxation not restored after concurrent set/read storm — state wedged")
+	}
+	if got := cn.EffectiveReadTimeout(time.Millisecond); got != 2*time.Second {
+		t.Fatalf("effective read = %v, want restored relaxed 2s", got)
+	}
+}
+
 func TestRelaxedTimeoutCounterRaceCondition(t *testing.T) {
 	netConn := &net.TCPConn{}
 	cn := NewConn(netConn)
@@ -57,7 +298,7 @@ func TestRelaxedTimeoutCounterRaceCondition(t *testing.T) {
 	cn.SetRelaxedTimeout(time.Second, time.Second)
 
 	// Verify counter is 1
-	if count := cn.relaxedCounter.Load(); count != 1 {
+	if count := cn.relaxedHolderCount(); count != 1 {
 		t.Errorf("Expected relaxed counter to be 1, got %d", count)
 	}
 
@@ -75,18 +316,18 @@ func TestRelaxedTimeoutCounterRaceCondition(t *testing.T) {
 	wg.Wait()
 
 	// Verify final state is consistent
-	if count := cn.relaxedCounter.Load(); count != 0 {
+	if count := cn.relaxedHolderCount(); count != 0 {
 		t.Errorf("Expected relaxed counter to be 0 after concurrent clearing, got %d", count)
 	}
 
 	// Verify timeouts are actually cleared
-	if timeout := cn.relaxedReadTimeoutNs.Load(); timeout != 0 {
+	if timeout := cn.relaxedReadNs(); timeout != 0 {
 		t.Errorf("Expected relaxed read timeout to be cleared, got %d", timeout)
 	}
-	if timeout := cn.relaxedWriteTimeoutNs.Load(); timeout != 0 {
+	if timeout := cn.relaxedWriteNs(); timeout != 0 {
 		t.Errorf("Expected relaxed write timeout to be cleared, got %d", timeout)
 	}
-	if deadline := cn.relaxedDeadlineNs.Load(); deadline != 0 {
+	if deadline := cn.relaxedDeadline(); deadline != 0 {
 		t.Errorf("Expected relaxed deadline to be cleared, got %d", deadline)
 	}
 }

local_cache.go

@@ -41,6 +41,18 @@ type cacheEntry struct {
 	// Written under Lock (Set/Fulfill), read under RLock (get).
 	validAt time.Time
 
+	// fetchSeq is the global cscFetchSeq value at the moment this entry's fetch was
+	// ISSUED (Reserve), carried unchanged through fulfill. It lets a batched
+	// invalidation tell "this value predates me" from "this value was refetched
+	// after me": an invalidation snapshots cscFetchSeq at OBSERVE time, and a delete
+	// is skipped when entry.fetchSeq > that snapshot (the fetch was issued after the
+	// invalidate, so it reached a server that had already applied the write). Fetch-
+	// ISSUE order is used, not fulfill-COMPLETION order, because the invalidation and
+	// the reply travel on different connections with no ordering — a stale reply can
+	// fulfill after the invalidate is observed (see deleteByRedisKey). Set/read under
+	// the shard Lock.
+	fetchSeq uint64
+
 	// ownerConnID is the conn that fetched this entry (set by FulfillOwned; 0 =
 	// none). Default CLIENT TRACKING sends a key's invalidation only to that
 	// conn, so the entry must be evicted when it goes away (see EvictByConn).
@@ -56,6 +68,17 @@ func nextLRUToken() int64 {
 	return lruSequence.Add(1)
 }
 
+// cscFetchSeq is the global monotonic counter feeding cacheEntry.fetchSeq. It
+// totally orders fetch-ISSUE (Reserve) events against invalidation OBSERVE
+// events so a batched delete can skip an entry refetched after the invalidation
+// (see cacheEntry.fetchSeq).
+var cscFetchSeq atomic.Uint64
+
+// nextFetchSeq returns the next strictly-greater fetch-issue sequence.
+func nextFetchSeq() uint64 {
+	return cscFetchSeq.Add(1)
+}
+
 // CacheSizer calculates estimated memory usage in bytes for a cache entry.
 //
 // Experimental: this API may change in a minor release.
@@ -211,6 +234,17 @@ func NewLocalCache(cfg CacheConfig) *LocalCache {
 	return c
 }
 
+// effectiveMaxStaleness reports the cache's staleness bound (0 = none). Every
+// shard carries the same value, so shard 0 is authoritative. Used by
+// Options.init to run the batch-window-vs-staleness sanity warning for an
+// INJECTED *LocalCache too, where no ClientSideCacheConfig exists to read.
+func (c *LocalCache) effectiveMaxStaleness() time.Duration {
+	if len(c.shards) == 0 {
+		return 0
+	}
+	return c.shards[0].maxStaleness
+}
+
 // LocalCache is the built-in sharded approximate-LRU cache.
 //
 // Experimental: this API may change in a minor release.
@@ -223,6 +257,16 @@ type LocalCache struct {
 	nextToken atomic.Uint64
 	hits      atomic.Uint64
 	misses    atomic.Uint64
+
+	// Invalidation accounting for refresh-on-invalidate (see CSCRefreshStats).
+	// invalidations counts keys named in INCOMING pushes, tallied once at the
+	// handler choke point before dedup/batching. deletions/deletionsNoop count
+	// APPLIED deletes (post-dedup) and the subset that matched no live entry. The
+	// gap between invalidations and deletions is the direct measure of dedup +
+	// duplicate invalidations (and, under a flood, the spill-cap full-Flush).
+	invalidations atomic.Uint64
+	deletions     atomic.Uint64
+	deletionsNoop atomic.Uint64
 }
 
 var _ Cache = (*LocalCache)(nil)
@@ -420,6 +464,10 @@ func (c *LocalCache) Reserve(cacheKey string, redisKeys []string) (token uint64,
 		reservedAt: reservedAt,
 		waitCh:     waitCh,
 		sizeBytes:  sizeBytes,
+		// Stamp fetch-ISSUE order now so a later invalidation can tell a value
+		// refetched after it (keep) from one that predates it (evict). Carried
+		// through fulfill unchanged. See cacheEntry.fetchSeq.
+		fetchSeq: nextFetchSeq(),
 	}
 	entry.lastAccessNs.Store(nextLRUToken())
 
@@ -484,6 +532,24 @@ func (c *LocalCache) fulfill(cacheKey string, token, ownerConnID uint64, value [
 	return stillExists && current == entry && entry.state == cacheEntryValid
 }
 
+// restoreAccessToken resets a Valid entry's reader-access recency (lastAccessNs)
+// to accessNs, undoing the fresh token fulfill stamps. The refresh republish calls
+// this so a background refresh does NOT count as a reader access: otherwise the
+// refreshed key stays above the refresh horizon and every later invalidation
+// refreshes it again even after all readers stop — a self-sustaining refetch loop
+// contrary to the cold-key guard. A reader that get()s between the fulfill and this
+// call has its newer token overwritten, which only UNDER-refreshes that key (the
+// next reader refetches), never serves stale. No-op if the entry is gone or not
+// Valid.
+func (c *LocalCache) restoreAccessToken(cacheKey string, accessNs int64) {
+	s := c.shardFor(cacheKey)
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	if entry, ok := s.entries[cacheKey]; ok && entry.state == cacheEntryValid {
+		entry.lastAccessNs.Store(accessNs)
+	}
+}
+
 // EvictByConn removes every entry fetched by connID and returns the count.
 // Called when a conn is removed/swapped: the server stops delivering those
 // keys' invalidations, so keeping them risks stale serves. Errs toward a miss.
@@ -544,12 +610,19 @@ func (c *LocalCache) Cancel(cacheKey string, token uint64) bool {
 	return true
 }
 
-// DeleteByRedisKey removes entries associated with redisKey.
+// DeleteByRedisKey removes entries associated with redisKey. It is the applied-
+// delete path used when refresh-on-invalidate is off (the collecting variant is
+// used when it is on); counting deletions here keeps DeletionStats accurate on
+// both paths. The two are disjoint — neither calls the other — so no double count.
 func (c *LocalCache) DeleteByRedisKey(redisKey string) int {
 	removed := 0
 	for i := range c.shards {
 		removed += c.shards[i].deleteByRedisKey(redisKey)
 	}
+	c.deletions.Add(1)
+	if removed == 0 {
+		c.deletionsNoop.Add(1)
+	}
 	return removed
 }
 

local_cache_test.go

@@ -91,6 +91,40 @@ func TestLocalCache_DeleteByRedisKey_MultiKey(t *testing.T) {
 	}
 }
 
+// TestLocalCache_DeletionStats locks the applied-delete counters at both cache
+// choke points: DeleteByRedisKey (refresh-off path) and
+// deleteByRedisKeyCollectingHot (refresh-on path). A delete that matches a live
+// entry bumps deletions; one that matches nothing bumps deletions AND noop. The
+// incoming-push counter (InvalidationStats) is independent and must not move.
+func TestLocalCache_DeletionStats(t *testing.T) {
+	cache := NewLocalCache(CacheConfig{MaxEntries: 16})
+	cache.set("get:a", []string{"a"}, []byte("1"))
+
+	if removed := cache.DeleteByRedisKey("a"); removed != 1 {
+		t.Fatalf("DeleteByRedisKey(matching) removed = %d, want 1", removed)
+	}
+	if removed := cache.DeleteByRedisKey("a"); removed != 0 {
+		t.Fatalf("DeleteByRedisKey(missing) removed = %d, want 0", removed)
+	}
+	if del, noop := cache.DeletionStats(); del != 2 || noop != 1 {
+		t.Fatalf("after DeleteByRedisKey: DeletionStats = (%d, %d), want (2, 1)", del, noop)
+	}
+
+	// The collecting variant counts on the SAME counters (no double count: the two
+	// delete paths are disjoint).
+	cache.set("get:b", []string{"b"}, []byte("2"))
+	cache.deleteByRedisKeyCollectingHot("b", 0, ^uint64(0), nil) // matches -> deletion, not noop
+	cache.deleteByRedisKeyCollectingHot("b", 0, ^uint64(0), nil) // missing -> deletion + noop
+	if del, noop := cache.DeletionStats(); del != 4 || noop != 2 {
+		t.Fatalf("after collecting deletes: DeletionStats = (%d, %d), want (4, 2)", del, noop)
+	}
+
+	// Applied deletes are NOT incoming pushes: the invalidation counter stays 0.
+	if inv := cache.InvalidationStats(); inv != 0 {
+		t.Fatalf("InvalidationStats = %d, want 0 (deletes must not bump the incoming count)", inv)
+	}
+}
+
 func TestLocalCache_Flush(t *testing.T) {
 	cache := NewLocalCache(CacheConfig{MaxEntries: 16})
 	cache.set("get:a", []string{"a"}, []byte("1"))

options.go

@@ -136,6 +136,7 @@ type Options struct {
 	DialTimeout time.Duration
 
 	// DialerRetries is the maximum number of retry attempts when dialing fails.
+	// A value <= 0 uses the default.
 	//
 	// default: 5
 	DialerRetries int
@@ -428,6 +429,19 @@ type Options struct {
 	// namespace is selected. Fixed Username/Password values are supported and
 	// included in the cache namespace.
 	//
+	// Invalidation freshness: a server invalidation is applied when the client
+	// next reads from the connection that carries it — at arrival on a
+	// full-duplex autopipeline connection, at the next command on an active
+	// connection, and at the next background drainer tick on an idle pooled
+	// connection. In every case a cached entry is never served past the cache's
+	// MaxStaleness, which is the hard upper bound.
+	//
+	// Requires the built-in push processor. Setting a custom
+	// PushNotificationProcessor together with CSC is not supported: the
+	// invalidation mechanism relies on the built-in processor to consume
+	// kernel-only readability and tolerate boundary read timeouts, so behavior is
+	// otherwise undefined (the client logs a warning at init in that case).
+	//
 	// Experimental: this API may change in a minor release.
 	ClientSideCacheConfig *ClientSideCacheConfig
 
@@ -453,6 +467,88 @@ type Options struct {
 	//
 	// Experimental: this API may change in a minor release.
 	ClientSideCacheStrategy CSCStrategy
+
+	// ClientSideCacheRefreshOnInvalidate re-fetches recently-read keys as soon as
+	// their invalidation arrives, instead of waiting for a reader to miss.
+	//
+	// Requires the built-in cache (ClientSideCacheConfig, or ClientSideCache set
+	// to a *LocalCache), like the other CSC knobs: the refresher's hot-entry
+	// collection and publish path are LocalCache-specific. With a custom Cache
+	// implementation the option is ignored.
+	//
+	// Experimental: this API may change in a minor release.
+	ClientSideCacheRefreshOnInvalidate bool
+
+	// ClientSideCacheRefreshRecencyWindow bounds ClientSideCacheRefreshOnInvalidate
+	// to keys read within this window before their invalidation arrived. 0
+	// (default) refreshes every invalidated Valid entry, regardless of how long
+	// ago it was last read. A positive value only refreshes entries whose last
+	// read falls inside the window; an entry outside it is just evicted (an
+	// ordinary miss on the next read, same as refresh being off).
+	//
+	// Tradeoff: refreshing re-registers the key with the server's tracking
+	// table, which manufactures the NEXT invalidation on the next write — so
+	// the default (refresh everything) turns a write-heavy, rarely-read key
+	// into a self-sustaining refresh loop driven by write traffic, not read
+	// traffic, for as long as its entry survives capacity eviction. Set a
+	// window to bound that blast radius to keys actually being read.
+	//
+	// Recency is tracked at the existing 200ms tick resolution
+	// (cscRefreshRecencyTick): a nonzero window is rounded up to the tick
+	// boundary that guarantees AT LEAST the requested window is covered
+	// regardless of where an invalidation lands relative to the tick phase, so
+	// the enforced window is somewhere in [window, window+200ms). It also takes
+	// up to one tick to reach full accuracy right after the refresher starts.
+	//
+	// Ignored unless ClientSideCacheRefreshOnInvalidate is set, and requires
+	// the built-in cache like that option does.
+	//
+	// Experimental: this API may change in a minor release.
+	ClientSideCacheRefreshRecencyWindow time.Duration
+
+	// ClientSideCacheCoalesceMisses coalesces concurrent cache misses so they
+	// stream on a held tracked full-duplex connection instead of each taking a
+	// pool connection: a lone miss is written immediately (no batching delay —
+	// the caller is waiting), concurrent misses share writes opportunistically,
+	// and new misses go out while earlier replies are still in flight (~1 RTT
+	// per miss, no batch phase-lock). Cuts pool contention and the churn p99
+	// tail at a small pool.
+	//
+	// Requires the built-in cache (ClientSideCacheConfig, or ClientSideCache set
+	// to a *LocalCache): the coalescer's publish path (fetch capture, refresh
+	// integration, hot-entry collection) is LocalCache-specific. With a custom
+	// Cache implementation the option is ignored and every miss fetches on its
+	// caller's connection as usual.
+	//
+	// Pool sizing: under sustained miss traffic the engine holds one pool
+	// connection (released after a 1s idle gap and at the recycle age). Size
+	// PoolSize for that held connection — with PoolSize 1, continuous misses
+	// can make unrelated non-cacheable commands wait on the pool.
+	//
+	// Limiter: a coalescer session holds ONE connection and serves MANY misses on
+	// it, so Options.Limiter is admitted (Allow/ReportResult) once PER SESSION —
+	// per held connection — not per coalesced miss, unlike the plain per-command
+	// path. This is inherent to the held-connection model (a per-miss Allow would
+	// defeat the coalescing and re-admit a connection already held). A caller that
+	// needs strict per-command admission or circuit-breaking should not enable
+	// miss coalescing.
+	//
+	// Experimental: this API may change in a minor release.
+	ClientSideCacheCoalesceMisses bool
+
+	// ClientSideCacheInvalidationBatchWindow coalesces invalidation-driven cache
+	// deletes into windowed background batches instead of applying them inline on
+	// the connection reader. 0 (default) applies invalidations inline. Set it no
+	// larger than the cache MaxStaleness: deferring a delete by up to the window
+	// lets a reader see the pre-invalidation value for up to that long.
+	//
+	// Requires the built-in cache (ClientSideCacheConfig, or ClientSideCache set
+	// to a *LocalCache), like ClientSideCacheCoalesceMisses: the batcher's
+	// hot-entry refresh integration is LocalCache-specific. With a custom Cache
+	// implementation the window is ignored and deletes apply inline.
+	//
+	// Experimental: this API may change in a minor release.
+	ClientSideCacheInvalidationBatchWindow time.Duration
 }
 
 // CSCStrategy selects the client-side caching invalidation architecture. Set via
@@ -517,6 +613,27 @@ func (opt *Options) init() {
 			opt.ClientSideCacheStrategy)
 		opt.ClientSideCacheStrategy = CSCStrategySharedTracking
 	}
+	// Deferring invalidation deletes by more than the cache's MaxStaleness lets a
+	// reader see a value past the point a received invalidation should have evicted
+	// it — beyond the staleness contract. Warn (not clamp): the field is
+	// experimental and a caller may accept it knowingly, but a window larger than
+	// MaxStaleness is almost always a misconfiguration. Checkable for the built-in
+	// cache via its config AND for an injected *LocalCache via its effective bound;
+	// a custom Cache implementation has no MaxStaleness to compare against.
+	if opt.ClientSideCacheInvalidationBatchWindow > 0 {
+		staleness := time.Duration(0)
+		if opt.ClientSideCacheConfig != nil {
+			staleness = opt.ClientSideCacheConfig.MaxStaleness
+		} else if lc, ok := opt.ClientSideCache.(*LocalCache); ok && lc != nil {
+			staleness = lc.effectiveMaxStaleness()
+		}
+		if staleness > 0 && opt.ClientSideCacheInvalidationBatchWindow > staleness {
+			internal.Logger.Printf(context.Background(),
+				"redis: ClientSideCacheInvalidationBatchWindow (%s) exceeds the cache MaxStaleness (%s); "+
+					"invalidations can be deferred past the staleness bound, serving stale values",
+				opt.ClientSideCacheInvalidationBatchWindow, staleness)
+		}
+	}
 	if opt.Network == "" {
 		if strings.HasPrefix(opt.Addr, "/") {
 			opt.Network = "unix"
@@ -536,7 +653,11 @@ func (opt *Options) init() {
 	if opt.DialTimeout == 0 {
 		opt.DialTimeout = 5 * time.Second
 	}
-	if opt.DialerRetries == 0 {
+	// <= 0, not == 0: the pool already treats a nonpositive value as the default
+	// (internal/pool ConnPool.dialConn), so normalize here too — code that
+	// derives a budget from opt.DialerRetries (the miss-coalescer's acquire
+	// deadline) must see the same count the pool will actually use.
+	if opt.DialerRetries <= 0 {
 		opt.DialerRetries = 5
 	}
 	if opt.DialerRetryTimeout == 0 {

push/processor.go

@@ -125,17 +125,18 @@ func (p *Processor) processPendingNotifications(
 		// see if we should skip this notification
 		notificationName, err := rd.PeekPushNotificationName()
 		if err != nil {
-			// Name too long to peek: consume & dispatch below rather than leave the
-			// frame at the buffer head (which would stall and desync the next reply).
-			if !errors.Is(err, proto.ErrPushNotificationNameTooLong) {
-				if bufferedContinuation {
-					if isTimeoutError(err) {
-						return nil
-					}
-					return err
-				}
-				break
+			// A buffered probe stops cleanly when no full frame is available yet.
+			if bufferedContinuation && isTimeoutError(err) {
+				return nil
 			}
+			// The frame is a CONFIRMED push (peeked above) but its name cannot be
+			// peeked — too long, or a non-string / malformed name. Fall through to
+			// ReadReply to CONSUME it. Do NOT break: that leaves the push at the
+			// buffer head for the caller's reply read to eat as the command value
+			// (cache poison / one-frame reply shift). A genuine mid-frame read error
+			// surfaces at ReadReply below (fatal → the reply-expected drain retires
+			// the conn); a non-string name is gracefully ignored (the type assertion
+			// below fails, so no handler runs).
 		} else if willHandleNotificationInClient(notificationName) {
 			break
 		}

push/push_test.go

@@ -71,6 +71,29 @@ func TestProcessor_ConsumesFragmentedPushFrame(t *testing.T) {
 	}
 }
 
+// TestProcessor_ConsumesUnpeekableNamePush: a confirmed push whose NAME cannot be
+// peeked (here a non-string/integer name) must be CONSUMED by the blocking drain,
+// not left at the buffer head. If it is left, a reply-expected reader (the CSC
+// refresh / miss-coalescer / single-command drain) reads the push as the command
+// value — cache poison and a one-frame reply shift (the intermittent `redis: nil`).
+func TestProcessor_ConsumesUnpeekableNamePush(t *testing.T) {
+	// A push with an integer (non-string) name, then the real command reply.
+	stream := []byte(">2\r\n:123\r\n$4\r\ndata\r\n$2\r\nOK\r\n")
+	rd := proto.NewReader(bytes.NewReader(stream))
+	p := NewProcessor()
+	if err := p.ProcessPendingNotifications(context.Background(), NotificationHandlerContext{}, rd); err != nil {
+		t.Fatalf("ProcessPendingNotifications: %v", err)
+	}
+	// The push was consumed, so the next frame is the command reply — not the push.
+	reply, err := rd.ReadString()
+	if err != nil {
+		t.Fatalf("read reply after drain: %v (the unpeekable-name push was left and misread)", err)
+	}
+	if reply != "OK" {
+		t.Fatalf("reply after drain = %q, want OK (the push was left at the buffer head)", reply)
+	}
+}
+
 func TestProcessor_BufferedPeekTimeoutEndsBatch(t *testing.T) {
 	t.Run("empty boundary", func(t *testing.T) {
 		rd := proto.NewReader(&timeoutAfterReader{read: true})

redis.go

@@ -9,6 +9,7 @@ import (
 	"sync"
 	"sync/atomic"
 	"time"
+	"weak"
 
 	"github.com/redis/go-redis/v9/auth"
 	"github.com/redis/go-redis/v9/internal"
@@ -180,6 +181,16 @@ func (h *hooks) setDefaults() {
 //
 // Please note: "next(ctx, cmd)" is very important, it will call the next hook,
 // if "next(ctx, cmd)" is not executed, the redis command will not be executed.
+//
+// Contract. A hook runs inside the client's command machinery and must be
+// well-behaved, or behavior is undefined:
+//   - Call next (as above) unless deliberately terminating the command.
+//   - Do not call Close or other client control methods from a hook — the hook
+//     runs on the goroutine Close waits for, so this deadlocks.
+//   - Do not panic. A panicking hook can crash the process or leave an operation
+//     unsettled; wrap fallible work and return an error instead.
+//   - Do not mutate client or connection state. A hook may observe and wrap
+//     errors (call cmd.SetErr) but must not reconfigure the client mid-flight.
 func (hs *hooksMixin) AddHook(hook Hook) {
 	hs.hooksMu.Lock()
 	defer hs.hooksMu.Unlock()
@@ -416,6 +427,24 @@ type baseClient struct {
 	// identity. It is computed once during attachment and copied with the cache.
 	cscKeyPrefix string
 
+	// Refresh-on-invalidate + reader-miss coalescing (nil unless enabled).
+	// cscRefreshQueue IS copied by clone() (a clone signals demand on the owner's
+	// queue, see clone()); cscRefreshHandle is owner-only (it drives the goroutine
+	// that owns/stops the queue) and stays nil in a clone, so a clone never stops it.
+	cscRefreshQueue  *cscRefreshQueue
+	cscRefreshHandle *cscRevalidateHandle
+	// cscClientWeak points (weakly) back at the canonical *Client wrapper, so
+	// the CSC push-handler adapter can close through Client.Close — which also
+	// stops the cached autopipeliners — instead of baseClient.Close. WEAK on
+	// purpose: a strong back-reference would make the wrapper reachable from
+	// the drainer goroutine and the drop-without-Close cleanup
+	// (cscRegisterCleanups) could never fire.
+	cscClientWeak weak.Pointer[Client]
+	// cscMissCoalescer is atomic: a miss (processCached) races Close, which
+	// Swaps it to nil — readers Load once (a double-load could call fetch on a
+	// nil receiver) and exactly one closer wins the Swap.
+	cscMissCoalescer atomic.Pointer[cscMissCoalescer]
+
 	// allowClientTracking exempts a client from the CLIENT TRACKING guard (see
 	// process and generalProcessPipeline). Set only on initConn's internal conn
 	// wrapper, whose init pipeline legitimately issues CLIENT TRACKING ON;
@@ -475,7 +504,22 @@ func (c *baseClient) clone() *baseClient {
 		cscPoolHook:  c.cscPoolHook,
 		cscActive:    c.cscActive,
 		cscKeyPrefix: c.cscKeyPrefix,
+		// cscRefreshQueue is SHARED (pointer copy), like cscPoolHook: processCached
+		// calls signalDemand on it so a miss for a key still in the refresher's
+		// window flushes that window early. Without the share a clone's field is
+		// nil and signalDemand no-ops, so the clone's miss waits the full window
+		// (#3965 F4). Lifecycle stays owner-only: signalDemand is a nil-safe,
+		// non-blocking buffered send that does no I/O, and teardown keys off the
+		// owner-only cscRefreshHandle/cscDrainHandle (both nil in a clone), so a
+		// clone only SIGNALS the queue and never stops it — even a clone outliving
+		// the owner's Close just sends into a buffered channel whose worker has
+		// exited, which is harmless.
+		cscRefreshQueue: c.cscRefreshQueue,
 	}
+	// The miss coalescer travels with the cache (an atomic.Pointer cannot appear
+	// in the literal above); a clone shares the OWNER's coalescer, whose
+	// lifecycle stays owner-only — a clone's Close does not stop it.
+	clone.cscMissCoalescer.Store(c.cscMissCoalescer.Load())
 	return clone
 }
 
@@ -496,6 +540,21 @@ func (c *baseClient) withTimeout(timeout time.Duration) *baseClient {
 	clone := c.clone()
 	clone.opt = opt
 
+	// Do not route the clone's misses through the OWNER's coalescer when the
+	// timeouts diverge: the shared engine's writer/reader/acquire budgets and
+	// backstops all read the owner's options, so a coalesced miss would honor
+	// the owner's deadlines, not the clone's — defeating WithTimeout's whole
+	// point. The clone still serves cache hits and still caches its uncached
+	// fetches (the pre-coalescer CSC path); only miss COALESCING is bypassed.
+	if timeout != c.opt.ReadTimeout || timeout != c.opt.WriteTimeout {
+		clone.cscMissCoalescer.Store(nil)
+	}
+	// cscRefreshQueue is NOT dropped here even when timeouts diverge: unlike the
+	// coalescer it does no I/O on this client's behalf — signalDemand is a
+	// non-blocking buffered send — so there is no deadline to inherit, and the
+	// refresh work itself runs on the owner's connection under the owner's options
+	// regardless of who nudged it.
+
 	return clone
 }
 
@@ -504,22 +563,32 @@ func (c *baseClient) String() string {
 }
 
 func (c *baseClient) getConn(ctx context.Context) (*pool.Conn, error) {
+	cn, _, err := c.getConnLimited(ctx)
+	return cn, err
+}
+
+// getConnLimited is getConn with the Limiter admission split from the dial: the
+// `limited` return reports that Limiter.Allow REJECTED the operation (before any
+// dial), so a caller can tell an explicit admission denial apart from a
+// dial/transport failure. getConn wraps it for the common case, leaving every
+// other caller unchanged. Allow is reported (ReportResult) only for dial results,
+// never for admission denials — matching the original getConn.
+func (c *baseClient) getConnLimited(ctx context.Context) (cn *pool.Conn, limited bool, err error) {
 	if c.opt.Limiter != nil {
-		err := c.opt.Limiter.Allow()
-		if err != nil {
-			return nil, err
+		if err := c.opt.Limiter.Allow(); err != nil {
+			return nil, true, err
 		}
 	}
 
-	cn, err := c._getConn(ctx)
+	cn, err = c._getConn(ctx)
 	if err != nil {
 		if c.opt.Limiter != nil {
 			c.opt.Limiter.ReportResult(err)
 		}
-		return nil, err
+		return nil, false, err
 	}
 
-	return cn, nil
+	return cn, false, nil
 }
 
 func (c *baseClient) _getConn(ctx context.Context) (*pool.Conn, error) {
@@ -1494,9 +1563,21 @@ func (c *baseClient) _process(ctx context.Context, cmd Cmder, attempt int, captu
 	var retryTimeout atomic.Uint32
 	if err := c.withConn(ctx, func(ctx context.Context, cn *pool.Conn) error {
 		usedConn = cn
-		// Process any pending push notifications before executing the command
+		// Process any pending push notifications before executing the command. A
+		// non-nil error means the drain may have stopped mid-frame (a fragmented
+		// push straddling its hard read cap), leaving the reader desynced. Do NOT
+		// ignore it: reading this command's reply on the conn would then return
+		// shifted bytes — a wrong result, or a wrong-key cache on the CSC capture
+		// path. Close the conn so releaseConn removes it, mark the error retryable,
+		// and let processWithRetry re-run on a fresh conn. Benign cases return nil
+		// (peekAndProcessPushNotifications gates on MaybeHasData; the built-in
+		// processor propagates only genuine mid-frame errors and a custom processor
+		// is peeked first), so this fires only on a real desync.
 		if err := c.processPushNotifications(ctx, cn); err != nil {
-			internal.Logger.Printf(ctx, "push: error processing pending notifications before command: %v", err)
+			internal.Logger.Printf(ctx, "push: pre-command drain failed, retiring conn: %v", err)
+			_ = cn.Close()
+			retryTimeout.Store(1)
+			return err
 		}
 
 		// HIMPORT bookkeeping: pending discards for this session and the
@@ -2176,6 +2257,25 @@ func NewClient(opt *Options) *Client {
 			// We constructed it, so we own it (may flush on drainer stop).
 			c.baseClient.cscOwnsCache = true
 		}
+		// CSC is only supported with the built-in push processor: it depends on the
+		// built-in draining kernel-only readability and treating a boundary read timeout
+		// as benign. Warn (do not fail) when a custom processor is paired with CSC — the
+		// contract is that behavior is otherwise undefined. See .claude/specs/push.md.
+		if cache != nil {
+			if _, builtin := c.pushProcessor.(*push.Processor); !builtin {
+				internal.Logger.Printf(context.Background(),
+					"redis: client-side caching enabled with a custom PushNotificationProcessor; "+
+						"CSC is only supported with the built-in processor, behavior is otherwise undefined "+
+						"(invalidation freshness and idle-connection health not guaranteed)")
+			}
+			// Publish the weak back-reference to the canonical *Client BEFORE attachCSC
+			// starts the drainer. The push-handler adapter's canonical close reads
+			// cscClientWeak on the drainer goroutine; setting it afterwards (it used to be
+			// set only later, in cscRegisterCleanups) raced that read and could miss the
+			// close for a push that arrived in the window. Weak so a dropped *Client stays
+			// collectible and its GC cleanup still fires.
+			c.baseClient.cscClientWeak = weak.Make(&c)
+		}
 		c.baseClient.attachCSC(context.Background(), cache)
 
 		// Safety net for a client dropped without Close: the goroutines hold
@@ -2245,6 +2345,17 @@ func (c *Client) WithTimeout(timeout time.Duration) *Client {
 		clone.cscLifecycleOwner = c
 	}
 	clone.baseClient = c.baseClient.withTimeout(timeout)
+	// Route the clone's CSC push-handler Close through the OWNER wrapper.
+	// clone() copies neither cscDrainHandle nor cscClientWeak, so without this a
+	// custom push handler calling Close() on the cscHandlerClient installed for a
+	// held-conn drain on this clone would fall through to baseClient.Close —
+	// closing the SHARED pools while the owner's drainer and cached autopipeliners
+	// keep running against them. cscLifecycleOwner is the canonical wrapper and is
+	// kept alive by this clone's strong ref, so closeCanonical resolves it and
+	// calls owner.Close() (the full CSC + autopipeliner teardown).
+	if clone.cscLifecycleOwner != nil {
+		clone.baseClient.cscClientWeak = weak.Make(clone.cscLifecycleOwner)
+	}
 	clone.init()
 	return &clone
 }
@@ -2709,23 +2820,197 @@ func (c *baseClient) peekAndProcessPushNotifications(ctx context.Context, cn *po
 		return nil
 	}
 
-	if !cn.MaybeHasData() {
+	// Also drain when the reader already holds buffered bytes (HasBufferedData),
+	// not only when the socket is readable (MaybeHasData). A reply and a trailing
+	// invalidation can arrive in one socket read. The invalidation then stays in
+	// the reader buffer while the socket is empty. MaybeHasData alone would skip
+	// it and serve stale data until the next miss or MaxStaleness (#3965).
+	buffered := cn.HasBufferedData()
+	if !buffered && !cn.MaybeHasData() {
 		return nil
 	}
 
-	// Short read timeout: MaybeHasData confirmed kernel-buffered bytes, so
-	// the first read returns immediately — the deadline only needs to cover
-	// scheduler pauses, not network waits. 10us was routinely lost to
-	// scheduling on loaded machines: the peek then timed out with nothing
-	// consumed, the processor treated that as "no pending data", and a
-	// connection with buffered push bytes was returned to the pool instead
-	// of being drained (or removed, when the frame turns out partial).
-	return cn.WithReader(ctx, time.Millisecond, func(rd *proto.Reader) error {
-		handlerCtx := c.pushNotificationHandlerContext(cn)
-		return c.pushProcessor.ProcessPendingNotifications(ctx, handlerCtx, rd)
+	// Drain even for a custom processor only when the reader already holds buffered
+	// bytes. NOTE: HasBufferedData is byte-level (rd.Buffered() > 0), NOT proof of a
+	// COMPLETE frame — a reply plus a partial trailing push in one socket read
+	// leaves a partial push buffered. Handing that to a custom processor lets it
+	// block to complete the frame under the hard cap; a mid-frame timeout surfaces
+	// an error and the caller retires the (now-desynced) connection. That outcome
+	// is SAFE (retiring a desynced conn is correct), but it does NOT eliminate
+	// conn churn for custom processors — the earlier "buffered == a complete, safe
+	// frame" claim was wrong.
+	//
+	// The gate is still worth it against the WORSE case: MaybeHasData WITHOUT
+	// buffered data proves only that the raw socket is readable, which over TLS can
+	// be a post-handshake control record with zero RESP bytes (conn_check.go
+	// refuses to unwrap TLS for this reason; maybeHasData does). A custom processor
+	// handed that empty input times out on an empty read → the FD session reader
+	// treats the timeout as fatal and closes a HEALTHY connection, failing its
+	// in-flight misses. The built-in buffered processor accepts the empty read. So
+	// a custom processor drains only on real buffered data. Tradeoff: a push that
+	// arrives as bare socket readiness on a custom, idle FD session waits for the
+	// next reply read, the session recycle, or the MaxStaleness backstop; the
+	// connection stays open. Mirrors the built-in-only guard on the opaque-
+	// transport probe in the FD session tick (csc_miss_coalesce_modes.go).
+	if !buffered {
+		if _, builtin := c.pushProcessor.(*push.Processor); !builtin {
+			return nil
+		}
+	}
+
+	// Readiness is established, so a frame is (probably) present: drain under the
+	// longer fragmented-frame budget the background drainer uses, NOT the 1ms
+	// no-data probe cap. A push can arrive in fragments more than 1ms apart —
+	// notably over TLS, where MaybeHasData only proves that some ciphertext is
+	// readable — and the 1ms cap would then time out after consuming the prefix,
+	// which is treated as fatal and fatally closes a healthy session (failing its
+	// in-flight misses). The 1ms cap is reserved for speculative no-data probes
+	// (timedPushDrain, the opaque-transport fallback).
+	return c.pushDrainWithin(ctx, cn, cscDrainHardReadCap)
+}
+
+// timedPushDrain runs a speculative no-data push probe under a 1ms hard read
+// deadline: for callers WITHOUT a readiness signal (the opaque-transport
+// fallback on a held full-duplex connection), which pay at most 1ms when
+// nothing is pending. Callers that HAVE established readiness use
+// pushDrainWithin(cscDrainHardReadCap) instead, to tolerate fragmented frames.
+func (c *baseClient) timedPushDrain(ctx context.Context, cn *pool.Conn) error {
+	return c.pushDrainWithin(ctx, cn, time.Millisecond)
+}
+
+// pushDrainWithin runs one push drain on cn. HARD read deadlines (not WithReader) so
+// an ordinary drain cannot silently inherit an unrelated deadline, and
+// WithReaderHardDeadline clears the deadline on exit, so no residue poisons a later
+// deadline-less read.
+//
+// The FIRST-byte wait is bounded by the short cap d; only a frame already begun gets
+// the relaxation-aware budget (pushDrainBudget) to finish. Applying the relaxed budget
+// to the first-byte wait would let a TLS control record — socket-readable with zero
+// RESP bytes — block a speculative drain for the full relaxed timeout (seconds),
+// stalling a ready command or the idle drainer (#3989). So under active relaxation,
+// probe one byte under d first; a timeout there means no frame began (benign). The
+// relaxed budget exists only to tolerate a fragmented frame mid-flight during a
+// failover — see pushDrainBudget.
+func (c *baseClient) pushDrainWithin(ctx context.Context, cn *pool.Conn, d time.Duration) error {
+	budget := pushDrainBudget(cn, d)
+	if budget > d {
+		// Relaxation active: confirm a RESP byte under the short cap before granting the
+		// longer budget. The peeked byte stays buffered for the drain below.
+		if err := cn.WithReaderHardDeadline(d, func(rd *proto.Reader) error {
+			_, perr := rd.Peek(1)
+			return perr
+		}); err != nil {
+			if isTimeout, hasFlag := isTimeoutError(err); isTimeout && hasFlag {
+				return nil // no frame began within the short cap
+			}
+			return err
+		}
+	}
+	return cn.WithReaderHardDeadline(budget, func(rd *proto.Reader) error {
+		// A speculative probe on a possibly-idle conn: stop when the reader buffer
+		// empties (Buffered variant) rather than block waiting for a reply.
+		return c.drainPushFrames(ctx, cn, rd, false)
 	})
 }
 
+// pushDrainBudget returns the read budget for one push drain: the caller's hard
+// cap d, raised to the connection's relaxed timeout while maintenance relaxation
+// is active. The raise matters exactly when a push frame is mid-flight on a slow
+// server: during a failover/migration — the very window relaxation covers — a
+// frame can fragment past a small cap, and a mid-frame timeout is a desync
+// (partial frame consumed), so the caller retires the connection and, on the
+// pre-command path, fails the command outright when MaxRetries=0. Holding the
+// conn up to the relaxed budget is the lesser cost, and it is paid only when
+// bytes are actually mid-frame: an empty or completed drain returns without
+// blocking regardless of the budget. Without active relaxation
+// EffectiveReadTimeout returns d unchanged, keeping the small hard cap that
+// protects a small pool from a parked speculative drain.
+func pushDrainBudget(cn *pool.Conn, d time.Duration) time.Duration {
+	if rel := cn.EffectiveReadTimeout(d); rel > d {
+		return rel
+	}
+	return d
+}
+
+// drainPushFrames processes pending push notifications on rd. The caller already
+// holds rd under a read deadline (WithReader or WithReaderHardDeadline); this
+// helper does not set one, so it is safe to call from a reader that must keep
+// reading afterward (the CSC miss reader reads the command reply next).
+//
+// blocking selects the built-in processor's drain discipline:
+//
+//   - blocking=true (reply-expected readers: the CSC miss and refresh readers):
+//     block on the socket and skip push frames until a NON-push frame — the
+//     command reply — is next, then return leaving it buffered. This is the same
+//     non-buffered discipline the full-duplex reader already uses. The Buffered
+//     variant instead stops the instant the reader buffer empties; if a second
+//     invalidation is still on the socket ahead of the reply, the caller's
+//     ReadRawReply would then read THAT push as the reply and cache it under the
+//     wrong key — a one-frame shift that cascades to every later reply. Note
+//     PeekReplyType is attribute-aware (it discards a RESP3 attribute prefix and
+//     recurses), so a fragmented attribute needs no separate buffered scan. A
+//     boundary-peek TIMEOUT the non-buffered loop swallows is caught by the
+//     caller's shared read deadline: ReadRawReply hits the same expired deadline
+//     and fails the session. This is the same non-buffered discipline the
+//     full-duplex reader already runs; a swallowed NON-timeout peek error would
+//     need a malformed RESP3 attribute mid-push (a server protocol bug), so this
+//     path is no weaker than that one. Pub/sub-named pushes, which the drain
+//     leaves buffered, cannot reach a CSC-held conn: subscriptions route to
+//     PubSub-owned connections.
+//
+//   - blocking=false (readiness-established probes: pushDrainWithin, the FD
+//     session tick, the background drainer): use the Buffered variant, which
+//     drains the current batch and stops when the reader buffer empties so the
+//     probe never blocks waiting for a reply that will not come on an idle conn.
+//     An error AFTER partially consuming a fragmented frame is PROPAGATED, not
+//     swallowed, so a mid-frame desync cannot later be read as a command reply.
+//
+// A custom processor is handed only a confirmed push frame in either mode: the
+// interface does not promise the built-in's peek-and-consume-only-push behavior,
+// and the next frame can be a coalesced REPLY. Peek the type first, propagate the
+// peek error (PeekReplyType can partially consume an attribute via DiscardNext
+// before it errors), and skip a non-push frame so a custom processor cannot
+// consume the reply; ProcessPendingNotifications then blocks and skips pushes
+// until a non-push is next, matching the built-in blocking discipline.
+func (c *baseClient) drainPushFrames(ctx context.Context, cn *pool.Conn, rd *proto.Reader, blocking bool) error {
+	handlerCtx := c.pushNotificationHandlerContext(cn)
+	// Nonblocking Close adapter: this drain runs on CSC-held paths (the FD
+	// session reader and tick among them), where a custom handler calling Close()
+	// on the raw client would deadlock — Close waits on the coalescer's
+	// WaitGroup, which includes the very goroutine parked in the handler.
+	handlerCtx.Client = cscHandlerClient{baseClient: c}
+	if processor, ok := c.pushProcessor.(*push.Processor); ok {
+		if blocking {
+			return processor.ProcessPendingNotifications(ctx, handlerCtx, rd)
+		}
+		return processor.ProcessPendingNotificationsBuffered(ctx, handlerCtx, rd)
+	}
+	// Custom processor: peek-then-process in a LOOP for blocking mode. A single
+	// peek+process drained only the first push (and whatever that one invocation
+	// consumed); a SECOND invalidation still on the socket ahead of the reply would
+	// then be read by the caller's ReadRawReply as the command value — published to
+	// the cache under the wrong key and shifting every later reply. PeekReplyType
+	// blocks on the socket for the next frame, so looping skips every push (buffered
+	// OR socket-pending) until a non-push (the reply) is next, matching the built-in
+	// blocking discipline. Non-blocking probes keep the single-pass behavior (one
+	// drain, never block waiting for a reply that will not come on an idle conn).
+	for {
+		t, err := rd.PeekReplyType()
+		if err != nil {
+			return err
+		}
+		if t != proto.RespPush {
+			return nil
+		}
+		if err := c.pushProcessor.ProcessPendingNotifications(ctx, handlerCtx, rd); err != nil {
+			return err
+		}
+		if !blocking {
+			return nil
+		}
+	}
+}
+
 // cscFallbackProbeInterval bounds how often an idle connection without a
 // portable readiness mechanism is subjected to a timed read. Post-command
 // probes remain immediate; this is only the eventual invalidation fallback.
@@ -2751,10 +3036,24 @@ func (c *baseClient) drainPushNotifications(cn *pool.Conn) (processorSucceeded b
 	if socketErr != nil {
 		return false, socketErr
 	}
-	hasData := cn.HasBufferedData() || socketData
+	// Capture the already-buffered state BEFORE the probe below: it decides whether a
+	// custom processor may run (see the custom-processor gate). The probe can peek a
+	// byte into the buffer, but a byte fetched from the socket/wrapper is NOT the
+	// "real bytes already buffered" the spec requires for a custom processor.
+	buffered := cn.HasBufferedData()
+	hasData := buffered || socketData
 	if !readPending && !periodicReadPending && !hasData {
 		return false, nil
 	}
+
+	// No deadline cleanup is needed here: WithReaderHardDeadline (the probe and
+	// drain reads below) already restores a cleared read deadline in its own defer
+	// (SetReadDeadline(time.Time{}) — see its doc). A previous WithReader(ctx, 0)
+	// cleanup here was both redundant and wrong: under an active relaxed
+	// maintenance timeout it re-armed a relaxed deadline (getEffectiveReadTimeout
+	// returns the relaxed value even for timeout 0), which a ReadTimeout<0 conn
+	// then never clears on its next read, causing a spurious timeout later.
+
 	if !hasData {
 		// TLS and opaque wrappers can hide bytes from the socket readiness
 		// check. Probe one byte without consuming it under a tiny deadline;
@@ -2771,12 +3070,54 @@ func (c *baseClient) drainPushNotifications(cn *pool.Conn) (processorSucceeded b
 		}
 	}
 
+	// Custom-processor gate (spec: .claude/specs/push.md): a custom processor
+	// implements only the blocking loop, so it gets work only when real RESP bytes
+	// were ALREADY buffered — never on kernel-only readability, which over TLS can be
+	// a control record with zero RESP bytes, and never on a byte the probe above just
+	// fetched from the socket/wrapper. Handing it non-buffered data would run the
+	// blocking loop into the hard cap; its error is always fatal for a custom
+	// processor and would retire a healthy conn (the spurious-retire the spec warns
+	// of). Skipping is bounded: the periodic fallback re-probes and MaxStaleness
+	// backstops. The built-in buffered processor accepts kernel-only readability (a
+	// boundary timeout is benign), so it still proceeds. Gate on `buffered` captured
+	// before the probe, mirroring peekAndProcessPushNotifications.
+	if !buffered {
+		if _, builtin := c.pushProcessor.(*push.Processor); !builtin {
+			return false, nil
+		}
+	}
+
 	handlerCtx := c.pushNotificationHandlerContext(cn)
 	handlerCtx.Client = cscHandlerClient{baseClient: c}
-	err = cn.WithReaderHardDeadline(cscDrainHardReadCap, func(rd *proto.Reader) error {
+	// Relaxation-aware cap (spec: pushDrainBudget): during a failover/migration a push
+	// frame can fragment past the small hard cap, and a mid-frame timeout is a fatal
+	// desync that would evict this conn's CSC coverage in the very window relaxation
+	// covers. Raise the cap to the relaxed timeout while relaxation is active; without
+	// it EffectiveReadTimeout returns the unchanged hard cap.
+	budget := pushDrainBudget(cn, cscDrainHardReadCap)
+	if budget > cscDrainHardReadCap {
+		// Relaxation active: confirm a RESP byte under the short cap before granting the
+		// relaxed budget. Otherwise a TLS control record — socket-readable (hasData) with
+		// zero RESP bytes and nothing buffered — makes the drain's Peek(1) block for the
+		// full relaxed timeout, and the single-goroutine idle drainer stalls the whole
+		// pass for seconds (#3989). A byte already buffered (the common case, including
+		// the !hasData probe above) makes this Peek return immediately. Mirrors
+		// pushDrainWithin, which the reply-path checkpoint already routes through.
+		if perr := cn.WithReaderHardDeadline(cscDrainHardReadCap, func(rd *proto.Reader) error {
+			_, e := rd.Peek(1)
+			return e
+		}); perr != nil {
+			if isTimeout, hasFlag := isTimeoutError(perr); isTimeout && hasFlag {
+				return false, nil // no frame began within the short cap — benign
+			}
+			return false, perr
+		}
+	}
+	err = cn.WithReaderHardDeadline(budget, func(rd *proto.Reader) error {
 		if processor, ok := c.pushProcessor.(*push.Processor); ok {
 			return processor.ProcessPendingNotificationsBuffered(
-				context.Background(), handlerCtx, rd)
+				context.Background(), handlerCtx, rd,
+			)
 		}
 		return c.pushProcessor.ProcessPendingNotifications(context.Background(), handlerCtx, rd)
 	})

search_commands.go

@@ -3181,6 +3181,9 @@ func (cmd *FTHybridCmd) readReply(rd *proto.Reader) (err error) {
 		if err != nil {
 			return err
 		}
+		// Populate rawVal on the RESP2 path too, so RawVal()/RawResult() behave
+		// the same regardless of protocol (the RESP3 branch sets it above).
+		cmd.rawVal = data
 	}
 
 	result, cursorResult, err := parseFTHybrid(data, cmd.withCursor)

universal.go

@@ -185,6 +185,30 @@ type UniversalOptions struct {
 	//
 	// Experimental: this API may change in a minor release.
 	ClientSideCacheStrategy CSCStrategy
+
+	// ClientSideCacheRefreshOnInvalidate re-fetches recently-read keys as soon as
+	// their invalidation arrives. See Options.ClientSideCacheRefreshOnInvalidate.
+	//
+	// Experimental: this API may change in a minor release.
+	ClientSideCacheRefreshOnInvalidate bool
+
+	// ClientSideCacheRefreshRecencyWindow bounds ClientSideCacheRefreshOnInvalidate
+	// to recently-read keys. See Options.ClientSideCacheRefreshRecencyWindow.
+	//
+	// Experimental: this API may change in a minor release.
+	ClientSideCacheRefreshRecencyWindow time.Duration
+
+	// ClientSideCacheCoalesceMisses coalesces concurrent cache misses onto a held
+	// full-duplex connection. See Options.ClientSideCacheCoalesceMisses.
+	//
+	// Experimental: this API may change in a minor release.
+	ClientSideCacheCoalesceMisses bool
+
+	// ClientSideCacheInvalidationBatchWindow batches invalidation-driven deletes.
+	// See Options.ClientSideCacheInvalidationBatchWindow.
+	//
+	// Experimental: this API may change in a minor release.
+	ClientSideCacheInvalidationBatchWindow time.Duration
 }
 
 // Cluster returns cluster options created from the universal options.
@@ -391,6 +415,11 @@ func (o *UniversalOptions) Simple() *Options {
 		ClientSideCacheConfig:     o.ClientSideCacheConfig,
 		ClientSideCache:           o.ClientSideCache,
 		ClientSideCacheStrategy:   o.ClientSideCacheStrategy,
+
+		ClientSideCacheRefreshOnInvalidate:     o.ClientSideCacheRefreshOnInvalidate,
+		ClientSideCacheRefreshRecencyWindow:    o.ClientSideCacheRefreshRecencyWindow,
+		ClientSideCacheCoalesceMisses:          o.ClientSideCacheCoalesceMisses,
+		ClientSideCacheInvalidationBatchWindow: o.ClientSideCacheInvalidationBatchWindow,
 	}
 }