mirror of
https://github.com/redis/go-redis.git
synced 2026-10-24 07:59:20 +03:00
Merge branch 'master' into entry-array-length-reconcile
This commit is contained in:
Nedyalko Dyakov
GitHub
25
.claude/README.md
Normal file
25
.claude/README.md
Normal file
@@ -0,0 +1,25 @@
|
||||
# `.claude/`
|
||||
|
||||
Claude Code configuration for the go-redis repository.
|
||||
|
||||
## `settings.json` is project-shared
|
||||
|
||||
`settings.json` is committed and applies to **every maintainer's** Claude Code
|
||||
session. Keep it to team-wide policy only — permission allowlists and other
|
||||
settings that should hold for everyone working in this repo.
|
||||
|
||||
Per-user preferences (personal model choice, local permission grants, machine
|
||||
-specific paths, etc.) belong in `settings.local.json`, which is gitignored.
|
||||
Do not add them to `settings.json`. Additions here are a deliberate policy
|
||||
choice for the whole team.
|
||||
|
||||
> Note: `settings.json` is schema-validated by Claude Code and does not accept
|
||||
> comments or unknown keys, which is why this guidance lives in the README
|
||||
> rather than inline.
|
||||
|
||||
## Layout
|
||||
|
||||
- `settings.json` — shared, committed Claude Code settings.
|
||||
- `commands/` — repo-specific slash commands (e.g. `check-ci`).
|
||||
- `skills/` — repo-specific skills (e.g. `add-command`).
|
||||
- `specs/` — architecture specs referenced by skills and by maintainers.
|
||||
27
.claude/commands/check-ci.md
Normal file
27
.claude/commands/check-ci.md
Normal file
@@ -0,0 +1,27 @@
|
||||
---
|
||||
description: Summarize GitHub Actions CI status for the current branch's PR (build, lint, e2e, govulncheck, ...)
|
||||
argument-hint: "[pr-number]"
|
||||
allowed-tools: Bash(gh pr checks:*), Bash(gh pr view:*), Bash(gh run view:*), Bash(gh pr list:*), Bash(git branch:*)
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
- Branch: !`git branch --show-current`
|
||||
- PR: !`gh pr view $1 --json number,title,url,state 2>/dev/null || echo "NO_PR"`
|
||||
- Checks (name / state / elapsed / url): !`gh pr checks $1 2>&1 || true`
|
||||
|
||||
## Task
|
||||
|
||||
Report the CI state for this PR. If the PR line is `NO_PR`, say there is no open PR for this branch and stop.
|
||||
|
||||
Otherwise, from the checks block:
|
||||
|
||||
1. **Counts first.** One line: `N passing · M failing · K pending` (and skipped/cancelled if any).
|
||||
2. **Failures and pending only.** For each check that is not passing, list its name, state, and the log URL. Skip the passing ones, they are noise.
|
||||
3. **Prioritize.** Order failures by what blocks a merge most: `test-redis-ce` (the main correctness matrix) and `E2E Tests (Mock Proxy)` first, then `lint` and `govulncheck`, then `CodeQL` / `doctests` / `check-spelling`, then everything else (benchmarks last).
|
||||
4. **Verdict.** One closing line:
|
||||
- All green → "All checks pass, nothing to do."
|
||||
- Any red → name the single most important failing check to look at first.
|
||||
- Only pending → say which are still running and that it is not done yet.
|
||||
|
||||
Keep it short. No preamble, no restating this prompt. The argument `$1` is an optional PR number; when omitted, the current branch's PR is used.
|
||||
24
.claude/settings.json
Normal file
24
.claude/settings.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(go version)",
|
||||
"Bash(go env:*)",
|
||||
"Bash(go list:*)",
|
||||
"Bash(go doc:*)",
|
||||
"Bash(docker info)",
|
||||
"Read",
|
||||
"Grep",
|
||||
"Glob",
|
||||
"Edit",
|
||||
"Write",
|
||||
"Bash(make:*)",
|
||||
"Bash(go test:*)",
|
||||
"Bash(go build:*)",
|
||||
"Bash(go vet:*)",
|
||||
"Bash(gofumpt:*)",
|
||||
"Bash(git status)",
|
||||
"Bash(git diff:*)",
|
||||
"Bash(git log:*)"
|
||||
]
|
||||
}
|
||||
}
|
||||
78
.claude/skills/add-command/SKILL.md
Normal file
78
.claude/skills/add-command/SKILL.md
Normal file
@@ -0,0 +1,78 @@
|
||||
---
|
||||
name: add-command
|
||||
description: Use when adding a new Redis command (or RediSearch / TimeSeries / VectorSet / module subcommand) to go-redis — covers fetching the command spec and docs, the Cmder type, Cmdable interface wiring, RESP parsing, tests, and the custom-vet rule that enforces SetVal.
|
||||
allowed-tools: WebFetch
|
||||
---
|
||||
|
||||
# Adding a new Redis command
|
||||
|
||||
Router for adding a Redis command to the root `redis` package. Read the reference file for the area you're touching — don't load all of them.
|
||||
|
||||
## Step 0 — Get the command spec AND docs FIRST
|
||||
|
||||
Before writing any Go, know the exact command shape (arguments, optional flags, reply structure, since-version, key positions) **and** its documented semantics (what each reply field means, RESP2-vs-RESP3 differences, examples). Fetch **two** sources — the machine-readable spec and the prose docs. They cover different gaps: the spec nails arguments and key positions, the docs nail what the reply actually looks like.
|
||||
|
||||
### A. Machine-readable spec (arguments, key specs, reply schema)
|
||||
|
||||
Resolve the spec in this order:
|
||||
|
||||
1. **Spec file argument** — if the user passed a path to a spec/JSON file, read it.
|
||||
2. **PR URL argument** — if the user passed a `github.com/redis/redis` PR (or other repo) URL, fetch the diff and read the command definition + `src/commands/<cmd>.json` it adds.
|
||||
3. **No argument** — fetch the machine-readable spec from redis/redis:
|
||||
```
|
||||
https://raw.githubusercontent.com/redis/redis/unstable/src/commands/<command>.json
|
||||
```
|
||||
Container subcommands use `<container>-<sub>.json` (e.g. `client-info.json`). **A 404 means it's a module command** (RediSearch, TimeSeries, VectorSet, Bloom) — those specs live in the **module's own repo**, not redis/redis. Switch to the module repo (see `references/module-commands.md` §1) or ask the user for the spec/PR. Don't retry the redis/redis URL.
|
||||
|
||||
### B. Prose docs (semantics, return value, RESP2/RESP3 reply, examples)
|
||||
|
||||
The JSON `reply_schema` is often thin or missing; the docs spell out what the reply actually is — including separate **RESP2 Reply** and **RESP3 Reply** sections, which decide your `readReply` and any module RESP2-vs-RESP3 handling. Fetch the command's doc page:
|
||||
|
||||
```
|
||||
https://redis.io/docs/latest/commands/<command>/
|
||||
```
|
||||
|
||||
Raw markdown source (alternative, good for diffing or when the rendered page is noisy):
|
||||
`https://raw.githubusercontent.com/redis/redis-doc/master/commands/<command>.md`.
|
||||
**Module commands** (RediSearch, TimeSeries, …) are documented under their own
|
||||
path on redis.io (e.g. `/docs/latest/commands/ft.search/`) or in the module repo
|
||||
— see `references/module-commands.md`.
|
||||
|
||||
Read the **Return value / RESP2 Reply / RESP3 Reply** sections plus the examples, and reconcile them against the JSON `reply_schema`. When the two disagree, the docs' reply description (and a quick `redis-cli` check) win.
|
||||
|
||||
### Map both sources to the implementation
|
||||
|
||||
| Source | Drives |
|
||||
|--------|--------|
|
||||
| spec `arguments` | method signature, args-slice build order, optional `FooArgs` struct |
|
||||
| spec `reply_schema` + docs **Return value** | the `readReply` parser and Cmder result type |
|
||||
| docs **RESP2 Reply / RESP3 Reply** | RESP2-vs-RESP3 branching in `readReply` (see `references/module-commands.md`) |
|
||||
| spec `since` + docs `@history` | `SkipBeforeRedisVersion(...)` in the integration test, doc comment |
|
||||
| spec `key_specs` | key-position maps in `command.go`; cluster routing |
|
||||
| spec `command_flags` (e.g. `READONLY`, no key) | keyless / fan-out handling, cluster routing |
|
||||
| docs examples | integration-test cases and expected values |
|
||||
|
||||
If you can't get either source, STOP and ask the user — guessing the reply shape produces a broken `readReply`.
|
||||
|
||||
## Decide before you start
|
||||
|
||||
1. **Which `*_commands.go` file?** — pick the existing file matching the data type (`string_commands.go`, `hash_commands.go`, `search_commands.go`, …). Only create a new file for a genuinely new category. New files need a matching `XxxCmdable` interface embedded in `Cmdable` (`commands.go`).
|
||||
2. **Reuse an existing Cmder, or define a new one?** — simple replies (`int`, `string`, `bool`, `[]string`, map) reuse `*IntCmd`, `*StringCmd`, `*BoolCmd`, `*StringSliceCmd`, `*MapStringStringCmd`, … Define a new Cmder only for a structured reply that no existing type fits.
|
||||
3. **`Foo` vs `FooWithArgs`?** — for commands with optional flags, expose a positional `Foo(...)` for the common path plus `FooWithArgs(ctx, key, *FooArgs)` for the full surface.
|
||||
|
||||
## Reference files
|
||||
|
||||
| Read this | When |
|
||||
|-----------|------|
|
||||
| `references/core-command-pattern.md` | Always — the 7-step Go pattern (interface method, Cmder type, RESP parsing, tests, vet, fmt). Worked example: `LCS`. |
|
||||
| `references/module-commands.md` | Adding a RediSearch / TimeSeries / VectorSet / Bloom subcommand — naming, RESP2-vs-RESP3 shape differences, where the Cmd type lives. |
|
||||
| `references/cluster-routing-wiring.md` | Command is keyless, fans out, multi-key, or aggregates across shards. |
|
||||
|
||||
When several apply, read in order: core → module → cluster.
|
||||
|
||||
## Pitfalls (full list in core-command-pattern.md)
|
||||
|
||||
- Forgetting to embed the new `XxxCmdable` interface in `Cmdable` — compiles on `*Client`, silently unreachable via `UniversalClient`.
|
||||
- Forgetting `Clone()` — pipelines reuse Cmders; shared `val` causes cross-execution bugs.
|
||||
- Skipping `SetVal` because "nothing calls it" — hooks do, and the `setval` custom-vet check fails the build.
|
||||
- Using `time.Duration` directly in args — server gets nanoseconds. Convert per spec.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Cluster routing for new commands
|
||||
|
||||
Read this only when the new command is keyless, fans out, targets multiple keys, or aggregates results across shards. **Most simple single-key commands need no cluster-side change** — slot routing is inferred from the key argument automatically.
|
||||
|
||||
This file (the skill reference) is a wiring checklist. For the full routing *design* — slot computation, MOVED/ASK, aggregator internals — read the separate architecture spec at the repo path `.claude/specs/cluster-routing.md` before changing routing code.
|
||||
|
||||
## When a command needs wiring
|
||||
|
||||
| Command shape | What to wire |
|
||||
|---------------|--------------|
|
||||
| Keyless (e.g. `DBSIZE`, `FLUSHALL`, server-info commands) | Add the lowercase name to `keylessCommands` (`command.go:25`). Otherwise `ClusterClient` can't pick a node. |
|
||||
| Unusual key position (key not the first arg) | Update the key-position / key-spec maps in `command.go`. |
|
||||
| Fan-out across all shards, results aggregated | Add a request/response policy in `command_policy_resolver.go` and the matching aggregator in `internal/routing/aggregator.go` (policy declared in `internal/routing/policy.go`). |
|
||||
| Multi-key, cross-slot | Confirm the cross-slot rules in `.claude/specs/cluster-routing.md`; commands spanning slots error unless the command supports it. |
|
||||
|
||||
## Source of truth = the command spec
|
||||
|
||||
The spec's `key_specs` and `command_flags` tell you the routing class:
|
||||
- empty `key_specs` → keyless → `keylessCommands`.
|
||||
- `command_flags` with no write flag → `READONLY` → eligible for replica routing.
|
||||
- multiple key specs / `numkeys`-style args → multi-key; check cross-slot handling.
|
||||
|
||||
## Verify
|
||||
|
||||
After wiring, exercise against the cluster profile:
|
||||
|
||||
```sh
|
||||
go test -run TestGinkgoSuite . -ginkgo.focus="cluster"
|
||||
```
|
||||
106
.claude/skills/add-command/references/core-command-pattern.md
Normal file
106
.claude/skills/add-command/references/core-command-pattern.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# Core command pattern
|
||||
|
||||
The 7-step Go pattern for adding a command to the root `redis` package. Worked example: **LCS** (grep `func (c cmdable) LCS` in `string_commands.go` and `type LCSCmd struct` in `command.go`) — it uses a `cmdable` receiver, a query struct, a custom Cmder, and a registered `CmdType`. Copy its shape exactly.
|
||||
|
||||
## 1. Method on the `cmdable` interface
|
||||
|
||||
In the matching `*_commands.go` file. Receiver is `cmdable` — this is what makes the command available on every client type (`Client`, `ClusterClient`, `Ring`, sentinel, pipelines). A `*Client` receiver compiles but is unreachable via `UniversalClient` / `Cmdable`.
|
||||
|
||||
```go
|
||||
func (c cmdable) LCS(ctx context.Context, q *LCSQuery) *LCSCmd {
|
||||
cmd := NewLCSCmd(ctx, q)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
```
|
||||
|
||||
For commands with many optional flags, expose `Foo(...)` for the common path plus `FooWithArgs(ctx, key, *FooArgs)` for the full surface (see `FTHybrid` / `FTHybridWithArgs` in `search_commands.go`).
|
||||
|
||||
Conventions:
|
||||
- Lowercase the command name in the args slice (`"lcs"`, not `"LCS"`); subcommand flags are uppercase (`"LEN"`, `"IDX"`, `"MINMATCHLEN"`).
|
||||
- Pre-size the args slice (`make([]interface{}, 3, 7)`).
|
||||
- Convert `time.Duration` to whatever the server expects — never pass it raw (server gets nanoseconds). Use `formatMs` / `formatSec` in `commands.go`.
|
||||
- Doc comment: state `Redis X.Y+` (from the spec `since`) and link the redis.io command page.
|
||||
|
||||
## 2. New `XxxCmdable` interface goes into `Cmdable`
|
||||
|
||||
Only if you added a **new** `*_commands.go` file with its own interface. Embed it in `Cmdable` (`commands.go`, alphabetical in the embed list — e.g. `SearchCmdable` at `commands.go:242`). Without this the method is unreachable through `UniversalClient`.
|
||||
|
||||
## 3. Custom Cmder type
|
||||
|
||||
Define only when no existing Cmd type (`*IntCmd`, `*StringCmd`, `*BoolCmd`, `*StringSliceCmd`, `*MapStringStringCmd`, …) fits the reply. Put it at the bottom of `command.go` (module commands may keep it in their own `*_commands.go`, like `FTHybridCmd`). Follow `LCSCmd`:
|
||||
|
||||
```go
|
||||
type LCSCmd struct {
|
||||
baseCmd
|
||||
readType uint8 // command-specific parse state
|
||||
val *LCSMatch
|
||||
}
|
||||
|
||||
var _ Cmder = (*LCSCmd)(nil)
|
||||
|
||||
func NewLCSCmd(ctx context.Context, q *LCSQuery) *LCSCmd {
|
||||
args := make([]interface{}, 3, 7)
|
||||
args[0], args[1], args[2] = "lcs", q.Key1, q.Key2
|
||||
// ...append optional flags...
|
||||
cmd := &LCSCmd{readType: 1}
|
||||
cmd.baseCmd = baseCmd{ctx: ctx, args: args, cmdType: CmdTypeLCS}
|
||||
return cmd
|
||||
}
|
||||
|
||||
func (cmd *LCSCmd) SetVal(val *LCSMatch) { cmd.val = val }
|
||||
func (cmd *LCSCmd) Val() *LCSMatch { return cmd.val }
|
||||
func (cmd *LCSCmd) Result() (*LCSMatch, error) { return cmd.val, cmd.err }
|
||||
func (cmd *LCSCmd) String() string { return cmdString(cmd, cmd.val) }
|
||||
func (cmd *LCSCmd) readReply(rd *proto.Reader) error { /* parse RESP */ }
|
||||
func (cmd *LCSCmd) Clone() Cmder { /* deep-copy val */ }
|
||||
```
|
||||
|
||||
Required methods — the `setval` analyzer (`internal/customvet/checks/setval`) fails the build if `Result` exists without `SetVal`:
|
||||
- `SetVal(...)` — hooks mutate results through it. (`SetErr` is inherited from `baseCmd`.)
|
||||
- `Val()`, `Result()` — public accessors.
|
||||
- `String()` — via `cmdString`.
|
||||
- `readReply(rd *proto.Reader) error` — RESP parser. Use `rd.ReadArrayLen`, `rd.ReadInt`, `rd.ReadString`, `rd.ReadFloat`, `rd.ReadFixedMapLen`, `rd.ReadFixedArrayLen`. Validate element counts when the protocol guarantees a fixed shape. Always propagate the reader's error — never return `nil` after a partial read.
|
||||
- `Clone()` — deep-copy `val` (including slices/maps) so pipeline/transaction reuse is safe.
|
||||
|
||||
**CmdType:** set `cmdType: CmdTypeXxx` in the constructor. If you add a new `CmdTypeXxx` constant (`command.go`), also add the matching case in the `ExtractCommandValue` switch further down `command.go` — this is what lets hooks read the typed value. Grep `CmdTypeLCS` to find both the const declaration and its `case` in `ExtractCommandValue`; copy that shape.
|
||||
|
||||
## 4. RESP3 vs RESP2
|
||||
|
||||
If the reply differs between protocols, the parser must handle both — this applies to **any** command with a protocol-dependent shape, core or module, not just modules. Typical case: a map in RESP3, a flat key/value array in RESP2. Branch on the reply type with `rd.PeekReplyType()` (check `proto.RespMap` for RESP3) or `rd.IsResp3()`. Tests run RESP3 by default but RESP2 must still parse for older servers. See `references/module-commands.md` for the FTHybrid worked example.
|
||||
|
||||
## 5. Tests — both layers
|
||||
|
||||
- **Unit test** in `xxx_commands_unit_test.go` (no Redis) — assert the args slice your method builds for the interesting flag combinations. Pattern: `search_commands_unit_test.go:120` captures `m.lastCmd.(*FTHybridCmd)` and checks args.
|
||||
- **Integration test** in `xxx_commands_test.go` (Ginkgo, docker stack). Gate version-specific commands: `SkipBeforeRedisVersion(8.8, "X requires Redis 8.8+")` (version from spec `since`). Cover happy path, errors, optional-arg combos.
|
||||
|
||||
```sh
|
||||
go test -run TestFTHybridWithArgsRejectsUnsupportedVectors .
|
||||
go test -run TestGinkgoSuite . -ginkgo.focus="LCS"
|
||||
```
|
||||
|
||||
## 6. Custom vet check
|
||||
|
||||
```sh
|
||||
cd internal/customvet && go build .
|
||||
go vet -vettool ./internal/customvet/customvet ./...
|
||||
```
|
||||
|
||||
Same check CI runs at the end of `make test.ci`; catches a missing `SetVal`.
|
||||
|
||||
## 7. Format
|
||||
|
||||
```sh
|
||||
make fmt
|
||||
```
|
||||
|
||||
(`gofumpt` + `goimports -local github.com/redis/go-redis`.)
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Forgetting to embed the new `XxxCmdable` interface in `Cmdable` — compiles on `*Client`, silently unreachable via `UniversalClient`.
|
||||
- Forgetting `Clone()` — pipelines reuse Cmders; shared `val` causes cross-execution bugs.
|
||||
- Skipping `SetVal` because "nothing calls it" — hooks do, and custom-vet fails the build.
|
||||
- Returning `nil` from `readReply` after a partial read — always propagate the reader's error.
|
||||
- `time.Duration` passed raw — server gets nanoseconds; convert per spec.
|
||||
- New `CmdTypeXxx` added but not registered in the type-routing switch in `command.go`.
|
||||
49
.claude/skills/add-command/references/module-commands.md
Normal file
49
.claude/skills/add-command/references/module-commands.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# Module commands (RediSearch / TimeSeries / VectorSet / Bloom)
|
||||
|
||||
Module subcommands (`FT.*`, `TS.*`, `V*`, `BF.*`/`CF.*`) follow the core pattern but with three differences. Read `references/core-command-pattern.md` first. Worked example: **FTHybrid** (`search_commands.go`).
|
||||
|
||||
## 1. Where the spec lives
|
||||
|
||||
Module command specs are **not** in `redis/redis/src/commands/`. They live in the module's own repo:
|
||||
- RediSearch → `RediSearch/RediSearch`
|
||||
- TimeSeries → `RedisTimeSeries/RedisTimeSeries`
|
||||
- VectorSet / Bloom → their respective repos
|
||||
|
||||
If Step 0's redis/redis fetch 404s, the command is a module command — get the spec from the module repo or ask the user for the spec file / PR.
|
||||
|
||||
## 2. Where the Cmder type lives
|
||||
|
||||
Module Cmd types are defined in the **module's** `*_commands.go` file, not `command.go`. `FTHybridCmd` lives in `search_commands.go` (grep `type FTHybridCmd`), registered on the `SearchCmdable` interface at the top of that file, which is embedded in `Cmdable` (`commands.go`). TimeSeries → `timeseries_commands.go`, VectorSet → `vectorset_commands.go`. The required-method set is identical to core (SetVal/Val/Result/String/readReply/Clone).
|
||||
|
||||
## 3. RESP2 vs RESP3 reply shapes
|
||||
|
||||
Module commands frequently return a **map in RESP3** and a **flat array in RESP2** for the same logical reply. The parser must handle both. FTHybrid reads the raw slice, then `parseFTHybrid` walks it tolerating three shapes:
|
||||
|
||||
```go
|
||||
func (cmd *FTHybridCmd) readReply(rd *proto.Reader) (err error) {
|
||||
data, err := rd.ReadSlice() // generic slice read
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
result, cursorResult, err := parseFTHybrid(data, cmd.withCursor)
|
||||
// ...assign cmd.val / cmd.cursorVal...
|
||||
}
|
||||
|
||||
// inside parseFTHybrid, per result item:
|
||||
if itemMap, ok := item.(map[string]interface{}); ok { // RESP3 map
|
||||
results = append(results, itemMap)
|
||||
} else if rawMap, ok := item.(map[interface{}]interface{}); ok { // alt RESP3
|
||||
// stringify keys
|
||||
} else if itemData, ok := item.([]interface{}); ok { // RESP2 array of k/v pairs
|
||||
// walk pairs
|
||||
}
|
||||
```
|
||||
|
||||
Pattern: don't assume RESP3. Try the map shape, fall back to the array (key-value pairs) shape. Tests run RESP3 by default but RESP2 must still parse for older servers.
|
||||
|
||||
## 4. Common module conventions
|
||||
|
||||
- Subcommand keyword is uppercase in args (`"SEARCH"`, `"VSIM"`, `"WITHCURSOR"`); the command token is lowercase (`"ft.hybrid"`).
|
||||
- Cursor-style commands carry parse state on the Cmd struct (FTHybrid's `withCursor`, `cursorVal`) — set it in the constructor from the options.
|
||||
- Module commands are almost always `FooWithArgs(ctx, index, *FooOptions)` plus a thin positional `Foo(...)`.
|
||||
- Version-gate integration tests against the **module** version, not the core Redis version.
|
||||
32
.claude/skills/commit-style/SKILL.md
Normal file
32
.claude/skills/commit-style/SKILL.md
Normal file
@@ -0,0 +1,32 @@
|
||||
---
|
||||
name: commit-style
|
||||
description: Use when writing a commit message or opening a PR in the go-redis repo — covers the Conventional-Commit format, the repo's scope vocabulary (pool, conn, sentinel, command, …), breaking-change syntax, and the rule that NO AI-attribution trailer is added.
|
||||
---
|
||||
|
||||
# go-redis commit messages
|
||||
|
||||
Conventional Commits, short and exact — `<type>(<scope>): <imperative summary>`. Subject ≤50 chars (hard cap 72), imperative mood ("add", not "added"), no trailing period. Body only when the *why* isn't obvious from the diff (breaking change, security fix, migration, non-obvious rationale); wrap at 72, bullets `-`. State why, not what — the diff says what.
|
||||
|
||||
- **Types:** `feat`, `fix`, `refactor`, `perf`, `docs`, `test`, `chore` (also `build`, `ci`, `style`, `revert`).
|
||||
- **Scope** = the subsystem touched, lowercase: `pool`, `conn`, `pubsub`, `sentinel`, `retry`, `command`/`cmd`, `vectorset`, `otel`, `streams`, `push`, `deps`, `ci`, `tests`, `docs`. Omit scope only for genuinely cross-cutting changes.
|
||||
- **Breaking change:** `feat(scope)!: ...` plus a `BREAKING CHANGE:` body line.
|
||||
- **Issues/PRs:** reference at the end — `Closes #42`, `Refs #17`.
|
||||
|
||||
## No attribution trailer
|
||||
|
||||
Do **not** add `Co-Authored-By: Claude …`, "Generated with Claude Code", or any AI-attribution line to commits or PR bodies in this repo. This overrides both the default harness behavior (which appends such trailers) and any other commit skill.
|
||||
|
||||
## Examples
|
||||
|
||||
```
|
||||
fix(pool): release wantConn slot on dial timeout
|
||||
|
||||
The waiter stayed queued after the dial deadline fired, leaking a
|
||||
slot per timed-out connection under load.
|
||||
|
||||
Closes #3812
|
||||
```
|
||||
|
||||
```
|
||||
feat(streams): support explicit LIMIT 0 in XTRIM trimming
|
||||
```
|
||||
103
.claude/skills/prepare-release/SKILL.md
Normal file
103
.claude/skills/prepare-release/SKILL.md
Normal file
@@ -0,0 +1,103 @@
|
||||
---
|
||||
name: prepare-release
|
||||
description: Use when preparing a go-redis release — cutting a new version, bumping version.go, updating the go-redis dependency version in submodule go.mod files, or writing the RELEASE-NOTES.md entry for a new vX.Y.Z. Covers choosing the next semver, finding the last release and gathering merged PRs since then, the release-notes format and what to exclude, running scripts/release.sh to bump versions, and verifying with the scripts/tag.sh dry-run. Does NOT create tags, push, or commit — publishing stays a manual maintainer step.
|
||||
allowed-tools: Bash(git describe:*), Bash(gh pr list:*)
|
||||
---
|
||||
|
||||
# Preparing a go-redis release
|
||||
|
||||
Stage everything for a release locally, then stop before anything is published.
|
||||
|
||||
## This skill never publishes
|
||||
|
||||
- **Never** run `scripts/tag.sh ... -t` — the `-t` flag creates and pushes git tags.
|
||||
- **Never** `git push` anything (no commits, no tags).
|
||||
- **Never** commit on the maintainer's behalf unless they explicitly ask.
|
||||
|
||||
The deliverable is a reviewable diff: bumped versions plus a new `RELEASE-NOTES.md`
|
||||
entry. A human reviews it and runs the publish step.
|
||||
|
||||
## 1. Pick the next version
|
||||
|
||||
The current version is the source of truth in `version.go`:
|
||||
|
||||
```sh
|
||||
grep 'return' version.go # e.g. return "9.21.0"
|
||||
```
|
||||
|
||||
Choose the next `vX.Y.Z` by semver, based on what shipped since the last release:
|
||||
|
||||
- **patch** (`Z`) — bug fixes only; drop-in upgrade.
|
||||
- **minor** (`Y`) — new features, no breaking changes; drop-in upgrade.
|
||||
- **major** (`X`) — breaking changes.
|
||||
|
||||
Confirm the level with the user if the changeset is ambiguous.
|
||||
|
||||
## 2. Find the last release and gather changes
|
||||
|
||||
`scripts/tag.sh` also tags every public submodule (`extra/redisotel/vX.Y.Z`, …),
|
||||
so a naive `git describe` returns a submodule tag. Match the **root** tag only:
|
||||
|
||||
```sh
|
||||
LAST=$(git describe --tags --abbrev=0 --match 'v[0-9]*') # e.g. v9.21.0
|
||||
git log "$LAST"..HEAD --oneline
|
||||
gh pr list --state merged --limit 100 \
|
||||
--json number,title,author,mergedAt,url --search "merged:>=<last-release-date>"
|
||||
```
|
||||
|
||||
Categorize the PRs: highlights, new features, bug fixes, performance,
|
||||
testing/infrastructure. Exclude dependabot bumps, typo-only doc fixes, internal
|
||||
refactors with no user-facing effect, and `dependabot[bot]` from the contributor
|
||||
list.
|
||||
|
||||
## 3. Write the release notes
|
||||
|
||||
Prepend a new `# X.Y.Z (YYYY-MM-DD)` section to the **top** of `RELEASE-NOTES.md`
|
||||
(newest first; leave older entries untouched). Follow
|
||||
[`.github/RELEASE_NOTES_TEMPLATE.md`](../../../.github/RELEASE_NOTES_TEMPLATE.md)
|
||||
exactly — section order, emoji headers, the `([#PR](url)) by [@user](url)` link
|
||||
format, and the `**Full Changelog**` compare link `${LAST}...vX.Y.Z`. Open the
|
||||
lead line with the release type and whether it is a drop-in upgrade, matching the
|
||||
existing entries.
|
||||
|
||||
That template carries the full "what to exclude" and formatting rules — read it,
|
||||
don't reinvent them. `release-drafter` separately auto-drafts a GitHub release
|
||||
from PR labels (`.github/release-drafter-config.yml`); `RELEASE-NOTES.md` is the
|
||||
curated, hand-written record and is the file you edit.
|
||||
|
||||
## 4. Bump versions
|
||||
|
||||
Run the repo's bump script. It rewrites the go-redis dependency version in every
|
||||
submodule `go.mod`, runs `go mod tidy`, and bumps `version.go`. It does **not**
|
||||
commit, push, or switch branches:
|
||||
|
||||
```sh
|
||||
TAG=vX.Y.Z ./scripts/release.sh
|
||||
```
|
||||
|
||||
## 5. Verify (dry run only)
|
||||
|
||||
Run the tag script's **dry run** — it checks that `version.go` and every `go.mod`
|
||||
already match the tag, and prints the tags it *would* push. No `-t`:
|
||||
|
||||
```sh
|
||||
./scripts/tag.sh vX.Y.Z # DRY RUN — must pass cleanly; never add -t here
|
||||
make build
|
||||
git diff --stat # review version.go, submodule go.mod, RELEASE-NOTES.md
|
||||
```
|
||||
|
||||
Fix anything the dry run flags before handing off.
|
||||
|
||||
## 6. Hand off
|
||||
|
||||
Report that the release is staged and list the publish steps for the maintainer
|
||||
to run themselves (this skill does not do them):
|
||||
|
||||
1. Review `git diff`, then commit (`chore(release): vX.Y.Z` — see the
|
||||
`commit-style` skill; no AI-attribution trailer).
|
||||
2. Open and merge the release PR.
|
||||
3. After merge, tag and push: `./scripts/tag.sh vX.Y.Z -t`.
|
||||
4. `release-drafter` publishes the GitHub release; reconcile it with the
|
||||
`RELEASE-NOTES.md` entry if needed.
|
||||
|
||||
Stop after step 5. Do not commit, tag, or push.
|
||||
62
.claude/skills/testing/SKILL.md
Normal file
62
.claude/skills/testing/SKILL.md
Normal file
@@ -0,0 +1,62 @@
|
||||
---
|
||||
name: testing
|
||||
description: Use when running go-redis tests of any kind — the full suite, a single test, or a filtered/focused subset (e.g. only cluster, sentinel, or pool specs); bringing up or tearing down the Docker test stack; the e2e maintenance-notifications suite (including the proxy-free logic tests); building or formatting; or setting the REDIS_VERSION / RE_CLUSTER / REDIS_PORT knobs that drive the test image and version gating. Covers how to focus a Ginkgo spec and how to run plain go tests.
|
||||
---
|
||||
|
||||
# Running tests, builds, and the Docker stack
|
||||
|
||||
Tests run against a Redis stack started via Docker Compose. Profiles in `docker-compose.yml` control which services come up (`standalone`, `cluster`, `sentinel`, `all`, `e2e`, …).
|
||||
|
||||
## Make targets
|
||||
|
||||
```sh
|
||||
make docker.start # bring up the full test stack (profile: all)
|
||||
make docker.stop
|
||||
make test # docker.start -> test.ci -> docker.stop
|
||||
make test.ci # run tests assuming containers are already up
|
||||
make test.ci.skip-vectorsets # used when REDIS_VERSION < 8
|
||||
make bench # go test -bench=. (root module only)
|
||||
make fmt # gofumpt + goimports (-local github.com/redis/go-redis)
|
||||
make build
|
||||
make go_mod_tidy # go mod tidy across every module
|
||||
```
|
||||
|
||||
The Makefile iterates over every `go.mod` (`GO_MOD_DIRS`) for `test.ci`, `go_mod_tidy`, etc.
|
||||
|
||||
## E2E (maintenance notifications)
|
||||
|
||||
Needs an extra `cae-resp-proxy` service:
|
||||
|
||||
```sh
|
||||
make test.e2e # auto-starts the e2e profile, runs ./maintnotifications/e2e/, tears down
|
||||
make test.e2e.docker # subset that runs inside docker
|
||||
make test.e2e.logic # logic-only tests, no proxy required
|
||||
```
|
||||
|
||||
## Env knobs (passed through the Makefile)
|
||||
|
||||
- `REDIS_VERSION` — e.g. `8.8`. Drives both the test image tag and `main_test.go` version-gating helpers (`SkipBeforeRedisVersion` / `SkipAfterRedisVersion`).
|
||||
- `CLIENT_LIBS_TEST_IMAGE` — full image ref, e.g. `redislabs/client-libs-test:8.8-m03`. (To change the default tag, use the `update-ci-image` skill — it lives in several files.)
|
||||
- `RE_CLUSTER=true` — point tests at a Redis Enterprise cluster instead of the docker-compose stack. The suite then skips ring/sentinel/TLS-cluster setup.
|
||||
- `RCE_DOCKER=true` — using Redis CE in docker (default for `make test`).
|
||||
- `REDIS_PORT` — override the default standalone port (`6380`) used by `main_test.go`.
|
||||
|
||||
## Running a single test
|
||||
|
||||
The root suite is Ginkgo-based (`bsm/ginkgo` + `bsm/gomega` forks). `go test -run` matches the Go-level wrapper, so to focus a Ginkgo spec use the Ginkgo focus flag:
|
||||
|
||||
```sh
|
||||
go test -run TestGinkgoSuite . -ginkgo.focus="ZAdd"
|
||||
go test -run TestGinkgoSuite . -ginkgo.focus="cluster"
|
||||
```
|
||||
|
||||
Plain `go test`-style tests (most files outside the Ginkgo suite, e.g. `internal/...`, `maintnotifications/...`) work the usual way:
|
||||
|
||||
```sh
|
||||
go test -run TestConnStateMachine ./internal/pool/...
|
||||
go test -race -run TestCircuitBreaker ./maintnotifications/...
|
||||
```
|
||||
|
||||
## Version-gating
|
||||
|
||||
Gate Redis-version-specific tests with `SkipBeforeRedisVersion` / `SkipAfterRedisVersion` (driven by `REDIS_VERSION`) rather than skipping at the suite level.
|
||||
71
.claude/skills/update-ci-image/SKILL.md
Normal file
71
.claude/skills/update-ci-image/SKILL.md
Normal file
@@ -0,0 +1,71 @@
|
||||
---
|
||||
name: update-ci-image
|
||||
description: Use when adding, updating, or removing a `redislabs/client-libs-test` image tag used by the test stack — covers every place a Redis version → image-tag mapping lives so local `make test` and GitHub CI stay in sync.
|
||||
---
|
||||
|
||||
# Updating the CI test image
|
||||
|
||||
The `redislabs/client-libs-test` image is referenced in **six** places (five live config + one doc). Miss one and either local `make test` and CI diverge, or a single CI job uses the wrong image. Always edit all relevant locations in one change.
|
||||
|
||||
## The six locations
|
||||
|
||||
| # | File | What lives here |
|
||||
|---|------|-----------------|
|
||||
| 1 | `Makefile` | `CLIENT_LIBS_TEST_IMAGE ?= redislabs/client-libs-test:<tag>` — default for local `make test` / `make docker.start`. |
|
||||
| 2 | `docker-compose.yml` | `x-default-image: &default-image ${CLIENT_LIBS_TEST_IMAGE:-redislabs/client-libs-test:<tag>}` — fallback when the env var is unset. Should match the Makefile default. |
|
||||
| 3 | `.github/workflows/build.yml` | `redis_version_mapping` bash assoc array (`["8.X.x"]="<tag>"`). Drives the `benchmark` job and the `test-redis-ce` matrix via `run-tests`. |
|
||||
| 4 | `.github/actions/run-tests/action.yml` | Identical `redis_version_mapping` array — the composite action that the matrix jobs invoke. Keep in lockstep with `build.yml`. |
|
||||
| 5 | `.github/workflows/doctests.yaml` | Hardcoded `image: redislabs/client-libs-test:<tag>` on the `redis-stack` service — doctests do not consult the mapping. |
|
||||
| 6 | `CONTRIBUTING.md` | Prose line "By default the docker image … is `redislabs/client-libs-test:<tag>`." Documentation only — no CI impact, but goes stale silently. Update it to match the Makefile default. |
|
||||
|
||||
## Tasks
|
||||
|
||||
### Update the tag for an existing Redis version (e.g. bump 8.8 → a new release)
|
||||
|
||||
1. `Makefile` — replace the tag after `redislabs/client-libs-test:` on the `CLIENT_LIBS_TEST_IMAGE` line.
|
||||
2. `docker-compose.yml` — replace the tag in the `:-redislabs/client-libs-test:<tag>` default.
|
||||
3. `.github/workflows/build.yml` — update the `["8.X.x"]="<tag>"` line in `redis_version_mapping`.
|
||||
4. `.github/actions/run-tests/action.yml` — update the matching `["8.X.x"]="<tag>"` line.
|
||||
5. `.github/workflows/doctests.yaml` — update the `image:` line **only if** doctests should run against this version. Doctests typically pin to the latest stable; check before changing.
|
||||
|
||||
If the version that doctests run against (`Makefile` default) and the mapping for that same minor version diverge, you have a bug — they should resolve to the same tag.
|
||||
|
||||
### Add a new Redis version to the CI matrix
|
||||
|
||||
1. `.github/workflows/build.yml`:
|
||||
- Add `- "<X.Y>.x"` to **both** `redis-version` matrices (the `benchmark` job and the `test-redis-ce` job — they have separate lists).
|
||||
- Add `["<X.Y>.x"]="<tag>"` to `redis_version_mapping`.
|
||||
2. `.github/actions/run-tests/action.yml` — add the same `["<X.Y>.x"]="<tag>"` entry to its `redis_version_mapping`.
|
||||
3. If this becomes the new default for local dev, also update `Makefile` (`REDIS_VERSION ?=` and `CLIENT_LIBS_TEST_IMAGE ?=`) and `docker-compose.yml`.
|
||||
|
||||
### Remove a Redis version
|
||||
|
||||
1. `.github/workflows/build.yml` — drop the `- "<X.Y>.x"` line from both `redis-version` matrices and the `["<X.Y>.x"]="..."` entry.
|
||||
2. `.github/actions/run-tests/action.yml` — drop the same `["<X.Y>.x"]="..."` mapping entry.
|
||||
3. If `Makefile` / `docker-compose.yml` / `doctests.yaml` were pinned to this version, pick a replacement (usually the next-newest version still in the matrix) and update them too.
|
||||
|
||||
### Switch to a custom build (e.g. `custom-26172898734-debian` for an unreleased server feature)
|
||||
|
||||
Same as "update the tag," but think about scope:
|
||||
- Custom tags usually correspond to a single Redis minor — only touch the mapping entry for that minor, not all five rows.
|
||||
- Custom tags often shouldn't be the local default if they're transient. Confirm with the user before touching `Makefile` / `docker-compose.yml`.
|
||||
- Leave `REDIS_VERSION` (the numeric `8.8` / `8.6` etc.) alone — it drives `SkipBeforeRedisVersion` gating in `main_test.go`, not the image. The custom image still reports its base version.
|
||||
|
||||
## Verification
|
||||
|
||||
After editing, search for any stale tag references:
|
||||
|
||||
```sh
|
||||
# Use the Grep tool, not bash grep.
|
||||
# Pattern: literal old tag, e.g. "8.8-rc1" or "client-libs-test:8.8"
|
||||
```
|
||||
|
||||
Expected matches: zero in `Makefile`, `docker-compose.yml`, `.github/**`. A reference in `AGENTS.md` or the `testing` skill documentation (e.g. "e.g. `redislabs/client-libs-test:8.8-m03`") is illustrative — leave it unless the doc is misleading after the change.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Two mappings, not one.** `build.yml` and `run-tests/action.yml` each declare their own `redis_version_mapping`. They are not deduped. Always edit both.
|
||||
- **Two matrices in `build.yml`.** The `benchmark` job and the `test-redis-ce` job each have their own `redis-version` list. Adding/removing a version means editing both.
|
||||
- **`doctests.yaml` is hardcoded.** It doesn't read the mapping. If the team intends "doctests follow the latest stable," verify the tag matches `Makefile`'s default after any change.
|
||||
- **Don't change `REDIS_VERSION` when only swapping the image.** The number gates version-specific test skips (`SkipBeforeRedisVersion(8.8, ...)`). A custom image built off 8.8 should keep `REDIS_VERSION=8.8`.
|
||||
- **Env var vs. default.** Local users can override with `CLIENT_LIBS_TEST_IMAGE=...`; `docker-compose.yml` uses that env var first and falls back to its baked-in default. So the docker-compose default only matters when the Makefile isn't driving things — but it's still load-bearing for direct `docker compose up` calls. Keep it in sync.
|
||||
153
.claude/specs/cluster-routing.md
Normal file
153
.claude/specs/cluster-routing.md
Normal file
@@ -0,0 +1,153 @@
|
||||
# Cluster routing
|
||||
|
||||
This is an architectural reference for how `ClusterClient` decides which node to send a command to and how it merges results from multi-shard fan-outs. Read this before changing `osscluster.go`, `osscluster_router.go`, `command_policy_resolver.go`, or anything under `internal/routing/` and `internal/hashtag/`.
|
||||
|
||||
The mental model: a command becomes a `(slot, policy, aggregator)` triple, executes against one or more nodes, and its responses are collapsed by the aggregator. Most bugs come from getting the slot wrong, picking the wrong policy, or aggregating responses incorrectly.
|
||||
|
||||
---
|
||||
|
||||
## Slot computation
|
||||
|
||||
Redis Cluster has 16384 slots. A key's slot is `CRC16(hashtag(key)) % 16384` where `hashtag(key)` returns the substring inside the first balanced `{...}` if present, otherwise the whole key (`internal/hashtag/hashtag.go`). Empty keys map to a random slot to avoid deterministic concentration.
|
||||
|
||||
Two non-obvious edge cases:
|
||||
|
||||
- **Byte-slice keys** (#3049): the cluster client used to cast args via `fmt.Sprintf("%s", v)` which doesn't handle `[]byte` consistently. `stringArg` in `command.go` now type-switches to handle `[]byte` keys passed through `client.Do(ctx, "GET", []byte("k"))`. If you add a generic-args path, route it through `stringArg`.
|
||||
- **Commands whose key is *not* the first arg, or which take the slot directly** — `COUNTKEYSINSLOT` is the canonical example: its argument is the slot integer, not a key (#3327). When you add such a command, register the correct key-spec in `command.go` or its slot will be computed from the wrong arg and the routing will land on the wrong shard.
|
||||
|
||||
`keylessCommands` in `command.go` is the override list for commands with no key at all (`PING`, `TIME`, `CONFIG`, `WAIT`, …). Routing them by "first arg" would either error or pick a random shard; instead, the router uses the configured `ShardPicker` (round-robin / random / static). Adding `WAIT` (#3615) and `TIME` (#3722) here fixed real bugs where pipelines and txpipelines mis-routed.
|
||||
|
||||
---
|
||||
|
||||
## MOVED and ASK redirection
|
||||
|
||||
`MOVED <slot> <addr>` means "this slot lives somewhere else now, permanently" — the client follows the redirect and triggers a topology reload (`loadState`). `ASK <slot> <addr>` means "this slot is *currently* migrating; just for this command, send `ASKING` then retry against `<addr>`" — no topology reload, only a one-shot redirect. Detection is via typed errors (`error.go`, `IsMovedError`/`IsAskError`) which return the address as well as a bool.
|
||||
|
||||
Two design decisions worth knowing:
|
||||
|
||||
- **No backoff after MOVED/ASK** (#3048). Redirection is informational; the new node is ready to serve. Adding exponential backoff between the redirect and the retry just adds latency for no benefit. If you're tempted to add a delay, you're probably solving a different problem (e.g. a thundering herd during failover) and should solve it elsewhere.
|
||||
- **`MASTERDOWN` is retriable** (#3164). It means a primary stepped down and a replica is being promoted; the right behaviour is to wait briefly and retry (with the existing topology), not to fail. `dial tcp` errors are now also retriable as redirects (#3786) so we don't lose connectivity during DNS flips.
|
||||
|
||||
When `MOVED` points at the same address we just used, that's a stale DNS cache, not a real redirect. The pool/dialer treats this as a bad connection and removes it (#3219 made `clusterNodes.Addrs()` return a copy so this race no longer corrupts the address list under the GC).
|
||||
|
||||
---
|
||||
|
||||
## Command policies (#3422)
|
||||
|
||||
`internal/routing/policy.go` defines the policy taxonomy. A command has a `RequestPolicy` (where it goes) and a `ResponsePolicy` (how multiple responses are combined).
|
||||
|
||||
### Request policies
|
||||
|
||||
| Policy | Meaning |
|
||||
|---|---|
|
||||
| `ReqDefault` | Single shard. Slot from first key (or the configured ShardPicker for keyless commands). |
|
||||
| `ReqAllNodes` | Broadcast to all masters **and** replicas. (`DEBUG OBJECT`, etc.) |
|
||||
| `ReqAllShards` | Broadcast to all masters only. (`DBSIZE`, `KEYS`, `SCAN` orchestration.) |
|
||||
| `ReqMultiShard` | Group keys by slot, fan out per slot. (`MGET`, `MSET`, `DEL` with cross-shard keys.) |
|
||||
| `ReqSpecial` | Custom resolver; e.g. `FT.CURSOR` is sticky-routed by cursor ID. |
|
||||
|
||||
### Response policies
|
||||
|
||||
| Policy | Meaning |
|
||||
|---|---|
|
||||
| `RespDefaultKeyless` | Return the first non-error result. |
|
||||
| `RespDefaultHashSlot` | Reassemble in original key order (for `MGET` etc.). |
|
||||
| `RespAllSucceeded` | Fail the whole call if any shard errored. |
|
||||
| `RespOneSucceeded` | Succeed if any shard succeeded. |
|
||||
| `RespAggSum` / `RespAggMin` / `RespAggMax` | Numeric aggregates. |
|
||||
| `RespAggLogicalAnd` / `RespAggLogicalOr` | Boolean aggregates. |
|
||||
| `RespSpecial` | Custom — e.g. `ZUNION` with `COUNT` (#3802). |
|
||||
|
||||
`CommandPolicy.CanBeUsedInPipeline()` returns `false` for `ReqAllNodes`, `ReqAllShards`, and `ReqMultiShard` — these can't sit inside a regular pipeline because they fan out across multiple connections. If you mark a new command with one of these request policies, expect pipeline tests to need updates.
|
||||
|
||||
`command_policy_resolver.go` resolves policies for a given command. Resolution order: explicit policy in the resolver → server-supplied tips (RESP3 `command info`) → defaults. There's a fallback chain you can extend with `SetFallbackResolver` (used by some test scaffolding and module-specific resolvers).
|
||||
|
||||
---
|
||||
|
||||
## Aggregators (`internal/routing/aggregator.go`)
|
||||
|
||||
The aggregator is the runtime side of `ResponsePolicy`. Two families:
|
||||
|
||||
- **Keyless aggregators** — collect responses without per-key bookkeeping. `DefaultKeylessAggregator`, `AllSucceededAggregator`, `AggSumAggregator`, etc.
|
||||
- **Keyed aggregators** — preserve key order. `DefaultKeyedAggregator` is the one you want for `MGET`, `MSET`, `DEL`: each shard returns results for its subset of keys, and the aggregator stitches them back in the user's original order. Reordering bugs here surface as "MGET returned values in the wrong slots."
|
||||
|
||||
When you add a new fan-out command, the typical workflow is:
|
||||
|
||||
1. Pick the right `RequestPolicy` (almost always `ReqMultiShard` for multi-key, `ReqAllShards` for global state).
|
||||
2. Pick the right `ResponsePolicy`. If none fit, register a `RespSpecial` aggregator.
|
||||
3. Test it with **at least 3 shards** and **keys distributed across all of them**, including the empty case (no keys land on a particular shard).
|
||||
|
||||
The `add aggregator case for new command` style commit shows up regularly because step 2 is easy to forget. Pair it with `add-command` in your mental checklist; the `add-command` skill points at this doc for a reason.
|
||||
|
||||
---
|
||||
|
||||
## Shard picker
|
||||
|
||||
For keyless commands and tie-breaking among replicas, `ShardPicker` (`internal/routing/shard_picker.go`) selects a node. Three implementations: round-robin (default), random, static. The picker is configured on `ClusterOptions.ShardPicker` and does *not* affect commands that have a slot.
|
||||
|
||||
The dispatch function `cmdNodeWithShardPicker` uses slot `-1` as the sentinel for "use the picker." Don't pass `-1` accidentally for a real command — that's a routing bug waiting to happen.
|
||||
|
||||
---
|
||||
|
||||
## Replica routing
|
||||
|
||||
`ReadOnly`, `RouteRandomly`, `RouteByLatency`, `ReplicaOnly` are the four read-routing knobs. Their interaction is:
|
||||
|
||||
- `ReadOnly = false` (default): always go to master.
|
||||
- `ReadOnly = true`, no other knob: go to master, but allow replicas as fallback. Reads never go to replicas in this mode unless the master is down.
|
||||
- `RouteRandomly = true`: pick a random non-failing replica for read-only commands.
|
||||
- `RouteByLatency = true`: pick the lowest-latency healthy replica.
|
||||
- `ReplicaOnly = true`: only ever talk to replicas.
|
||||
|
||||
Two history points:
|
||||
|
||||
- **Latency throttling** (#2795): `RouteByLatency` would otherwise ping every replica before every command. Latencies are now measured at most once per `minLatencyMeasurementInterval` (10s, defined in `osscluster.go`).
|
||||
- **`slotClosestNode` fail-safe** (#3043): if all replicas are unhealthy, fall back to the lowest-latency *failing* replica rather than a random one; if all pings have timed out, fall back to a random replica. This was a real bug that produced "no nodes available" errors during partial outages.
|
||||
- **PubSub on replicas** (#3480): `ClusterClient.Subscribe` can target replica nodes, with the connection running in `READONLY` mode.
|
||||
- **`ReplicaOnly` + `NewFailoverClusterClient`** (#3482): the option used to be silently dropped on failover-cluster construction. Watch for this when extending failover.
|
||||
|
||||
---
|
||||
|
||||
## Topology reload
|
||||
|
||||
`clusterState` carries `Masters`, `Slaves`, and slot→node mappings, built from `CLUSTER SLOTS`. It's stored under `clusterStateHolder` (an `atomic.Value`) so reads are lock-free. Reloads run in the background, triggered by:
|
||||
|
||||
- `MOVED` errors (canonical case).
|
||||
- Periodic refresh.
|
||||
- `SMIGRATED` notifications from the maintenance-notifications subsystem (it can target a partial reload by passing the affected slot ranges).
|
||||
|
||||
`loadState` tries every known address in random order until one returns a usable topology. If all fail, `activeAddrs` is cleared so DNS gets re-resolved on the next attempt. Stale node generations get GC'd one minute after a reload (so old node addresses don't keep accumulating).
|
||||
|
||||
The `IsClusterMode` option (#3255) flips routing behaviour for ElastiCache cluster mode, which emulates `CLUSTER SLOTS` but with cluster-mode semantics layered on a different topology. The flag exists because we can't always detect cluster mode reliably from the server response alone.
|
||||
|
||||
---
|
||||
|
||||
## Cross-slot, transactions, and pipelines
|
||||
|
||||
`ClusterClient` enforces these:
|
||||
|
||||
- **`Watch`/`MULTI`/`EXEC` (TxPipeline)**: all keys must hash to the same slot. Cross-slot returns `ErrCrossSlot` immediately and sets the error on every queued command. This is a Redis Cluster protocol limit, not a client choice.
|
||||
- **`Pipeline`**: no restriction. Commands are grouped by slot and dispatched in parallel; each shard sees only the commands targeting it. This is why `Pipeline` can succeed where `TxPipeline` would fail.
|
||||
- **`Eval`/`EvalSha`**: routed by the **first key in `KEYS`**. Scripts that read multiple keys must declare them all in `KEYS` and all of them must hash to the same slot. If your script touches a key it didn't declare, you'll get cluster-routing errors at runtime.
|
||||
- **`FT.CURSOR` and similar paginated commands**: sticky routing by cursor ID via `ReqSpecial`, so pagination stays on the node that owns the cursor.
|
||||
|
||||
---
|
||||
|
||||
## History worth knowing
|
||||
|
||||
Recurring shape of cluster bugs:
|
||||
|
||||
- **Wrong slot computation**: byte-slice keys (#3049), commands with non-key first arg (`COUNTKEYSINSLOT` #3327), missing keyless markers (`WAIT` #3615, `TIME` #3722).
|
||||
- **Race in node accounting**: `clusterNodes.Addrs()` returning a shared slice (#3219).
|
||||
- **Replica routing fallbacks**: `slotClosestNode` not handling all-failing case (#3043).
|
||||
- **Treating recoverable failures as fatal**: `MASTERDOWN` (#3164), `dial tcp` errors during failover (#3786).
|
||||
- **Unnecessary delays on recoverable redirects**: backoff after MOVED/ASK (#3048).
|
||||
- **Failover variants forgetting options**: `ReplicaOnly` (#3482), `MaintNotificationsConfig` (#3600).
|
||||
|
||||
When you add a feature or fix a bug here, add a test that:
|
||||
|
||||
1. Spins a 6-node cluster (the test scaffolding in `main_test.go` already provides `clusterScenario` with 16600–16605).
|
||||
2. Uses keys that you've verified land on different shards (use `CLUSTER KEYSLOT` if unsure).
|
||||
3. Exercises both the happy path and the redirect path.
|
||||
|
||||
The cluster integration tests are slow but catch real protocol issues — the unit-only tests don't.
|
||||
204
.claude/specs/maintnotifications.md
Normal file
204
.claude/specs/maintnotifications.md
Normal file
@@ -0,0 +1,204 @@
|
||||
# Maintenance notifications (smart client handoff)
|
||||
|
||||
This is an architectural reference for `maintnotifications/`. Read this before changing notification handling, the handoff worker pool, the per-conn state machine, or anything that touches `MaintNotificationsConfig`.
|
||||
|
||||
The mental model: Redis Enterprise pushes RESP3 notifications when nodes are about to move or fail over. The client either (a) **relaxes timeouts** so in-flight commands don't false-time-out during the disruption, or (b) **hands off** an existing TCP connection to a new endpoint without dropping pending operations. The user-visible promise is "no spurious errors during planned maintenance."
|
||||
|
||||
---
|
||||
|
||||
## Why this exists
|
||||
|
||||
During a Redis Enterprise upgrade or scale event, individual nodes get drained, slots get migrated, and primaries fail over to replicas. Without help, a client sees this as connection drops, slow commands, and timeout errors. With maintenance notifications, the server tells the client *what's about to happen* via RESP3 push frames, and the client compensates: extends timeouts during the disruption, opens a new connection to the new endpoint, and migrates pending commands across.
|
||||
|
||||
This only works on Redis Enterprise (and Redis Cloud) — open-source Redis doesn't emit these notifications. It also only works on RESP3 (`Protocol: 3`), since RESP2 has no concept of out-of-band push frames.
|
||||
|
||||
---
|
||||
|
||||
## Notifications and what they mean
|
||||
|
||||
Two families of frames:
|
||||
|
||||
### Per-connection (standalone clients)
|
||||
|
||||
| Frame | Meaning | Client action |
|
||||
|---|---|---|
|
||||
| `["MOVING", seqID, timeS, endpoint]` | This connection's node is being drained; reconnect to `endpoint` within `timeS`. | Hand off the connection. |
|
||||
| `["MIGRATING", seqID, ...]` | A slot migration is starting; expect latency spikes. | Apply `RelaxedTimeout`. |
|
||||
| `["MIGRATED", seqID, ...]` | Migration done. | Clear relaxed timeout. |
|
||||
| `["FAILING_OVER", seqID, ...]` | This primary is about to fail over. | Apply `RelaxedTimeout`. |
|
||||
| `["FAILED_OVER", seqID, ...]` | Failover done. | Clear relaxed timeout. |
|
||||
|
||||
`MOVING` is the only frame that triggers a real connection handoff. The others only adjust timeouts on the existing connection.
|
||||
|
||||
### Cluster-level
|
||||
|
||||
| Frame | Meaning | Client action |
|
||||
|---|---|---|
|
||||
| `["SMIGRATING", seqID, slot/range, ...]` | Cluster slot migration starting. | Relax timeouts on connections to the source node. |
|
||||
| `["SMIGRATED", seqID, [[src, dst, slots], ...]]` | Cluster slot migration done. | Clear relaxed timeouts; trigger lazy `clusterState` reload (selectively, using the slot ranges). |
|
||||
|
||||
ClusterClient supports only `SMIGRATING`/`SMIGRATED`. Per-connection `MOVING` is meaningless in cluster mode because cluster clients discover endpoint changes via `MOVED`/`ASK` redirection — the cluster routing path already handles node moves. Adding `MOVING` to the cluster path would duplicate (and potentially conflict with) the routing layer's redirection logic.
|
||||
|
||||
`SMIGRATED` deduplication is by `seqID` (`Manager.MarkSMigratedSeqIDProcessed`) — Redis can re-send the same notification on multiple connections, and we only want to reload topology once.
|
||||
|
||||
---
|
||||
|
||||
## Modes
|
||||
|
||||
`MaintNotificationsConfig.Mode`:
|
||||
|
||||
- `ModeDisabled` — feature off; no handshake sent. Choose this when running against open-source Redis.
|
||||
- `ModeEnabled` — `CLIENT MAINT_NOTIFICATIONS ON` is mandatory. Hard error if the server rejects it.
|
||||
- `ModeAuto` (default) — try the handshake, silently downgrade if the server doesn't understand it.
|
||||
|
||||
The handshake lives inside `initConn`. The order is: `HELLO 3` → `CLIENT MAINT_NOTIFICATIONS ON`. If `HELLO 3` fails (e.g., RESP2-only server), `ModeAuto` skips the second step and the connection runs without notifications. `ModeEnabled` errors out — the user explicitly asked for the feature.
|
||||
|
||||
Three real bugs trace to this handshake:
|
||||
|
||||
- **#3788** — `ModeAuto` would still attempt the maintenance handshake after `HELLO` failed, panicking on the unexpected reply shape.
|
||||
- **#3707** — `MaintNotificationsConfig` could be nil (when disabled), and `initConn` dereferenced it. The fix nil-checks before every access; preserve that pattern when adding fields.
|
||||
- **#3789** — Endpoint-type auto-detection was flaky over slow DNS; a 2-second resolution timeout and clearer IP-range classification (RFC1918, RFC4193, loopback, RFC6598) fixed it.
|
||||
|
||||
---
|
||||
|
||||
## Per-connection state
|
||||
|
||||
`maintnotifications/state.go` defines a tiny manager-level enum: `StateIdle` (no upgrade in progress) and `StateMoving` (handoff initiated). The richer per-connection state lives in `internal/pool` via the conn state machine — `StateUnusable` is what the pool uses to keep handoff-targeted connections out of `Get` while the worker is operating on them.
|
||||
|
||||
The flow for a `MOVING` notification on conn X:
|
||||
|
||||
1. Push handler reads `["MOVING", seqID, timeS, endpoint]` and calls `conn.MarkForHandoff(endpoint, seqID)` on X.
|
||||
2. The next `Put` to return X to the pool sees `ShouldHandoff()` true and routes X to `MarkQueuedForHandoff` (transitioning to `StateUnusable`) and enqueues a handoff request.
|
||||
3. The pool no longer hands X out via `Get` (the `OnGet` hook rejects unusable conns).
|
||||
4. A handoff worker picks up the request, dials the new endpoint, runs `initConn` against the new socket, and swaps the underlying conn.
|
||||
5. On success, `ClearHandoffState()` returns the conn to `StateIdle`. On terminal failure, the worker removes it.
|
||||
|
||||
The whole sequence is async w.r.t. the user's request that triggered the `Put`. The user does not block on handoff.
|
||||
|
||||
---
|
||||
|
||||
## Handoff worker pool (`handoff_worker.go`)
|
||||
|
||||
Workers are created on demand and self-terminate after 15 seconds idle. Auto-sizing (when not explicitly configured):
|
||||
|
||||
```
|
||||
MaxWorkers = min(PoolSize/2, max(10, PoolSize/3))
|
||||
HandoffQueueSize = max(20 * MaxWorkers, PoolSize)
|
||||
capped by min(MaxActiveConns + 1, 5 * PoolSize)
|
||||
```
|
||||
|
||||
Examples:
|
||||
- `PoolSize=100` → 33 workers, 660 queue (capped at 500).
|
||||
- `PoolSize=100, MaxActiveConns=150` → 33 workers, 151 queue.
|
||||
|
||||
These bounds are tuned to absorb a "whole pool moves at once" event — during a node drain you can get a `MOVING` notification on every connection at roughly the same time.
|
||||
|
||||
### Queue-full behavior
|
||||
|
||||
If the handoff queue is full, the worker manager waits 100ms for space; if still full, it returns `ErrHandoffQueueFull` and the connection is removed from the pool (no retry at this layer). It will be re-dialed by the pool when needed. **This is intentional**: queueing forever would mask a fundamental misconfiguration, and dropping the conn is recoverable.
|
||||
|
||||
### Retries
|
||||
|
||||
A failed handoff retries up to `MaxHandoffRetries` (default 3). The retry delay is flat, not exponential: the remainder of the handoff deadline, falling back to `HandoffTimeout/3` when no deadline is set, with a 500ms floor. Some errors are non-retryable (e.g., `initConn` failure — the new endpoint is unreachable or rejecting auth). After max retries, the conn is permanently removed.
|
||||
|
||||
### Post-handoff relaxed duration
|
||||
|
||||
After a successful handoff, the *new* conn has its read/write deadlines extended for `PostHandoffRelaxedDuration` (default `2 * RelaxedTimeout`). Why: even after the new endpoint accepts the connection, the cluster is still settling — first commands may be slow as topology propagates. The relaxed window covers the post-stabilization tail. This is applied *before* `initConn` so the auth handshake also benefits.
|
||||
|
||||
### Past pitfall
|
||||
|
||||
#3633 — the worker's logging path could call `closeConnFromRequest` with a nil error and then `err.Error()` panic-ed. Always nil-check errors before logging in this code; the worker runs on a long-lived goroutine and a panic kills future handoffs.
|
||||
|
||||
---
|
||||
|
||||
## Pool hook integration (`pool_hook.go`)
|
||||
|
||||
The maintnotifications subsystem plugs into the pool via `pool.PoolHook`:
|
||||
|
||||
- `OnGet` — reject the conn if `IsUsable()` is false or `ShouldHandoff()` is true.
|
||||
- `OnPut` — if the conn needs handoff, queue it (transitioning to `StateUnusable` so the pool keeps it but won't hand it out).
|
||||
- `OnRemove` / `Shutdown` — clean up.
|
||||
|
||||
`enableMaintNotificationsUpgrades` (called from `redis.go` after `initConn` confirms RESP3) constructs the `Manager`, registers the pool hook, and registers push handlers on `push.Registry`. On client `Close`, the manager calls `pool.RemovePoolHook` and shuts down its workers.
|
||||
|
||||
The "ReAuth must not interfere with handoff" invariant (#3547) lives at this layer: both subsystems use `StateUnusable`, so whichever one transitions first wins and the other observes the unusable state via `OnGet`. See [pool.md](pool.md) for the full coexistence contract.
|
||||
|
||||
---
|
||||
|
||||
## Circuit breaker (`circuit_breaker.go`)
|
||||
|
||||
Per-endpoint circuit breaker keyed by destination. States: `Closed` (allow), `Open` (reject), `HalfOpen` (probe). Defaults: open after 5 consecutive failures, half-open after 60s, close after 3 consecutive successes from half-open.
|
||||
|
||||
Workflow: before a handoff dial, check `IsOpen(endpoint)`. If open, fail fast — no point burning the retry budget on an endpoint that's known-broken. After each handoff, record the result.
|
||||
|
||||
A background goroutine GCs unused circuit breakers (>30 min unused) every 5 minutes so a long-lived client doesn't accumulate state for one-off endpoint failures.
|
||||
|
||||
---
|
||||
|
||||
## Endpoint type resolution (`config.go`)
|
||||
|
||||
`EndpointType` selects which form the client uses when reconnecting after a `MOVING` notification (Redis Enterprise emits multiple endpoint forms in topology):
|
||||
|
||||
- `EndpointTypeAuto` — auto-detect based on the current connection (TLS preference, IP ranges).
|
||||
- `EndpointTypeInternalIP` / `EndpointTypeInternalFQDN`.
|
||||
- `EndpointTypeExternalIP` / `EndpointTypeExternalFQDN`.
|
||||
- `EndpointTypeNone` — reconnect with the originally configured address.
|
||||
|
||||
Auto-detection logic in `DetectEndpointType`:
|
||||
|
||||
- TLS enabled → prefer FQDN (so SNI and cert validation work).
|
||||
- TLS disabled → prefer IP (avoids unnecessary DNS).
|
||||
- Internal vs external is decided by IP range (RFC1918 / RFC4193 / loopback / RFC6598) for IP literals, or by resolution-and-classification for hostnames (with a 2-second DNS timeout, post #3789).
|
||||
|
||||
Choose explicitly if your network topology requires a specific form — e.g., a client running outside the VPC must use `EndpointTypeExternalFQDN`.
|
||||
|
||||
---
|
||||
|
||||
## Manager (`manager.go`)
|
||||
|
||||
Central coordinator. Owns:
|
||||
|
||||
- The push notification handler registrations (one per notification type).
|
||||
- A `sync.Map` of in-flight `MOVING` operations keyed by conn ID, so duplicate notifications don't double-handoff.
|
||||
- The `SMIGRATED` seqID dedup set.
|
||||
- The cluster-state-reload callback (set by `ClusterClient` so the manager can trigger a partial reload on `SMIGRATED` without holding a back-reference to the cluster client).
|
||||
|
||||
Lock-free state where possible: `atomic.Int64` for active operation count, `atomic.Bool` for closed flag. Callbacks fire on the push notification handler goroutine — keep them fast and avoid blocking I/O directly inside them.
|
||||
|
||||
---
|
||||
|
||||
## E2E testing
|
||||
|
||||
`maintnotifications/e2e/` has its own `go.mod` and a separate test entrypoint. It drives a fault-injecting RESP proxy (`cae-resp-proxy`) that sits between the client and Redis and can synthesize maintenance notifications on demand. Scenarios are written as Go tests that:
|
||||
|
||||
1. Start the proxy with a `DEFAULT_INTERCEPTORS` profile (`cluster`, `hitless`).
|
||||
2. Create a real client.
|
||||
3. Issue commands while the proxy injects `MOVING`/`SMIGRATING`/etc.
|
||||
4. Assert no spurious errors and correct handoff timing.
|
||||
|
||||
Run with `make test.e2e` (auto-starts the proxy via the `e2e` docker profile). `make test.e2e.logic` runs only the logic-layer tests (no proxy). `make test.e2e.docker` runs the proxy-based subset that fits in CI's docker-network constraints.
|
||||
|
||||
If you're changing the subsystem in ways that could affect handoff timing, run E2E locally — unit tests don't catch races between notification arrival and request lifecycle.
|
||||
|
||||
---
|
||||
|
||||
## History worth knowing
|
||||
|
||||
Recurring failure modes:
|
||||
|
||||
- **Nil `MaintNotificationsConfig`** dereferenced in `initConn` when the feature is disabled (#3707). Always nil-check before access.
|
||||
- **HELLO failure not handled** before the maintenance handshake (#3788). The handshake assumes RESP3 — if `HELLO 3` failed, skip it.
|
||||
- **Endpoint-type detection over slow DNS** (#3789). Use the bounded resolver, not bare `net.LookupHost`.
|
||||
- **Failover client wiring gaps** (#3600). When you add fields to `MaintNotificationsConfig`, check that `FailoverClient` and `FailoverClusterClient` propagate them.
|
||||
- **Pool/handoff/re-auth interaction** (#3547). Always use `StateUnusable` for exclusive ownership, never a separate flag.
|
||||
- **Logging panics on nil error** (#3633) in long-lived worker goroutines. Defensive nil-checks in log paths.
|
||||
- **Test flakes from notification-injection timing** (#3641). Synchronize injection with client state, don't rely on sleeps.
|
||||
|
||||
If you add a new notification type, the checklist is roughly:
|
||||
|
||||
1. Define the frame shape and document it in `README.md` / this file.
|
||||
2. Add a parser in `push_notification_handler.go`.
|
||||
3. Decide: per-connection action (mark-for-handoff, relax timeout) or manager-level (topology reload, dedup)?
|
||||
4. If per-connection: route through the pool hook so the existing state-machine guarantees apply. Don't invent a new exclusion mechanism.
|
||||
5. Add an E2E scenario in `maintnotifications/e2e/`.
|
||||
6. Decide whether `ClusterClient` should also handle it — if it duplicates routing-layer behaviour (`MOVED`/`ASK`), probably no.
|
||||
146
.claude/specs/pool.md
Normal file
146
.claude/specs/pool.md
Normal file
@@ -0,0 +1,146 @@
|
||||
# Connection pool
|
||||
|
||||
This is an architectural reference for `internal/pool/`. Read this before changing connection lifecycle, the wantConn queue, the conn state machine, or hooks. The pool is the single most common source of bugs in this codebase (`fix(pool):` is the top commit-prefix in the history) — almost always because someone violated an invariant below without realizing it.
|
||||
|
||||
This doc captures **invariants and the "why"**, not a tour of the code. File names anchor each section but the line numbers will drift.
|
||||
|
||||
---
|
||||
|
||||
## What the pool actually does
|
||||
|
||||
`ConnPool` (`internal/pool/pool.go`) hands out `*Conn` to callers, recycles them when returned, dials new ones when needed, and replaces them when they die. It's used by every client type (`Client`, `ClusterClient`, `Ring`, sentinel) — they all share the same pool implementation. There are two thin wrappers:
|
||||
|
||||
- `SingleConnPool` (`pool_single.go`) — wraps exactly one `*Conn`, no concurrency. Used during `initConn` (`HELLO`, `AUTH`, `CLIENT SETNAME`) and re-auth, where we must run commands on a specific connection that is *not* yet in the regular pool.
|
||||
- `StickyConnPool` (`pool_sticky.go`) — wraps a Pooler and pins one connection across many `Get`/`Put` cycles. Used by PubSub, where the subscription state lives on a specific TCP connection.
|
||||
|
||||
Concurrency from clients is bounded by `PoolSize` via a semaphore (`waitTurn`/`freeTurn`). `MaxActiveConns` is a hard ceiling that includes excess (non-pooled) connections; if 0, only `PoolSize` matters.
|
||||
|
||||
---
|
||||
|
||||
## The four primary invariants
|
||||
|
||||
### 1. Every `waitTurn` that succeeds must call `freeTurn`
|
||||
|
||||
The semaphore is the only thing keeping concurrency bounded. A leaked turn is permanently lost capacity. Every error path in `Get`, every retry loop, every panic-recovery must release. Audit new exit paths in `Get`/`Put` carefully — most pool bugs in history trace back to a missed `freeTurn`.
|
||||
|
||||
### 2. FIFO order on `wantConnQueue`
|
||||
|
||||
When the pool is exhausted, callers queue as `wantConn` requests in `wantConnQueue` (`want_conn.go`). They're served in arrival order. Two real bugs hinged on this:
|
||||
|
||||
- **#3777 — FIFO violation in `notifyWaiters`**: the wakeup loop re-read the atomic state on each iteration, so a goroutine that woke up fast could "skip the line" by re-entering `notifyWaiters` ahead of an earlier waiter. Fix: snapshot state locally inside `notifyWaiters` and only consider transitions made within this call. See the comment near `conn_state.go:300` ("atomic would let a fast goroutine's `Transition(StateIdle)` leak into our…").
|
||||
- **#3680 — zombie `wantConn` elements**: cancelled or timed-out waiters stayed in the queue head until the next `discardDoneAtFront`, which delayed real waiters and caused spurious retries. Fix: scan and prune done elements proactively.
|
||||
|
||||
If you change `wantConnQueue`, write a test that asserts a slow first waiter still gets the connection before a faster second waiter.
|
||||
|
||||
### 3. The `ConnState` machine is the single source of truth for liveness
|
||||
|
||||
`conn_state.go` defines six states:
|
||||
|
||||
```
|
||||
StateCreated → StateInitializing → StateIdle ⇄ StateInUse
|
||||
↓
|
||||
StateUnusable (handoff or re-auth)
|
||||
↓
|
||||
StateIdle / StateClosed
|
||||
```
|
||||
|
||||
- `StateCreated` — just allocated, no I/O yet.
|
||||
- `StateInitializing` — running `initConn` (`HELLO`, `AUTH`, etc.). Reachable only via `SingleConnPool`.
|
||||
- `StateIdle` — sitting in `idleConns`, eligible to be popped by `Get`.
|
||||
- `StateInUse` — handed out to a caller.
|
||||
- `StateUnusable` — owned by a background worker (handoff, re-auth). The pool will *not* return this conn from `Get` and will *not* close it from `Put`.
|
||||
- `StateClosed` — terminal.
|
||||
|
||||
There used to also be an atomic `Conn.closed` bool. It was removed in #3783 because it duplicated `StateClosed` and the duplication caused missed updates. **Do not re-introduce shadow flags.** If you need a new connection-level fact, add a state or use the existing fields under the state machine's lock.
|
||||
|
||||
`StateUnusable` is the contract that lets handoff and re-auth coexist with normal traffic without races. `OnGet` rejects unusable conns, `OnPut` keeps them in the pool (instead of recycling/closing), and the background worker drives them back to `StateIdle` (or removes them) when its work finishes.
|
||||
|
||||
### 4. Dial timeout is per-attempt, not cumulative
|
||||
|
||||
`DialerRetries` (default 5), `DialerRetryTimeout` (default 100ms), and the optional `DialerRetryBackoff` function shape retry behaviour. `DialTimeout` is applied **per attempt**: each retry gets a fresh window (#3705 — `fix(timeout): make dial timeout per-retry`). The caller's context deadline is honored only when it's tighter than the per-attempt budget. This prevents a long backoff from eating the next attempt's I/O time.
|
||||
|
||||
If all attempts fail, `tryDial` keeps probing in the background once a second so the pool can recover without a foreground request having to retry the whole sequence.
|
||||
|
||||
---
|
||||
|
||||
## Get / Put semantics
|
||||
|
||||
`Get`:
|
||||
1. `waitTurn` — acquire a semaphore token (blocks).
|
||||
2. `popIdle` — pop an idle conn, retrying past `StateUnusable`/`StateClosed` ones.
|
||||
3. If no idle conn, queue a `wantConn` and let `queuedNewConn` dial in a background goroutine.
|
||||
4. Run `OnGet` hooks (auth, handoff guard).
|
||||
5. Transition `StateIdle → StateInUse` and return.
|
||||
|
||||
`Put`:
|
||||
1. If the conn has unread bytes, peek for a RESP3 push notification. If it's a push frame, leave the buffered data in place (the push processor will consume it on the next `Get`); otherwise, treat the conn as broken and remove it.
|
||||
2. Run `OnPut` hooks. A hook may transition the conn to `StateUnusable` to schedule a background operation; if so, `Put` returns the conn to the pool **without** recycling so the worker can pick it up.
|
||||
3. If the conn is past `ConnMaxIdleTime` / `ConnMaxLifetime` (with jitter), close it.
|
||||
4. Otherwise transition `StateInUse → StateIdle` and prepend to `idleConns`.
|
||||
|
||||
The "unread data on Put → check for push frame" path is what makes RESP3 maintenance notifications work without dedicated reader goroutines. Don't simplify it.
|
||||
|
||||
### Errors on close are noise for stale conns
|
||||
|
||||
#3778 (`fix(pool): suppress pool Close() errors for stale connections`): a connection that exceeded `ConnMaxIdleTime` is half-broken by definition; reporting `EBADF`/`ECONNRESET` from its `Close` is alarm-fatigue. The pool only surfaces close errors for connections that were healthy when we decided to close them.
|
||||
|
||||
---
|
||||
|
||||
## `conn_check`: detecting dead sockets and push frames
|
||||
|
||||
`conn_check.go` (Unix) does a non-blocking `MSG_PEEK` on the socket to distinguish three situations on an idle connection:
|
||||
|
||||
- 0 bytes available, no error → healthy.
|
||||
- EOF → peer closed; remove the conn.
|
||||
- Bytes available → either a push notification (RESP3, leave it) or unsolicited data (broken protocol, remove it).
|
||||
|
||||
`conn_check_dummy.go` (non-Unix) is a no-op stub. Push notification handling on Windows is therefore best-effort; the `Put` peek-on-unread-data path remains correct cross-platform but is the only line of defense. If you change `conn_check`, check both files.
|
||||
|
||||
---
|
||||
|
||||
## Re-auth and handoff coexistence (#3547)
|
||||
|
||||
Both re-auth (driven by `internal/auth/streaming`) and handoff (driven by `maintnotifications`) need exclusive access to a connection while running on it. Both use `StateUnusable` to tell the pool "don't touch this." The bug fixed in #3547 was that re-auth could grab a connection that handoff had already targeted, causing the handoff worker to run against a connection whose credentials had changed.
|
||||
|
||||
The contract now: a conn that is `StateUnusable` *for handoff* must not be picked up by re-auth. Whichever subsystem marks the conn first wins; the other one observes the unusable state via `OnGet` and skips it. The auth pool hook lives in `internal/auth/streaming/pool_hook.go`; the handoff pool hook lives in `maintnotifications/pool_hook.go`. They share the pool's hook list — order of registration does not matter for correctness because the state-machine transitions are atomic, but be careful when adding a third subsystem that needs the same exclusion.
|
||||
|
||||
---
|
||||
|
||||
## Pool configuration
|
||||
|
||||
| Option | Default | Notes |
|
||||
|---|---|---|
|
||||
| `PoolSize` | `10 * GOMAXPROCS` | Soft cap on pooled conns. |
|
||||
| `MinIdleConns` | 0 | Pre-warmed at init; replenished after `Remove`. Background goroutine fills these via `dialHook`, hence the read-lock in `dialHook` (#3225). |
|
||||
| `MaxActiveConns` | 0 (unlimited) | Hard cap including excess (non-pooled) conns used during bursts. |
|
||||
| `PoolTimeout` | `ReadTimeout + 1s` | How long `waitTurn` will block. |
|
||||
| `ConnMaxIdleTime` | 30 min | Eviction in `Put`. |
|
||||
| `ConnMaxLifetime` | 0 (none) | Absolute age limit; checked on `Put`. |
|
||||
| `ConnMaxLifetimeJitter` | 0 | Random ±jitter on `ConnMaxLifetime` to avoid all conns expiring together (#3666). |
|
||||
| `ReadBufferSize` / `WriteBufferSize` | 32 KiB (since v9.12) | Was 4 KiB before. Sentinel client uses 4 KiB (#3476) because its connections are short-lived. If you see odd performance regressions, flip back to 4 KiB and compare. |
|
||||
|
||||
`ConnMaxLifetimeJitter` matters in deployments where many clients started at the same time: without jitter, all their connections expire in the same second and reconnect simultaneously, hammering the server (thundering herd).
|
||||
|
||||
---
|
||||
|
||||
## Hook integration (`hooks.go`)
|
||||
|
||||
`PoolHookManager` carries `OnGet`, `OnPut`, `OnRemove`. It's stored as an `atomic.Value` on `ConnPool` so reads in the hot path don't need a mutex. Hooks are called *outside* the pool's internal lock — never call back into the pool from inside a hook in a way that requires the lock you're not holding.
|
||||
|
||||
`OnGet` is allowed to reject a conn (e.g. handoff hook seeing `ShouldHandoff()` true). `OnPut` is allowed to mutate state (mark unusable, schedule background work). `OnRemove` is for cleanup (per-conn registries, listener unsubscribes).
|
||||
|
||||
When you add a new subsystem that needs per-connection state, prefer registering a pool hook over reaching into `ConnPool` directly. The hook is the seam.
|
||||
|
||||
---
|
||||
|
||||
## History worth knowing
|
||||
|
||||
The recurring shape of pool bugs:
|
||||
|
||||
- **State duplication**: shadow flags (`closed`, old `usable`/`Inited`/`used` atomics) drift out of sync with the real state. Removed in #3559 / #3783.
|
||||
- **Lost waiters / lost turns**: missing `freeTurn` on an error path, or `wantConn` left in the queue with no one to deliver to. #3680, #3777, #3626 (`putIdleConn` turn management).
|
||||
- **Race between background work and `Close`**: pool tries to close a conn that handoff/re-auth still owns. Fixed by `StateUnusable` + hook contract.
|
||||
- **Errors that aren't really errors**: stale-conn close errors (#3778), failed dial-tcp errors that should be treated as redirectable (#3786 — these are routing-layer, but the pool surfaces them).
|
||||
- **Identity verification (HELLO/SETINFO) failures coupling into pool init**: #3295, #3294, #3788. The pattern: feature handshakes during `initConn` should fail soft when the server doesn't support them, and must not deadlock or leave the conn half-initialized.
|
||||
|
||||
When you write a pool fix, add a regression test that fails without your change. The pool's behaviour is too easy to verify by inspection only.
|
||||
1
.github/actions/run-tests/action.yml
vendored
1
.github/actions/run-tests/action.yml
vendored
@@ -47,6 +47,7 @@ runs:
|
||||
elif [[ -n "$REDIS_VERSION" ]]; then
|
||||
# Mapping of redis version to redis testing containers
|
||||
declare -A redis_version_mapping=(
|
||||
["8.10.x"]="8.10-rc2"
|
||||
["8.8.x"]="8.8.0"
|
||||
["8.6.x"]="8.6.1"
|
||||
["8.4.x"]="8.4.0"
|
||||
|
||||
5
.github/workflows/build.yml
vendored
5
.github/workflows/build.yml
vendored
@@ -18,6 +18,7 @@ jobs:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
redis-version:
|
||||
- "8.10.x" # Redis CE 8.10
|
||||
- "8.8.x" # Redis CE 8.8
|
||||
- "8.6.x" # Redis CE 8.6
|
||||
- "8.4.x" # Redis CE 8.4
|
||||
@@ -30,7 +31,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Set up ${{ matrix.go-version }}
|
||||
uses: actions/setup-go@v6
|
||||
uses: actions/setup-go@v7
|
||||
with:
|
||||
go-version: ${{ matrix.go-version }}
|
||||
|
||||
@@ -47,6 +48,7 @@ jobs:
|
||||
|
||||
# Mapping of redis version to redis testing containers
|
||||
declare -A redis_version_mapping=(
|
||||
["8.10.x"]="8.10-rc2"
|
||||
["8.8.x"]="8.8.0"
|
||||
["8.6.x"]="8.6.1"
|
||||
["8.4.x"]="8.4.0"
|
||||
@@ -79,6 +81,7 @@ jobs:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
redis-version:
|
||||
- "8.10.x" # Redis CE 8.10
|
||||
- "8.8.x" # Redis CE 8.8
|
||||
- "8.6.x" # Redis CE 8.6
|
||||
- "8.4.x" # Redis CE 8.4
|
||||
|
||||
4
.github/workflows/doctests.yaml
vendored
4
.github/workflows/doctests.yaml
vendored
@@ -16,7 +16,7 @@ jobs:
|
||||
|
||||
services:
|
||||
redis-stack:
|
||||
image: redislabs/client-libs-test:8.8.0
|
||||
image: redislabs/client-libs-test:8.10-rc2
|
||||
env:
|
||||
TLS_ENABLED: no
|
||||
REDIS_CLUSTER: no
|
||||
@@ -31,7 +31,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Set up ${{ matrix.go-version }}
|
||||
uses: actions/setup-go@v6
|
||||
uses: actions/setup-go@v7
|
||||
with:
|
||||
go-version: ${{ matrix.go-version }}
|
||||
|
||||
|
||||
2
.github/workflows/govulncheck.yml
vendored
2
.github/workflows/govulncheck.yml
vendored
@@ -27,7 +27,7 @@ jobs:
|
||||
uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@v6
|
||||
uses: actions/setup-go@v7
|
||||
with:
|
||||
go-version: "1.26.x"
|
||||
cache: true
|
||||
|
||||
2
.github/workflows/test-e2e.yml
vendored
2
.github/workflows/test-e2e.yml
vendored
@@ -24,7 +24,7 @@ jobs:
|
||||
uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Go ${{ matrix.go-version }}
|
||||
uses: actions/setup-go@v6
|
||||
uses: actions/setup-go@v7
|
||||
with:
|
||||
go-version: ${{ matrix.go-version }}
|
||||
|
||||
|
||||
2
.github/workflows/test-redis-enterprise.yml
vendored
2
.github/workflows/test-redis-enterprise.yml
vendored
@@ -30,7 +30,7 @@ jobs:
|
||||
path: redis-ee
|
||||
|
||||
- name: Set up ${{ matrix.go-version }}
|
||||
uses: actions/setup-go@v6
|
||||
uses: actions/setup-go@v7
|
||||
with:
|
||||
go-version: ${{ matrix.go-version }}
|
||||
|
||||
|
||||
3
.gitignore
vendored
3
.gitignore
vendored
@@ -17,3 +17,6 @@ maintenanceNotifications/docs/
|
||||
# Docker-generated files (TLS certificates, cluster data, etc.)
|
||||
dockers/*/tls/
|
||||
dockers/osscluster-tls/
|
||||
|
||||
# Per-user Claude Code settings (machine/personal overrides, not shared policy)
|
||||
.claude/settings.local.json
|
||||
|
||||
228
AGENTS.md
Normal file
228
AGENTS.md
Normal file
@@ -0,0 +1,228 @@
|
||||
# AGENTS.md
|
||||
|
||||
Guidance for AI coding agents (and humans) working in this repository. This is
|
||||
the shared, tool-agnostic source of truth: Claude Code loads it through
|
||||
`CLAUDE.md` (which imports this file), and other agents (Codex, Cursor, Aider,
|
||||
Zed, …) read `AGENTS.md` directly. Edit repository guidance here, not in
|
||||
`CLAUDE.md`.
|
||||
|
||||
## Repository
|
||||
|
||||
go-redis is the official Redis client for Go. Module path:
|
||||
`github.com/redis/go-redis/v9` (Go 1.24+). The repo is a multi-module workspace
|
||||
— every directory containing a `go.mod` is built and tested independently:
|
||||
|
||||
- root (`github.com/redis/go-redis/v9`) — the client library.
|
||||
- `extra/redisotel`, `extra/redisotel-native`, `extra/redisprometheus`,
|
||||
`extra/rediscensus`, `extra/rediscmd` — instrumentation adapters with their
|
||||
own module paths (so they can pin large telemetry deps without forcing them on
|
||||
root consumers).
|
||||
- `internal/customvet` — custom `go vet` analyzers (also its own module).
|
||||
- `maintnotifications/e2e`, `doctests`, `fuzz`, examples under `example/` —
|
||||
separate modules.
|
||||
|
||||
The Makefile iterates over every `go.mod` (`GO_MOD_DIRS`) when running
|
||||
`test.ci`, `go_mod_tidy`, etc. When you add a dependency in one module, you
|
||||
almost never need to update the others.
|
||||
|
||||
## Common commands
|
||||
|
||||
Tests run against a Redis stack started via Docker Compose. Profiles in
|
||||
`docker-compose.yml` control which services come up (`standalone`, `cluster`,
|
||||
`sentinel`, `all`, `e2e`).
|
||||
|
||||
```sh
|
||||
make docker.start # bring up the full test stack (profile: all)
|
||||
make docker.stop
|
||||
make test # docker.start -> test.ci -> docker.stop
|
||||
make test.ci # run tests assuming containers are already up
|
||||
make test.ci.skip-vectorsets # when REDIS_VERSION < 8
|
||||
make bench # go test -bench=. (root module only)
|
||||
make fmt # gofumpt + goimports -local github.com/redis/go-redis
|
||||
make build
|
||||
make go_mod_tidy # go mod tidy across every module
|
||||
```
|
||||
|
||||
E2E (maintenance notifications) needs the extra `cae-resp-proxy` service:
|
||||
|
||||
```sh
|
||||
make test.e2e # starts e2e profile, runs ./maintnotifications/e2e/, tears down
|
||||
make test.e2e.docker # subset that runs inside docker
|
||||
make test.e2e.logic # logic-only tests, no proxy required
|
||||
```
|
||||
|
||||
Run a single test. The root suite is Ginkgo-based (`bsm/ginkgo` + `bsm/gomega`
|
||||
forks), so `go test -run` matches the Go-level wrapper and you focus a spec with
|
||||
the Ginkgo flag:
|
||||
|
||||
```sh
|
||||
go test -run TestGinkgoSuite . -ginkgo.focus="ZAdd"
|
||||
go test -run TestGinkgoSuite . -ginkgo.focus="cluster"
|
||||
```
|
||||
|
||||
Plain `go test` tests (most files outside the Ginkgo suite, e.g. `internal/...`,
|
||||
`maintnotifications/...`) work the usual way:
|
||||
|
||||
```sh
|
||||
go test -run TestConnStateMachine ./internal/pool/...
|
||||
go test -race -run TestCircuitBreaker ./maintnotifications/...
|
||||
```
|
||||
|
||||
Env knobs (passed through the Makefile):
|
||||
|
||||
- `REDIS_VERSION` — e.g. `8.8`. Drives both the test image tag and
|
||||
`main_test.go` version-gating (`SkipBeforeRedisVersion` /
|
||||
`SkipAfterRedisVersion`).
|
||||
- `CLIENT_LIBS_TEST_IMAGE` — full image ref, e.g.
|
||||
`redislabs/client-libs-test:8.8-m03`.
|
||||
- `RE_CLUSTER=true` — run against a Redis Enterprise cluster instead of the
|
||||
docker-compose stack (the suite then skips ring/sentinel/TLS-cluster setup).
|
||||
- `RCE_DOCKER=true` — Redis CE in docker (default for `make test`).
|
||||
- `REDIS_PORT` — override the default standalone port (`6380`).
|
||||
|
||||
CI also runs the custom vet tool:
|
||||
`go vet -vettool ./internal/customvet/customvet ./...`. The `setval` analyzer
|
||||
requires every `Cmder` with a `Result()` to also have a `SetVal()`.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Client types (root package)
|
||||
|
||||
All clients are in the root package and share most plumbing:
|
||||
|
||||
- `Client` (`redis.go`) — single-node client.
|
||||
- `ClusterClient` (`osscluster.go`) — Redis Cluster aware. `osscluster_router.go`
|
||||
routes commands to the right shard; `internal/routing/` handles cluster-wide
|
||||
aggregation policies (e.g. fan-out for `KEYS`, `DBSIZE`).
|
||||
- `Ring` (`ring.go`) — client-side sharding across independent Redis nodes
|
||||
(consistent hashing, no cluster protocol).
|
||||
- Failover client (`sentinel.go`) — Sentinel-managed failover.
|
||||
- `UniversalClient` (`universal.go`) — wrapper that picks one of the above based
|
||||
on options.
|
||||
|
||||
Command surface lives in topical files: `string_commands.go`, `hash_commands.go`,
|
||||
`stream_commands.go`, `search_commands.go`, `vectorset_commands.go`, etc. Each
|
||||
file defines methods on the shared `Cmdable` interface so every client type gets
|
||||
the same API.
|
||||
|
||||
### Hooks (`redis.go` `hooksMixin`)
|
||||
|
||||
Three hook chains run around every operation: `DialHook`, `ProcessHook`,
|
||||
`ProcessPipelineHook`. Hooks are registered via `client.AddHook(...)` and chain
|
||||
in FIFO order; each hook must call `next` to continue. When a hook wraps an
|
||||
error, it must call `cmd.SetErr(wrappedErr)` so the typed-error helpers
|
||||
(`redis.IsLoadingError`, `IsMovedError`, etc. in `error.go`) keep working through
|
||||
`errors.As`. The README has a longer pipeline-hook example.
|
||||
|
||||
### Connection pool (`internal/pool`)
|
||||
|
||||
Owns dialing, idle/active connection bookkeeping, conn state (`conn_state.go`),
|
||||
pubsub-conn lifecycle (`pubsub.go`), and the dial-retry/backoff logic that powers
|
||||
`DialerRetries` / `DialerRetryBackoff` (also exposed at `dial_retry_backoff.go`
|
||||
in the root). `OnConnect`, `MinIdleConns`, and the buffer-size options
|
||||
(`ReadBufferSize`/`WriteBufferSize`, default 32 KiB since v9.12) flow through
|
||||
here.
|
||||
|
||||
### Protocol (`internal/proto`)
|
||||
|
||||
RESP2/RESP3 reader and writer. Push notifications (RESP3 `>`-prefixed frames) are
|
||||
peeked here and dispatched via the `push/` package. The `push.Registry` lets
|
||||
callers register handlers for specific notification names;
|
||||
`maintnotifications/push_notification_handler.go` is how `maintnotifications`
|
||||
plugs in.
|
||||
|
||||
### Maintenance notifications (`maintnotifications/`)
|
||||
|
||||
This is a non-trivial subsystem worth understanding before touching
|
||||
cluster/handoff code. It listens for RESP3 push notifications about cluster
|
||||
maintenance (`MOVING`, `MIGRATING`, `MIGRATED`, `FAILING_OVER`, `FAILED_OVER`
|
||||
for standalone; `SMIGRATING`, `SMIGRATED` for cluster) and performs seamless
|
||||
connection handoff to new endpoints. Key pieces:
|
||||
|
||||
- `manager.go` — coordinates state transitions.
|
||||
- `handoff_worker.go` — moves in-flight ops to new connections.
|
||||
- `pool_hook.go` — integrates with `internal/pool` to mark/replace connections.
|
||||
- `circuit_breaker.go` — backs off when the upstream is unhealthy.
|
||||
- `state.go` — per-connection state machine.
|
||||
- E2E coverage lives in `maintnotifications/e2e/` and drives a fault-injector /
|
||||
RESP proxy (`cae-resp-proxy`).
|
||||
|
||||
Configuration is via `redis.Options.MaintNotificationsConfig`; modes are
|
||||
`ModeAuto` (default), `ModeEnabled` (require server support), `ModeDisabled`.
|
||||
RESP3 (`Protocol: 3`) is required.
|
||||
|
||||
### Authentication (`auth/`, `internal/auth/streaming`)
|
||||
|
||||
Four credential sources, in priority order: streaming provider (e.g. Entra ID
|
||||
via `go-redis-entraid`), context-based provider, function provider, static
|
||||
`Username`/`Password`. The streaming provider is what enables token rotation
|
||||
without reconnecting — the listener in `auth/reauth_credentials_listener.go`
|
||||
issues `AUTH` on each refresh.
|
||||
|
||||
### Internal helpers
|
||||
|
||||
- `internal/hscan` — struct scanning for `HGETALL` results (`Scan` interface
|
||||
re-exported as `redis.Scanner`).
|
||||
- `internal/hashtag` — extracts `{tag}` segments for cluster slot routing.
|
||||
- `internal/routing` — aggregator policies and shard pickers used by
|
||||
`ClusterClient` for multi-shard commands.
|
||||
- `internal/otel` — small OpenTelemetry shim used to keep root free of telemetry
|
||||
deps; full instrumentation lives in `extra/redisotel-native`.
|
||||
|
||||
## Architectural specs
|
||||
|
||||
Read the relevant design doc **before** changing code in that subsystem. They
|
||||
cover invariants and decisions that aren't obvious from the code, and are plain
|
||||
markdown any tool or editor can open:
|
||||
|
||||
- `.claude/specs/pool.md` — connection pool: `wantConn` queue and FIFO
|
||||
discipline, `ConnState` machine, dial retry/backoff, hook integration, the
|
||||
re-auth/handoff coexistence contract.
|
||||
- `.claude/specs/cluster-routing.md` — slot computation, MOVED/ASK redirection,
|
||||
request/response policies, aggregators, replica routing, topology reload,
|
||||
cross-slot rules.
|
||||
- `.claude/specs/maintnotifications.md` — RESP3 push notification protocol, mode
|
||||
handshake, per-conn state, handoff worker pool, circuit breaker, endpoint-type
|
||||
resolution, cluster vs. standalone differences.
|
||||
|
||||
## Conventions
|
||||
|
||||
- New `Cmder` type → also implement `SetVal` (the custom vet `setval` check
|
||||
enforces this; `SetErr` is on the embedded `baseCmd`).
|
||||
- Wrap errors with custom error types that implement `Unwrap`, or use
|
||||
`fmt.Errorf("...: %w", err)`. Always call `cmd.SetErr(...)` after wrapping so
|
||||
typed-error checks still pass.
|
||||
- `gofumpt` + `goimports -local github.com/redis/go-redis` is the formatter
|
||||
(`make fmt`); CI runs both.
|
||||
- Don't log directly — use `internal.Logger` (set via `redis.SetLogger`);
|
||||
`logging.Disable()` is called in tests.
|
||||
- Version-gate Redis-version-specific tests with `SkipBeforeRedisVersion` /
|
||||
`SkipAfterRedisVersion` rather than skipping at the suite level.
|
||||
|
||||
### Commits and PRs
|
||||
|
||||
Conventional Commits, short and exact — `<type>(<scope>): <imperative summary>`.
|
||||
Subject ≤50 chars (hard cap 72), imperative ("add", not "added"), no trailing
|
||||
period. Body only when the *why* isn't obvious from the diff; wrap at 72.
|
||||
|
||||
- Types: `feat`, `fix`, `refactor`, `perf`, `docs`, `test`, `chore` (also
|
||||
`build`, `ci`, `style`, `revert`).
|
||||
- Scope = the subsystem touched, lowercase: `pool`, `conn`, `pubsub`,
|
||||
`sentinel`, `retry`, `command`/`cmd`, `vectorset`, `otel`, `streams`, `push`,
|
||||
`deps`, `ci`, `tests`, `docs`. Omit only for genuinely cross-cutting changes.
|
||||
- Breaking change: `feat(scope)!: ...` plus a `BREAKING CHANGE:` body line.
|
||||
Reference issues/PRs at the end — `Closes #42`, `Refs #17`.
|
||||
- **No AI-attribution trailer.** Do not add `Co-Authored-By: …`, "Generated with
|
||||
…", or any AI-attribution line to commits or PR bodies in this repo.
|
||||
|
||||
## Repo-specific tooling
|
||||
|
||||
`.claude/` holds shared AI config:
|
||||
|
||||
- `commands/` — slash commands (e.g. `check-ci`, which summarizes a PR's CI).
|
||||
- `skills/` — task playbooks: `testing`, `add-command`, `commit-style`,
|
||||
`update-ci-image`, `prepare-release`.
|
||||
- `specs/` — the architecture docs listed above.
|
||||
|
||||
For Claude Code, the skills auto-trigger from their descriptions. For other
|
||||
tools, each `SKILL.md` is plain markdown you can open and follow directly.
|
||||
19
CLAUDE.md
Normal file
19
CLAUDE.md
Normal file
@@ -0,0 +1,19 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
Repository guidance — layout, commands, architecture, and conventions — is
|
||||
maintained in [`AGENTS.md`](AGENTS.md) so it stays shared across AI tools. It is
|
||||
imported below; edit `AGENTS.md`, not this file, for that content.
|
||||
|
||||
@AGENTS.md
|
||||
|
||||
## Claude-specific
|
||||
|
||||
- Repo-local **skills** under `.claude/skills/` auto-trigger from their
|
||||
descriptions — no need to invoke them manually (the set is listed in
|
||||
`AGENTS.md`).
|
||||
- **Slash commands** under `.claude/commands/` (e.g. `/check-ci`) are available
|
||||
in-session.
|
||||
- Architectural **specs** under `.claude/specs/` are read on demand; open the
|
||||
relevant one before changing that subsystem.
|
||||
@@ -37,7 +37,7 @@ Here's how to get started with your code contribution:
|
||||
> Note: this clones and builds the docker containers specified in `docker-compose.yml`, to understand more about
|
||||
> the infrastructure that will be started you can check the `docker-compose.yml`. You also have the possiblity
|
||||
> to specify the redis image that will be pulled with the env variable `CLIENT_LIBS_TEST_IMAGE`.
|
||||
> By default the docker image that will be pulled and started is `redislabs/client-libs-test:8.2.1-pre`.
|
||||
> By default the docker image that will be pulled and started is `redislabs/client-libs-test:8.10-rc2`.
|
||||
> If you want to test with newer Redis version, using a newer version of `redislabs/client-libs-test` should work out of the box.
|
||||
|
||||
4. While developing, make sure the tests pass by running `make test` (if you have the docker containers running, `make test.ci` may be sufficient).
|
||||
|
||||
4
Makefile
4
Makefile
@@ -1,8 +1,8 @@
|
||||
GO_MOD_DIRS := $(shell find . -type f -name 'go.mod' -exec dirname {} \; | sort)
|
||||
REDIS_VERSION ?= 8.8
|
||||
REDIS_VERSION ?= 8.10
|
||||
RE_CLUSTER ?= false
|
||||
RCE_DOCKER ?= true
|
||||
CLIENT_LIBS_TEST_IMAGE ?= redislabs/client-libs-test:8.8.0
|
||||
CLIENT_LIBS_TEST_IMAGE ?= redislabs/client-libs-test:8.10-rc2
|
||||
|
||||
docker.start:
|
||||
export RE_CLUSTER=$(RE_CLUSTER) && \
|
||||
|
||||
@@ -265,7 +265,7 @@ var _ = Describe("ACL permissions", Label("NonRedisEnterprise"), func() {
|
||||
})
|
||||
|
||||
It("set permissions for module commands", func() {
|
||||
SkipBeforeRedisVersion(8, "permissions for modules are supported for Redis Version >=8")
|
||||
SkipBeforeRedisVersion("8", "permissions for modules are supported for Redis Version >=8")
|
||||
Expect(client.FlushDB(ctx).Err()).NotTo(HaveOccurred())
|
||||
val, err := client.FTCreate(ctx, "txt", &redis.FTCreateOptions{}, &redis.FieldSchema{FieldName: "txt", FieldType: redis.SearchFieldTypeText}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
@@ -345,7 +345,7 @@ var _ = Describe("ACL permissions", Label("NonRedisEnterprise"), func() {
|
||||
})
|
||||
|
||||
It("set permissions for module categories", func() {
|
||||
SkipBeforeRedisVersion(8, "permissions for modules are supported for Redis Version >=8")
|
||||
SkipBeforeRedisVersion("8", "permissions for modules are supported for Redis Version >=8")
|
||||
Expect(client.FlushDB(ctx).Err()).NotTo(HaveOccurred())
|
||||
val, err := client.FTCreate(ctx, "txt", &redis.FTCreateOptions{}, &redis.FieldSchema{FieldName: "txt", FieldType: redis.SearchFieldTypeText}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
@@ -442,7 +442,7 @@ var _ = Describe("ACL Categories", func() {
|
||||
})
|
||||
|
||||
It("lists acl categories and subcategories with Modules", func() {
|
||||
SkipBeforeRedisVersion(8, "modules are included in acl for redis version >= 8")
|
||||
SkipBeforeRedisVersion("8", "modules are included in acl for redis version >= 8")
|
||||
aclTestCase := map[string]string{
|
||||
"search": "FT.CREATE",
|
||||
"bloom": "bf.add",
|
||||
|
||||
@@ -17,7 +17,7 @@ var _ = Describe("Array Commands", Label("array"), func() {
|
||||
var client *redis.Client
|
||||
|
||||
BeforeEach(func() {
|
||||
SkipBeforeRedisVersion(8.8, "Redis 8.8.0 introduces support for Array")
|
||||
SkipBeforeRedisVersion("8.8", "Redis 8.8.0 introduces support for Array")
|
||||
client = redis.NewClient(redisOptions())
|
||||
Expect(client.FlushDB(ctx).Err()).NotTo(HaveOccurred())
|
||||
})
|
||||
|
||||
63
command.go
63
command.go
@@ -155,6 +155,7 @@ const (
|
||||
CmdTypeFTSearch
|
||||
CmdTypeTSTimestampValue
|
||||
CmdTypeTSTimestampValueSlice
|
||||
CmdTypeTSNRangePivotRowSlice
|
||||
CmdTypeHotKeys
|
||||
CmdTypeIncrEXInt
|
||||
CmdTypeIncrEXFloat
|
||||
@@ -1322,8 +1323,13 @@ func (cmd *IntSliceCmd) readReply(rd *proto.Reader) error {
|
||||
}
|
||||
cmd.val = make([]int64, n)
|
||||
for i := 0; i < len(cmd.val); i++ {
|
||||
if cmd.val[i], err = rd.ReadInt(); err != nil {
|
||||
switch num, err := rd.ReadInt(); {
|
||||
case err == Nil:
|
||||
cmd.val[i] = 0
|
||||
case err != nil:
|
||||
return err
|
||||
default:
|
||||
cmd.val[i] = num
|
||||
}
|
||||
}
|
||||
return nil
|
||||
@@ -1382,8 +1388,13 @@ func (cmd *UintSliceCmd) readReply(rd *proto.Reader) error {
|
||||
}
|
||||
cmd.val = make([]uint64, n)
|
||||
for i := range cmd.val {
|
||||
if cmd.val[i], err = rd.ReadUint(); err != nil {
|
||||
switch num, err := rd.ReadUint(); {
|
||||
case err == Nil:
|
||||
cmd.val[i] = 0
|
||||
case err != nil:
|
||||
return err
|
||||
default:
|
||||
cmd.val[i] = num
|
||||
}
|
||||
}
|
||||
return nil
|
||||
@@ -2110,8 +2121,13 @@ func (cmd *BoolSliceCmd) readReply(rd *proto.Reader) error {
|
||||
}
|
||||
cmd.val = make([]bool, n)
|
||||
for i := 0; i < len(cmd.val); i++ {
|
||||
if cmd.val[i], err = rd.ReadBool(); err != nil {
|
||||
switch b, err := rd.ReadBool(); {
|
||||
case err == Nil:
|
||||
cmd.val[i] = false
|
||||
case err != nil:
|
||||
return err
|
||||
default:
|
||||
cmd.val[i] = b
|
||||
}
|
||||
}
|
||||
return nil
|
||||
@@ -5334,6 +5350,10 @@ type SlowLog struct {
|
||||
// https://redis.io/commands/slowlog#output-format
|
||||
ClientAddr string
|
||||
ClientName string
|
||||
// CommandArgc is the command's total argument count (including the command
|
||||
// name), emitted only by Redis 8.10 or greater. It may exceed len(Args) when
|
||||
// the slow log truncates the stored arguments (slowlog-max-argc, default 32).
|
||||
CommandArgc int64
|
||||
}
|
||||
|
||||
type SlowLogCmd struct {
|
||||
@@ -5385,6 +5405,9 @@ func (cmd *SlowLogCmd) readReply(rd *proto.Reader) error {
|
||||
if nn < 4 {
|
||||
return fmt.Errorf("redis: got %d elements in slowlog get, expected at least 4", nn)
|
||||
}
|
||||
if nn > 7 {
|
||||
return fmt.Errorf("redis: got %d elements in slowlog get, expected at most 7", nn)
|
||||
}
|
||||
|
||||
if cmd.val[i].ID, err = rd.ReadInt(); err != nil {
|
||||
return err
|
||||
@@ -5429,13 +5452,19 @@ func (cmd *SlowLogCmd) readReply(rd *proto.Reader) error {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// Drain any elements past the 6 this parser knows about so a server
|
||||
// that declares a longer entry array doesn't leave frames on the wire.
|
||||
for j := 6; j < nn; j++ {
|
||||
if err = rd.DiscardNext(); err != nil {
|
||||
|
||||
// Redis 8.10+ appends a 7th field: the command's total argument count.
|
||||
if nn >= 7 {
|
||||
if cmd.val[i].CommandArgc, err = rd.ReadInt(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// Drain any elements past the 7 this parser knows about so a server
|
||||
// that declares a longer entry array doesn't leave frames on the wire.
|
||||
for j := 7; j < nn; j++ {
|
||||
if err = rd.DiscardNext(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5448,11 +5477,12 @@ func (cmd *SlowLogCmd) Clone() Cmder {
|
||||
val = make([]SlowLog, len(cmd.val))
|
||||
for i, log := range cmd.val {
|
||||
val[i] = SlowLog{
|
||||
ID: log.ID,
|
||||
Time: log.Time,
|
||||
Duration: log.Duration,
|
||||
ClientAddr: log.ClientAddr,
|
||||
ClientName: log.ClientName,
|
||||
ID: log.ID,
|
||||
Time: log.Time,
|
||||
Duration: log.Duration,
|
||||
ClientAddr: log.ClientAddr,
|
||||
ClientName: log.ClientName,
|
||||
CommandArgc: log.CommandArgc,
|
||||
}
|
||||
if log.Args != nil {
|
||||
val[i].Args = make([]string, len(log.Args))
|
||||
@@ -8998,6 +9028,13 @@ func ExtractCommandValue(cmd interface{}) (interface{}, error) {
|
||||
}); ok {
|
||||
return tsTimestampValueSliceCmd.Val(), tsTimestampValueSliceCmd.Err()
|
||||
}
|
||||
case CmdTypeTSNRangePivotRowSlice:
|
||||
if tsNRangePivotRowSliceCmd, ok := cmd.(interface {
|
||||
Val() []TSNRangePivotRow
|
||||
Err() error
|
||||
}); ok {
|
||||
return tsNRangePivotRowSliceCmd.Val(), tsNRangePivotRowSliceCmd.Err()
|
||||
}
|
||||
case CmdTypeStringSlice:
|
||||
if stringSliceCmd, ok := cmd.(interface {
|
||||
Val() []string
|
||||
|
||||
67
command_nil_element_desync_test.go
Normal file
67
command_nil_element_desync_test.go
Normal file
@@ -0,0 +1,67 @@
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/redis/go-redis/v9/internal/proto"
|
||||
)
|
||||
|
||||
// A flat scalar array can carry a nil in a non-final position. BITFIELD with
|
||||
// OVERFLOW FAIL returns nil for an operation that would overflow, e.g.
|
||||
// `BITFIELD k OVERFLOW FAIL INCRBY u8 0 255 INCRBY u8 0 255 GET u8 0` replies
|
||||
// [255, nil, 255]. IntSliceCmd, UintSliceCmd and BoolSliceCmd used to return on
|
||||
// the nil element, leaving the trailing elements on the wire and desyncing the
|
||||
// pooled connection so the next command read them as its reply. They must
|
||||
// consume the whole array like FloatSliceCmd and StringSliceCmd already do.
|
||||
func TestSliceCmdNilElementNoDesync(t *testing.T) {
|
||||
// Each reply is [ v, nil, v ] followed by the next command's reply :999.
|
||||
// A correct parser consumes all three elements and leaves :999 for the next read.
|
||||
t.Run("IntSliceCmd", func(t *testing.T) {
|
||||
rd := proto.NewReader(strings.NewReader("*3\r\n:255\r\n_\r\n:100\r\n:999\r\n"))
|
||||
cmd := NewIntSliceCmd(context.Background())
|
||||
if err := cmd.readReply(rd); err != nil {
|
||||
t.Fatalf("readReply: %v", err)
|
||||
}
|
||||
if got := cmd.val; len(got) != 3 || got[0] != 255 || got[1] != 0 || got[2] != 100 {
|
||||
t.Fatalf("unexpected val: %+v", got)
|
||||
}
|
||||
assertNextReplyInt(t, rd, 999)
|
||||
})
|
||||
|
||||
t.Run("UintSliceCmd", func(t *testing.T) {
|
||||
rd := proto.NewReader(strings.NewReader("*3\r\n:255\r\n_\r\n:100\r\n:999\r\n"))
|
||||
cmd := NewUintSliceCmd(context.Background())
|
||||
if err := cmd.readReply(rd); err != nil {
|
||||
t.Fatalf("readReply: %v", err)
|
||||
}
|
||||
if got := cmd.val; len(got) != 3 || got[0] != 255 || got[1] != 0 || got[2] != 100 {
|
||||
t.Fatalf("unexpected val: %+v", got)
|
||||
}
|
||||
assertNextReplyInt(t, rd, 999)
|
||||
})
|
||||
|
||||
t.Run("BoolSliceCmd", func(t *testing.T) {
|
||||
rd := proto.NewReader(strings.NewReader("*3\r\n:1\r\n_\r\n:1\r\n:999\r\n"))
|
||||
cmd := NewBoolSliceCmd(context.Background())
|
||||
if err := cmd.readReply(rd); err != nil {
|
||||
t.Fatalf("readReply: %v", err)
|
||||
}
|
||||
if got := cmd.val; len(got) != 3 || !got[0] || got[1] || !got[2] {
|
||||
t.Fatalf("unexpected val: %+v", got)
|
||||
}
|
||||
assertNextReplyInt(t, rd, 999)
|
||||
})
|
||||
}
|
||||
|
||||
func assertNextReplyInt(t *testing.T, rd *proto.Reader, want int64) {
|
||||
t.Helper()
|
||||
got, err := rd.ReadInt()
|
||||
if err != nil {
|
||||
t.Fatalf("next reply read failed (connection desynced): %v", err)
|
||||
}
|
||||
if got != want {
|
||||
t.Fatalf("connection desynced: next reply = %d, want %d", got, want)
|
||||
}
|
||||
}
|
||||
@@ -707,7 +707,7 @@ func (c cmdable) ReplicaOf(ctx context.Context, host, port string) *StatusCmd {
|
||||
}
|
||||
|
||||
func (c cmdable) SlowLogGet(ctx context.Context, num int64) *SlowLogCmd {
|
||||
cmd := NewSlowLogCmd(context.Background(), "slowlog", "get", num)
|
||||
cmd := NewSlowLogCmd(ctx, "slowlog", "get", num)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
@@ -798,6 +798,11 @@ func (c *ModuleLoadexConfig) toArgs() []interface{} {
|
||||
|
||||
// ModuleLoadex Redis `MODULE LOADEX path [CONFIG name value [CONFIG name value ...]] [ARGS args [args ...]]` command.
|
||||
func (c cmdable) ModuleLoadex(ctx context.Context, conf *ModuleLoadexConfig) *StringCmd {
|
||||
if conf == nil {
|
||||
cmd := NewStringCmd(ctx)
|
||||
cmd.SetErr(errors.New("redis: ModuleLoadex nil config"))
|
||||
return cmd
|
||||
}
|
||||
cmd := NewStringCmd(ctx, conf.toArgs()...)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
|
||||
536
commands_test.go
536
commands_test.go
File diff suppressed because it is too large
Load Diff
@@ -3,7 +3,6 @@ package redis_test
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -12,19 +11,17 @@ import (
|
||||
)
|
||||
|
||||
func init() {
|
||||
// Initialize RedisVersion from environment variable for regular Go tests
|
||||
// (Ginkgo tests initialize this in BeforeSuite)
|
||||
if version := os.Getenv("REDIS_VERSION"); version != "" {
|
||||
if v, err := strconv.ParseFloat(strings.Trim(version, "\""), 64); err == nil && v > 0 {
|
||||
RedisVersion = v
|
||||
}
|
||||
// Initialize version vars from environment for regular Go tests
|
||||
// (Ginkgo tests initialize these in BeforeSuite)
|
||||
if version := strings.Trim(os.Getenv("REDIS_VERSION"), "\""); version != "" {
|
||||
redisMajorVersion, redisMinorVersion = parseRedisVersionStr(version)
|
||||
}
|
||||
}
|
||||
|
||||
// skipIfRedisBelow84 checks if Redis version is below 8.4 and skips the test if so
|
||||
func skipIfRedisBelow84(t *testing.T) {
|
||||
if RedisVersion < 8.4 {
|
||||
t.Skipf("Skipping test: Redis version %.1f < 8.4 (DIGEST command requires Redis 8.4+)", RedisVersion)
|
||||
if !redisVersionAtLeast("8.4") {
|
||||
t.Skipf("Skipping test: redis version < 8.4 (DIGEST command requires Redis 8.4+)")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
|
||||
x-default-image: &default-image ${CLIENT_LIBS_TEST_IMAGE:-redislabs/client-libs-test:8.8.0}
|
||||
x-default-image: &default-image ${CLIENT_LIBS_TEST_IMAGE:-redislabs/client-libs-test:8.10-rc2}
|
||||
|
||||
services:
|
||||
redis:
|
||||
|
||||
3
error.go
3
error.go
@@ -168,6 +168,9 @@ func shouldRetry(err error, retryTimeout bool) bool {
|
||||
return true
|
||||
}
|
||||
|
||||
// Other server errors are not retried. This includes the logical
|
||||
// -SEARCH_TIMEOUT (search-on-timeout fail): retrying would just repeat the
|
||||
// same expensive query.
|
||||
return false
|
||||
}
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@ package redis_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
|
||||
@@ -49,6 +50,8 @@ var _ = Describe("error", func() {
|
||||
proto.ParseErrorReply([]byte("-TRYAGAIN Command cannot be processed, please try again")): true,
|
||||
proto.ParseErrorReply([]byte("-NOREPLICAS Not enough good replicas to write")): true,
|
||||
proto.ParseErrorReply([]byte("-ERR other")): false,
|
||||
// Logical search timeout (search-on-timeout fail): never retried.
|
||||
proto.ParseErrorReply([]byte("-SEARCH_TIMEOUT Timeout limit was reached")): false,
|
||||
}
|
||||
|
||||
for err, expected := range data {
|
||||
@@ -66,4 +69,15 @@ var _ = Describe("error", func() {
|
||||
Expect(redis.ShouldRetry(t2, true)).To(Equal(true))
|
||||
Expect(redis.ShouldRetry(t2, false)).To(Equal(true))
|
||||
})
|
||||
|
||||
It("should not retry a logical search timeout", func() {
|
||||
// -SEARCH_TIMEOUT (search-on-timeout fail) is never retried, plain or wrapped.
|
||||
err := proto.ParseErrorReply([]byte("-SEARCH_TIMEOUT Timeout limit was reached"))
|
||||
Expect(redis.ShouldRetry(err, false)).To(BeFalse())
|
||||
Expect(redis.ShouldRetry(err, true)).To(BeFalse())
|
||||
|
||||
wrapped := fmt.Errorf("query failed: %w", err)
|
||||
Expect(redis.ShouldRetry(wrapped, false)).To(BeFalse())
|
||||
Expect(redis.ShouldRetry(wrapped, true)).To(BeFalse())
|
||||
})
|
||||
})
|
||||
|
||||
@@ -33,6 +33,15 @@ type config struct {
|
||||
mp metric.MeterProvider
|
||||
meter metric.Meter
|
||||
|
||||
// semconvCompliantMetrics, when true, registers the cumulative pool-stat
|
||||
// metrics (waits, timeouts, hits, misses) as monotonic Counters as
|
||||
// required by the OpenTelemetry database semantic conventions. It defaults
|
||||
// to false to preserve backwards compatibility: existing users have these
|
||||
// exported as UpDownCounter (which some backends surface as a Gauge), and
|
||||
// switching the instrument type is a breaking change for them. New users
|
||||
// that want spec-compliant metrics can opt in. See #3612.
|
||||
semconvCompliantMetrics bool
|
||||
|
||||
poolName string
|
||||
|
||||
closeChan chan struct{}
|
||||
@@ -228,3 +237,18 @@ func WithCloseChan(closeChan chan struct{}) MetricsOption {
|
||||
conf.closeChan = closeChan
|
||||
})
|
||||
}
|
||||
|
||||
// WithSemConvCompliantMetrics registers the cumulative connection-pool metrics
|
||||
// (db.client.connections.waits, .timeouts, .hits, .misses) as monotonic
|
||||
// Counters, matching the OpenTelemetry database semantic conventions.
|
||||
//
|
||||
// It is opt-in and defaults to off. By default these metrics remain
|
||||
// UpDownCounters for backwards compatibility, because changing an existing
|
||||
// instrument's type is a breaking change for current OTel users (e.g. the
|
||||
// Prometheus exporter would change the exported metric type). Enable this when
|
||||
// starting fresh or when you can tolerate the metric-type change. See #3612.
|
||||
func WithSemConvCompliantMetrics(on bool) MetricsOption {
|
||||
return metricsOption(func(conf *config) {
|
||||
conf.semconvCompliantMetrics = on
|
||||
})
|
||||
}
|
||||
|
||||
@@ -12,6 +12,7 @@ require (
|
||||
go.opentelemetry.io/otel v1.43.0
|
||||
go.opentelemetry.io/otel/metric v1.43.0
|
||||
go.opentelemetry.io/otel/sdk v1.43.0
|
||||
go.opentelemetry.io/otel/sdk/metric v1.43.0
|
||||
go.opentelemetry.io/otel/trace v1.43.0
|
||||
)
|
||||
|
||||
|
||||
@@ -140,6 +140,25 @@ func poolStatsAttrs(conf *config) (poolAttrs, idleAttrs, usedAttrs attribute.Set
|
||||
return
|
||||
}
|
||||
|
||||
// cumulativePoolStat creates an observable instrument for a monotonically
|
||||
// increasing pool statistic. By default it uses Int64ObservableUpDownCounter to
|
||||
// preserve backwards compatibility with existing exporters; when
|
||||
// conf.semconvCompliantMetrics is set it uses Int64ObservableCounter as the
|
||||
// OpenTelemetry database semantic conventions require. Both instrument types
|
||||
// satisfy metric.Int64Observable, so callers observe them uniformly. See #3612.
|
||||
func cumulativePoolStat(conf *config, name, description string) (metric.Int64Observable, error) {
|
||||
if conf.semconvCompliantMetrics {
|
||||
return conf.meter.Int64ObservableCounter(
|
||||
name,
|
||||
metric.WithDescription(description),
|
||||
)
|
||||
}
|
||||
return conf.meter.Int64ObservableUpDownCounter(
|
||||
name,
|
||||
metric.WithDescription(description),
|
||||
)
|
||||
}
|
||||
|
||||
func reportPoolStats(rdb *redis.Client, conf *config) (metric.Registration, error) {
|
||||
poolAttrs, idleAttrs, usedAttrs := poolStatsAttrs(conf)
|
||||
|
||||
@@ -175,9 +194,9 @@ func reportPoolStats(rdb *redis.Client, conf *config) (metric.Registration, erro
|
||||
return nil, err
|
||||
}
|
||||
|
||||
waits, err := conf.meter.Int64ObservableUpDownCounter(
|
||||
waits, err := cumulativePoolStat(conf,
|
||||
"db.client.connections.waits",
|
||||
metric.WithDescription("The number of times a connection was waited for"),
|
||||
"The number of times a connection was waited for",
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -192,25 +211,25 @@ func reportPoolStats(rdb *redis.Client, conf *config) (metric.Registration, erro
|
||||
return nil, err
|
||||
}
|
||||
|
||||
timeouts, err := conf.meter.Int64ObservableUpDownCounter(
|
||||
timeouts, err := cumulativePoolStat(conf,
|
||||
"db.client.connections.timeouts",
|
||||
metric.WithDescription("The number of connection timeouts that have occurred trying to obtain a connection from the pool"),
|
||||
"The number of connection timeouts that have occurred trying to obtain a connection from the pool",
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
hits, err := conf.meter.Int64ObservableUpDownCounter(
|
||||
hits, err := cumulativePoolStat(conf,
|
||||
"db.client.connections.hits",
|
||||
metric.WithDescription("The number of times free connection was found in the pool"),
|
||||
"The number of times free connection was found in the pool",
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
misses, err := conf.meter.Int64ObservableUpDownCounter(
|
||||
misses, err := cumulativePoolStat(conf,
|
||||
"db.client.connections.misses",
|
||||
metric.WithDescription("The number of times free connection was not found in the pool"),
|
||||
"The number of times free connection was not found in the pool",
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
101
extra/redisotel/metrics_semconv_test.go
Normal file
101
extra/redisotel/metrics_semconv_test.go
Normal file
@@ -0,0 +1,101 @@
|
||||
package redisotel
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/redis/go-redis/v9"
|
||||
sdkmetric "go.opentelemetry.io/otel/sdk/metric"
|
||||
"go.opentelemetry.io/otel/sdk/metric/metricdata"
|
||||
)
|
||||
|
||||
// cumulativeMetricNames are the connection-pool statistics that the
|
||||
// OpenTelemetry database semantic conventions define as monotonic Counters.
|
||||
var cumulativeMetricNames = map[string]struct{}{
|
||||
"db.client.connections.waits": {},
|
||||
"db.client.connections.timeouts": {},
|
||||
"db.client.connections.hits": {},
|
||||
"db.client.connections.misses": {},
|
||||
}
|
||||
|
||||
// collectPoolMetrics wires a manual reader to a fresh client instrumented with
|
||||
// the given options, then returns the collected instrument for each cumulative
|
||||
// pool-stat metric name.
|
||||
func collectPoolMetrics(t *testing.T, opts ...MetricsOption) map[string]metricdata.Aggregation {
|
||||
t.Helper()
|
||||
|
||||
reader := sdkmetric.NewManualReader()
|
||||
mp := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader))
|
||||
t.Cleanup(func() { _ = mp.Shutdown(context.Background()) })
|
||||
|
||||
rdb := redis.NewClient(&redis.Options{Addr: "localhost:6379"})
|
||||
t.Cleanup(func() { _ = rdb.Close() })
|
||||
|
||||
allOpts := append([]MetricsOption{WithMeterProvider(mp)}, opts...)
|
||||
if err := InstrumentMetrics(rdb, allOpts...); err != nil {
|
||||
t.Fatalf("InstrumentMetrics: %v", err)
|
||||
}
|
||||
|
||||
var rm metricdata.ResourceMetrics
|
||||
if err := reader.Collect(context.Background(), &rm); err != nil {
|
||||
t.Fatalf("Collect: %v", err)
|
||||
}
|
||||
|
||||
got := make(map[string]metricdata.Aggregation)
|
||||
for _, sm := range rm.ScopeMetrics {
|
||||
for _, m := range sm.Metrics {
|
||||
if _, ok := cumulativeMetricNames[m.Name]; ok {
|
||||
got[m.Name] = m.Data
|
||||
}
|
||||
}
|
||||
}
|
||||
return got
|
||||
}
|
||||
|
||||
// TestCumulativePoolStats_DefaultIsUpDownCounter verifies the backwards-compatible
|
||||
// default: the cumulative pool metrics are exported as non-monotonic sums
|
||||
// (UpDownCounter), matching pre-#3612 behaviour so existing exporters do not
|
||||
// see an instrument-type change.
|
||||
func TestCumulativePoolStats_DefaultIsUpDownCounter(t *testing.T) {
|
||||
got := collectPoolMetrics(t)
|
||||
|
||||
for name := range cumulativeMetricNames {
|
||||
data, ok := got[name]
|
||||
if !ok {
|
||||
t.Errorf("metric %q not collected", name)
|
||||
continue
|
||||
}
|
||||
sum, ok := data.(metricdata.Sum[int64])
|
||||
if !ok {
|
||||
t.Errorf("metric %q: expected Sum[int64], got %T", name, data)
|
||||
continue
|
||||
}
|
||||
if sum.IsMonotonic {
|
||||
t.Errorf("metric %q: expected non-monotonic (UpDownCounter) by default, got monotonic", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestCumulativePoolStats_OptInIsCounter verifies that enabling
|
||||
// WithSemConvCompliantMetrics switches the cumulative pool metrics to monotonic
|
||||
// sums (Counter), as required by the OpenTelemetry database semantic
|
||||
// conventions. See #3612.
|
||||
func TestCumulativePoolStats_OptInIsCounter(t *testing.T) {
|
||||
got := collectPoolMetrics(t, WithSemConvCompliantMetrics(true))
|
||||
|
||||
for name := range cumulativeMetricNames {
|
||||
data, ok := got[name]
|
||||
if !ok {
|
||||
t.Errorf("metric %q not collected", name)
|
||||
continue
|
||||
}
|
||||
sum, ok := data.(metricdata.Sum[int64])
|
||||
if !ok {
|
||||
t.Errorf("metric %q: expected Sum[int64], got %T", name, data)
|
||||
continue
|
||||
}
|
||||
if !sum.IsMonotonic {
|
||||
t.Errorf("metric %q: expected monotonic (Counter) when semconv-compliant, got non-monotonic", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -129,6 +129,9 @@ func (c cmdable) ExpireAt(ctx context.Context, key string, tm time.Time) *BoolCm
|
||||
return cmd
|
||||
}
|
||||
|
||||
// ExpireTime returns the absolute expiration time of key as a Unix timestamp
|
||||
// encoded in *DurationCmd (seconds since the epoch), not a remaining TTL.
|
||||
// Convert with: time.Unix(int64(d/time.Second), 0). Use TTL/PTTL for remaining TTL.
|
||||
func (c cmdable) ExpireTime(ctx context.Context, key string) *DurationCmd {
|
||||
cmd := NewDurationCmd(ctx, time.Second, "expiretime", key)
|
||||
_ = c(ctx, cmd)
|
||||
@@ -209,6 +212,9 @@ func (c cmdable) PExpireAt(ctx context.Context, key string, tm time.Time) *BoolC
|
||||
return cmd
|
||||
}
|
||||
|
||||
// PExpireTime returns the absolute expiration time of key as a Unix timestamp
|
||||
// encoded in *DurationCmd (milliseconds since the epoch), not a remaining TTL.
|
||||
// Convert with: time.UnixMilli(int64(d/time.Millisecond)). Use TTL/PTTL for remaining TTL.
|
||||
func (c cmdable) PExpireTime(ctx context.Context, key string) *DurationCmd {
|
||||
cmd := NewDurationCmd(ctx, time.Millisecond, "pexpiretime", key)
|
||||
_ = c(ctx, cmd)
|
||||
|
||||
@@ -25,7 +25,7 @@ var _ = Describe("HotKeys Commands", func() {
|
||||
|
||||
Describe("HOTKEYS", func() {
|
||||
It("should start, get, stop, and reset hotkeys tracking", func() {
|
||||
SkipBeforeRedisVersion(8.6, "HOTKEYS commands require Redis >= 8.6")
|
||||
SkipBeforeRedisVersion("8.6", "HOTKEYS commands require Redis >= 8.6")
|
||||
|
||||
startArgs := &redis.HotKeysStartArgs{
|
||||
Metrics: []redis.HotKeysMetric{redis.HotKeysMetricCPU, redis.HotKeysMetricNET},
|
||||
@@ -89,7 +89,7 @@ var _ = Describe("HotKeys Commands", func() {
|
||||
})
|
||||
|
||||
It("should start hotkeys tracking with CPU metric only", func() {
|
||||
SkipBeforeRedisVersion(8.6, "HOTKEYS commands require Redis >= 8.6")
|
||||
SkipBeforeRedisVersion("8.6", "HOTKEYS commands require Redis >= 8.6")
|
||||
|
||||
startArgs := &redis.HotKeysStartArgs{
|
||||
Metrics: []redis.HotKeysMetric{redis.HotKeysMetricCPU},
|
||||
@@ -104,7 +104,7 @@ var _ = Describe("HotKeys Commands", func() {
|
||||
})
|
||||
|
||||
It("should start hotkeys tracking with NET metric only", func() {
|
||||
SkipBeforeRedisVersion(8.6, "HOTKEYS commands require Redis >= 8.6")
|
||||
SkipBeforeRedisVersion("8.6", "HOTKEYS commands require Redis >= 8.6")
|
||||
|
||||
startArgs := &redis.HotKeysStartArgs{
|
||||
Metrics: []redis.HotKeysMetric{redis.HotKeysMetricNET},
|
||||
@@ -119,7 +119,7 @@ var _ = Describe("HotKeys Commands", func() {
|
||||
})
|
||||
|
||||
It("should start hotkeys tracking with duration", func() {
|
||||
SkipBeforeRedisVersion(8.6, "HOTKEYS commands require Redis >= 8.6")
|
||||
SkipBeforeRedisVersion("8.6", "HOTKEYS commands require Redis >= 8.6")
|
||||
|
||||
startArgs := &redis.HotKeysStartArgs{
|
||||
Metrics: []redis.HotKeysMetric{redis.HotKeysMetricCPU, redis.HotKeysMetricNET},
|
||||
@@ -140,7 +140,7 @@ var _ = Describe("HotKeys Commands", func() {
|
||||
})
|
||||
|
||||
It("should start hotkeys tracking with sampling", func() {
|
||||
SkipBeforeRedisVersion(8.6, "HOTKEYS commands require Redis >= 8.6")
|
||||
SkipBeforeRedisVersion("8.6", "HOTKEYS commands require Redis >= 8.6")
|
||||
|
||||
startArgs := &redis.HotKeysStartArgs{
|
||||
Metrics: []redis.HotKeysMetric{redis.HotKeysMetricCPU, redis.HotKeysMetricNET},
|
||||
@@ -156,7 +156,7 @@ var _ = Describe("HotKeys Commands", func() {
|
||||
})
|
||||
|
||||
It("should error when using slots in non-cluster mode", func() {
|
||||
SkipBeforeRedisVersion(8.6, "HOTKEYS commands require Redis >= 8.6")
|
||||
SkipBeforeRedisVersion("8.6", "HOTKEYS commands require Redis >= 8.6")
|
||||
|
||||
startArgs := &redis.HotKeysStartArgs{
|
||||
Metrics: []redis.HotKeysMetric{redis.HotKeysMetricCPU, redis.HotKeysMetricNET},
|
||||
@@ -178,7 +178,7 @@ var _ = Describe("HotKeys Commands", func() {
|
||||
})
|
||||
|
||||
It("should error when starting tracking while already active", func() {
|
||||
SkipBeforeRedisVersion(8.6, "HOTKEYS commands require Redis >= 8.6")
|
||||
SkipBeforeRedisVersion("8.6", "HOTKEYS commands require Redis >= 8.6")
|
||||
|
||||
startArgs := &redis.HotKeysStartArgs{
|
||||
Metrics: []redis.HotKeysMetric{redis.HotKeysMetricCPU},
|
||||
@@ -195,7 +195,7 @@ var _ = Describe("HotKeys Commands", func() {
|
||||
})
|
||||
|
||||
It("should error when resetting while tracking is active", func() {
|
||||
SkipBeforeRedisVersion(8.6, "HOTKEYS commands require Redis >= 8.6")
|
||||
SkipBeforeRedisVersion("8.6", "HOTKEYS commands require Redis >= 8.6")
|
||||
|
||||
startArgs := &redis.HotKeysStartArgs{
|
||||
Metrics: []redis.HotKeysMetric{redis.HotKeysMetricCPU},
|
||||
|
||||
@@ -265,3 +265,93 @@ func TestCloseConnInvokesOnRemove(t *testing.T) {
|
||||
t.Errorf("Expected OnRemove to fire once for CloseConn, got %d", hook.OnRemoveCalled)
|
||||
}
|
||||
}
|
||||
|
||||
// closeOnPutHook is a PoolHook whose OnPut drives the connection to StateClosed while still
|
||||
// asking for it to be pooled, so that Put reaches the StateClosed-on-Put branch of putConn.
|
||||
type closeOnPutHook struct {
|
||||
onRemoveCalled int
|
||||
}
|
||||
|
||||
func (h *closeOnPutHook) OnGet(ctx context.Context, conn *Conn, isNewConn bool) (bool, error) {
|
||||
return true, nil
|
||||
}
|
||||
|
||||
func (h *closeOnPutHook) OnPut(ctx context.Context, conn *Conn) (bool, bool, error) {
|
||||
conn.GetStateMachine().Transition(StateClosed)
|
||||
return true, false, nil
|
||||
}
|
||||
|
||||
func (h *closeOnPutHook) OnRemove(ctx context.Context, conn *Conn, reason error) {
|
||||
h.onRemoveCalled++
|
||||
}
|
||||
|
||||
func TestPutConnEvictsOnIdleFullInvokesOnRemove(t *testing.T) {
|
||||
opt := &Options{
|
||||
Dialer: func(ctx context.Context) (net.Conn, error) {
|
||||
return &net.TCPConn{}, nil
|
||||
},
|
||||
PoolSize: 2,
|
||||
MaxIdleConns: 1,
|
||||
MaxConcurrentDials: 2,
|
||||
DialTimeout: time.Second,
|
||||
}
|
||||
|
||||
p := NewConnPool(opt)
|
||||
defer p.Close()
|
||||
|
||||
hook := &TestHook{ShouldPool: true, ShouldAccept: true}
|
||||
p.AddPoolHook(hook)
|
||||
|
||||
ctx := context.Background()
|
||||
cn1, err := p.newConn(ctx, true)
|
||||
if err != nil {
|
||||
t.Fatalf("newConn cn1 failed: %v", err)
|
||||
}
|
||||
cn1.GetStateMachine().Transition(StateInUse)
|
||||
cn2, err := p.newConn(ctx, true)
|
||||
if err != nil {
|
||||
t.Fatalf("newConn cn2 failed: %v", err)
|
||||
}
|
||||
cn2.GetStateMachine().Transition(StateInUse)
|
||||
|
||||
p.putConnWithoutTurn(ctx, cn1) // pools: idle=1
|
||||
p.putConnWithoutTurn(ctx, cn2) // idle pool full -> evicted on Put
|
||||
|
||||
if hook.OnRemoveCalled != 1 {
|
||||
t.Errorf("Expected OnRemove to fire once for idle-full eviction, got %d", hook.OnRemoveCalled)
|
||||
}
|
||||
if got := p.Stats().StaleConns; got != 1 {
|
||||
t.Errorf("Expected StaleConns=1, got %d", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPutConnStateClosedByHookInvokesOnRemove(t *testing.T) {
|
||||
opt := &Options{
|
||||
Dialer: func(ctx context.Context) (net.Conn, error) {
|
||||
return &net.TCPConn{}, nil
|
||||
},
|
||||
PoolSize: 1,
|
||||
MaxIdleConns: 1,
|
||||
MaxConcurrentDials: 1,
|
||||
DialTimeout: time.Second,
|
||||
}
|
||||
|
||||
p := NewConnPool(opt)
|
||||
defer p.Close()
|
||||
|
||||
hook := &closeOnPutHook{}
|
||||
p.AddPoolHook(hook)
|
||||
|
||||
ctx := context.Background()
|
||||
cn, err := p.newConn(ctx, true)
|
||||
if err != nil {
|
||||
t.Fatalf("newConn failed: %v", err)
|
||||
}
|
||||
cn.GetStateMachine().Transition(StateInUse)
|
||||
|
||||
p.putConnWithoutTurn(ctx, cn) // OnPut sets StateClosed -> StateClosed-on-Put branch
|
||||
|
||||
if hook.onRemoveCalled != 1 {
|
||||
t.Errorf("Expected OnRemove to fire once for StateClosed-on-Put, got %d", hook.onRemoveCalled)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -72,6 +72,11 @@ var (
|
||||
|
||||
// errConnNotPooled is returned when trying to return a non-pooled connection to the pool.
|
||||
errConnNotPooled = errors.New("connection not pooled")
|
||||
|
||||
// errConnEvictedIdle is passed to OnRemove hooks when a pooled connection is evicted on
|
||||
// Put because the idle pool is already at MaxIdleConns.
|
||||
errConnEvictedIdle = errors.New("connection evicted: idle pool at capacity")
|
||||
|
||||
// metricCallbackMu protects all global metric callback functions for thread-safe access.
|
||||
metricCallbackMu sync.RWMutex
|
||||
|
||||
@@ -1293,6 +1298,9 @@ func (p *ConnPool) putConn(ctx context.Context, cn *Conn, freeTurn bool) {
|
||||
// expected state, don't log it
|
||||
case StateClosed:
|
||||
internal.Logger.Printf(ctx, "Unexpected conn[%d] state changed by hook to %v, closing it", cn.GetID(), currentState)
|
||||
if hookManager != nil {
|
||||
hookManager.ProcessOnRemove(ctx, cn, errHookRequestedRemoval)
|
||||
}
|
||||
shouldCloseConn = true
|
||||
removedFromPool = p.removeConnWithLock(cn)
|
||||
default:
|
||||
@@ -1360,6 +1368,9 @@ func (p *ConnPool) putConn(ctx context.Context, cn *Conn, freeTurn bool) {
|
||||
}
|
||||
} else {
|
||||
shouldCloseConn = true
|
||||
if hookManager != nil {
|
||||
hookManager.ProcessOnRemove(ctx, cn, errConnEvictedIdle)
|
||||
}
|
||||
removedFromPool = p.removeConnWithLock(cn)
|
||||
|
||||
// Only emit if we actually removed it from the map (not already taken by Close()).
|
||||
|
||||
@@ -85,7 +85,7 @@ var _ = Describe("ScanIterator", func() {
|
||||
})
|
||||
|
||||
It("should hscan across multiple pages", func() {
|
||||
SkipBeforeRedisVersion(7.4, "doesn't work with older redis stack images")
|
||||
SkipBeforeRedisVersion("7.4", "doesn't work with older redis stack images")
|
||||
Expect(hashSeed(71)).NotTo(HaveOccurred())
|
||||
|
||||
var vals []string
|
||||
@@ -101,7 +101,7 @@ var _ = Describe("ScanIterator", func() {
|
||||
})
|
||||
|
||||
It("should hscan without values across multiple pages", Label("NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "doesn't work with older redis stack images")
|
||||
SkipBeforeRedisVersion("7.4", "doesn't work with older redis stack images")
|
||||
Expect(hashSeed(71)).NotTo(HaveOccurred())
|
||||
|
||||
var vals []string
|
||||
|
||||
@@ -264,7 +264,7 @@ var _ = Describe("JSON Commands", Label("json"), func() {
|
||||
})
|
||||
|
||||
It("should JSONSetWithArgs with FPHA", Label("json.set", "json"), func() {
|
||||
SkipBeforeRedisVersion(8.8, "FPHA argument requires Redis 8.8+")
|
||||
SkipBeforeRedisVersion("8.8", "FPHA argument requires Redis 8.8+")
|
||||
|
||||
fpArray := `[1.1, 2.2, 3.3, 4.4]`
|
||||
for _, fpha := range []redis.FPHAType{
|
||||
@@ -285,7 +285,7 @@ var _ = Describe("JSON Commands", Label("json"), func() {
|
||||
})
|
||||
|
||||
It("should JSONSetWithArgs with FPHA and NX/XX", Label("json.set", "json"), func() {
|
||||
SkipBeforeRedisVersion(8.8, "FPHA argument requires Redis 8.8+")
|
||||
SkipBeforeRedisVersion("8.8", "FPHA argument requires Redis 8.8+")
|
||||
|
||||
key := "fpha_nx"
|
||||
fpArray := `[1.5, 2.5, 3.5]`
|
||||
|
||||
@@ -34,6 +34,8 @@ type ListCmdable interface {
|
||||
RPushX(ctx context.Context, key string, values ...interface{}) *IntCmd
|
||||
LMove(ctx context.Context, source, destination, srcpos, destpos string) *StringCmd
|
||||
BLMove(ctx context.Context, source, destination, srcpos, destpos string, timeout time.Duration) *StringCmd
|
||||
LMoveM(ctx context.Context, source, destination, srcpos, destpos string, args LMoveMArgs) *StringSliceCmd
|
||||
BLMoveM(ctx context.Context, source, destination, srcpos, destpos string, timeout time.Duration, args LMoveMArgs) *StringSliceCmd
|
||||
}
|
||||
|
||||
func (c cmdable) BLPop(ctx context.Context, timeout time.Duration, keys ...string) *StringSliceCmd {
|
||||
@@ -156,6 +158,45 @@ type LPosArgs struct {
|
||||
Rank, MaxLen int64
|
||||
}
|
||||
|
||||
// LMoveMMode is the count semantics for LMOVEM/BLMOVEM.
|
||||
type LMoveMMode string
|
||||
|
||||
const (
|
||||
LMoveMCount LMoveMMode = "COUNT" // up to Count
|
||||
LMoveMExactly LMoveMMode = "EXACTLY" // exactly Count, or nothing
|
||||
)
|
||||
|
||||
// LMoveMOrder is the destination ordering for LMOVEM/BLMOVEM.
|
||||
type LMoveMOrder string
|
||||
|
||||
const (
|
||||
LMoveMOBO LMoveMOrder = "OBO" // one-by-one, order reversed
|
||||
LMoveMBulk LMoveMOrder = "BULK" // preserve order
|
||||
)
|
||||
|
||||
// LMoveMArgs configures the optional count group of LMOVEM/BLMOVEM.
|
||||
// Count <= 0 moves a single element. Mode defaults to COUNT, Order to BULK.
|
||||
type LMoveMArgs struct {
|
||||
Mode LMoveMMode
|
||||
Count int64
|
||||
Order LMoveMOrder
|
||||
}
|
||||
|
||||
func (a LMoveMArgs) appendArgs(args []interface{}) []interface{} {
|
||||
if a.Count <= 0 {
|
||||
return args
|
||||
}
|
||||
mode := a.Mode
|
||||
if mode == "" {
|
||||
mode = LMoveMCount
|
||||
}
|
||||
order := a.Order
|
||||
if order == "" {
|
||||
order = LMoveMBulk
|
||||
}
|
||||
return append(args, string(mode), a.Count, string(order))
|
||||
}
|
||||
|
||||
func (c cmdable) LPos(ctx context.Context, key string, value string, a LPosArgs) *IntCmd {
|
||||
args := []interface{}{"lpos", key, value}
|
||||
if a.Rank != 0 {
|
||||
@@ -295,3 +336,27 @@ func (c cmdable) BLMove(
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
// LMoveM atomically moves multiple elements between lists (Redis 8.10+).
|
||||
// srcpos/destpos are "LEFT" or "RIGHT". Returns moved elements, or redis.Nil if none.
|
||||
func (c cmdable) LMoveM(ctx context.Context, source, destination, srcpos, destpos string, a LMoveMArgs) *StringSliceCmd {
|
||||
args := make([]interface{}, 5, 8)
|
||||
args[0], args[1], args[2], args[3], args[4] = "lmovem", source, destination, srcpos, destpos
|
||||
args = a.appendArgs(args)
|
||||
cmd := NewStringSliceCmd(ctx, args...)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
// BLMoveM is the blocking variant of LMoveM (Redis 8.10+); timeout 0 blocks forever.
|
||||
// Returns moved elements, or redis.Nil on timeout.
|
||||
func (c cmdable) BLMoveM(ctx context.Context, source, destination, srcpos, destpos string, timeout time.Duration, a LMoveMArgs) *StringSliceCmd {
|
||||
args := make([]interface{}, 6, 9)
|
||||
args[0], args[1], args[2], args[3], args[4] = "blmovem", source, destination, srcpos, destpos
|
||||
args[5] = formatSec(ctx, timeout)
|
||||
args = a.appendArgs(args)
|
||||
cmd := NewStringSliceCmd(ctx, args...)
|
||||
cmd.setReadTimeout(timeout)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
47
main_test.go
47
main_test.go
@@ -80,16 +80,37 @@ var RCEDocker = false
|
||||
// This can be used before we change the bsm fork of ginkgo for one,
|
||||
// which have support for label sets, so we can filter tests per redis version.
|
||||
var RedisVersion float64 = 8.4
|
||||
var redisMajorVersion = 8
|
||||
var redisMinorVersion = 4
|
||||
|
||||
func SkipBeforeRedisVersion(version float64, msg string) {
|
||||
if RedisVersion < version {
|
||||
Skip(fmt.Sprintf("(redis version < %f) %s", version, msg))
|
||||
// parseRedisVersionStr splits "major.minor" into two ints. Minor defaults to 0.
|
||||
func parseRedisVersionStr(v string) (int, int) {
|
||||
parts := strings.SplitN(v, ".", 2)
|
||||
major, _ := strconv.Atoi(parts[0])
|
||||
if len(parts) < 2 {
|
||||
return major, 0
|
||||
}
|
||||
minor, _ := strconv.Atoi(parts[1])
|
||||
return major, minor
|
||||
}
|
||||
|
||||
// redisVersionAtLeast returns true when the running Redis is >= major.minor.
|
||||
func redisVersionAtLeast(version string) bool {
|
||||
major, minor := parseRedisVersionStr(version)
|
||||
return redisMajorVersion > major || (redisMajorVersion == major && redisMinorVersion >= minor)
|
||||
}
|
||||
|
||||
func SkipBeforeRedisVersion(version string, msg string) {
|
||||
major, minor := parseRedisVersionStr(version)
|
||||
if redisMajorVersion < major || (redisMajorVersion == major && redisMinorVersion < minor) {
|
||||
Skip(fmt.Sprintf("(redis version < %s) %s", version, msg))
|
||||
}
|
||||
}
|
||||
|
||||
func SkipAfterRedisVersion(version float64, msg string) {
|
||||
if RedisVersion > version {
|
||||
Skip(fmt.Sprintf("(redis version > %f) %s", version, msg))
|
||||
func SkipAfterRedisVersion(version string, msg string) {
|
||||
major, minor := parseRedisVersionStr(version)
|
||||
if redisMajorVersion > major || (redisMajorVersion == major && redisMinorVersion > minor) {
|
||||
Skip(fmt.Sprintf("(redis version > %s) %s", version, msg))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -103,19 +124,25 @@ var _ = BeforeSuite(func() {
|
||||
RECluster, _ = strconv.ParseBool(os.Getenv("RE_CLUSTER"))
|
||||
RCEDocker, _ = strconv.ParseBool(os.Getenv("RCE_DOCKER"))
|
||||
|
||||
RedisVersion, _ = strconv.ParseFloat(strings.Trim(os.Getenv("REDIS_VERSION"), "\""), 64)
|
||||
vStr := strings.Trim(os.Getenv("REDIS_VERSION"), "\"")
|
||||
if vStr != "" {
|
||||
redisMajorVersion, redisMinorVersion = parseRedisVersionStr(vStr)
|
||||
RedisVersion, _ = strconv.ParseFloat(vStr, 64)
|
||||
}
|
||||
|
||||
if RedisVersion == 0 {
|
||||
if redisMajorVersion == 0 {
|
||||
redisMajorVersion = 8
|
||||
redisMinorVersion = 4
|
||||
RedisVersion = 8.4
|
||||
}
|
||||
|
||||
fmt.Printf("RECluster: %v\n", RECluster)
|
||||
fmt.Printf("RCEDocker: %v\n", RCEDocker)
|
||||
fmt.Printf("REDIS_VERSION: %.1f\n", RedisVersion)
|
||||
fmt.Printf("REDIS_VERSION: %d.%d\n", redisMajorVersion, redisMinorVersion)
|
||||
fmt.Printf("CLIENT_LIBS_TEST_IMAGE: %v\n", os.Getenv("CLIENT_LIBS_TEST_IMAGE"))
|
||||
logging.Disable()
|
||||
|
||||
if RedisVersion < 7.0 || RedisVersion > 9 {
|
||||
if redisMajorVersion < 7 || redisMajorVersion > 9 {
|
||||
panic("incorrect or not supported redis version")
|
||||
}
|
||||
|
||||
|
||||
17
moduleloadex_nil_test.go
Normal file
17
moduleloadex_nil_test.go
Normal file
@@ -0,0 +1,17 @@
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestModuleLoadexNilConfig(t *testing.T) {
|
||||
rdb := NewClient(&Options{Addr: "localhost:6379"})
|
||||
cmd := rdb.ModuleLoadex(context.Background(), nil)
|
||||
if cmd.Err() == nil {
|
||||
t.Fatal("expected error for nil config")
|
||||
}
|
||||
if cmd.Err().Error() != "redis: ModuleLoadex nil config" {
|
||||
t.Fatalf("got %v", cmd.Err())
|
||||
}
|
||||
}
|
||||
29
options.go
29
options.go
@@ -121,12 +121,12 @@ type Options struct {
|
||||
// MinRetryBackoff is the minimum backoff between each retry.
|
||||
// -1 disables backoff.
|
||||
//
|
||||
// default: 8 milliseconds
|
||||
// default: 10 milliseconds
|
||||
MinRetryBackoff time.Duration
|
||||
|
||||
// MaxRetryBackoff is the maximum backoff between each retry.
|
||||
// -1 disables backoff.
|
||||
// default: 512 milliseconds;
|
||||
// default: 1 second;
|
||||
MaxRetryBackoff time.Duration
|
||||
|
||||
// DialTimeout for establishing new connections.
|
||||
@@ -157,7 +157,7 @@ type Options struct {
|
||||
// - `-1` - no timeout (block indefinitely).
|
||||
// - `-2` - disables SetReadDeadline calls completely.
|
||||
//
|
||||
// default: 3 seconds
|
||||
// default: 5 seconds
|
||||
ReadTimeout time.Duration
|
||||
|
||||
// WriteTimeout for socket writes. If reached, commands will fail
|
||||
@@ -166,7 +166,7 @@ type Options struct {
|
||||
// - `-1` - no timeout (block indefinitely).
|
||||
// - `-2` - disables SetWriteDeadline calls completely.
|
||||
//
|
||||
// default: 3 seconds
|
||||
// default: 5 seconds (same as ReadTimeout, which it follows when unset)
|
||||
WriteTimeout time.Duration
|
||||
|
||||
// ContextTimeoutEnabled controls whether the client respects context timeouts and deadlines.
|
||||
@@ -367,7 +367,7 @@ func (opt *Options) init() {
|
||||
case -1:
|
||||
opt.ReadTimeout = 0
|
||||
case 0:
|
||||
opt.ReadTimeout = 3 * time.Second
|
||||
opt.ReadTimeout = 5 * time.Second
|
||||
}
|
||||
switch opt.WriteTimeout {
|
||||
case -2:
|
||||
@@ -400,13 +400,13 @@ func (opt *Options) init() {
|
||||
case -1:
|
||||
opt.MinRetryBackoff = 0
|
||||
case 0:
|
||||
opt.MinRetryBackoff = 8 * time.Millisecond
|
||||
opt.MinRetryBackoff = 10 * time.Millisecond
|
||||
}
|
||||
switch opt.MaxRetryBackoff {
|
||||
case -1:
|
||||
opt.MaxRetryBackoff = 0
|
||||
case 0:
|
||||
opt.MaxRetryBackoff = 512 * time.Millisecond
|
||||
opt.MaxRetryBackoff = time.Second
|
||||
}
|
||||
|
||||
if opt.FailingTimeoutSeconds == 0 {
|
||||
@@ -442,13 +442,24 @@ func (opt *Options) NewDialer() func(context.Context, string, string) (net.Conn,
|
||||
return NewDialer(opt)
|
||||
}
|
||||
|
||||
// defaultKeepAliveConfig is the TCP keep-alive policy of the default dialers
|
||||
// here and in sentinel.go: start probing after 30s idle (below typical LB/NAT
|
||||
// idle timeouts), then declare the peer dead after 3 unanswered probes 5s
|
||||
// apart.
|
||||
var defaultKeepAliveConfig = net.KeepAliveConfig{
|
||||
Enable: true,
|
||||
Idle: 30 * time.Second,
|
||||
Interval: 5 * time.Second,
|
||||
Count: 3,
|
||||
}
|
||||
|
||||
// NewDialer returns a function that will be used as the default dialer
|
||||
// when none is specified in Options.Dialer.
|
||||
func NewDialer(opt *Options) func(context.Context, string, string) (net.Conn, error) {
|
||||
return func(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||
netDialer := &net.Dialer{
|
||||
Timeout: opt.DialTimeout,
|
||||
KeepAlive: 5 * time.Minute,
|
||||
Timeout: opt.DialTimeout,
|
||||
KeepAliveConfig: defaultKeepAliveConfig,
|
||||
}
|
||||
if opt.TLSConfig == nil {
|
||||
return netDialer.DialContext(ctx, network, addr)
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
//go:build go1.7
|
||||
|
||||
package redis
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
"errors"
|
||||
"net"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -270,7 +269,7 @@ func comprareOptions(t *testing.T, actual, expected *Options) {
|
||||
func TestReadTimeoutOptions(t *testing.T) {
|
||||
testDataInputOutputMap := map[time.Duration]time.Duration{
|
||||
-1: 0 * time.Second,
|
||||
0: 3 * time.Second,
|
||||
0: 5 * time.Second,
|
||||
1: 1 * time.Nanosecond,
|
||||
3: 3 * time.Nanosecond,
|
||||
}
|
||||
@@ -288,6 +287,70 @@ func TestReadTimeoutOptions(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// Pin the retry backoff defaults shared by Options, ClusterOptions and
|
||||
// RingOptions, including the -1 escape hatch.
|
||||
func TestRetryBackoffOptions(t *testing.T) {
|
||||
check := func(t *testing.T, kind string, min, max time.Duration) {
|
||||
t.Helper()
|
||||
if min != 10*time.Millisecond {
|
||||
t.Errorf("%s: got %s as default MinRetryBackoff, want 10ms", kind, min)
|
||||
}
|
||||
if max != time.Second {
|
||||
t.Errorf("%s: got %s as default MaxRetryBackoff, want 1s", kind, max)
|
||||
}
|
||||
}
|
||||
|
||||
o := &Options{}
|
||||
o.init()
|
||||
check(t, "Options", o.MinRetryBackoff, o.MaxRetryBackoff)
|
||||
|
||||
co := &ClusterOptions{}
|
||||
co.init()
|
||||
check(t, "ClusterOptions", co.MinRetryBackoff, co.MaxRetryBackoff)
|
||||
|
||||
ro := &RingOptions{}
|
||||
ro.init()
|
||||
check(t, "RingOptions", ro.MinRetryBackoff, ro.MaxRetryBackoff)
|
||||
|
||||
disabled := &Options{MinRetryBackoff: -1, MaxRetryBackoff: -1}
|
||||
disabled.init()
|
||||
if disabled.MinRetryBackoff != 0 || disabled.MaxRetryBackoff != 0 {
|
||||
t.Errorf("-1 must disable backoff, got min=%s max=%s",
|
||||
disabled.MinRetryBackoff, disabled.MaxRetryBackoff)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClusterStateReloadIntervalOption(t *testing.T) {
|
||||
o := &ClusterOptions{}
|
||||
o.init()
|
||||
if o.ClusterStateReloadInterval != 60*time.Second {
|
||||
t.Errorf("got %s as default ClusterStateReloadInterval, want 60s",
|
||||
o.ClusterStateReloadInterval)
|
||||
}
|
||||
|
||||
o = &ClusterOptions{ClusterStateReloadInterval: 5 * time.Minute}
|
||||
o.init()
|
||||
if o.ClusterStateReloadInterval != 5*time.Minute {
|
||||
t.Errorf("explicit ClusterStateReloadInterval overridden: got %s",
|
||||
o.ClusterStateReloadInterval)
|
||||
}
|
||||
}
|
||||
|
||||
// The keep-alive policy is shared by the default dialer (options.go) and the
|
||||
// sentinel master/replica dialer (sentinel.go); pin it so a change shows up
|
||||
// in review instead of drifting silently.
|
||||
func TestDefaultKeepAliveConfig(t *testing.T) {
|
||||
want := net.KeepAliveConfig{
|
||||
Enable: true,
|
||||
Idle: 30 * time.Second,
|
||||
Interval: 5 * time.Second,
|
||||
Count: 3,
|
||||
}
|
||||
if defaultKeepAliveConfig != want {
|
||||
t.Errorf("defaultKeepAliveConfig = %+v, want %+v", defaultKeepAliveConfig, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProtocolOptions(t *testing.T) {
|
||||
testCasesMap := map[int]int{
|
||||
0: 3,
|
||||
|
||||
473
osscluster.go
473
osscluster.go
@@ -190,7 +190,9 @@ type ClusterOptions struct {
|
||||
ShardPicker routing.ShardPicker
|
||||
|
||||
// ClusterStateReloadInterval is the interval for reloading the cluster state.
|
||||
// Default is 10 seconds.
|
||||
// MOVED/ASK redirects still trigger an immediate reactive reload, so this
|
||||
// only bounds how stale a topology can get without traffic errors.
|
||||
// Default is 60 seconds.
|
||||
ClusterStateReloadInterval time.Duration
|
||||
}
|
||||
|
||||
@@ -235,7 +237,7 @@ func (opt *ClusterOptions) init() {
|
||||
case -1:
|
||||
opt.ReadTimeout = 0
|
||||
case 0:
|
||||
opt.ReadTimeout = 3 * time.Second
|
||||
opt.ReadTimeout = 5 * time.Second
|
||||
}
|
||||
switch opt.WriteTimeout {
|
||||
case -1:
|
||||
@@ -251,13 +253,13 @@ func (opt *ClusterOptions) init() {
|
||||
case -1:
|
||||
opt.MinRetryBackoff = 0
|
||||
case 0:
|
||||
opt.MinRetryBackoff = 8 * time.Millisecond
|
||||
opt.MinRetryBackoff = 10 * time.Millisecond
|
||||
}
|
||||
switch opt.MaxRetryBackoff {
|
||||
case -1:
|
||||
opt.MaxRetryBackoff = 0
|
||||
case 0:
|
||||
opt.MaxRetryBackoff = 512 * time.Millisecond
|
||||
opt.MaxRetryBackoff = time.Second
|
||||
}
|
||||
|
||||
if opt.NewClient == nil {
|
||||
@@ -273,7 +275,7 @@ func (opt *ClusterOptions) init() {
|
||||
}
|
||||
|
||||
if opt.ClusterStateReloadInterval == 0 {
|
||||
opt.ClusterStateReloadInterval = 10 * time.Second
|
||||
opt.ClusterStateReloadInterval = 60 * time.Second
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1837,18 +1839,108 @@ func (c *ClusterClient) TxPipelined(ctx context.Context, fn func(Pipeliner) erro
|
||||
return c.TxPipeline().Pipelined(ctx, fn)
|
||||
}
|
||||
|
||||
func (c *ClusterClient) processTxPipeline(ctx context.Context, cmds []Cmder) error {
|
||||
// Only call time.Now() if pipeline operation duration callback is set to avoid overhead
|
||||
// A cluster tx pipeline sends MULTI, c1..cN, EXEC — N+2 commands, or N+3 with a
|
||||
// leading ASKING — and always receives exactly that many replies, so every
|
||||
// redirect/abort path leaves the connection clean.
|
||||
//
|
||||
// Possible reply sequences:
|
||||
// 1. Slot owned here, no migration:
|
||||
// +OK, +QUEUED x N, *N (array of N results) -> success
|
||||
// 2. Slot already migrated away:
|
||||
// +OK, -MOVED x N, -EXECABORT -> re-route whole tx
|
||||
// 3. Slot in migrating state (still owned here, keys draining out). Per
|
||||
// cmd, the queue reply is +QUEUED / -ASK / -TRYAGAIN (keys present /
|
||||
// all gone / some gone); any -ASK or -TRYAGAIN dirties the tx, so
|
||||
// EXEC is -EXECABORT. Still N+2 replies, like the cases above:
|
||||
// +OK, (+QUEUED|-ASK|-TRYAGAIN) x N, -EXECABORT -> follow first redirect
|
||||
// 4. Narrow race (all +QUEUED, slot moves before EXEC):
|
||||
// +OK, +QUEUED x N, -MOVED <slot> <addr> -> re-route whole tx
|
||||
// 5. Non-cluster command error (arity / ACL / unknown):
|
||||
// +OK, +QUEUED..., -ERR..., -EXECABORT -> surface, not retryable
|
||||
// 6. Narrow race (all +QUEUED, slot still migrating, keys drain before EXEC):
|
||||
// +OK, +QUEUED x N, -ASK / -TRYAGAIN -> re-route on -ASK, back off on -TRYAGAIN
|
||||
//
|
||||
// EXEC reply — the reply that decides the outcome:
|
||||
//
|
||||
// *N success; read N per-command results
|
||||
// -EXECABORT a queue-stage command failed; follow the first queue
|
||||
// redirect (MOVED/ASK/TRYAGAIN), else surface the trigger
|
||||
// -MOVED <slot> <addr> case 4; re-route whole tx to addr, reload topology
|
||||
// -ASK <slot> <addr> race: slot entered migrating state; re-route to addr
|
||||
// with a top-level ASKING before MULTI
|
||||
// -TRYAGAIN race: migrating with split keys, or slot being trimmed
|
||||
// (CLUSTER_REDIR_TRIMMING on a write); back off and retry
|
||||
// the whole tx (same node still owns it)
|
||||
// -CLUSTERDOWN cluster degraded; back off and retry whole tx
|
||||
//
|
||||
// ASK retry: the ASKING flag is NOT cleared between commands inside a MULTI
|
||||
// so one top-level ASKING before MULTI covers the whole tx and lets the importing
|
||||
// slot serve at EXEC. ASKING placed inside the MULTI would be queued and leave
|
||||
// the flag unset during queueing, so the keyed commands would still get MOVED.
|
||||
//
|
||||
// Out of scope: WATCH's null-array EXEC and -CROSSSLOT;
|
||||
// cluster TxPipeline is not used with WATCH and cross-slot is rejected client-side.
|
||||
|
||||
type txOutcomeKind int
|
||||
|
||||
const (
|
||||
txSuccess txOutcomeKind = iota // transaction executed; per-command results are set
|
||||
txRetryMoved // MOVED: reload topology and re-route the whole tx
|
||||
txRetryAsk // ASK: re-route to the target with a top-level ASKING
|
||||
txRetryTryAgain // TRYAGAIN: back off and re-route the whole tx
|
||||
txRetryConn // connection/write/read failure: re-route the whole tx
|
||||
txFatal // non-retryable error; surface to the caller
|
||||
)
|
||||
|
||||
// txOutcome is the result of a single tx attempt. err is the error to report
|
||||
// when the redirect/retry loop is exhausted (or the fatal error to surface);
|
||||
// addr is the ASK target; execErr is the EXEC reply error used to mark
|
||||
// aborted commands; unreadReplies forces the connection to be discarded
|
||||
// when the read loop exited before consuming all N+2 replies, leaving bytes
|
||||
// on the wire.
|
||||
type txOutcome struct {
|
||||
kind txOutcomeKind
|
||||
err error
|
||||
addr string
|
||||
execErr error
|
||||
unreadReplies bool
|
||||
}
|
||||
|
||||
// txRedirect records the first queue-stage redirect (MOVED/ASK/TRYAGAIN) seen
|
||||
// while reading +QUEUED replies. Redis dirties and aborts the transaction on
|
||||
// any such reply, so the EXEC reply will be EXECABORT and the client must
|
||||
// follow the recorded redirect with the whole transaction.
|
||||
type txRedirect struct {
|
||||
moved bool
|
||||
ask bool
|
||||
tryAgain bool
|
||||
addr string
|
||||
err error
|
||||
}
|
||||
|
||||
// errTxDirtyConn forces releaseConn to discard a connection that may still have
|
||||
// unread transaction replies on it (an early exit before consuming all N+2).
|
||||
var errTxDirtyConn = errors.New("redis: connection has unread transaction replies")
|
||||
|
||||
func (c *ClusterClient) processTxPipeline(ctx context.Context, cmds []Cmder) (retErr error) {
|
||||
var operationStart time.Time
|
||||
pipelineOpDurationCallback := otel.GetPipelineOperationDurationCallback()
|
||||
if pipelineOpDurationCallback != nil {
|
||||
operationStart = time.Now()
|
||||
}
|
||||
totalAttempts := 0
|
||||
var lastErr error
|
||||
|
||||
defer func() {
|
||||
if pipelineOpDurationCallback == nil {
|
||||
return
|
||||
}
|
||||
finalErr := cmp.Or(retErr, cmdsFirstErr(cmds), lastErr)
|
||||
pipelineOpDurationCallback(ctx, time.Since(operationStart), "MULTI", len(cmds), totalAttempts, finalErr, nil, 0)
|
||||
}()
|
||||
|
||||
// Trim multi .. exec.
|
||||
cmds = cmds[1 : len(cmds)-1]
|
||||
|
||||
if len(cmds) == 0 {
|
||||
return nil
|
||||
}
|
||||
@@ -1856,10 +1948,6 @@ func (c *ClusterClient) processTxPipeline(ctx context.Context, cmds []Cmder) err
|
||||
state, err := c.state.Get(ctx)
|
||||
if err != nil {
|
||||
setCmdsErr(cmds, err)
|
||||
if pipelineOpDurationCallback != nil {
|
||||
operationDuration := time.Since(operationStart)
|
||||
pipelineOpDurationCallback(ctx, operationDuration, "MULTI", len(cmds), 1, err, nil, 0)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -1871,77 +1959,85 @@ func (c *ClusterClient) processTxPipeline(ctx context.Context, cmds []Cmder) err
|
||||
case 1:
|
||||
for sl := range keyedCmdsBySlot {
|
||||
slot = sl
|
||||
break
|
||||
}
|
||||
default:
|
||||
// TxPipeline does not support cross slot transaction.
|
||||
setCmdsErr(cmds, ErrCrossSlot)
|
||||
if pipelineOpDurationCallback != nil {
|
||||
operationDuration := time.Since(operationStart)
|
||||
pipelineOpDurationCallback(ctx, operationDuration, "MULTI", len(cmds), 1, ErrCrossSlot, nil, 0)
|
||||
}
|
||||
return ErrCrossSlot
|
||||
}
|
||||
|
||||
node, err := state.slotMasterNode(slot)
|
||||
if err != nil {
|
||||
setCmdsErr(cmds, err)
|
||||
if pipelineOpDurationCallback != nil {
|
||||
operationDuration := time.Since(operationStart)
|
||||
pipelineOpDurationCallback(ctx, operationDuration, "MULTI", len(cmds), 1, err, nil, 0)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
var lastErr error
|
||||
cmdsMap := map[*clusterNode][]Cmder{node: cmds}
|
||||
asking := false
|
||||
// MOVED/ASK are routing changes, not transient failures: follow them immediately.
|
||||
redirected := false
|
||||
for attempt := 0; attempt <= c.opt.MaxRedirects; attempt++ {
|
||||
totalAttempts++
|
||||
if attempt > 0 {
|
||||
if attempt > 0 && !redirected {
|
||||
if err := internal.Sleep(ctx, c.retryBackoff(attempt)); err != nil {
|
||||
setCmdsErr(cmds, err)
|
||||
if pipelineOpDurationCallback != nil {
|
||||
operationDuration := time.Since(operationStart)
|
||||
pipelineOpDurationCallback(ctx, operationDuration, "MULTI", len(cmds), totalAttempts, err, nil, 0)
|
||||
}
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
failedCmds := newCmdsMap()
|
||||
var wg sync.WaitGroup
|
||||
|
||||
for node, cmds := range cmdsMap {
|
||||
wg.Add(1)
|
||||
go func(node *clusterNode, cmds []Cmder) {
|
||||
defer wg.Done()
|
||||
c.processTxPipelineNode(ctx, node, cmds, failedCmds)
|
||||
}(node, cmds)
|
||||
outcome := c.processTxPipelineNode(ctx, node, cmds, asking)
|
||||
lastErr = outcome.err
|
||||
redirected = false
|
||||
switch outcome.kind {
|
||||
case txSuccess:
|
||||
return cmdsFirstErr(cmds)
|
||||
case txRetryMoved:
|
||||
// Route directly to the authoritative addr from the MOVED; the
|
||||
// cached slot state may be stale until LazyReload lands.
|
||||
redirected = true
|
||||
asking = false
|
||||
c.state.LazyReload()
|
||||
if node, err = c.nodes.GetOrCreate(outcome.addr); err != nil {
|
||||
setCmdsErr(cmds, err)
|
||||
return err
|
||||
}
|
||||
case txRetryAsk:
|
||||
redirected = true
|
||||
asking = true
|
||||
if node, err = c.nodes.GetOrCreate(outcome.addr); err != nil {
|
||||
setCmdsErr(cmds, err)
|
||||
return err
|
||||
}
|
||||
case txRetryTryAgain, txRetryConn:
|
||||
// Same node, fresh connection: TRYAGAIN comes from the migrating
|
||||
// source (still the owner), and a conn failure only needs a new
|
||||
// connection. Preserve a prior ASKING flag: if we followed an ASK
|
||||
// to the importing target, the retry must still send ASKING (the
|
||||
// slot is still importing). ASKING is harmless if the migration
|
||||
// has since completed, since the flag is only consulted for
|
||||
// importing slots.
|
||||
case txFatal:
|
||||
// Mark every queued-but-never-executed command with the abort
|
||||
// error; the command that triggered EXECABORT already has its
|
||||
// own error and keeps it, so callers can tell what went wrong.
|
||||
abortErr := cmp.Or(outcome.execErr, outcome.err)
|
||||
for _, cmd := range cmds {
|
||||
if cmd.Err() == nil {
|
||||
cmd.SetErr(abortErr)
|
||||
}
|
||||
}
|
||||
return lastErr
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
if len(failedCmds.m) == 0 {
|
||||
break
|
||||
}
|
||||
cmdsMap = failedCmds.m
|
||||
lastErr = cmdsFirstErr(cmds)
|
||||
}
|
||||
|
||||
if pipelineOpDurationCallback != nil {
|
||||
operationDuration := time.Since(operationStart)
|
||||
finalErr := cmdsFirstErr(cmds)
|
||||
if finalErr == nil {
|
||||
finalErr = lastErr
|
||||
}
|
||||
pipelineOpDurationCallback(ctx, operationDuration, "MULTI", len(cmds), totalAttempts, finalErr, nil, 0)
|
||||
if lastErr != nil {
|
||||
setCmdsErr(cmds, lastErr)
|
||||
}
|
||||
|
||||
return cmdsFirstErr(cmds)
|
||||
}
|
||||
|
||||
// slottedKeyedCommands returns a map of slot to commands taking into account
|
||||
// only commands that have keys.
|
||||
func (c *ClusterClient) slottedKeyedCommands(ctx context.Context, cmds []Cmder) map[int][]Cmder {
|
||||
func (c *ClusterClient) slottedKeyedCommands(_ context.Context, cmds []Cmder) map[int][]Cmder {
|
||||
cmdsSlots := map[int][]Cmder{}
|
||||
|
||||
// Peek once outside the loop, one RLock for the whole batch instead of
|
||||
@@ -1972,151 +2068,218 @@ func (c *ClusterClient) slottedKeyedCommands(ctx context.Context, cmds []Cmder)
|
||||
}
|
||||
|
||||
func (c *ClusterClient) processTxPipelineNode(
|
||||
ctx context.Context, node *clusterNode, cmds []Cmder, failedCmds *cmdsMap,
|
||||
) {
|
||||
cmds = wrapMultiExec(ctx, cmds)
|
||||
_ = node.Client.withProcessPipelineHook(ctx, cmds, func(ctx context.Context, cmds []Cmder) error {
|
||||
ctx context.Context, node *clusterNode, cmds []Cmder, asking bool,
|
||||
) *txOutcome {
|
||||
wire := wrapMultiExec(ctx, cmds)
|
||||
if asking {
|
||||
// ASKING must precede MULTI so the flag stays set for the whole tx.
|
||||
wire = append([]Cmder{NewCmd(ctx, "asking")}, wire...)
|
||||
}
|
||||
|
||||
var outcome *txOutcome
|
||||
_ = node.Client.withProcessPipelineHook(ctx, wire, func(ctx context.Context, wire []Cmder) error {
|
||||
cn, err := node.Client.getConn(ctx)
|
||||
if err != nil {
|
||||
_ = c.mapCmdsByNode(ctx, failedCmds, cmds)
|
||||
setCmdsErr(cmds, err)
|
||||
if shouldRetry(err, true) && !cmdsContainNoRetry(cmds) {
|
||||
outcome = &txOutcome{kind: txRetryConn, err: err}
|
||||
} else {
|
||||
outcome = &txOutcome{kind: txFatal, err: err}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
var processErr error
|
||||
defer func() {
|
||||
node.Client.releaseConn(ctx, cn, processErr)
|
||||
}()
|
||||
processErr = c.processTxPipelineNodeConn(ctx, node, cn, cmds, failedCmds)
|
||||
|
||||
return processErr
|
||||
outcome = c.processTxPipelineNodeConn(ctx, node, cn, wire, cmds, asking)
|
||||
connErr := outcome.err
|
||||
if isRedisError(outcome.err) {
|
||||
connErr = nil
|
||||
}
|
||||
if outcome.unreadReplies {
|
||||
connErr = errTxDirtyConn
|
||||
}
|
||||
node.Client.releaseConn(ctx, cn, connErr)
|
||||
return connErr
|
||||
})
|
||||
|
||||
if outcome == nil {
|
||||
outcome = &txOutcome{kind: txFatal, err: fmt.Errorf("redis: tx pipeline produced no outcome")}
|
||||
}
|
||||
return outcome
|
||||
}
|
||||
|
||||
func (c *ClusterClient) processTxPipelineNodeConn(
|
||||
ctx context.Context, node *clusterNode, cn *pool.Conn, cmds []Cmder, failedCmds *cmdsMap,
|
||||
) error {
|
||||
ctx context.Context, node *clusterNode, cn *pool.Conn, wire []Cmder, cmds []Cmder, asking bool,
|
||||
) *txOutcome {
|
||||
if err := cn.WithWriter(c.context(ctx), c.opt.WriteTimeout, func(wr *proto.Writer) error {
|
||||
return writeCmds(wr, cmds)
|
||||
return writeCmds(wr, wire)
|
||||
}); err != nil {
|
||||
// Write failure: re-route the whole tx on a fresh connection.
|
||||
if shouldRetry(err, true) && !cmdsContainNoRetry(cmds) {
|
||||
_ = c.mapCmdsByNode(ctx, failedCmds, cmds)
|
||||
return &txOutcome{kind: txRetryConn, err: err}
|
||||
}
|
||||
setCmdsErr(cmds, err)
|
||||
return err
|
||||
return &txOutcome{kind: txFatal, err: err}
|
||||
}
|
||||
|
||||
return cn.WithReader(c.context(ctx), c.opt.ReadTimeout, func(rd *proto.Reader) error {
|
||||
statusCmd := cmds[0].(*StatusCmd)
|
||||
// Trim multi and exec.
|
||||
trimmedCmds := cmds[1 : len(cmds)-1]
|
||||
|
||||
if err := c.txPipelineReadQueued(
|
||||
ctx, node, cn, rd, statusCmd, trimmedCmds, failedCmds,
|
||||
); err != nil {
|
||||
setCmdsErr(cmds, err)
|
||||
|
||||
moved, ask, addr := isMovedError(err)
|
||||
if moved || ask {
|
||||
return c.cmdsMoved(ctx, trimmedCmds, moved, ask, addr, failedCmds)
|
||||
}
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
return node.Client.pipelineReadCmds(ctx, cn, rd, trimmedCmds)
|
||||
var outcome *txOutcome
|
||||
readErr := cn.WithReader(c.context(ctx), c.opt.ReadTimeout, func(rd *proto.Reader) error {
|
||||
outcome = c.readTxPipelineReplies(ctx, node, cn, rd, cmds, asking)
|
||||
return nil
|
||||
})
|
||||
|
||||
if readErr != nil {
|
||||
// Reader-level failure (deadline setup, nil conn) around the read loop.
|
||||
// The batch was already written, so the server may have committed;
|
||||
// surface the error as fatal and discard the suspect connection rather
|
||||
// than re-executing the transaction.
|
||||
return c.txReadFatal(readErr)
|
||||
}
|
||||
return outcome
|
||||
}
|
||||
|
||||
func (c *ClusterClient) txPipelineReadQueued(
|
||||
ctx context.Context,
|
||||
node *clusterNode,
|
||||
cn *pool.Conn,
|
||||
rd *proto.Reader,
|
||||
statusCmd *StatusCmd,
|
||||
cmds []Cmder,
|
||||
failedCmds *cmdsMap,
|
||||
) error {
|
||||
// Parse queued replies.
|
||||
// To be sure there are no buffered push notifications, we process them before reading the reply
|
||||
if err := node.Client.processPendingPushNotificationWithReader(ctx, cn, rd); err != nil {
|
||||
// Log the error but don't fail the command execution
|
||||
// Push notification processing errors shouldn't break normal Redis operations
|
||||
internal.Logger.Printf(ctx, "push: error processing pending notifications before reading reply: %v", err)
|
||||
}
|
||||
if err := statusCmd.readReply(rd); err != nil {
|
||||
return err
|
||||
// readTxPipelineReplies reads the replies of one MULTI..EXEC unit and
|
||||
// classifies the outcome. The reply count always matches the number of sent
|
||||
// commands, so success/redirect paths leave the connection clean; only an early
|
||||
// MULTI read failure can leave unread replies.
|
||||
func (c *ClusterClient) readTxPipelineReplies(
|
||||
ctx context.Context, node *clusterNode, cn *pool.Conn, rd *proto.Reader, cmds []Cmder, asking bool,
|
||||
) *txOutcome {
|
||||
scratch := NewStatusCmd(ctx)
|
||||
|
||||
readStatus := func() error {
|
||||
c.txProcessPush(ctx, node, cn, rd)
|
||||
return scratch.readReply(rd)
|
||||
}
|
||||
|
||||
// Optional top-level ASKING reply (+OK, or a retryable error such as -LOADING).
|
||||
if asking {
|
||||
if err := readStatus(); err != nil {
|
||||
return c.txPreQueueErrorOutcome(err, cmds)
|
||||
}
|
||||
}
|
||||
|
||||
// MULTI reply (+OK, or an error such as -LOADING during failover).
|
||||
if err := readStatus(); err != nil {
|
||||
return c.txPreQueueErrorOutcome(err, cmds)
|
||||
}
|
||||
|
||||
// Queue replies: +QUEUED, or a redirect / command error that dirties the tx.
|
||||
var firstRedirect *txRedirect
|
||||
var firstFatal error
|
||||
for _, cmd := range cmds {
|
||||
// To be sure there are no buffered push notifications, we process them before reading the reply
|
||||
if err := node.Client.processPendingPushNotificationWithReader(ctx, cn, rd); err != nil {
|
||||
// Log the error but don't fail the command execution
|
||||
// Push notification processing errors shouldn't break normal Redis operations
|
||||
internal.Logger.Printf(ctx, "push: error processing pending notifications before reading reply: %v", err)
|
||||
err := readStatus()
|
||||
if err == nil {
|
||||
continue // +QUEUED
|
||||
}
|
||||
err := statusCmd.readReply(rd)
|
||||
if err != nil {
|
||||
if c.checkMovedErr(ctx, cmd, err, failedCmds) {
|
||||
// will be processed later
|
||||
continue
|
||||
if !isRedisError(err) {
|
||||
return c.txReadFatal(err) // IO error
|
||||
}
|
||||
if moved, ask, addr := isMovedError(err); moved || ask {
|
||||
if firstRedirect == nil {
|
||||
firstRedirect = &txRedirect{moved: moved, ask: ask, addr: addr, err: err}
|
||||
}
|
||||
cmd.SetErr(err)
|
||||
if !isRedisError(err) {
|
||||
return err
|
||||
continue
|
||||
}
|
||||
if proto.IsTryAgainError(err) {
|
||||
if firstRedirect == nil {
|
||||
firstRedirect = &txRedirect{tryAgain: true, err: err}
|
||||
}
|
||||
continue
|
||||
}
|
||||
// Non-redirect command error (e.g. wrong arity) dirties the tx.
|
||||
cmd.SetErr(err)
|
||||
if firstFatal == nil {
|
||||
firstFatal = err
|
||||
}
|
||||
}
|
||||
|
||||
// To be sure there are no buffered push notifications, we process them before reading the reply
|
||||
if err := node.Client.processPendingPushNotificationWithReader(ctx, cn, rd); err != nil {
|
||||
// Log the error but don't fail the command execution
|
||||
// Push notification processing errors shouldn't break normal Redis operations
|
||||
internal.Logger.Printf(ctx, "push: error processing pending notifications before reading reply: %v", err)
|
||||
}
|
||||
// Parse number of replies.
|
||||
// EXEC reply. ReadLine parses error lines into typed errors, so a non-nil
|
||||
// err means EXEC returned an error rather than the result array.
|
||||
c.txProcessPush(ctx, node, cn, rd)
|
||||
line, err := rd.ReadLine()
|
||||
if err != nil {
|
||||
if err == Nil {
|
||||
err = TxFailedErr
|
||||
if !isRedisError(err) {
|
||||
return c.txReadFatal(err) // IO error
|
||||
}
|
||||
return err
|
||||
return c.classifyExecError(err, firstRedirect, firstFatal)
|
||||
}
|
||||
|
||||
if line[0] != proto.RespArray {
|
||||
return fmt.Errorf("redis: expected '*', but got line %q", line)
|
||||
err := fmt.Errorf("redis: unexpected EXEC reply %q", line)
|
||||
setCmdsErr(cmds, err)
|
||||
// A non-array aggregate reply may carry an unread payload.
|
||||
return &txOutcome{kind: txFatal, err: err, unreadReplies: true}
|
||||
}
|
||||
|
||||
return nil
|
||||
// Success: read the N command results.
|
||||
if err := node.Client.pipelineReadCmds(ctx, cn, rd, cmds); err != nil && !isRedisError(err) {
|
||||
return c.txReadFatal(err) // IO error mid-results
|
||||
}
|
||||
return &txOutcome{kind: txSuccess}
|
||||
}
|
||||
|
||||
func (c *ClusterClient) cmdsMoved(
|
||||
ctx context.Context, cmds []Cmder,
|
||||
moved, ask bool,
|
||||
addr string,
|
||||
failedCmds *cmdsMap,
|
||||
) error {
|
||||
node, err := c.nodes.GetOrCreate(addr)
|
||||
if err != nil {
|
||||
return err
|
||||
func (c *ClusterClient) txProcessPush(ctx context.Context, node *clusterNode, cn *pool.Conn, rd *proto.Reader) {
|
||||
if err := node.Client.processPendingPushNotificationWithReader(ctx, cn, rd); err != nil {
|
||||
internal.Logger.Printf(ctx, "push: error processing pending notifications before reading reply: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
if moved {
|
||||
c.state.LazyReload()
|
||||
for _, cmd := range cmds {
|
||||
failedCmds.Add(node, cmd)
|
||||
// txReadFatal classifies a read-phase IO error. The MULTI..EXEC batch was
|
||||
// already written, so the server may have committed the transaction; retrying
|
||||
// would re-execute it, double-applying non-idempotent commands (INCR/APPEND,
|
||||
// which are not NoRetry). Surface the error as fatal and discard the
|
||||
// connection, since replies may still be unread on the wire.
|
||||
func (c *ClusterClient) txReadFatal(err error) *txOutcome {
|
||||
return &txOutcome{kind: txFatal, err: err, unreadReplies: true}
|
||||
}
|
||||
|
||||
// txPreQueueErrorOutcome classifies a setup-phase reply error: the top-level
|
||||
// ASKING reply or the MULTI reply. The transaction body never executes (EXEC
|
||||
// returns -EXECABORT), so retryable errors such as -LOADING are safe to retry
|
||||
// on a fresh connection. A failed setup reply still leaves the remaining
|
||||
// replies on the wire -- the server replies to each following command and to
|
||||
// EXEC regardless -- so the connection is always discarded.
|
||||
func (c *ClusterClient) txPreQueueErrorOutcome(err error, cmds []Cmder) *txOutcome {
|
||||
if !isRedisError(err) {
|
||||
return c.txReadFatal(err)
|
||||
}
|
||||
if shouldRetry(err, true) && !cmdsContainNoRetry(cmds) {
|
||||
return &txOutcome{kind: txRetryConn, err: err, unreadReplies: true}
|
||||
}
|
||||
return &txOutcome{kind: txFatal, err: err, unreadReplies: true}
|
||||
}
|
||||
|
||||
// classifyExecError turns an EXEC reply error into a retry/fatal outcome.
|
||||
func (c *ClusterClient) classifyExecError(execErr error, firstRedirect *txRedirect, firstFatal error) *txOutcome {
|
||||
if moved, ask, addr := isMovedError(execErr); moved || ask {
|
||||
// Narrow race: the slot moved after every command was queued.
|
||||
if ask {
|
||||
return &txOutcome{kind: txRetryAsk, err: execErr, addr: addr}
|
||||
}
|
||||
return nil
|
||||
return &txOutcome{kind: txRetryMoved, err: execErr, addr: addr}
|
||||
}
|
||||
|
||||
if ask {
|
||||
for _, cmd := range cmds {
|
||||
failedCmds.Add(node, NewCmd(ctx, "asking"), cmd)
|
||||
if proto.IsTryAgainError(execErr) {
|
||||
return &txOutcome{kind: txRetryTryAgain, err: execErr}
|
||||
}
|
||||
if proto.IsClusterDownError(execErr) {
|
||||
// Cluster degraded: back off and retry. Replies were fully consumed.
|
||||
return &txOutcome{kind: txRetryConn, err: execErr}
|
||||
}
|
||||
if proto.IsExecAbortError(execErr) {
|
||||
if firstFatal != nil {
|
||||
return &txOutcome{kind: txFatal, err: firstFatal, execErr: execErr}
|
||||
}
|
||||
return nil
|
||||
if firstRedirect != nil {
|
||||
switch {
|
||||
case firstRedirect.moved:
|
||||
return &txOutcome{kind: txRetryMoved, err: firstRedirect.err, addr: firstRedirect.addr}
|
||||
case firstRedirect.ask:
|
||||
return &txOutcome{kind: txRetryAsk, err: firstRedirect.err, addr: firstRedirect.addr}
|
||||
case firstRedirect.tryAgain:
|
||||
return &txOutcome{kind: txRetryTryAgain, err: firstRedirect.err}
|
||||
}
|
||||
}
|
||||
return &txOutcome{kind: txFatal, err: execErr, execErr: execErr}
|
||||
}
|
||||
|
||||
return nil
|
||||
return &txOutcome{kind: txFatal, err: execErr}
|
||||
}
|
||||
|
||||
func (c *ClusterClient) Watch(ctx context.Context, fn func(*Tx) error, keys ...string) error {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package redis_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"errors"
|
||||
@@ -11,10 +12,12 @@ import (
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
. "github.com/bsm/ginkgo/v2"
|
||||
. "github.com/bsm/gomega"
|
||||
|
||||
"github.com/redis/go-redis/v9"
|
||||
"github.com/redis/go-redis/v9/internal/hashtag"
|
||||
"github.com/redis/go-redis/v9/internal/routing"
|
||||
@@ -257,6 +260,170 @@ func slotEqual(s1, s2 redis.ClusterSlot) bool {
|
||||
return true
|
||||
}
|
||||
|
||||
// txWriteRecorder captures, per node addr, each Write payload that carries a
|
||||
// MULTI/EXEC transaction. txWriteCount uses it to prove whether the client
|
||||
// retried a transaction (e.g. after a transient write/read failure or an ASK).
|
||||
type txWriteRecorder struct {
|
||||
mu sync.Mutex
|
||||
writes map[string][][]byte // addr -> tx write payloads
|
||||
}
|
||||
|
||||
func newTxWriteRecorder() *txWriteRecorder {
|
||||
return &txWriteRecorder{writes: map[string][][]byte{}}
|
||||
}
|
||||
|
||||
func (r *txWriteRecorder) record(addr string, b []byte) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.writes[addr] = append(r.writes[addr], bytes.Clone(b))
|
||||
}
|
||||
|
||||
// txWriteCount returns the number of tx writes (MULTI-containing batches) sent
|
||||
// to addr. Used to prove whether the client retried a transaction.
|
||||
func (r *txWriteRecorder) txWriteCount(addr string) int {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
n := 0
|
||||
for _, b := range r.writes[addr] {
|
||||
if countRespCmd(b, "MULTI") > 0 {
|
||||
n++
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// countRespCmd counts occurrences of a RESP command name (e.g. "MULTI") within a
|
||||
// raw Write payload by matching the bulk-string framing "$<len>\r\n<NAME>\r\n".
|
||||
// Matching is case-insensitive: go-redis writes tx commands in lower case
|
||||
// ("multi"/"exec") but other commands may differ.
|
||||
func countRespCmd(b []byte, cmd string) int {
|
||||
needle := []byte("$" + strconv.Itoa(len(cmd)) + "\r\n" + strings.ToUpper(cmd) + "\r\n")
|
||||
return bytes.Count(bytes.ToUpper(b), needle)
|
||||
}
|
||||
|
||||
// txFaultConn wraps a real redis connection. Non-tx traffic is forwarded
|
||||
// unchanged so the cluster client can still load topology; tx writes (those
|
||||
// containing MULTI) are always recorded. When failOnce is non-nil
|
||||
// (transient-retry test), the first MULTI write across all sharing conns
|
||||
// fails with io.EOF and subsequent ones are forwarded, so the client must retry the
|
||||
// whole tx and then succeed.
|
||||
type txFaultConn struct {
|
||||
net.Conn
|
||||
addr string
|
||||
rec *txWriteRecorder
|
||||
failOnce *atomic.Int32
|
||||
// failAddr, when set, restricts failOnce to writes whose addr matches; an
|
||||
// empty failAddr (the default) fires on the first MULTI write to any addr.
|
||||
failAddr string
|
||||
// failRead makes reads return io.EOF once a MULTI write has been seen on
|
||||
// this conn, so the tx batch is committed server-side but the client cannot
|
||||
// read the reply. Used to prove a post-commit read failure is not retried.
|
||||
failRead bool
|
||||
sawMulti atomic.Bool
|
||||
}
|
||||
|
||||
func (c *txFaultConn) Write(b []byte) (int, error) {
|
||||
if countRespCmd(b, "MULTI") > 0 {
|
||||
c.sawMulti.Store(true)
|
||||
c.rec.record(c.addr, b)
|
||||
if c.failOnce != nil && (c.failAddr == "" || c.addr == c.failAddr) &&
|
||||
c.failOnce.CompareAndSwap(0, 1) {
|
||||
return 0, io.EOF
|
||||
}
|
||||
}
|
||||
return c.Conn.Write(b)
|
||||
}
|
||||
|
||||
func (c *txFaultConn) Read(b []byte) (int, error) {
|
||||
if c.failRead && c.sawMulti.Load() {
|
||||
return 0, io.EOF
|
||||
}
|
||||
return c.Conn.Read(b)
|
||||
}
|
||||
|
||||
// slotMasterPort returns the port of the master owning `slot` in the test
|
||||
// cluster's fixed distribution (0-5460 -> 16600, 5461-10922 -> 16601,
|
||||
// 10923-16383 -> 16602), matching clusterScenario.slots().
|
||||
func slotMasterPort(slot int) string {
|
||||
switch {
|
||||
case slot < 5461:
|
||||
return cluster.ports[0]
|
||||
case slot < 10923:
|
||||
return cluster.ports[1]
|
||||
default:
|
||||
return cluster.ports[2]
|
||||
}
|
||||
}
|
||||
|
||||
// slotOtherMasterPort returns the port of a master that does NOT own `slot`,
|
||||
// matching the migration target that setupAskMigration picks (the first master
|
||||
// port != the slot's owner).
|
||||
func slotOtherMasterPort(slot int) string {
|
||||
src := slotMasterPort(slot)
|
||||
for _, p := range cluster.ports[:3] {
|
||||
if p != src {
|
||||
return p
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// resetSlot restores `slot` to a clean, fully-owned state on its master and
|
||||
// deletes `keys` from all masters. It is idempotent and best-effort, used to
|
||||
// recover from a prior run that left the slot migrating/importing.
|
||||
func resetSlot(ctx context.Context, slot int, keys ...string) {
|
||||
srcPort := slotMasterPort(slot)
|
||||
srcCli := redis.NewClient(&redis.Options{Addr: "127.0.0.1:" + srcPort})
|
||||
srcID, _ := srcCli.ClusterMyID(ctx).Result()
|
||||
_ = srcCli.Close()
|
||||
for _, p := range cluster.ports[:3] {
|
||||
c := redis.NewClient(&redis.Options{Addr: "127.0.0.1:" + p})
|
||||
_ = c.Do(ctx, "cluster", "setslot", slot, "node", srcID).Err()
|
||||
for _, k := range keys {
|
||||
_ = c.Del(ctx, k).Err()
|
||||
}
|
||||
_ = c.Close()
|
||||
}
|
||||
}
|
||||
|
||||
// setupAskMigration puts `slot` into MIGRATING on its owner (source) and IMPORTING
|
||||
// on a different master (target), leaving the slot's keys absent from both. The
|
||||
// source then returns -ASK for any absent key in the slot. The returned cleanup
|
||||
// restores slot ownership on all masters.
|
||||
func setupAskMigration(ctx context.Context, slot int) func() {
|
||||
resetSlot(ctx, slot)
|
||||
srcPort := slotMasterPort(slot)
|
||||
tgtPort := ""
|
||||
for _, p := range cluster.ports[:3] {
|
||||
if p != srcPort {
|
||||
tgtPort = p
|
||||
break
|
||||
}
|
||||
}
|
||||
srcCli := redis.NewClient(&redis.Options{Addr: "127.0.0.1:" + srcPort})
|
||||
tgtCli := redis.NewClient(&redis.Options{Addr: "127.0.0.1:" + tgtPort})
|
||||
|
||||
srcID, err := srcCli.ClusterMyID(ctx).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
dstID, err := tgtCli.ClusterMyID(ctx).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
Expect(srcCli.Do(ctx, "cluster", "setslot", slot, "migrating", dstID).Err()).
|
||||
NotTo(HaveOccurred())
|
||||
Expect(tgtCli.Do(ctx, "cluster", "setslot", slot, "importing", srcID).Err()).
|
||||
NotTo(HaveOccurred())
|
||||
|
||||
return func() {
|
||||
for _, p := range cluster.ports[:3] {
|
||||
c := redis.NewClient(&redis.Options{Addr: "127.0.0.1:" + p})
|
||||
_ = c.Do(ctx, "cluster", "setslot", slot, "node", srcID).Err()
|
||||
_ = c.Close()
|
||||
}
|
||||
_ = srcCli.Close()
|
||||
_ = tgtCli.Close()
|
||||
}
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------------
|
||||
|
||||
var _ = Describe("ClusterClient", func() {
|
||||
@@ -859,8 +1026,198 @@ var _ = Describe("ClusterClient", func() {
|
||||
Expect(client.Close()).NotTo(HaveOccurred())
|
||||
})
|
||||
|
||||
// A TxPipeline whose slot is initially routed at a replica gets -MOVED on
|
||||
// the write; the client must follow the redirect and run the whole tx on
|
||||
// the real master, with the written values visible afterwards.
|
||||
It("should follow MOVED redirect for TxPipeline", func() {
|
||||
// {s} hashtag forces a single-slot, single-node transaction.
|
||||
_, err := client.TxPipelined(ctx, func(pipe redis.Pipeliner) error {
|
||||
pipe.Set(ctx, "A{s}", "1", 0)
|
||||
pipe.Set(ctx, "B{s}", "2", 0)
|
||||
return nil
|
||||
})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
// Route the slot at a replica so the write tx is MOVED back to the real master.
|
||||
Eventually(func() error {
|
||||
return client.SwapNodes(ctx, "A{s}")
|
||||
}, 30*time.Second).ShouldNot(HaveOccurred())
|
||||
|
||||
_, err = client.TxPipelined(ctx, func(pipe redis.Pipeliner) error {
|
||||
pipe.Set(ctx, "A{s}", "11", 0)
|
||||
pipe.Set(ctx, "B{s}", "22", 0)
|
||||
return nil
|
||||
})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
// The MOVED triggers a topology reload that un-swaps the state, so
|
||||
// reads eventually land on the real master with the new values.
|
||||
Eventually(func() string {
|
||||
return client.Get(ctx, "A{s}").Val()
|
||||
}, 30*time.Second).Should(Equal("11"))
|
||||
Eventually(func() string {
|
||||
return client.Get(ctx, "B{s}").Val()
|
||||
}, 30*time.Second).Should(Equal("22"))
|
||||
})
|
||||
|
||||
// When a queued command fails with a non-redirect error (here GET with
|
||||
// wrong arity), Redis aborts the tx with -EXECABORT. The surfaced error
|
||||
// should name the triggering command's error ("wrong number of
|
||||
// arguments"), not a generic parse error.
|
||||
It("should surface the triggering error on TxPipeline EXECABORT instead of 'expected'", func() {
|
||||
_, err := client.TxPipelined(ctx, func(pipe redis.Pipeliner) error {
|
||||
pipe.Set(ctx, "A{s}", "1", 0)
|
||||
pipe.Do(ctx, "GET", "A{s}", "extra")
|
||||
pipe.Set(ctx, "B{s}", "2", 0)
|
||||
return nil
|
||||
})
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).NotTo(ContainSubstring("expected '*'"))
|
||||
Expect(err.Error()).To(ContainSubstring("wrong number of arguments"))
|
||||
})
|
||||
|
||||
// With redirects disabled (MaxRedirects = -1), a TxPipeline that hits
|
||||
// -MOVED must surface a MOVED error so the caller can tell it was a
|
||||
// redirect, not a generic parse error.
|
||||
It("should return MOVED (not 'expected') for TxPipeline when redirects are exhausted", func() {
|
||||
opt := redisClusterOptions()
|
||||
opt.MaxRedirects = -1
|
||||
redirectClient := cluster.newClusterClient(ctx, opt)
|
||||
defer func() { Expect(redirectClient.Close()).NotTo(HaveOccurred()) }()
|
||||
|
||||
Eventually(func() error {
|
||||
return redirectClient.SwapNodes(ctx, "A{s}")
|
||||
}, 30*time.Second).ShouldNot(HaveOccurred())
|
||||
|
||||
_, err := redirectClient.TxPipelined(ctx, func(pipe redis.Pipeliner) error {
|
||||
pipe.Set(ctx, "A{s}", "1", 0)
|
||||
pipe.Set(ctx, "B{s}", "2", 0)
|
||||
return nil
|
||||
})
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("MOVED"))
|
||||
Expect(err.Error()).NotTo(ContainSubstring("expected '*'"))
|
||||
})
|
||||
|
||||
// A transient write failure during a tx must be retried as a whole
|
||||
// transaction and ultimately succeed, with both values actually written.
|
||||
// The first tx write is failed with io.EOF and the retry is let through.
|
||||
It("retries the whole tx after a transient write failure and succeeds", func() {
|
||||
rec := newTxWriteRecorder()
|
||||
var failOnce atomic.Int32 // first MULTI write fails, subsequent ones forward
|
||||
opt := redisClusterOptions()
|
||||
opt.Dialer = func(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||
var d net.Dialer
|
||||
cn, err := d.DialContext(ctx, network, addr)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &txFaultConn{Conn: cn, addr: addr, rec: rec, failOnce: &failOnce}, nil
|
||||
}
|
||||
retryClient := cluster.newClusterClient(ctx, opt)
|
||||
defer func() { Expect(retryClient.Close()).NotTo(HaveOccurred()) }()
|
||||
|
||||
_, err := retryClient.TxPipelined(ctx, func(pipe redis.Pipeliner) error {
|
||||
pipe.Set(ctx, "retry1{s}", "1", 0)
|
||||
pipe.Set(ctx, "retry2{s}", "2", 0)
|
||||
return nil
|
||||
})
|
||||
Expect(err).NotTo(HaveOccurred(),
|
||||
"a transient write failure should be retried as a whole tx and succeed")
|
||||
|
||||
// The retried tx must have executed: both values are present on the
|
||||
// slot master.
|
||||
Expect(retryClient.Get(ctx, "retry1{s}").Val()).To(Equal("1"))
|
||||
Expect(retryClient.Get(ctx, "retry2{s}").Val()).To(Equal("2"))
|
||||
|
||||
// The slot master received more than one tx write, so the first
|
||||
// attempt really failed and the tx was retried.
|
||||
srcAddr := "127.0.0.1:" + slotMasterPort(hashtag.Slot("retry1{s}"))
|
||||
Expect(rec.txWriteCount(srcAddr)).To(BeNumerically(">=", 2),
|
||||
"the first tx write failed and the client should have retried the whole tx")
|
||||
})
|
||||
|
||||
// During an ASK redirect, a transient connection failure to the
|
||||
// importing target must be retried with the ASKING flag preserved, so the
|
||||
// tx still executes on the target.
|
||||
It("preserves ASKING when retrying a conn failure on the ASK target", func() {
|
||||
key := "askretry{s}"
|
||||
slot := hashtag.Slot(key)
|
||||
defer resetSlot(ctx, slot, key)
|
||||
cleanup := setupAskMigration(ctx, slot)
|
||||
defer cleanup()
|
||||
|
||||
tgtAddr := "127.0.0.1:" + slotOtherMasterPort(slot)
|
||||
rec := newTxWriteRecorder()
|
||||
var failOnce atomic.Int32 // first MULTI write to the target fails, then forwards
|
||||
opt := redisClusterOptions()
|
||||
opt.MaxRedirects = 2 // source-ASK (0) + target-connfail (1) + target-success (2)
|
||||
opt.Dialer = func(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||
var d net.Dialer
|
||||
cn, err := d.DialContext(ctx, network, addr)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &txFaultConn{Conn: cn, addr: addr, rec: rec, failOnce: &failOnce, failAddr: tgtAddr}, nil
|
||||
}
|
||||
askClient := cluster.newClusterClient(ctx, opt)
|
||||
defer func() { Expect(askClient.Close()).NotTo(HaveOccurred()) }()
|
||||
|
||||
_, err := askClient.TxPipelined(ctx, func(pipe redis.Pipeliner) error {
|
||||
pipe.Set(ctx, key, "v", 0)
|
||||
return nil
|
||||
})
|
||||
Expect(err).NotTo(HaveOccurred(),
|
||||
"a transient write failure on the ASK target should be retried with ASKING preserved and succeed")
|
||||
|
||||
// The tx executed on the target (with ASKING); a Get follows the
|
||||
// ASK back to the target and reads the value.
|
||||
Expect(askClient.Get(ctx, key).Val()).To(Equal("v"))
|
||||
|
||||
// The target received >=2 tx writes: the failed first attempt and
|
||||
// the successful retry, both with a leading ASKING.
|
||||
Expect(rec.txWriteCount(tgtAddr)).To(BeNumerically(">=", 2))
|
||||
})
|
||||
|
||||
// A read failure after the tx batch was written must not be retried: the
|
||||
// server may have committed the transaction, and re-sending the batch
|
||||
// would double-execute non-idempotent commands. The batch should be sent
|
||||
// exactly once.
|
||||
It("does not retry a read failure after the tx batch is written", func() {
|
||||
rec := newTxWriteRecorder()
|
||||
opt := redisClusterOptions()
|
||||
opt.MaxRedirects = 3
|
||||
opt.Dialer = func(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||
var d net.Dialer
|
||||
cn, err := d.DialContext(ctx, network, addr)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// failRead returns EOF from every read once a MULTI write has
|
||||
// been seen on the conn: the batch is committed server-side but
|
||||
// the client cannot read the reply.
|
||||
return &txFaultConn{Conn: cn, addr: addr, rec: rec, failRead: true}, nil
|
||||
}
|
||||
readClient := cluster.newClusterClient(ctx, opt)
|
||||
defer func() { Expect(readClient.Close()).NotTo(HaveOccurred()) }()
|
||||
|
||||
_, err := readClient.TxPipelined(ctx, func(pipe redis.Pipeliner) error {
|
||||
pipe.Set(ctx, "rdretry1{s}", "1", 0)
|
||||
pipe.Set(ctx, "rdretry2{s}", "2", 0)
|
||||
return nil
|
||||
})
|
||||
Expect(err).To(HaveOccurred(),
|
||||
"a read failure after the batch was written should surface an error")
|
||||
|
||||
// The batch was written exactly once: the read failure is fatal, so
|
||||
// the tx is not re-sent (no double-execution).
|
||||
srcAddr := "127.0.0.1:" + slotMasterPort(hashtag.Slot("rdretry1{s}"))
|
||||
Expect(rec.txWriteCount(srcAddr)).To(Equal(1),
|
||||
"a post-commit read failure must not re-send the tx batch")
|
||||
})
|
||||
|
||||
It("should call fn for every master node", func() {
|
||||
for i := 0; i < 10; i++ {
|
||||
for i := range 10 {
|
||||
Expect(client.Set(ctx, strconv.Itoa(i), "", 0).Err()).NotTo(HaveOccurred())
|
||||
}
|
||||
|
||||
@@ -1766,7 +2123,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should verify COMMAND tips match router policy types", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
expectedPolicies := map[string]struct {
|
||||
RequestPolicy string
|
||||
ResponsePolicy string
|
||||
@@ -1801,7 +2158,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
|
||||
Describe("Explicit Routing Policy Tests", func() {
|
||||
It("should test explicit routing policy for TOUCH", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Verify TOUCH command has multi_shard policy
|
||||
cmds, err := client.Command(ctx).Result()
|
||||
@@ -1825,7 +2182,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test explicit routing policy for FLUSHALL", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Verify FLUSHALL command has all_shards policy
|
||||
cmds, err := client.Command(ctx).Result()
|
||||
@@ -1853,7 +2210,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test explicit routing policy for PING", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Verify PING command has all_shards policy
|
||||
cmds, err := client.Command(ctx).Result()
|
||||
@@ -1870,7 +2227,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test explicit routing policy for DBSIZE", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Verify DBSIZE command has all_shards policy with agg_sum response
|
||||
cmds, err := client.Command(ctx).Result()
|
||||
@@ -1902,7 +2259,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test DDL commands routing policy for FT.CREATE", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Verify FT.CREATE command routing policy
|
||||
cmds, err := client.Command(ctx).Result()
|
||||
@@ -1941,7 +2298,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test DDL commands routing policy for FT.ALTER", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Verify FT.ALTER command routing policy
|
||||
cmds, err := client.Command(ctx).Result()
|
||||
@@ -1979,7 +2336,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should route keyed commands to correct shard based on hash slot", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
type masterNode struct {
|
||||
client *redis.Client
|
||||
@@ -2055,7 +2412,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should aggregate responses according to explicit aggregation policies", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
type masterNode struct {
|
||||
client *redis.Client
|
||||
@@ -2216,7 +2573,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should verify command aggregation policies", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
cmds, err := client.Command(ctx).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
@@ -2246,7 +2603,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should properly aggregate responses from keyless commands executed on multiple shards", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
type masterNode struct {
|
||||
client *redis.Client
|
||||
@@ -2322,7 +2679,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should properly aggregate responses from keyed commands executed on multiple shards", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
type masterNode struct {
|
||||
client *redis.Client
|
||||
addr string
|
||||
@@ -2418,7 +2775,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should propagate coordinator errors to client without modification", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
type masterNode struct {
|
||||
client *redis.Client
|
||||
@@ -2545,7 +2902,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
|
||||
Describe("Routing Policies Comprehensive Test Suite", func() {
|
||||
It("should test MGET command with multi-slot routing and key order preservation", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Set up test data across multiple shards
|
||||
testData := map[string]string{
|
||||
@@ -2590,7 +2947,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test MGET with non-existent keys across multiple shards", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Set up some keys
|
||||
Expect(client.Set(ctx, "mget_exists_1111", "value1", 0).Err()).NotTo(HaveOccurred())
|
||||
@@ -2618,7 +2975,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test TOUCH command with multi-slot routing", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Set up keys across multiple shards
|
||||
keys := []string{
|
||||
@@ -2648,7 +3005,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test DEL command with multi-slot routing", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Set up keys across multiple shards
|
||||
keys := []string{
|
||||
@@ -2683,7 +3040,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test DBSIZE command with agg_sum aggregation across all shards", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Set keys across multiple shards
|
||||
keys := []string{
|
||||
@@ -2709,7 +3066,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test PING command with all_shards routing", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// PING should be sent to all shards and return one successful response
|
||||
result := client.Ping(ctx)
|
||||
@@ -2718,7 +3075,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test MGET with single shard optimization", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Use hash tags to ensure all keys are on the same shard
|
||||
keys := []string{
|
||||
@@ -2750,7 +3107,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test empty MGET command returns error", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// MGET with no keys should return an error
|
||||
result := client.MGet(ctx)
|
||||
@@ -2759,7 +3116,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test MGET integration with MSET across multiple shards", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Create test data
|
||||
testData := map[string]string{
|
||||
@@ -2801,7 +3158,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test multi-shard commands cannot be used in pipeline", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Create keys across multiple shards
|
||||
keys := []string{
|
||||
@@ -2827,7 +3184,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test DisableRoutingPolicies option disables routing policies", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Test 1: With routing policies enabled (default), MGET should work across slots
|
||||
testData := map[string]string{
|
||||
@@ -2872,7 +3229,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should test large MGET with many keys across all shards", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Create many keys to ensure coverage across all shards
|
||||
numKeys := 100
|
||||
@@ -2909,7 +3266,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should route keyless commands to arbitrary shards using round robin", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
var numMasters int
|
||||
var numMastersMu sync.Mutex
|
||||
@@ -3170,7 +3527,7 @@ var _ = Describe("Command Tips tests", func() {
|
||||
})
|
||||
|
||||
It("should distribute keyless commands randomly across shards using random shard picker", func() {
|
||||
SkipBeforeRedisVersion(7.9, "The tips are included from Redis 8")
|
||||
SkipBeforeRedisVersion("7.9", "The tips are included from Redis 8")
|
||||
|
||||
// Create a cluster client with random shard picker
|
||||
opt := redisClusterOptions()
|
||||
|
||||
@@ -82,7 +82,7 @@ var _ = Describe("PubSub", func() {
|
||||
})
|
||||
|
||||
It("should receive hash field subkey notifications", Label("hash-expiration", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.8, "subkeyspace notifications require Redis 8.8")
|
||||
SkipBeforeRedisVersion("8.8", "subkeyspace notifications require Redis 8.8")
|
||||
|
||||
config, err := client.ConfigGet(ctx, "notify-keyspace-events").Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
4
ring.go
4
ring.go
@@ -194,13 +194,13 @@ func (opt *RingOptions) init() {
|
||||
case -1:
|
||||
opt.MinRetryBackoff = 0
|
||||
case 0:
|
||||
opt.MinRetryBackoff = 8 * time.Millisecond
|
||||
opt.MinRetryBackoff = 10 * time.Millisecond
|
||||
}
|
||||
switch opt.MaxRetryBackoff {
|
||||
case -1:
|
||||
opt.MaxRetryBackoff = 0
|
||||
case 0:
|
||||
opt.MaxRetryBackoff = 512 * time.Millisecond
|
||||
opt.MaxRetryBackoff = time.Second
|
||||
}
|
||||
|
||||
if opt.ReadBufferSize == 0 {
|
||||
|
||||
@@ -494,8 +494,10 @@ type FTSynDumpCmd struct {
|
||||
// FTAggregateResult represents the result of an aggregate operation
|
||||
// NOTE: For RESP3 Total is not reliable (before Redis 8.8)
|
||||
type FTAggregateResult struct {
|
||||
Total int
|
||||
Rows []AggregateRow
|
||||
Total int
|
||||
Rows []AggregateRow
|
||||
// Warnings holds server warnings for a partial result (search-on-timeout
|
||||
// return/return-strict). RESP3 only; the fail policy returns an error instead.
|
||||
Warnings []string
|
||||
}
|
||||
|
||||
@@ -626,8 +628,10 @@ type SpellCheckSuggestion struct {
|
||||
}
|
||||
|
||||
type FTSearchResult struct {
|
||||
Total int
|
||||
Docs []Document
|
||||
Total int
|
||||
Docs []Document
|
||||
// Warnings holds server warnings for a partial result (search-on-timeout
|
||||
// return/return-strict). RESP3 only; the fail policy returns an error instead.
|
||||
Warnings []string
|
||||
}
|
||||
|
||||
@@ -2897,8 +2901,10 @@ func (cmd *FTSearchCmd) Clone() Cmder {
|
||||
|
||||
// FTHybridResult represents the result of a hybrid search operation
|
||||
type FTHybridResult struct {
|
||||
TotalResults int
|
||||
Results []map[string]interface{}
|
||||
TotalResults int
|
||||
Results []map[string]interface{}
|
||||
// Warnings holds server warnings for a partial result (search-on-timeout
|
||||
// return/return-strict), on RESP2 and RESP3; the fail policy returns an error.
|
||||
Warnings []string
|
||||
ExecutionTime float64
|
||||
}
|
||||
@@ -3041,9 +3047,13 @@ func parseFTHybrid(data []interface{}, withCursor bool) (FTHybridResult, *FTHybr
|
||||
results = append(results, itemMap)
|
||||
}
|
||||
|
||||
// Parse warnings (optional field)
|
||||
// Optional warnings; accept both "warning" (as FT.SEARCH/FT.AGGREGATE) and "warnings".
|
||||
var warnings []string
|
||||
if warningsData, ok := resultMap["warnings"].([]interface{}); ok {
|
||||
warningsData, ok := resultMap["warning"].([]interface{})
|
||||
if !ok {
|
||||
warningsData, ok = resultMap["warnings"].([]interface{})
|
||||
}
|
||||
if ok {
|
||||
warnings = make([]string, 0, len(warningsData))
|
||||
for _, w := range warningsData {
|
||||
if ws, ok := w.(string); ok {
|
||||
|
||||
@@ -461,3 +461,57 @@ func TestFTHybridWithArgs_ShardKRatio(t *testing.T) {
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// TestParseFTHybridWarnings checks that FT.HYBRID warnings populate from either
|
||||
// the "warning" or "warnings" key.
|
||||
func TestParseFTHybridWarnings(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
data []interface{}
|
||||
expected []string
|
||||
}{
|
||||
{
|
||||
name: "singular warning key",
|
||||
data: []interface{}{
|
||||
"total_results", int64(0),
|
||||
"results", []interface{}{},
|
||||
"warning", []interface{}{"Timeout limit was reached"},
|
||||
},
|
||||
expected: []string{"Timeout limit was reached"},
|
||||
},
|
||||
{
|
||||
name: "plural warnings key",
|
||||
data: []interface{}{
|
||||
"total_results", int64(0),
|
||||
"results", []interface{}{},
|
||||
"warnings", []interface{}{"Timeout limit was reached"},
|
||||
},
|
||||
expected: []string{"Timeout limit was reached"},
|
||||
},
|
||||
{
|
||||
name: "no warnings",
|
||||
data: []interface{}{
|
||||
"total_results", int64(0),
|
||||
"results", []interface{}{},
|
||||
},
|
||||
expected: nil,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
result, _, err := parseFTHybrid(tt.data, false)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if len(result.Warnings) != len(tt.expected) {
|
||||
t.Fatalf("len(Warnings) = %d, want %d (%v)", len(result.Warnings), len(tt.expected), result.Warnings)
|
||||
}
|
||||
for i, w := range result.Warnings {
|
||||
if w != tt.expected[i] {
|
||||
t.Errorf("Warnings[%d] = %q, want %q", i, w, tt.expected[i])
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
146
search_test.go
146
search_test.go
@@ -409,7 +409,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
// up until redis 8 the default scorer was TFIDF, in redis 8 it is BM25
|
||||
// this test expect redis major version >= 8
|
||||
It("should FTSearch WithScores", Label("search", "ftsearch"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "default scorer is not BM25STD")
|
||||
SkipBeforeRedisVersion("7.9", "default scorer is not BM25STD")
|
||||
|
||||
text1 := &redis.FieldSchema{FieldName: "description", FieldType: redis.SearchFieldTypeText}
|
||||
val, err := client.FTCreate(ctx, "idx1", &redis.FTCreateOptions{}, text1).Result()
|
||||
@@ -452,7 +452,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
// up until redis 8 the default scorer was TFIDF, in redis 8 it is BM25
|
||||
// this test expect redis version < 8.0
|
||||
It("should FTSearch WithScores", Label("search", "ftsearch"), func() {
|
||||
SkipAfterRedisVersion(7.9, "default scorer is not TFIDF")
|
||||
SkipAfterRedisVersion("7.9", "default scorer is not TFIDF")
|
||||
text1 := &redis.FieldSchema{FieldName: "description", FieldType: redis.SearchFieldTypeText}
|
||||
val, err := client.FTCreate(ctx, "idx1", &redis.FTCreateOptions{}, text1).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
@@ -695,7 +695,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should FTAggregate with scorer and addscores", Label("search", "ftaggregate", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "no addscores support")
|
||||
SkipBeforeRedisVersion("7.4", "no addscores support")
|
||||
title := &redis.FieldSchema{FieldName: "title", FieldType: redis.SearchFieldTypeText, Sortable: false}
|
||||
description := &redis.FieldSchema{FieldName: "description", FieldType: redis.SearchFieldTypeText, Sortable: false}
|
||||
val, err := client.FTCreate(ctx, "idx1", &redis.FTCreateOptions{OnHash: true, Prefix: []interface{}{"product:"}}, title, description).Result()
|
||||
@@ -1508,7 +1508,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test dialect 4", Label("search", "ftcreate", "ftsearch", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "doesn't work with older redis stack images")
|
||||
SkipBeforeRedisVersion("7.4", "doesn't work with older redis stack images")
|
||||
val, err := client.FTCreate(ctx, "idx1", &redis.FTCreateOptions{
|
||||
Prefix: []interface{}{"resource:"},
|
||||
}, &redis.FieldSchema{
|
||||
@@ -1641,7 +1641,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should create search index with FLOAT16 and BFLOAT16 vectors", Label("search", "ftcreate", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "doesn't work with older redis stack images")
|
||||
SkipBeforeRedisVersion("7.4", "doesn't work with older redis stack images")
|
||||
val, err := client.FTCreate(ctx, "index", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "float16", FieldType: redis.SearchFieldTypeVector, VectorArgs: &redis.FTVectorArgs{FlatOptions: &redis.FTFlatOptions{Type: "FLOAT16", Dim: 768, DistanceMetric: "COSINE"}}},
|
||||
&redis.FieldSchema{FieldName: "bfloat16", FieldType: redis.SearchFieldTypeVector, VectorArgs: &redis.FTVectorArgs{FlatOptions: &redis.FTFlatOptions{Type: "BFLOAT16", Dim: 768, DistanceMetric: "COSINE"}}},
|
||||
@@ -1652,7 +1652,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test geoshapes query intersects and disjoint", Label("NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "doesn't work with older redis stack images")
|
||||
SkipBeforeRedisVersion("7.4", "doesn't work with older redis stack images")
|
||||
_, err := client.FTCreate(ctx, "idx1", &redis.FTCreateOptions{}, &redis.FieldSchema{
|
||||
FieldName: "g",
|
||||
FieldType: redis.SearchFieldTypeGeoShape,
|
||||
@@ -1721,7 +1721,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should search missing fields", Label("search", "ftcreate", "ftsearch", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "doesn't work with older redis stack images")
|
||||
SkipBeforeRedisVersion("7.4", "doesn't work with older redis stack images")
|
||||
val, err := client.FTCreate(ctx, "idx1", &redis.FTCreateOptions{Prefix: []interface{}{"property:"}},
|
||||
&redis.FieldSchema{FieldName: "title", FieldType: redis.SearchFieldTypeText, Sortable: true},
|
||||
&redis.FieldSchema{FieldName: "features", FieldType: redis.SearchFieldTypeTag, IndexMissing: true},
|
||||
@@ -1766,7 +1766,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should search empty fields", Label("search", "ftcreate", "ftsearch", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "doesn't work with older redis stack images")
|
||||
SkipBeforeRedisVersion("7.4", "doesn't work with older redis stack images")
|
||||
val, err := client.FTCreate(ctx, "idx1", &redis.FTCreateOptions{Prefix: []interface{}{"property:"}},
|
||||
&redis.FieldSchema{FieldName: "title", FieldType: redis.SearchFieldTypeText, Sortable: true},
|
||||
&redis.FieldSchema{FieldName: "features", FieldType: redis.SearchFieldTypeTag, IndexEmpty: true},
|
||||
@@ -1813,7 +1813,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should FTCreate VECTOR with int8 and uint8 types", Label("search", "ftcreate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "doesn't work with older redis")
|
||||
SkipBeforeRedisVersion("7.9", "doesn't work with older redis")
|
||||
// Define INT8 vector field
|
||||
hnswOptionsInt8 := &redis.FTHNSWOptions{
|
||||
Type: "INT8",
|
||||
@@ -1869,7 +1869,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return special float scores in FT.SEARCH vecsim", Label("search", "ftsearch", "vecsim"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "doesn't work with older redis stack images")
|
||||
SkipBeforeRedisVersion("7.4", "doesn't work with older redis stack images")
|
||||
|
||||
vecField := &redis.FTFlatOptions{
|
||||
Type: "FLOAT32",
|
||||
@@ -1919,7 +1919,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should FTCreate VECTOR with VAMANA algorithm - basic", Label("search", "ftcreate"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 2,
|
||||
@@ -1949,7 +1949,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should FTCreate VECTOR with VAMANA algorithm - with compression", Label("search", "ftcreate"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT16",
|
||||
Dim: 256,
|
||||
@@ -1966,7 +1966,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should FTCreate VECTOR with VAMANA algorithm - advanced parameters", Label("search", "ftcreate"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 512,
|
||||
@@ -2041,7 +2041,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA L2 distance metric", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 3,
|
||||
@@ -2080,7 +2080,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA COSINE distance metric", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 3,
|
||||
@@ -2118,7 +2118,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA IP distance metric", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 3,
|
||||
@@ -2156,7 +2156,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA basic functionality", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 4,
|
||||
@@ -2194,7 +2194,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA FLOAT16 type", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT16",
|
||||
Dim: 4,
|
||||
@@ -2229,7 +2229,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA FLOAT32 type", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 4,
|
||||
@@ -2264,7 +2264,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA with default dialect", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 2,
|
||||
@@ -2292,7 +2292,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA with LVQ8 compression", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 8,
|
||||
@@ -2329,7 +2329,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA compression with both vector types", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
|
||||
// Test FLOAT16 with LVQ8
|
||||
vamanaOptions16 := &redis.FTVamanaOptions{
|
||||
@@ -2395,7 +2395,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA construction window size", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 6,
|
||||
@@ -2431,7 +2431,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA graph max degree", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 6,
|
||||
@@ -2467,7 +2467,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA search window size", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 6,
|
||||
@@ -2503,7 +2503,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA all advanced parameters", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 8,
|
||||
@@ -2544,7 +2544,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should fail when using a non-zero offset with a zero limit", Label("search", "ftsearch"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "testIdx", &redis.FTCreateOptions{}, &redis.FieldSchema{
|
||||
FieldName: "txt",
|
||||
FieldType: redis.SearchFieldTypeText,
|
||||
@@ -2564,7 +2564,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should evaluate exponentiation precedence in APPLY expressions correctly", Label("search", "ftaggregate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "txns", &redis.FTCreateOptions{}, &redis.FieldSchema{
|
||||
FieldName: "dummy",
|
||||
FieldType: redis.SearchFieldTypeText,
|
||||
@@ -2588,7 +2588,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return a syntax error when empty strings are used for numeric parameters", Label("search", "ftsearch"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "idx", &redis.FTCreateOptions{}, &redis.FieldSchema{
|
||||
FieldName: "n",
|
||||
FieldType: redis.SearchFieldTypeNumeric,
|
||||
@@ -2611,7 +2611,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return NaN as default for AVG reducer when no numeric values are present", Label("search", "ftaggregate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "aggTestAvg", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "grp", FieldType: redis.SearchFieldTypeText},
|
||||
&redis.FieldSchema{FieldName: "n", FieldType: redis.SearchFieldTypeNumeric},
|
||||
@@ -2637,7 +2637,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return 1 as default for COUNT reducer when no numeric values are present", Label("search", "ftaggregate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "aggTestCount", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "grp", FieldType: redis.SearchFieldTypeText},
|
||||
&redis.FieldSchema{FieldName: "n", FieldType: redis.SearchFieldTypeNumeric},
|
||||
@@ -2663,7 +2663,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return NaN as default for SUM reducer when no numeric values are present", Label("search", "ftaggregate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "aggTestSum", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "grp", FieldType: redis.SearchFieldTypeText},
|
||||
&redis.FieldSchema{FieldName: "n", FieldType: redis.SearchFieldTypeNumeric},
|
||||
@@ -2689,7 +2689,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return the full requested number of results by re-running the query when some results expire", Label("search", "ftsearch"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "aggExpired", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "order", FieldType: redis.SearchFieldTypeNumeric, Sortable: true},
|
||||
).Result()
|
||||
@@ -2722,7 +2722,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should stop processing and return an error when a timeout occurs", Label("search", "ftaggregate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "aggTimeoutHeavy", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "n", FieldType: redis.SearchFieldTypeNumeric, Sortable: true},
|
||||
).Result()
|
||||
@@ -2752,7 +2752,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return 0 as default for COUNT_DISTINCT reducer when no values are present", Label("search", "ftaggregate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "aggTestCountDistinct", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "grp", FieldType: redis.SearchFieldTypeText},
|
||||
&redis.FieldSchema{FieldName: "x", FieldType: redis.SearchFieldTypeText},
|
||||
@@ -2778,7 +2778,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return 0 as default for COUNT_DISTINCTISH reducer when no values are present", Label("search", "ftaggregate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "aggTestCountDistinctIsh", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "grp", FieldType: redis.SearchFieldTypeText},
|
||||
&redis.FieldSchema{FieldName: "y", FieldType: redis.SearchFieldTypeText},
|
||||
@@ -2804,7 +2804,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should use BM25 as the default scorer", Label("search", "ftsearch"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "scoringTest", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "description", FieldType: redis.SearchFieldTypeText},
|
||||
).Result()
|
||||
@@ -2830,7 +2830,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return 0 as default for STDDEV reducer when no numeric values are present", Label("search", "ftaggregate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "aggTestStddev", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "grp", FieldType: redis.SearchFieldTypeText},
|
||||
&redis.FieldSchema{FieldName: "n", FieldType: redis.SearchFieldTypeNumeric},
|
||||
@@ -2857,7 +2857,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return NaN as default for QUANTILE reducer when no numeric values are present", Label("search", "ftaggregate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "aggTestQuantile", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "grp", FieldType: redis.SearchFieldTypeText},
|
||||
&redis.FieldSchema{FieldName: "n", FieldType: redis.SearchFieldTypeNumeric},
|
||||
@@ -2883,7 +2883,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return nil as default for FIRST_VALUE reducer when no values are present", Label("search", "ftaggregate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "aggTestFirstValue", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "grp", FieldType: redis.SearchFieldTypeText},
|
||||
&redis.FieldSchema{FieldName: "t", FieldType: redis.SearchFieldTypeText},
|
||||
@@ -2909,7 +2909,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should fail to add an alias that is an existing index name", Label("search", "ftalias"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "idx1", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "name", FieldType: redis.SearchFieldTypeText},
|
||||
).Result()
|
||||
@@ -2968,7 +2968,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should reject deprecated configuration keys", Label("search", "ftconfig"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
// List of deprecated configuration keys.
|
||||
deprecatedKeys := []string{
|
||||
"_FREE_RESOURCE_ON_THREAD",
|
||||
@@ -3032,7 +3032,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should return INF for MIN reducer and -INF for MAX reducer when no numeric values are present", Label("search", "ftaggregate"), func() {
|
||||
SkipBeforeRedisVersion(7.9, "requires Redis 8.x")
|
||||
SkipBeforeRedisVersion("7.9", "requires Redis 8.x")
|
||||
val, err := client.FTCreate(ctx, "aggTestMinMax", &redis.FTCreateOptions{},
|
||||
&redis.FieldSchema{FieldName: "grp", FieldType: redis.SearchFieldTypeText},
|
||||
&redis.FieldSchema{FieldName: "n", FieldType: redis.SearchFieldTypeNumeric},
|
||||
@@ -3061,7 +3061,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA with LVQ4 compression", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 8,
|
||||
@@ -3099,7 +3099,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA with LeanVec4x8 compression and reduce parameter", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 8,
|
||||
@@ -3138,7 +3138,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA compression algorithms with FLOAT16 type", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
|
||||
compressionAlgorithms := []string{"LVQ4", "LVQ4x4", "LVQ4x8", "LeanVec4x8", "LeanVec8x8"}
|
||||
|
||||
@@ -3185,7 +3185,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA compression algorithms with FLOAT32 type", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
|
||||
compressionAlgorithms := []string{"LVQ4", "LVQ4x4", "LVQ4x8", "LeanVec4x8", "LeanVec8x8"}
|
||||
|
||||
@@ -3232,7 +3232,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA compression with different distance metrics", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
|
||||
compressionAlgorithms := []string{"LVQ4", "LVQ4x4", "LVQ4x8", "LeanVec4x8", "LeanVec8x8"}
|
||||
distanceMetrics := []string{"L2", "COSINE", "IP"}
|
||||
@@ -3282,7 +3282,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA compression with all advanced parameters", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
|
||||
compressionAlgorithms := []string{"LVQ4", "LVQ4x4", "LVQ4x8", "LeanVec4x8", "LeanVec8x8"}
|
||||
|
||||
@@ -3333,7 +3333,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should fail when using reduce parameter with non-LeanVec compression", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 8,
|
||||
@@ -3349,7 +3349,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA with LVQ4 compression in RESP3", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 8,
|
||||
@@ -3387,7 +3387,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA with LeanVec4x8 compression and reduce parameter in RESP3", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
vamanaOptions := &redis.FTVamanaOptions{
|
||||
Type: "FLOAT32",
|
||||
Dim: 8,
|
||||
@@ -3426,7 +3426,7 @@ var _ = Describe("RediSearch commands Resp 2", Label("search"), func() {
|
||||
})
|
||||
|
||||
It("should test VAMANA compression algorithms with FLOAT16 type in RESP3", Label("search", "ftcreate", "vamana"), func() {
|
||||
SkipBeforeRedisVersion(8.2, "VAMANA requires Redis 8.2+")
|
||||
SkipBeforeRedisVersion("8.2", "VAMANA requires Redis 8.2+")
|
||||
|
||||
compressionAlgorithms := []string{"LVQ4", "LVQ4x4", "LVQ4x8", "LeanVec4x8", "LeanVec8x8"}
|
||||
|
||||
@@ -3678,7 +3678,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform basic hybrid search", Label("search", "fthybrid"), func() {
|
||||
SkipBeforeRedisVersion(8.4, "no support")
|
||||
SkipBeforeRedisVersion("8.4", "no support")
|
||||
// Basic hybrid search combining text and vector search
|
||||
searchQuery := "@color:{red}"
|
||||
vectorData := encodeFloat32Vector([]float32{-100, -200, -200, -300})
|
||||
@@ -3698,7 +3698,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with scorer", Label("search", "fthybrid", "scorer"), func() {
|
||||
SkipBeforeRedisVersion(8.4, "no support")
|
||||
SkipBeforeRedisVersion("8.4", "no support")
|
||||
// Test with TFIDF scorer
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
@@ -3734,7 +3734,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with vector filter", Label("search", "fthybrid", "filter"), func() {
|
||||
SkipBeforeRedisVersion(8.4, "no support")
|
||||
SkipBeforeRedisVersion("8.4", "no support")
|
||||
// This query won't have results from search, so we can validate vector filter
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
@@ -3777,7 +3777,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with KNN method", Label("search", "fthybrid", "knn"), func() {
|
||||
SkipBeforeRedisVersion(8.4, "no support")
|
||||
SkipBeforeRedisVersion("8.4", "no support")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -3802,7 +3802,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with RANGE method", Label("search", "fthybrid", "range"), func() {
|
||||
SkipBeforeRedisVersion(8.4, "no support")
|
||||
SkipBeforeRedisVersion("8.4", "no support")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -3829,7 +3829,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with LINEAR combine method", Label("search", "fthybrid", "combine"), func() {
|
||||
SkipBeforeRedisVersion(8.4, "no support")
|
||||
SkipBeforeRedisVersion("8.4", "no support")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -3859,7 +3859,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with RRF combine method", Label("search", "fthybrid", "rrf"), func() {
|
||||
SkipBeforeRedisVersion(8.4, "no support")
|
||||
SkipBeforeRedisVersion("8.4", "no support")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -3887,7 +3887,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with LOAD and APPLY", Label("search", "fthybrid", "load", "apply"), func() {
|
||||
SkipBeforeRedisVersion(8.4, "no support")
|
||||
SkipBeforeRedisVersion("8.4", "no support")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -3929,7 +3929,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with LIMIT", Label("search", "fthybrid", "limit"), func() {
|
||||
SkipBeforeRedisVersion(8.4, "no support")
|
||||
SkipBeforeRedisVersion("8.4", "no support")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -3953,7 +3953,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with SORTBY", Label("search", "fthybrid", "sortby"), func() {
|
||||
SkipBeforeRedisVersion(8.4, "no support")
|
||||
SkipBeforeRedisVersion("8.4", "no support")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -4002,7 +4002,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
|
||||
// Redis 8.6+ tests using PARAMS-based vector approach
|
||||
It("should perform basic hybrid search (8.6+ PARAMS)", Label("search", "fthybrid", "params"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "PARAMS-based vector support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "PARAMS-based vector support requires Redis 8.6+")
|
||||
// Basic hybrid search combining text and vector search
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
@@ -4031,7 +4031,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with scorer (8.6+ PARAMS)", Label("search", "fthybrid", "scorer", "params"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "PARAMS-based vector support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "PARAMS-based vector support requires Redis 8.6+")
|
||||
// Test with TFIDF scorer
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
@@ -4066,7 +4066,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with vector filter (8.6+ PARAMS)", Label("search", "fthybrid", "filter", "params"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "PARAMS-based vector support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "PARAMS-based vector support requires Redis 8.6+")
|
||||
// This query won't have results from search, so we can validate vector filter
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
@@ -4108,7 +4108,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with KNN method (8.6+ PARAMS)", Label("search", "fthybrid", "knn", "params"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "PARAMS-based vector support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "PARAMS-based vector support requires Redis 8.6+")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -4132,7 +4132,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with RANGE method (8.6+ PARAMS)", Label("search", "fthybrid", "range", "params"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "PARAMS-based vector support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "PARAMS-based vector support requires Redis 8.6+")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -4158,7 +4158,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with LINEAR combine method (8.6+ PARAMS)", Label("search", "fthybrid", "combine", "params"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "PARAMS-based vector support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "PARAMS-based vector support requires Redis 8.6+")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -4187,7 +4187,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with RRF combine method (8.6+ PARAMS)", Label("search", "fthybrid", "rrf", "params"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "PARAMS-based vector support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "PARAMS-based vector support requires Redis 8.6+")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -4216,7 +4216,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with LOAD and APPLY (8.6+ PARAMS)", Label("search", "fthybrid", "load", "apply", "params"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "PARAMS-based vector support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "PARAMS-based vector support requires Redis 8.6+")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -4257,7 +4257,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with LIMIT (8.6+ PARAMS)", Label("search", "fthybrid", "limit", "params"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "PARAMS-based vector support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "PARAMS-based vector support requires Redis 8.6+")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
@@ -4280,7 +4280,7 @@ var _ = Describe("FT.HYBRID Commands", func() {
|
||||
})
|
||||
|
||||
It("should perform hybrid search with SORTBY (8.6+ PARAMS)", Label("search", "fthybrid", "sortby", "params"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "PARAMS-based vector support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "PARAMS-based vector support requires Redis 8.6+")
|
||||
options := &redis.FTHybridOptions{
|
||||
CountExpressions: 2,
|
||||
SearchExpressions: []redis.FTHybridSearchExpression{
|
||||
|
||||
@@ -599,8 +599,8 @@ func masterReplicaDialer(
|
||||
}
|
||||
|
||||
netDialer := &net.Dialer{
|
||||
Timeout: failover.opt.DialTimeout,
|
||||
KeepAlive: 5 * time.Minute,
|
||||
Timeout: failover.opt.DialTimeout,
|
||||
KeepAliveConfig: defaultKeepAliveConfig,
|
||||
}
|
||||
if failover.opt.TLSConfig == nil {
|
||||
return netDialer.DialContext(ctx, network, addr)
|
||||
|
||||
@@ -12,6 +12,7 @@ type SetCmdable interface {
|
||||
SAdd(ctx context.Context, key string, members ...interface{}) *IntCmd
|
||||
SCard(ctx context.Context, key string) *IntCmd
|
||||
SDiff(ctx context.Context, keys ...string) *StringSliceCmd
|
||||
SDiffCard(ctx context.Context, opts *SDiffCardOptions, keys ...string) *IntCmd
|
||||
SDiffStore(ctx context.Context, destination string, keys ...string) *IntCmd
|
||||
SInter(ctx context.Context, keys ...string) *StringSliceCmd
|
||||
SInterCard(ctx context.Context, limit int64, keys ...string) *IntCmd
|
||||
@@ -28,9 +29,21 @@ type SetCmdable interface {
|
||||
SRem(ctx context.Context, key string, members ...interface{}) *IntCmd
|
||||
SScan(ctx context.Context, key string, cursor uint64, match string, count int64) *ScanCmd
|
||||
SUnion(ctx context.Context, keys ...string) *StringSliceCmd
|
||||
SUnionCard(ctx context.Context, opts *SUnionCardOptions, keys ...string) *IntCmd
|
||||
SUnionStore(ctx context.Context, destination string, keys ...string) *IntCmd
|
||||
}
|
||||
|
||||
// SUnionCardOptions are the options for SUnionCard.
|
||||
type SUnionCardOptions struct {
|
||||
Approx bool // use an approximate (HyperLogLog) count.
|
||||
Limit int64 // cap the result; 0 means no limit.
|
||||
}
|
||||
|
||||
// SDiffCardOptions are the options for SDiffCard.
|
||||
type SDiffCardOptions struct {
|
||||
Limit int64 // cap the result; 0 means no limit.
|
||||
}
|
||||
|
||||
// Returns the number of elements that were added to the set, not including all
|
||||
// the elements already present in the set.
|
||||
//
|
||||
@@ -96,6 +109,30 @@ func (c cmdable) SDiffStore(ctx context.Context, destination string, keys ...str
|
||||
return cmd
|
||||
}
|
||||
|
||||
// Returns the cardinality of the difference of the first set and the rest.
|
||||
// Missing keys are treated as empty sets.
|
||||
//
|
||||
// For more information about the command please refer to [SDIFFCARD].
|
||||
//
|
||||
// [SDIFFCARD]: (https://redis.io/docs/latest/commands/sdiffcard/)
|
||||
func (c cmdable) SDiffCard(ctx context.Context, opts *SDiffCardOptions, keys ...string) *IntCmd {
|
||||
if opts == nil {
|
||||
opts = &SDiffCardOptions{}
|
||||
}
|
||||
numKeys := len(keys)
|
||||
args := make([]interface{}, 0, 4+numKeys)
|
||||
args = append(args, "sdiffcard", numKeys)
|
||||
for _, key := range keys {
|
||||
args = append(args, key)
|
||||
}
|
||||
args = append(args, "limit", opts.Limit)
|
||||
cmd := NewIntCmd(ctx, args...)
|
||||
// Keys start after the numkeys arg: ["sdiffcard", numKeys, key1, ...].
|
||||
cmd.SetFirstKeyPos(2)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
// Returns the members of the set resulting from the intersection of all the given sets.
|
||||
// Keys that do not exist are considered to be empty sets.
|
||||
// With one of the keys being an empty set, the resulting set is also empty.
|
||||
@@ -328,6 +365,33 @@ func (c cmdable) SUnionStore(ctx context.Context, destination string, keys ...st
|
||||
return cmd
|
||||
}
|
||||
|
||||
// Returns the cardinality of the union of all the given sets.
|
||||
// Missing keys are treated as empty sets.
|
||||
//
|
||||
// For more information about the command please refer to [SUNIONCARD].
|
||||
//
|
||||
// [SUNIONCARD]: (https://redis.io/docs/latest/commands/sunioncard/)
|
||||
func (c cmdable) SUnionCard(ctx context.Context, opts *SUnionCardOptions, keys ...string) *IntCmd {
|
||||
if opts == nil {
|
||||
opts = &SUnionCardOptions{}
|
||||
}
|
||||
numKeys := len(keys)
|
||||
args := make([]interface{}, 0, 4+numKeys+1)
|
||||
args = append(args, "sunioncard", numKeys)
|
||||
for _, key := range keys {
|
||||
args = append(args, key)
|
||||
}
|
||||
if opts.Approx {
|
||||
args = append(args, "approx")
|
||||
}
|
||||
args = append(args, "limit", opts.Limit)
|
||||
cmd := NewIntCmd(ctx, args...)
|
||||
// Keys start after the numkeys arg: ["sunioncard", numKeys, key1, ...].
|
||||
cmd.SetFirstKeyPos(2)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
// Incrementally iterates the set elements stored at key.
|
||||
// This is a cursor-based iterator that allows scanning large sets efficiently.
|
||||
//
|
||||
|
||||
@@ -227,14 +227,16 @@ func (c cmdable) XRevRangeN(ctx context.Context, stream, start, stop string, cou
|
||||
}
|
||||
|
||||
type XReadArgs struct {
|
||||
Streams []string // list of streams and ids, e.g. stream1 stream2 id1 id2
|
||||
Count int64
|
||||
Block time.Duration
|
||||
ID string
|
||||
Streams []string // list of streams and ids, e.g. stream1 stream2 id1 id2
|
||||
Count int64
|
||||
MaxCount int64 // cumulative cap on total entries across all streams (Redis >= 8.10)
|
||||
MaxSize int64 // soft cumulative cap on total reply size in bytes across all streams (Redis >= 8.10)
|
||||
Block time.Duration
|
||||
ID string
|
||||
}
|
||||
|
||||
func (c cmdable) XRead(ctx context.Context, a *XReadArgs) *XStreamSliceCmd {
|
||||
args := make([]interface{}, 0, 2*len(a.Streams)+6)
|
||||
args := make([]interface{}, 0, 2*len(a.Streams)+10)
|
||||
args = append(args, "xread")
|
||||
|
||||
keyPos := int8(1)
|
||||
@@ -243,6 +245,14 @@ func (c cmdable) XRead(ctx context.Context, a *XReadArgs) *XStreamSliceCmd {
|
||||
args = append(args, a.Count)
|
||||
keyPos += 2
|
||||
}
|
||||
if a.MaxCount > 0 {
|
||||
args = append(args, "maxcount", a.MaxCount)
|
||||
keyPos += 2
|
||||
}
|
||||
if a.MaxSize > 0 {
|
||||
args = append(args, "maxsize", a.MaxSize)
|
||||
keyPos += 2
|
||||
}
|
||||
if a.Block >= 0 {
|
||||
args = append(args, "block")
|
||||
args = append(args, int64(a.Block/time.Millisecond))
|
||||
@@ -322,13 +332,15 @@ type XReadGroupArgs struct {
|
||||
Consumer string
|
||||
Streams []string // list of streams and ids, e.g. stream1 stream2 id1 id2
|
||||
Count int64
|
||||
MaxCount int64 // cumulative cap on total entries across all streams (Redis >= 8.10)
|
||||
MaxSize int64 // soft cumulative cap on total reply size in bytes across all streams (Redis >= 8.10)
|
||||
Block time.Duration
|
||||
NoAck bool
|
||||
Claim time.Duration // Claim idle pending entries older than this duration
|
||||
}
|
||||
|
||||
func (c cmdable) XReadGroup(ctx context.Context, a *XReadGroupArgs) *XStreamSliceCmd {
|
||||
args := make([]interface{}, 0, 10+len(a.Streams))
|
||||
args := make([]interface{}, 0, 14+len(a.Streams))
|
||||
args = append(args, "xreadgroup", "group", a.Group, a.Consumer)
|
||||
|
||||
keyPos := int8(4)
|
||||
@@ -336,6 +348,14 @@ func (c cmdable) XReadGroup(ctx context.Context, a *XReadGroupArgs) *XStreamSlic
|
||||
args = append(args, "count", a.Count)
|
||||
keyPos += 2
|
||||
}
|
||||
if a.MaxCount > 0 {
|
||||
args = append(args, "maxcount", a.MaxCount)
|
||||
keyPos += 2
|
||||
}
|
||||
if a.MaxSize > 0 {
|
||||
args = append(args, "maxsize", a.MaxSize)
|
||||
keyPos += 2
|
||||
}
|
||||
if a.Block >= 0 {
|
||||
args = append(args, "block", int64(a.Block/time.Millisecond))
|
||||
keyPos += 2
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"reflect"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// These tests assert the argument lists built by the XTRIM/XADD trimming
|
||||
@@ -194,3 +195,143 @@ func TestXAdd_TrimLimitArgs(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// These tests assert the argument lists built by XREAD for the MAXCOUNT and
|
||||
// MAXSIZE options (Redis >= 8.10) without dispatching to a server:
|
||||
// - both default to unset and emit no token when zero;
|
||||
// - when set they are emitted in canonical order, after COUNT and before
|
||||
// BLOCK, and before the STREAMS keys;
|
||||
// - the first-key position (used for cluster routing) accounts for the new
|
||||
// tokens, since they appear before STREAMS.
|
||||
func TestXRead_MaxArgs(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
args *XReadArgs
|
||||
want []interface{}
|
||||
wantPos int8
|
||||
}{
|
||||
{
|
||||
name: "no_max_options",
|
||||
args: &XReadArgs{Streams: []string{"s1", "0"}, Block: -1},
|
||||
want: []interface{}{"xread", "streams", "s1", "0"},
|
||||
// keyPos: 1 (xread) + 1 (streams) = 2
|
||||
wantPos: 2,
|
||||
},
|
||||
{
|
||||
name: "maxcount_only",
|
||||
args: &XReadArgs{Streams: []string{"s1", "s2", "0", "0"}, MaxCount: 80, Block: -1},
|
||||
want: []interface{}{"xread", "maxcount", int64(80), "streams", "s1", "s2", "0", "0"},
|
||||
// keyPos: 1 + 2 (maxcount) + 1 (streams) = 4
|
||||
wantPos: 4,
|
||||
},
|
||||
{
|
||||
name: "maxsize_only",
|
||||
args: &XReadArgs{Streams: []string{"s1", "0"}, MaxSize: 65536, Block: -1},
|
||||
want: []interface{}{"xread", "maxsize", int64(65536), "streams", "s1", "0"},
|
||||
// keyPos: 1 + 2 (maxsize) + 1 (streams) = 4
|
||||
wantPos: 4,
|
||||
},
|
||||
{
|
||||
name: "count_maxcount_maxsize_block",
|
||||
args: &XReadArgs{
|
||||
Streams: []string{"s1", "s2", "0", "0"},
|
||||
Count: 50,
|
||||
MaxCount: 80,
|
||||
MaxSize: 65536,
|
||||
Block: 5 * time.Second,
|
||||
},
|
||||
want: []interface{}{
|
||||
"xread", "count", int64(50), "maxcount", int64(80), "maxsize", int64(65536),
|
||||
"block", int64(5000), "streams", "s1", "s2", "0", "0",
|
||||
},
|
||||
// keyPos: 1 + 2 (count) + 2 (maxcount) + 2 (maxsize) + 2 (block) + 1 (streams) = 10
|
||||
wantPos: 10,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var captured Cmder
|
||||
c := captureCmdable(&captured)
|
||||
cmd := c.XRead(ctx, tt.args)
|
||||
if cmd == nil {
|
||||
t.Fatalf("XRead returned nil")
|
||||
}
|
||||
if !reflect.DeepEqual(cmd.Args(), tt.want) {
|
||||
t.Errorf("args mismatch\n got: %#v\nwant: %#v", cmd.Args(), tt.want)
|
||||
}
|
||||
if got := cmd.firstKeyPos(); got != tt.wantPos {
|
||||
t.Errorf("firstKeyPos mismatch: got %d, want %d", got, tt.wantPos)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// These tests assert the argument lists built by XREADGROUP for the MAXCOUNT and
|
||||
// MAXSIZE options (Redis >= 8.10), including their interaction with the existing
|
||||
// NOACK and CLAIM options and the first-key position used for cluster routing.
|
||||
func TestXReadGroup_MaxArgs(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
args *XReadGroupArgs
|
||||
want []interface{}
|
||||
wantPos int8
|
||||
}{
|
||||
{
|
||||
name: "no_max_options",
|
||||
args: &XReadGroupArgs{Group: "g", Consumer: "c", Streams: []string{"s1", ">"}, Block: -1},
|
||||
want: []interface{}{"xreadgroup", "group", "g", "c", "streams", "s1", ">"},
|
||||
// keyPos: 4 (group/consumer) + 1 (streams) = 5
|
||||
wantPos: 5,
|
||||
},
|
||||
{
|
||||
name: "maxcount_only",
|
||||
args: &XReadGroupArgs{Group: "g", Consumer: "c", Streams: []string{"s1", ">"}, MaxCount: 80, Block: -1},
|
||||
want: []interface{}{"xreadgroup", "group", "g", "c", "maxcount", int64(80), "streams", "s1", ">"},
|
||||
// keyPos: 4 + 2 (maxcount) + 1 (streams) = 7
|
||||
wantPos: 7,
|
||||
},
|
||||
{
|
||||
name: "all_options",
|
||||
args: &XReadGroupArgs{
|
||||
Group: "g",
|
||||
Consumer: "c",
|
||||
Streams: []string{"s1", "s2", ">", ">"},
|
||||
Count: 50,
|
||||
MaxCount: 80,
|
||||
MaxSize: 65536,
|
||||
Block: 5 * time.Second,
|
||||
NoAck: true,
|
||||
Claim: 30 * time.Second,
|
||||
},
|
||||
want: []interface{}{
|
||||
"xreadgroup", "group", "g", "c", "count", int64(50), "maxcount", int64(80),
|
||||
"maxsize", int64(65536), "block", int64(5000), "noack", "claim", int64(30000),
|
||||
"streams", "s1", "s2", ">", ">",
|
||||
},
|
||||
// keyPos: 4 + 2 (count) + 2 (maxcount) + 2 (maxsize) + 2 (block) + 1 (noack) + 2 (claim) + 1 (streams) = 16
|
||||
wantPos: 16,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var captured Cmder
|
||||
c := captureCmdable(&captured)
|
||||
cmd := c.XReadGroup(ctx, tt.args)
|
||||
if cmd == nil {
|
||||
t.Fatalf("XReadGroup returned nil")
|
||||
}
|
||||
if !reflect.DeepEqual(cmd.Args(), tt.want) {
|
||||
t.Errorf("args mismatch\n got: %#v\nwant: %#v", cmd.Args(), tt.want)
|
||||
}
|
||||
if got := cmd.firstKeyPos(); got != tt.wantPos {
|
||||
t.Errorf("firstKeyPos mismatch: got %d, want %d", got, tt.wantPos)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/redis/go-redis/v9/internal/proto"
|
||||
"github.com/redis/go-redis/v9/internal/util"
|
||||
@@ -40,6 +41,27 @@ type TimeseriesCmdable interface {
|
||||
TSMRevRangeWithArgs(ctx context.Context, fromTimestamp int, toTimestamp int, filterExpr []string, options *TSMRevRangeOptions) *MapStringSliceInterfaceCmd
|
||||
TSMGet(ctx context.Context, filters []string) *MapStringSliceInterfaceCmd
|
||||
TSMGetWithArgs(ctx context.Context, filters []string, options *TSMGetOptions) *MapStringSliceInterfaceCmd
|
||||
TSNRange(ctx context.Context, keys []string, fromTimestamp interface{}, toTimestamp interface{}) *TSNRangePivotRowSliceCmd
|
||||
TSNRangeWithArgs(ctx context.Context, keys []string, fromTimestamp interface{}, toTimestamp interface{}, options *TSNRangeOptions) *TSNRangePivotRowSliceCmd
|
||||
TSNRevRange(ctx context.Context, keys []string, fromTimestamp interface{}, toTimestamp interface{}) *TSNRangePivotRowSliceCmd
|
||||
TSNRevRangeWithArgs(ctx context.Context, keys []string, fromTimestamp interface{}, toTimestamp interface{}, options *TSNRevRangeOptions) *TSNRangePivotRowSliceCmd
|
||||
TSRead(ctx context.Context, key string, timestamp interface{}) *TSTimestampValueSliceCmd
|
||||
TSReadWithArgs(ctx context.Context, key string, timestamp interface{}, options *TSReadOptions) *TSTimestampValueSliceCmd
|
||||
}
|
||||
|
||||
// TS.READ timestamp cursor sentinels.
|
||||
const (
|
||||
TSReadEarliest = "-" // read from the earliest sample
|
||||
TSReadLatest = "+" // latest sample, inclusive
|
||||
TSReadNew = "$" // only samples added after the call
|
||||
)
|
||||
|
||||
// TSReadOptions holds the optional TS.READ arguments.
|
||||
type TSReadOptions struct {
|
||||
Block bool // wait for samples (emits the BLOCK group)
|
||||
Timeout time.Duration // max wait; 0 blocks indefinitely
|
||||
MinCount int // unblock threshold; defaults to 1
|
||||
MaxCount int // reply cap; 0 is unlimited
|
||||
}
|
||||
|
||||
type TSOptions struct {
|
||||
@@ -145,6 +167,7 @@ func (a Aggregator) String() string {
|
||||
var (
|
||||
errTSMultiAggregationGroupBy = errors.New("redis: GROUPBY is not allowed when multiple aggregators are specified")
|
||||
errTSAggregationConflict = errors.New("redis: setting both Aggregator and Aggregators is not allowed; use Aggregators instead because Aggregator is deprecated")
|
||||
errTSExcludeEmptyGroupBy = errors.New("redis: EXCLUDEEMPTY is not allowed with GROUPBY")
|
||||
)
|
||||
|
||||
func formatAggregationArgs(aggregator Aggregator, aggregators []Aggregator) (string, int, error) {
|
||||
@@ -227,8 +250,10 @@ type TSMRangeOptions struct {
|
||||
BucketDuration int
|
||||
BucketTimestamp interface{}
|
||||
Empty bool
|
||||
GroupByLabel interface{}
|
||||
Reducer interface{}
|
||||
// ExcludeEmpty omits matching series that have no samples. Not allowed with GroupByLabel/Reducer. Redis 8.10+.
|
||||
ExcludeEmpty bool
|
||||
GroupByLabel interface{}
|
||||
Reducer interface{}
|
||||
}
|
||||
|
||||
type TSMRevRangeOptions struct {
|
||||
@@ -245,8 +270,10 @@ type TSMRevRangeOptions struct {
|
||||
BucketDuration int
|
||||
BucketTimestamp interface{}
|
||||
Empty bool
|
||||
GroupByLabel interface{}
|
||||
Reducer interface{}
|
||||
// ExcludeEmpty omits matching series that have no samples. Not allowed with GroupByLabel/Reducer. Redis 8.10+.
|
||||
ExcludeEmpty bool
|
||||
GroupByLabel interface{}
|
||||
Reducer interface{}
|
||||
}
|
||||
|
||||
type TSMGetOptions struct {
|
||||
@@ -255,6 +282,30 @@ type TSMGetOptions struct {
|
||||
SelectedLabels []interface{}
|
||||
}
|
||||
|
||||
type TSNRangeOptions struct {
|
||||
Latest bool
|
||||
FilterByTS []int
|
||||
FilterByValue []float64 // exactly two elements: [min, max]
|
||||
Count int
|
||||
Align interface{}
|
||||
Aggregators []Aggregator
|
||||
BucketDuration int
|
||||
BucketTimestamp interface{}
|
||||
Empty bool
|
||||
}
|
||||
|
||||
type TSNRevRangeOptions struct {
|
||||
Latest bool
|
||||
FilterByTS []int
|
||||
FilterByValue []float64 // exactly two elements: [min, max]
|
||||
Count int
|
||||
Align interface{}
|
||||
Aggregators []Aggregator
|
||||
BucketDuration int
|
||||
BucketTimestamp interface{}
|
||||
Empty bool
|
||||
}
|
||||
|
||||
// TSAdd - Adds one or more observations to a t-digest sketch.
|
||||
// For more information - https://redis.io/commands/ts.add/
|
||||
func (c cmdable) TSAdd(ctx context.Context, key string, timestamp interface{}, value float64) *IntCmd {
|
||||
@@ -790,6 +841,47 @@ func (c cmdable) TSRangeWithArgs(ctx context.Context, key string, fromTimestamp
|
||||
return cmd
|
||||
}
|
||||
|
||||
// TSRead - Returns samples at or after timestamp, in ascending order.
|
||||
// timestamp is a non-negative Unix-ms integer or a sentinel (TSReadEarliest,
|
||||
// TSReadLatest, TSReadNew).
|
||||
// For more information - https://redis.io/commands/ts.read/
|
||||
func (c cmdable) TSRead(ctx context.Context, key string, timestamp interface{}) *TSTimestampValueSliceCmd {
|
||||
args := []interface{}{"TS.READ", key, timestamp}
|
||||
cmd := newTSTimestampValueSliceCmd(ctx, args...)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
// TSReadWithArgs - TS.READ with the optional BLOCK and MAX_COUNT groups.
|
||||
// When options.Block is set it waits for options.MinCount samples or until
|
||||
// options.Timeout elapses. Blocking calls must not be used in a pipeline or MULTI.
|
||||
// For more information - https://redis.io/commands/ts.read/
|
||||
func (c cmdable) TSReadWithArgs(ctx context.Context, key string, timestamp interface{}, options *TSReadOptions) *TSTimestampValueSliceCmd {
|
||||
args := []interface{}{"TS.READ", key, timestamp}
|
||||
blocking := false
|
||||
var blockTimeout time.Duration
|
||||
if options != nil {
|
||||
if options.Block {
|
||||
blocking = true
|
||||
blockTimeout = options.Timeout
|
||||
minCount := options.MinCount
|
||||
if minCount <= 0 {
|
||||
minCount = 1
|
||||
}
|
||||
args = append(args, "BLOCK", formatMs(ctx, options.Timeout), minCount)
|
||||
}
|
||||
if options.MaxCount != 0 {
|
||||
args = append(args, "MAX_COUNT", options.MaxCount)
|
||||
}
|
||||
}
|
||||
cmd := newTSTimestampValueSliceCmd(ctx, args...)
|
||||
if blocking {
|
||||
cmd.setReadTimeout(blockTimeout)
|
||||
}
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
type TSTimestampValueSliceCmd struct {
|
||||
baseCmd
|
||||
val []TSTimestampValue
|
||||
@@ -896,11 +988,8 @@ func (c cmdable) TSMRange(ctx context.Context, fromTimestamp int, toTimestamp in
|
||||
return cmd
|
||||
}
|
||||
|
||||
// TSMRangeWithArgs - Returns a range of samples from multiple time-series keys with additional options.
|
||||
// This function allows for specifying additional options such as:
|
||||
// Latest, FilterByTS, FilterByValue, WithLabels, SelectedLabels,
|
||||
// Count, Align, Aggregator, BucketDuration, BucketTimestamp,
|
||||
// Empty, GroupByLabel and Reducer.
|
||||
// TSMRangeWithArgs - Returns a range of samples from multiple time-series keys.
|
||||
// Options are set via TSMRangeOptions.
|
||||
// For more information - https://redis.io/commands/ts.mrange/
|
||||
func (c cmdable) TSMRangeWithArgs(ctx context.Context, fromTimestamp int, toTimestamp int, filterExpr []string, options *TSMRangeOptions) *MapStringSliceInterfaceCmd {
|
||||
args := []interface{}{"TS.MRANGE", fromTimestamp, toTimestamp}
|
||||
@@ -953,12 +1042,20 @@ func (c cmdable) TSMRangeWithArgs(ctx context.Context, fromTimestamp int, toTime
|
||||
if options.Empty {
|
||||
args = append(args, "EMPTY")
|
||||
}
|
||||
if options.ExcludeEmpty {
|
||||
args = append(args, "EXCLUDEEMPTY")
|
||||
}
|
||||
}
|
||||
args = append(args, "FILTER")
|
||||
for _, f := range filterExpr {
|
||||
args = append(args, f)
|
||||
}
|
||||
if options != nil {
|
||||
if options.ExcludeEmpty && (options.GroupByLabel != nil || options.Reducer != nil) {
|
||||
cmd := NewMapStringSliceInterfaceCmd(ctx, args...)
|
||||
cmd.SetErr(errTSExcludeEmptyGroupBy)
|
||||
return cmd
|
||||
}
|
||||
if multiAggregationCount > 1 && (options.GroupByLabel != nil || options.Reducer != nil) {
|
||||
cmd := NewMapStringSliceInterfaceCmd(ctx, args...)
|
||||
cmd.SetErr(errTSMultiAggregationGroupBy)
|
||||
@@ -988,11 +1085,8 @@ func (c cmdable) TSMRevRange(ctx context.Context, fromTimestamp int, toTimestamp
|
||||
return cmd
|
||||
}
|
||||
|
||||
// TSMRevRangeWithArgs - Returns a range of samples from multiple time-series keys in reverse order with additional options.
|
||||
// This function allows for specifying additional options such as:
|
||||
// Latest, FilterByTS, FilterByValue, WithLabels, SelectedLabels,
|
||||
// Count, Align, Aggregator, BucketDuration, BucketTimestamp,
|
||||
// Empty, GroupByLabel and Reducer.
|
||||
// TSMRevRangeWithArgs - Returns a range of samples from multiple time-series keys in reverse order.
|
||||
// Options are set via TSMRevRangeOptions.
|
||||
// For more information - https://redis.io/commands/ts.mrevrange/
|
||||
func (c cmdable) TSMRevRangeWithArgs(ctx context.Context, fromTimestamp int, toTimestamp int, filterExpr []string, options *TSMRevRangeOptions) *MapStringSliceInterfaceCmd {
|
||||
args := []interface{}{"TS.MREVRANGE", fromTimestamp, toTimestamp}
|
||||
@@ -1045,12 +1139,20 @@ func (c cmdable) TSMRevRangeWithArgs(ctx context.Context, fromTimestamp int, toT
|
||||
if options.Empty {
|
||||
args = append(args, "EMPTY")
|
||||
}
|
||||
if options.ExcludeEmpty {
|
||||
args = append(args, "EXCLUDEEMPTY")
|
||||
}
|
||||
}
|
||||
args = append(args, "FILTER")
|
||||
for _, f := range filterExpr {
|
||||
args = append(args, f)
|
||||
}
|
||||
if options != nil {
|
||||
if options.ExcludeEmpty && (options.GroupByLabel != nil || options.Reducer != nil) {
|
||||
cmd := NewMapStringSliceInterfaceCmd(ctx, args...)
|
||||
cmd.SetErr(errTSExcludeEmptyGroupBy)
|
||||
return cmd
|
||||
}
|
||||
if multiAggregationCount > 1 && (options.GroupByLabel != nil || options.Reducer != nil) {
|
||||
cmd := NewMapStringSliceInterfaceCmd(ctx, args...)
|
||||
cmd.SetErr(errTSMultiAggregationGroupBy)
|
||||
@@ -1106,3 +1208,263 @@ func (c cmdable) TSMGetWithArgs(ctx context.Context, filters []string, options *
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
// TSNRangePivotRow represents a single row in the pivot response from TS.NRANGE / TS.NREVRANGE.
|
||||
// Timestamp is the row's timestamp, Values holds one float64 per input key in input-key order.
|
||||
// Missing samples and missing aggregation buckets are represented as NaN.
|
||||
type TSNRangePivotRow struct {
|
||||
Timestamp int64
|
||||
Values []float64
|
||||
}
|
||||
|
||||
type TSNRangePivotRowSliceCmd struct {
|
||||
baseCmd
|
||||
val []TSNRangePivotRow
|
||||
}
|
||||
|
||||
func newTSNRangePivotRowSliceCmd(ctx context.Context, args ...interface{}) *TSNRangePivotRowSliceCmd {
|
||||
return &TSNRangePivotRowSliceCmd{
|
||||
baseCmd: baseCmd{
|
||||
ctx: ctx,
|
||||
args: args,
|
||||
cmdType: CmdTypeTSNRangePivotRowSlice,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (cmd *TSNRangePivotRowSliceCmd) String() string {
|
||||
return cmdString(cmd, cmd.val)
|
||||
}
|
||||
|
||||
func (cmd *TSNRangePivotRowSliceCmd) SetVal(val []TSNRangePivotRow) {
|
||||
cmd.val = val
|
||||
}
|
||||
|
||||
func (cmd *TSNRangePivotRowSliceCmd) Result() ([]TSNRangePivotRow, error) {
|
||||
return cmd.val, cmd.err
|
||||
}
|
||||
|
||||
func (cmd *TSNRangePivotRowSliceCmd) Val() []TSNRangePivotRow {
|
||||
return cmd.val
|
||||
}
|
||||
|
||||
func (cmd *TSNRangePivotRowSliceCmd) readReply(rd *proto.Reader) error {
|
||||
n, err := rd.ReadArrayLen()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cmd.val = make([]TSNRangePivotRow, n)
|
||||
for i := 0; i < n; i++ {
|
||||
// Each row is a 2-element array: [timestamp, [value_0, value_1, ...]]
|
||||
if _, err = rd.ReadArrayLen(); err != nil {
|
||||
return err
|
||||
}
|
||||
timestamp, err := rd.ReadInt()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cmd.val[i].Timestamp = timestamp
|
||||
|
||||
valCount, err := rd.ReadArrayLen()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cmd.val[i].Values = make([]float64, valCount)
|
||||
for j := 0; j < valCount; j++ {
|
||||
s, err := rd.ReadString()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cmd.val[i].Values[j], err = util.ParseStringToFloat(s)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (cmd *TSNRangePivotRowSliceCmd) Clone() Cmder {
|
||||
var val []TSNRangePivotRow
|
||||
if cmd.val != nil {
|
||||
val = make([]TSNRangePivotRow, len(cmd.val))
|
||||
copy(val, cmd.val)
|
||||
for i := range cmd.val {
|
||||
if cmd.val[i].Values != nil {
|
||||
val[i].Values = make([]float64, len(cmd.val[i].Values))
|
||||
copy(val[i].Values, cmd.val[i].Values)
|
||||
}
|
||||
}
|
||||
}
|
||||
return &TSNRangePivotRowSliceCmd{
|
||||
baseCmd: cmd.cloneBaseCmd(),
|
||||
val: val,
|
||||
}
|
||||
}
|
||||
|
||||
// buildNRangeAggregationArgs validates and returns one aggregator string per key for TS.NRANGE / TS.NREVRANGE.
|
||||
// The number of aggregators must equal the number of keys. Aggregators are emitted as separate wire tokens.
|
||||
func buildNRangeAggregationArgs(keys []string, aggregators []Aggregator) ([]string, error) {
|
||||
if len(aggregators) != len(keys) {
|
||||
return nil, fmt.Errorf("redis: TS.NRANGE/TS.NREVRANGE requires exactly %d aggregator(s), got %d", len(keys), len(aggregators))
|
||||
}
|
||||
parts := make([]string, len(aggregators))
|
||||
for i, agg := range aggregators {
|
||||
if agg == Invalid {
|
||||
return nil, fmt.Errorf("redis: invalid timeseries aggregator at index %d: Invalid (%d)", i, agg)
|
||||
}
|
||||
s := agg.String()
|
||||
if s == "" {
|
||||
return nil, fmt.Errorf("redis: invalid timeseries aggregator at index %d: %d", i, agg)
|
||||
}
|
||||
parts[i] = s
|
||||
}
|
||||
return parts, nil
|
||||
}
|
||||
|
||||
// appendNRangeOptions appends optional TS.NRANGE / TS.NREVRANGE arguments to args.
|
||||
func appendNRangeOptions(
|
||||
args []interface{},
|
||||
keys []string,
|
||||
latest bool,
|
||||
filterByTS []int,
|
||||
filterByValue []float64,
|
||||
count int,
|
||||
align interface{},
|
||||
aggregators []Aggregator,
|
||||
bucketDuration int,
|
||||
bucketTimestamp interface{},
|
||||
empty bool,
|
||||
) ([]interface{}, error) {
|
||||
if latest {
|
||||
args = append(args, "LATEST")
|
||||
}
|
||||
if len(filterByTS) > 0 {
|
||||
args = append(args, "FILTER_BY_TS")
|
||||
for _, ts := range filterByTS {
|
||||
args = append(args, ts)
|
||||
}
|
||||
}
|
||||
if len(filterByValue) > 0 {
|
||||
if len(filterByValue) != 2 {
|
||||
return args, fmt.Errorf("redis: FILTER_BY_VALUE requires exactly 2 elements [min, max], got %d", len(filterByValue))
|
||||
}
|
||||
args = append(args, "FILTER_BY_VALUE", filterByValue[0], filterByValue[1])
|
||||
}
|
||||
if count != 0 {
|
||||
args = append(args, "COUNT", count)
|
||||
}
|
||||
if align != nil {
|
||||
args = append(args, "ALIGN", align)
|
||||
}
|
||||
if len(aggregators) > 0 {
|
||||
aggParts, err := buildNRangeAggregationArgs(keys, aggregators)
|
||||
if err != nil {
|
||||
return args, err
|
||||
}
|
||||
args = append(args, "AGGREGATION")
|
||||
for _, a := range aggParts {
|
||||
args = append(args, a)
|
||||
}
|
||||
if bucketDuration != 0 {
|
||||
args = append(args, bucketDuration)
|
||||
}
|
||||
if bucketTimestamp != nil {
|
||||
args = append(args, "BUCKETTIMESTAMP", bucketTimestamp)
|
||||
}
|
||||
if empty {
|
||||
args = append(args, "EMPTY")
|
||||
}
|
||||
}
|
||||
return args, nil
|
||||
}
|
||||
|
||||
// TSNRange - Queries multiple time-series keys and returns a pivot response in forward (ascending) order.
|
||||
// For more information - https://redis.io/commands/ts.nrange/
|
||||
func (c cmdable) TSNRange(ctx context.Context, keys []string, fromTimestamp interface{}, toTimestamp interface{}) *TSNRangePivotRowSliceCmd {
|
||||
args := make([]interface{}, 0, 3+len(keys))
|
||||
args = append(args, "TS.NRANGE", len(keys))
|
||||
for _, k := range keys {
|
||||
args = append(args, k)
|
||||
}
|
||||
args = append(args, fromTimestamp, toTimestamp)
|
||||
cmd := newTSNRangePivotRowSliceCmd(ctx, args...)
|
||||
cmd.SetFirstKeyPos(2)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
// TSNRangeWithArgs - Queries multiple time-series keys and returns a pivot response in forward (ascending) order with additional options.
|
||||
// This function allows for specifying additional options such as:
|
||||
// Latest, FilterByTS, FilterByValue, Count, Align, Aggregators, BucketDuration, BucketTimestamp and Empty.
|
||||
// Aggregators must contain exactly one entry per key; they are emitted as separate wire tokens.
|
||||
// For more information - https://redis.io/commands/ts.nrange/
|
||||
func (c cmdable) TSNRangeWithArgs(ctx context.Context, keys []string, fromTimestamp interface{}, toTimestamp interface{}, options *TSNRangeOptions) *TSNRangePivotRowSliceCmd {
|
||||
args := make([]interface{}, 0, 3+len(keys))
|
||||
args = append(args, "TS.NRANGE", len(keys))
|
||||
for _, k := range keys {
|
||||
args = append(args, k)
|
||||
}
|
||||
args = append(args, fromTimestamp, toTimestamp)
|
||||
if options != nil {
|
||||
var err error
|
||||
args, err = appendNRangeOptions(args, keys,
|
||||
options.Latest, options.FilterByTS, options.FilterByValue,
|
||||
options.Count, options.Align, options.Aggregators,
|
||||
options.BucketDuration, options.BucketTimestamp, options.Empty)
|
||||
if err != nil {
|
||||
cmd := newTSNRangePivotRowSliceCmd(ctx, args...)
|
||||
cmd.SetErr(err)
|
||||
return cmd
|
||||
}
|
||||
}
|
||||
cmd := newTSNRangePivotRowSliceCmd(ctx, args...)
|
||||
cmd.SetFirstKeyPos(2)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
// TSNRevRange - Queries multiple time-series keys and returns a pivot response in reverse (descending) order.
|
||||
// For more information - https://redis.io/commands/ts.nrevrange/
|
||||
func (c cmdable) TSNRevRange(ctx context.Context, keys []string, fromTimestamp interface{}, toTimestamp interface{}) *TSNRangePivotRowSliceCmd {
|
||||
args := make([]interface{}, 0, 3+len(keys))
|
||||
args = append(args, "TS.NREVRANGE", len(keys))
|
||||
for _, k := range keys {
|
||||
args = append(args, k)
|
||||
}
|
||||
args = append(args, fromTimestamp, toTimestamp)
|
||||
cmd := newTSNRangePivotRowSliceCmd(ctx, args...)
|
||||
cmd.SetFirstKeyPos(2)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
// TSNRevRangeWithArgs - Queries multiple time-series keys and returns a pivot response in reverse (descending) order with additional options.
|
||||
// This function allows for specifying additional options such as:
|
||||
// Latest, FilterByTS, FilterByValue, Count, Align, Aggregators, BucketDuration, BucketTimestamp and Empty.
|
||||
// Aggregators must contain exactly one entry per key; they are emitted as separate wire tokens.
|
||||
// For more information - https://redis.io/commands/ts.nrevrange/
|
||||
func (c cmdable) TSNRevRangeWithArgs(ctx context.Context, keys []string, fromTimestamp interface{}, toTimestamp interface{}, options *TSNRevRangeOptions) *TSNRangePivotRowSliceCmd {
|
||||
args := make([]interface{}, 0, 3+len(keys))
|
||||
args = append(args, "TS.NREVRANGE", len(keys))
|
||||
for _, k := range keys {
|
||||
args = append(args, k)
|
||||
}
|
||||
args = append(args, fromTimestamp, toTimestamp)
|
||||
if options != nil {
|
||||
var err error
|
||||
args, err = appendNRangeOptions(args, keys,
|
||||
options.Latest, options.FilterByTS, options.FilterByValue,
|
||||
options.Count, options.Align, options.Aggregators,
|
||||
options.BucketDuration, options.BucketTimestamp, options.Empty)
|
||||
if err != nil {
|
||||
cmd := newTSNRangePivotRowSliceCmd(ctx, args...)
|
||||
cmd.SetErr(err)
|
||||
return cmd
|
||||
}
|
||||
}
|
||||
cmd := newTSNRangePivotRowSliceCmd(ctx, args...)
|
||||
cmd.SetFirstKeyPos(2)
|
||||
_ = c(ctx, cmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"math"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
. "github.com/bsm/ginkgo/v2"
|
||||
. "github.com/bsm/gomega"
|
||||
@@ -45,7 +46,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should TSCreate and TSCreateWithArgs", Label("timeseries", "tscreate", "tscreateWithArgs", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "older redis stack has different results for timeseries module")
|
||||
SkipBeforeRedisVersion("7.4", "older redis stack has different results for timeseries module")
|
||||
result, err := client.TSCreate(ctx, "1").Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(result).To(BeEquivalentTo("OK"))
|
||||
@@ -142,7 +143,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
{Timestamp: 1013, Value: 10.0}}))
|
||||
})
|
||||
It("should TSAdd and TSAddWithArgs", Label("timeseries", "tsadd", "tsaddWithArgs", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "older redis stack has different results for timeseries module")
|
||||
SkipBeforeRedisVersion("7.4", "older redis stack has different results for timeseries module")
|
||||
result, err := client.TSAdd(ctx, "1", 1, 1).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(result).To(BeEquivalentTo(1))
|
||||
@@ -236,7 +237,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should TSAlter", Label("timeseries", "tsalter", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "older redis stack has different results for timeseries module")
|
||||
SkipBeforeRedisVersion("7.4", "older redis stack has different results for timeseries module")
|
||||
result, err := client.TSCreate(ctx, "1").Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(result).To(BeEquivalentTo("OK"))
|
||||
@@ -271,7 +272,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
if client.Options().Protocol == 2 {
|
||||
Expect(resultInfo["labels"].([]interface{})[0]).To(BeEquivalentTo([]interface{}{"Time", "Series"}))
|
||||
Expect(resultInfo["retentionTime"]).To(BeEquivalentTo(10))
|
||||
if RedisVersion >= 8 {
|
||||
if redisVersionAtLeast("8") {
|
||||
Expect(resultInfo["duplicatePolicy"]).To(BeEquivalentTo("block"))
|
||||
} else {
|
||||
// Older versions of Redis had a bug where the duplicate policy was not set correctly
|
||||
@@ -280,7 +281,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
} else {
|
||||
Expect(resultInfo["labels"].(map[interface{}]interface{})["Time"]).To(BeEquivalentTo("Series"))
|
||||
Expect(resultInfo["retentionTime"]).To(BeEquivalentTo(10))
|
||||
if RedisVersion >= 8 {
|
||||
if redisVersionAtLeast("8") {
|
||||
Expect(resultInfo["duplicatePolicy"]).To(BeEquivalentTo("block"))
|
||||
} else {
|
||||
// Older versions of Redis had a bug where the duplicate policy was not set correctly
|
||||
@@ -364,7 +365,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should TSIncrBy, TSIncrByWithArgs, TSDecrBy and TSDecrByWithArgs", Label("timeseries", "tsincrby", "tsdecrby", "tsincrbyWithArgs", "tsdecrbyWithArgs", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(7.4, "older redis stack has different results for timeseries module")
|
||||
SkipBeforeRedisVersion("7.4", "older redis stack has different results for timeseries module")
|
||||
for i := 0; i < 100; i++ {
|
||||
_, err := client.TSIncrBy(ctx, "1", 1).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
@@ -757,7 +758,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should TSRangeWithArgs support multiple aggregators", Label("timeseries", "tsrange", "tsrangeWithArgs", "aggregators", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.8, "multiple aggregators require Redis 8.8+")
|
||||
SkipBeforeRedisVersion("8.8", "multiple aggregators require Redis 8.8+")
|
||||
|
||||
_, err := client.TSCreate(ctx, "multi-range").Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
@@ -938,7 +939,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should TSRevRangeWithArgs support multiple aggregators", Label("timeseries", "tsrevrange", "tsrevrangeWithArgs", "aggregators", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.8, "multiple aggregators require Redis 8.8+")
|
||||
SkipBeforeRedisVersion("8.8", "multiple aggregators require Redis 8.8+")
|
||||
|
||||
_, err := client.TSCreate(ctx, "multi-revrange").Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
@@ -1153,7 +1154,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
}
|
||||
})
|
||||
It("should TSMRangeWithArgs support multiple aggregators", Label("timeseries", "tsmrange", "tsmrangeWithArgs", "aggregators", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.8, "multiple aggregators require Redis 8.8+")
|
||||
SkipBeforeRedisVersion("8.8", "multiple aggregators require Redis 8.8+")
|
||||
|
||||
_, err := client.TSCreateWithArgs(ctx, "multi-mrange-a", &redis.TSOptions{
|
||||
Labels: map[string]string{"type": "sensor", "name": "a"},
|
||||
@@ -1340,7 +1341,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should TSMRevRangeWithArgs support multiple aggregators", Label("timeseries", "tsmrevrange", "tsmrevrangeWithArgs", "aggregators", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.8, "multiple aggregators require Redis 8.8+")
|
||||
SkipBeforeRedisVersion("8.8", "multiple aggregators require Redis 8.8+")
|
||||
|
||||
_, err := client.TSCreateWithArgs(ctx, "multi-mrevrange-a", &redis.TSOptions{
|
||||
Labels: map[string]string{"type": "sensor", "name": "a"},
|
||||
@@ -1399,6 +1400,79 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
Expect(cmd.Err()).To(MatchError("redis: GROUPBY is not allowed when multiple aggregators are specified"))
|
||||
})
|
||||
|
||||
It("should TSMRange and TSMRevRange with ExcludeEmpty", Label("timeseries", "tsmrange", "tsmrevrange", "excludeempty", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion("8.10", "EXCLUDEEMPTY requires Redis 8.10+")
|
||||
|
||||
for _, key := range []string{"s", "t", "u"} {
|
||||
opt := &redis.TSOptions{Labels: map[string]string{"sensor": "1", "type": "demo"}}
|
||||
_, err := client.TSCreateWithArgs(ctx, key, opt).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
}
|
||||
_, err := client.TSMAdd(ctx, [][]interface{}{
|
||||
{"s", 100, 100}, {"t", 100, 100},
|
||||
{"s", 200, 200}, {"t", 300, 300},
|
||||
{"s", 400, 400}, {"t", 400, 400},
|
||||
{"u", 2000, 2000},
|
||||
}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
// Without ExcludeEmpty, "u" is returned with an empty samples array.
|
||||
result, err := client.TSMRange(ctx, 0, 500, []string{"sensor=1"}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(result).To(HaveLen(3))
|
||||
Expect(result).To(HaveKey("u"))
|
||||
|
||||
// With ExcludeEmpty, "u" is omitted because it has no samples in range.
|
||||
result, err = client.TSMRangeWithArgs(ctx, 0, 500, []string{"sensor=1"}, &redis.TSMRangeOptions{
|
||||
WithLabels: true,
|
||||
ExcludeEmpty: true,
|
||||
}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(result).To(HaveLen(2))
|
||||
Expect(result).To(HaveKey("s"))
|
||||
Expect(result).To(HaveKey("t"))
|
||||
Expect(result).NotTo(HaveKey("u"))
|
||||
|
||||
// ExcludeEmpty composes with AGGREGATION.
|
||||
result, err = client.TSMRangeWithArgs(ctx, 0, 500, []string{"sensor=1"}, &redis.TSMRangeOptions{
|
||||
Aggregator: redis.Min,
|
||||
BucketDuration: 100,
|
||||
ExcludeEmpty: true,
|
||||
}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(result).To(HaveLen(2))
|
||||
Expect(result).NotTo(HaveKey("u"))
|
||||
|
||||
// TSMRevRange with ExcludeEmpty behaves the same.
|
||||
result, err = client.TSMRevRangeWithArgs(ctx, 0, 500, []string{"sensor=1"}, &redis.TSMRevRangeOptions{
|
||||
ExcludeEmpty: true,
|
||||
}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(result).To(HaveLen(2))
|
||||
Expect(result).NotTo(HaveKey("u"))
|
||||
|
||||
// When every matching series is empty, the reply is empty.
|
||||
result, err = client.TSMRangeWithArgs(ctx, 1, 50, []string{"sensor=1"}, &redis.TSMRangeOptions{
|
||||
ExcludeEmpty: true,
|
||||
}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(result).To(BeEmpty())
|
||||
})
|
||||
It("should reject ExcludeEmpty combined with GroupBy", Label("timeseries", "tsmrange", "tsmrevrange", "excludeempty"), func() {
|
||||
mrangeCmd := client.TSMRangeWithArgs(ctx, 0, 500, []string{"sensor=1"}, &redis.TSMRangeOptions{
|
||||
ExcludeEmpty: true,
|
||||
GroupByLabel: "sensor",
|
||||
Reducer: "max",
|
||||
})
|
||||
Expect(mrangeCmd.Err()).To(MatchError("redis: EXCLUDEEMPTY is not allowed with GROUPBY"))
|
||||
|
||||
mrevrangeCmd := client.TSMRevRangeWithArgs(ctx, 0, 500, []string{"sensor=1"}, &redis.TSMRevRangeOptions{
|
||||
ExcludeEmpty: true,
|
||||
GroupByLabel: "sensor",
|
||||
Reducer: "max",
|
||||
})
|
||||
Expect(mrevrangeCmd.Err()).To(MatchError("redis: EXCLUDEEMPTY is not allowed with GROUPBY"))
|
||||
})
|
||||
It("should TSMRevRangeWithArgs Latest", Label("timeseries", "tsmrevrangeWithArgs", "tsmrevrangelatest", "NonRedisEnterprise"), func() {
|
||||
resultCreate, err := client.TSCreate(ctx, "a").Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
@@ -1454,7 +1528,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
|
||||
// NaN Value Support Tests
|
||||
It("should support NaN values in TSAdd and TSAddWithArgs", Label("timeseries", "tsadd", "nan", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "NaN support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "NaN support requires Redis 8.6+")
|
||||
|
||||
// Test basic NaN insertion with TSAdd
|
||||
result, err := client.TSAdd(ctx, "nan-test-1", 1000, math.NaN()).Result()
|
||||
@@ -1475,7 +1549,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should support NaN values in TSMAdd", Label("timeseries", "tsmadd", "nan", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "NaN support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "NaN support requires Redis 8.6+")
|
||||
|
||||
// Create time series
|
||||
_, err := client.TSCreate(ctx, "nan-madd-1").Result()
|
||||
@@ -1503,7 +1577,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should retrieve NaN values with TSGet and TSMGet", Label("timeseries", "tsget", "tsmget", "nan", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "NaN support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "NaN support requires Redis 8.6+")
|
||||
|
||||
// Add NaN values to multiple time series
|
||||
opt := &redis.TSOptions{Labels: map[string]string{"type": "sensor"}}
|
||||
@@ -1550,7 +1624,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should support NaN values in TSRange and TSRevRange", Label("timeseries", "tsrange", "tsrevrange", "nan", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "NaN support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "NaN support requires Redis 8.6+")
|
||||
|
||||
// Create time series with mixed NaN and regular values
|
||||
_, err := client.TSCreate(ctx, "mixed-values").Result()
|
||||
@@ -1590,7 +1664,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should support CountNaN and CountAll aggregators", Label("timeseries", "aggregator", "nan", "countnan", "countall", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "NaN aggregators require Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "NaN aggregators require Redis 8.6+")
|
||||
|
||||
// Create time series with mixed NaN and regular values
|
||||
_, err := client.TSCreate(ctx, "agg-test").Result()
|
||||
@@ -1630,7 +1704,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should ignore NaN values in existing aggregators", Label("timeseries", "aggregator", "nan", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "NaN support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "NaN support requires Redis 8.6+")
|
||||
|
||||
// Create time series with mixed NaN and regular values
|
||||
_, err := client.TSCreate(ctx, "agg-ignore-nan").Result()
|
||||
@@ -1700,7 +1774,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should support NaN values in TSMRange and TSMRevRange", Label("timeseries", "tsmrange", "tsmrevrange", "nan", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "NaN support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "NaN support requires Redis 8.6+")
|
||||
|
||||
// Create multiple time series with NaN values
|
||||
opt := &redis.TSOptions{Labels: map[string]string{"location": "sensor-room"}}
|
||||
@@ -1731,7 +1805,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should support NaN with CountNaN and CountAll in TSMRange", Label("timeseries", "tsmrange", "aggregator", "nan", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "NaN aggregators require Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "NaN aggregators require Redis 8.6+")
|
||||
|
||||
// Create multiple time series with NaN values
|
||||
opt := &redis.TSOptions{Labels: map[string]string{"device": "temp-sensor"}}
|
||||
@@ -1775,7 +1849,7 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
})
|
||||
|
||||
It("should handle duplicate policy with NaN values", Label("timeseries", "nan", "duplicatepolicy", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion(8.6, "NaN support requires Redis 8.6+")
|
||||
SkipBeforeRedisVersion("8.6", "NaN support requires Redis 8.6+")
|
||||
|
||||
// Test BLOCK duplicate policy with NaN (should work - just blocks duplicates)
|
||||
opt := &redis.TSOptions{DuplicatePolicy: "BLOCK"}
|
||||
@@ -1816,6 +1890,276 @@ var _ = Describe("RedisTimeseries commands", Label("timeseries"), func() {
|
||||
Expect(redis.Sum.String()).To(Equal("SUM"))
|
||||
Expect(redis.Count.String()).To(Equal("COUNT"))
|
||||
})
|
||||
|
||||
It("should TSNRange, TSNRangeWithArgs", Label("timeseries", "tsnrange", "tsnrangeWithArgs", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion("8.10", "TS.NRANGE requires Redis 8.10+")
|
||||
|
||||
keys := []string{"{ts}:open", "{ts}:high", "{ts}:low"}
|
||||
for _, k := range keys {
|
||||
Expect(client.TSCreate(ctx, k).Err()).NotTo(HaveOccurred())
|
||||
}
|
||||
Expect(client.TSMAdd(ctx, [][]interface{}{
|
||||
{"{ts}:open", 1000, 10.0},
|
||||
{"{ts}:high", 1000, 13.0},
|
||||
{"{ts}:low", 1000, 9.0},
|
||||
{"{ts}:open", 2000, 12.0},
|
||||
{"{ts}:high", 2000, 15.0},
|
||||
// {ts}:low intentionally missing at 2000
|
||||
{"{ts}:open", 3000, 11.0},
|
||||
{"{ts}:high", 3000, 14.0},
|
||||
{"{ts}:low", 3000, 8.0},
|
||||
}).Err()).NotTo(HaveOccurred())
|
||||
|
||||
// Basic forward range
|
||||
result, err := client.TSNRange(ctx, keys, 1000, 3000).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(len(result)).To(Equal(3))
|
||||
Expect(result[0].Timestamp).To(BeEquivalentTo(1000))
|
||||
Expect(result[2].Timestamp).To(BeEquivalentTo(3000))
|
||||
// Ascending order
|
||||
Expect(result[0].Timestamp).To(BeNumerically("<", result[1].Timestamp))
|
||||
Expect(result[1].Timestamp).To(BeNumerically("<", result[2].Timestamp))
|
||||
// Values follow key order
|
||||
Expect(result[0].Values[0]).To(BeEquivalentTo(10.0)) // open
|
||||
Expect(result[0].Values[1]).To(BeEquivalentTo(13.0)) // high
|
||||
Expect(result[0].Values[2]).To(BeEquivalentTo(9.0)) // low
|
||||
// Missing low at t=2000 is NaN
|
||||
Expect(math.IsNaN(result[1].Values[2])).To(BeTrue())
|
||||
|
||||
// COUNT limits rows
|
||||
opt := &redis.TSNRangeOptions{Count: 2}
|
||||
resultCount, err := client.TSNRangeWithArgs(ctx, keys, 1000, 3000, opt).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(len(resultCount)).To(Equal(2))
|
||||
// COUNT limits in ascending order: first two rows
|
||||
Expect(resultCount[0].Timestamp).To(BeEquivalentTo(1000))
|
||||
Expect(resultCount[1].Timestamp).To(BeEquivalentTo(2000))
|
||||
|
||||
// FILTER_BY_TS
|
||||
opt = &redis.TSNRangeOptions{FilterByTS: []int{1000, 3000}}
|
||||
resultFilter, err := client.TSNRangeWithArgs(ctx, keys, 1000, 3000, opt).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(len(resultFilter)).To(Equal(2))
|
||||
Expect(resultFilter[0].Timestamp).To(BeEquivalentTo(1000))
|
||||
Expect(resultFilter[1].Timestamp).To(BeEquivalentTo(3000))
|
||||
|
||||
// Sentinel timestamps ("-" and "+")
|
||||
resultSentinel, err := client.TSNRange(ctx, keys, "-", "+").Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(len(resultSentinel)).To(Equal(3))
|
||||
|
||||
// Aggregation: one aggregator per key
|
||||
optAgg := &redis.TSNRangeOptions{
|
||||
Aggregators: []redis.Aggregator{redis.Min, redis.Max, redis.Sum},
|
||||
BucketDuration: 2000,
|
||||
}
|
||||
resultAgg, err := client.TSNRangeWithArgs(ctx, keys, 1000, 3000, optAgg).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(len(resultAgg)).To(BeNumerically(">", 0))
|
||||
// Each row has exactly 3 values (one per key)
|
||||
for _, row := range resultAgg {
|
||||
Expect(len(row.Values)).To(Equal(3))
|
||||
}
|
||||
|
||||
// EMPTY flag (emit empty buckets)
|
||||
optEmpty := &redis.TSNRangeOptions{
|
||||
Aggregators: []redis.Aggregator{redis.Min, redis.Max, redis.Sum},
|
||||
BucketDuration: 500,
|
||||
Empty: true,
|
||||
}
|
||||
resultEmpty, err := client.TSNRangeWithArgs(ctx, keys, 1000, 3000, optEmpty).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(len(resultEmpty)).To(BeNumerically(">=", 3))
|
||||
})
|
||||
|
||||
It("should TSNRevRange, TSNRevRangeWithArgs", Label("timeseries", "tsnrevrange", "tsnrevrangeWithArgs", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion("8.10", "TS.NREVRANGE requires Redis 8.10+")
|
||||
|
||||
keys := []string{"{ts2}:a", "{ts2}:b"}
|
||||
for _, k := range keys {
|
||||
Expect(client.TSCreate(ctx, k).Err()).NotTo(HaveOccurred())
|
||||
}
|
||||
Expect(client.TSMAdd(ctx, [][]interface{}{
|
||||
{"{ts2}:a", 1000, 1.0},
|
||||
{"{ts2}:b", 1000, 2.0},
|
||||
{"{ts2}:a", 2000, 3.0},
|
||||
{"{ts2}:b", 2000, 4.0},
|
||||
{"{ts2}:a", 3000, 5.0},
|
||||
{"{ts2}:b", 3000, 6.0},
|
||||
}).Err()).NotTo(HaveOccurred())
|
||||
|
||||
// Basic reverse range: rows in descending order
|
||||
result, err := client.TSNRevRange(ctx, keys, 1000, 3000).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(len(result)).To(Equal(3))
|
||||
Expect(result[0].Timestamp).To(BeEquivalentTo(3000))
|
||||
Expect(result[1].Timestamp).To(BeEquivalentTo(2000))
|
||||
Expect(result[2].Timestamp).To(BeEquivalentTo(1000))
|
||||
// Descending order
|
||||
Expect(result[0].Timestamp).To(BeNumerically(">", result[1].Timestamp))
|
||||
Expect(result[1].Timestamp).To(BeNumerically(">", result[2].Timestamp))
|
||||
|
||||
// COUNT limits rows in descending order: most recent rows first
|
||||
opt := &redis.TSNRevRangeOptions{Count: 2}
|
||||
resultCount, err := client.TSNRevRangeWithArgs(ctx, keys, 1000, 3000, opt).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(len(resultCount)).To(Equal(2))
|
||||
Expect(resultCount[0].Timestamp).To(BeEquivalentTo(3000))
|
||||
Expect(resultCount[1].Timestamp).To(BeEquivalentTo(2000))
|
||||
|
||||
// Aggregation in reverse order
|
||||
optAgg := &redis.TSNRevRangeOptions{
|
||||
Aggregators: []redis.Aggregator{redis.Max, redis.Min},
|
||||
BucketDuration: 2000,
|
||||
}
|
||||
resultAgg, err := client.TSNRevRangeWithArgs(ctx, keys, 1000, 3000, optAgg).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(len(resultAgg)).To(BeNumerically(">", 0))
|
||||
for _, row := range resultAgg {
|
||||
Expect(len(row.Values)).To(Equal(2))
|
||||
}
|
||||
// Reverse: last bucket row first
|
||||
if len(resultAgg) > 1 {
|
||||
Expect(resultAgg[0].Timestamp).To(BeNumerically(">", resultAgg[1].Timestamp))
|
||||
}
|
||||
})
|
||||
|
||||
It("should TSNRange preserve key order and support duplicate keys", Label("timeseries", "tsnrange", "duplicatekeys", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion("8.10", "TS.NRANGE requires Redis 8.10+")
|
||||
|
||||
Expect(client.TSCreate(ctx, "{dup}:x").Err()).NotTo(HaveOccurred())
|
||||
Expect(client.TSMAdd(ctx, [][]interface{}{
|
||||
{"{dup}:x", 1000, 5.0},
|
||||
{"{dup}:x", 2000, 10.0},
|
||||
}).Err()).NotTo(HaveOccurred())
|
||||
|
||||
// Same key twice with different aggregators: two value columns from one physical series
|
||||
dupKeys := []string{"{dup}:x", "{dup}:x"}
|
||||
optAgg := &redis.TSNRangeOptions{
|
||||
Aggregators: []redis.Aggregator{redis.Min, redis.Max},
|
||||
BucketDuration: 2000,
|
||||
}
|
||||
result, err := client.TSNRangeWithArgs(ctx, dupKeys, 1000, 2000, optAgg).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(len(result)).To(BeNumerically(">", 0))
|
||||
for _, row := range result {
|
||||
Expect(len(row.Values)).To(Equal(2))
|
||||
}
|
||||
})
|
||||
|
||||
It("should TSNRangeWithArgs validate aggregator count", Label("timeseries", "tsnrange", "validation", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion("8.10", "TS.NRANGE requires Redis 8.10+")
|
||||
|
||||
keys := []string{"{val}:a", "{val}:b", "{val}:c"}
|
||||
// 2 aggregators for 3 keys: must error
|
||||
cmd := client.TSNRangeWithArgs(ctx, keys, 0, 1000, &redis.TSNRangeOptions{
|
||||
Aggregators: []redis.Aggregator{redis.Min, redis.Max},
|
||||
BucketDuration: 100,
|
||||
})
|
||||
Expect(cmd.Err()).To(MatchError(ContainSubstring("requires exactly 3 aggregator(s), got 2")))
|
||||
})
|
||||
|
||||
It("should TSNRangeWithArgs reject Invalid aggregators", Label("timeseries", "tsnrange", "validation", "NonRedisEnterprise"), func() {
|
||||
SkipBeforeRedisVersion("8.10", "TS.NRANGE requires Redis 8.10+")
|
||||
|
||||
keys := []string{"{val2}:a", "{val2}:b"}
|
||||
cmd := client.TSNRangeWithArgs(ctx, keys, 0, 1000, &redis.TSNRangeOptions{
|
||||
Aggregators: []redis.Aggregator{redis.Min, redis.Invalid},
|
||||
BucketDuration: 100,
|
||||
})
|
||||
Expect(cmd.Err()).To(MatchError(ContainSubstring("invalid timeseries aggregator at index 1: Invalid")))
|
||||
})
|
||||
|
||||
It("should TSRead and TSReadWithArgs", Label("timeseries", "tsread", "tsreadWithArgs", "NonRedisEnterprise"), func() {
|
||||
// TS.READ was added in Redis 8.10.
|
||||
SkipBeforeRedisVersion("8.10", "TS.READ was added in Redis 8.10")
|
||||
|
||||
_, err := client.TSCreate(ctx, "tsread:1").Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
for _, s := range []redis.TSTimestampValue{
|
||||
{Timestamp: 100, Value: 1.0},
|
||||
{Timestamp: 200, Value: 2.0},
|
||||
{Timestamp: 300, Value: 3.0},
|
||||
} {
|
||||
_, err := client.TSAdd(ctx, "tsread:1", s.Timestamp, s.Value).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
}
|
||||
|
||||
// Read everything at or after the cursor.
|
||||
samples, err := client.TSRead(ctx, "tsread:1", 0).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(samples).To(Equal([]redis.TSTimestampValue{
|
||||
{Timestamp: 100, Value: 1.0},
|
||||
{Timestamp: 200, Value: 2.0},
|
||||
{Timestamp: 300, Value: 3.0},
|
||||
}))
|
||||
|
||||
// Page in bounded batches, advancing the cursor to lastTimestamp + 1.
|
||||
page, err := client.TSReadWithArgs(ctx, "tsread:1", redis.TSReadEarliest, &redis.TSReadOptions{MaxCount: 2}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(page).To(Equal([]redis.TSTimestampValue{
|
||||
{Timestamp: 100, Value: 1.0},
|
||||
{Timestamp: 200, Value: 2.0},
|
||||
}))
|
||||
next := page[len(page)-1].Timestamp + 1
|
||||
page, err = client.TSReadWithArgs(ctx, "tsread:1", next, &redis.TSReadOptions{MaxCount: 2}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(page).To(Equal([]redis.TSTimestampValue{
|
||||
{Timestamp: 300, Value: 3.0},
|
||||
}))
|
||||
|
||||
// Past the newest sample, or a missing key, returns empty.
|
||||
empty, err := client.TSRead(ctx, "tsread:1", 301).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(empty).To(BeEmpty())
|
||||
empty, err = client.TSRead(ctx, "tsread-missing", 0).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(empty).To(BeEmpty())
|
||||
|
||||
// "+" returns the latest sample, inclusive, even without BLOCK.
|
||||
latest, err := client.TSRead(ctx, "tsread:1", redis.TSReadLatest).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(latest).To(Equal([]redis.TSTimestampValue{{Timestamp: 300, Value: 3.0}}))
|
||||
|
||||
// "$" without BLOCK always returns empty.
|
||||
fresh, err := client.TSRead(ctx, "tsread:1", redis.TSReadNew).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(fresh).To(BeEmpty())
|
||||
|
||||
// Timeout flush: MinCount can't be reached, so the samples >= 101
|
||||
// are returned after the timeout.
|
||||
flush, err := client.TSReadWithArgs(ctx, "tsread:1", 101, &redis.TSReadOptions{
|
||||
Block: true,
|
||||
Timeout: 500 * time.Millisecond,
|
||||
MinCount: 10,
|
||||
}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(flush).To(Equal([]redis.TSTimestampValue{
|
||||
{Timestamp: 200, Value: 2.0},
|
||||
{Timestamp: 300, Value: 3.0},
|
||||
}))
|
||||
|
||||
// Wake-up: a concurrent append unblocks a "$" tail read.
|
||||
producer := setupRedisClient(protocol)
|
||||
done := make(chan struct{})
|
||||
// Defers run LIFO: join the producer before closing its client.
|
||||
defer producer.Close()
|
||||
defer func() { <-done }()
|
||||
go func() {
|
||||
defer GinkgoRecover()
|
||||
defer close(done)
|
||||
time.Sleep(200 * time.Millisecond)
|
||||
_, addErr := producer.TSAdd(ctx, "tsread:1", 400, 4.0).Result()
|
||||
Expect(addErr).NotTo(HaveOccurred())
|
||||
}()
|
||||
tail, err := client.TSReadWithArgs(ctx, "tsread:1", redis.TSReadNew, &redis.TSReadOptions{
|
||||
Block: true,
|
||||
Timeout: 5 * time.Second,
|
||||
MinCount: 1,
|
||||
}).Result()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(tail).To(Equal([]redis.TSTimestampValue{{Timestamp: 400, Value: 4.0}}))
|
||||
})
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
@@ -5,7 +5,6 @@ import (
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -13,20 +12,18 @@ import (
|
||||
)
|
||||
|
||||
func init() {
|
||||
// Initialize RedisVersion from environment variable for regular Go tests
|
||||
// (Ginkgo tests initialize this in BeforeSuite)
|
||||
if version := os.Getenv("REDIS_VERSION"); version != "" {
|
||||
if v, err := strconv.ParseFloat(strings.Trim(version, "\""), 64); err == nil && v > 0 {
|
||||
RedisVersion = v
|
||||
}
|
||||
// Initialize version vars from environment for regular Go tests
|
||||
// (Ginkgo tests initialize these in BeforeSuite)
|
||||
if version := strings.Trim(os.Getenv("REDIS_VERSION"), "\""); version != "" {
|
||||
redisMajorVersion, redisMinorVersion = parseRedisVersionStr(version)
|
||||
}
|
||||
}
|
||||
|
||||
// skipBeforeRedisVersion checks if Redis version is below the specified version and skips the test if so
|
||||
func skipBeforeRedisVersion(t *testing.T, version float64, msg string) {
|
||||
func skipBeforeRedisVersion(t *testing.T, version string, msg string) {
|
||||
t.Helper()
|
||||
if RedisVersion < version {
|
||||
t.Skipf("Skipping test: Redis version %.1f < %.1f: %s", RedisVersion, version, msg)
|
||||
if !redisVersionAtLeast(version) {
|
||||
t.Skipf("Skipping test: redis version < %s: %s", version, msg)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -44,7 +41,7 @@ func skipBeforeRedisVersion(t *testing.T, version float64, msg string) {
|
||||
// 3. Connect using TLS with that certificate
|
||||
// 4. Verify that Redis automatically authenticates as that user (no AUTH command needed)
|
||||
func TestTLSCertificateAuthentication(t *testing.T) {
|
||||
skipBeforeRedisVersion(t, 8.6, "tls-auth-clients-user CN requires Redis 8.6+")
|
||||
skipBeforeRedisVersion(t, "8.6", "tls-auth-clients-user CN requires Redis 8.6+")
|
||||
|
||||
ctx := context.Background()
|
||||
testUsername := "testcertuser"
|
||||
@@ -174,7 +171,7 @@ func TestTLSCertificateAuthentication(t *testing.T) {
|
||||
// 2. Connects with a certificate that has CN=testcertuser
|
||||
// 3. Verifies that Redis authenticates as "default" (fallback behavior)
|
||||
func TestTLSCertificateAuthenticationNoUser(t *testing.T) {
|
||||
skipBeforeRedisVersion(t, 8.6, "tls-auth-clients-user CN requires Redis 8.6+")
|
||||
skipBeforeRedisVersion(t, "8.6", "tls-auth-clients-user CN requires Redis 8.6+")
|
||||
|
||||
ctx := context.Background()
|
||||
testUsername := "testcertuser"
|
||||
|
||||
@@ -63,7 +63,7 @@ var _ = Describe("Redis VectorSet commands", Label("vectorset"), func() {
|
||||
})
|
||||
|
||||
It("basic", func() {
|
||||
SkipBeforeRedisVersion(8.0, "Redis 8.0 introduces support for VectorSet")
|
||||
SkipBeforeRedisVersion("8.0", "Redis 8.0 introduces support for VectorSet")
|
||||
vecName := "basic"
|
||||
val := &redis.VectorValues{
|
||||
Val: []float64{1.5, 2.4, 3.3, 4.2},
|
||||
@@ -98,7 +98,7 @@ var _ = Describe("Redis VectorSet commands", Label("vectorset"), func() {
|
||||
})
|
||||
|
||||
It("basic similarity", func() {
|
||||
SkipBeforeRedisVersion(8.0, "Redis 8.0 introduces support for VectorSet")
|
||||
SkipBeforeRedisVersion("8.0", "Redis 8.0 introduces support for VectorSet")
|
||||
vecName := "basic_similarity"
|
||||
|
||||
ok, err := client.VAdd(ctx, vecName, "k1", &redis.VectorValues{
|
||||
@@ -132,7 +132,7 @@ var _ = Describe("Redis VectorSet commands", Label("vectorset"), func() {
|
||||
})
|
||||
|
||||
It("dimension operation", func() {
|
||||
SkipBeforeRedisVersion(8.0, "Redis 8.0 introduces support for VectorSet")
|
||||
SkipBeforeRedisVersion("8.0", "Redis 8.0 introduces support for VectorSet")
|
||||
vecName := "dimension_op"
|
||||
originalDim := 100
|
||||
reducedDim := 50
|
||||
@@ -167,7 +167,7 @@ var _ = Describe("Redis VectorSet commands", Label("vectorset"), func() {
|
||||
})
|
||||
|
||||
It("remove", func() {
|
||||
SkipBeforeRedisVersion(8.0, "Redis 8.0 introduces support for VectorSet")
|
||||
SkipBeforeRedisVersion("8.0", "Redis 8.0 introduces support for VectorSet")
|
||||
vecName := "remove"
|
||||
v1 := generateRandomVector(5)
|
||||
ok, err := client.VAdd(ctx, vecName, "k1", &v1).Result()
|
||||
@@ -188,7 +188,7 @@ var _ = Describe("Redis VectorSet commands", Label("vectorset"), func() {
|
||||
})
|
||||
|
||||
It("all operations", func() {
|
||||
SkipBeforeRedisVersion(8.0, "Redis 8.0 introduces support for VectorSet")
|
||||
SkipBeforeRedisVersion("8.0", "Redis 8.0 introduces support for VectorSet")
|
||||
vecName := "commands"
|
||||
vals := []struct {
|
||||
name string
|
||||
@@ -258,7 +258,7 @@ var _ = Describe("Redis VectorSet commands", Label("vectorset"), func() {
|
||||
expectNil(err)
|
||||
expectEqual(len(res), len(vals))
|
||||
|
||||
if RedisVersion >= 8.4 {
|
||||
if redisVersionAtLeast("8.4") {
|
||||
res, err = client.VRange(ctx, vecName, "[k1", "[k2", -1).Result()
|
||||
expectNil(err)
|
||||
expectEqual(len(res), 2)
|
||||
@@ -355,7 +355,7 @@ var _ = Describe("Redis VectorSet commands", Label("vectorset"), func() {
|
||||
expectEqual(simScores[0].Name, vals[0].name)
|
||||
expectEqual(simScores[0].Score, float64(1))
|
||||
|
||||
if RedisVersion >= 8.2 {
|
||||
if redisVersionAtLeast("8.2") {
|
||||
// test withattribs
|
||||
simAttribs, err := client.VSimWithArgsWithAttribs(ctx, vecName, &vals[0].v, &redis.VSimArgs{
|
||||
Filter: `.age < 30 or .age > 35`,
|
||||
@@ -414,7 +414,7 @@ var _ = Describe("Redis VectorSet commands", Label("vectorset"), func() {
|
||||
})
|
||||
|
||||
It("vlinks with single element", func() {
|
||||
SkipBeforeRedisVersion(8.0, "Redis 8.0 introduces support for VectorSet")
|
||||
SkipBeforeRedisVersion("8.0", "Redis 8.0 introduces support for VectorSet")
|
||||
vecName := "vlinks_single"
|
||||
|
||||
// Add only one vector
|
||||
|
||||
Reference in New Issue
Block a user