As of Go 1.20, this is the idiomatic way to convert from Bytes to
String and vice-versa.
This updates `go.mod` to 1.20, builds have already been running on
1.23.x and 1.24.x since #3274.
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* chore(go): update go version to 1.21
* chore(aggregators): make aggregators work with 1.21
* fix doctests
* address copilot comments
* use atomic bool for logic and/or aggregators
* Update .github/workflows/build.yml
Co-authored-by: ccoVeille <3875889+ccoVeille@users.noreply.github.com>
* use stable/oldstable, 1.23 and 1.21
* fix versions in README
* add oldstable in wordlist
---------
Co-authored-by: ccoVeille <3875889+ccoVeille@users.noreply.github.com>
* test: reduce flakiness in e2e tests
- Increase traffic analysis duration from 30s to 60s in endpoint types test to allow sufficient time for push notification analysis
- Fix TLS config test function name by replacing Cyrillic 'Т' with ASCII 'T' to ensure proper test discovery
* test: disable TLS configurations test
Skip TLS configurations test due to missing TLS environment in test setup
* Remove metrics with empty data
Because empty data cannot be considered an error.
* redis.Nil uses a separate state.
Better differentiation between values, null values, and errors.
There is no Stringer on the channel struct, so using %s causes an error.
Using %v to print the default representation instead.
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* fix(pool): prevent double freeTurn in queuedNewConn
This commit fixes a critical race condition where freeTurn() could be
called twice in the connection pool's queuedNewConn flow, causing turn
counter inconsistency.
Problem:
- When a new connection creation failed in queuedNewConn, both the
defer handler and the dialing goroutine could call freeTurn()
- This led to turn counter underflow and queue length inconsistency
Solution:
- Modified putIdleConn to return a boolean indicating whether the
caller needs to call freeTurn()
- Returns true: connection was put back to pool, caller must free turn
- Returns false: connection was delivered to a waiting request,
turn will be freed by the receiving goroutine
- Updated queuedNewConn to only call freeTurn() when putIdleConn
returns true
- Improved error handling flow in the dialing goroutine
Changes:
- putIdleConn now returns bool instead of void
- Added comprehensive documentation for putIdleConn behavior
- Refactored error handling in queuedNewConn goroutine
- Updated test cases to reflect correct turn state expectations
This ensures each turn is freed exactly once, preventing resource
leaks and maintaining correct queue state.
* fix: sync double freeturn bug fix and context calculation from upstream
Synced from https://github.com/redis/go-redis/tree/ndyakov/freeturn-fix
Changes include:
- Add comprehensive tests for double freeTurn bug detection
- Improve context timeout calculation using min(remaining time, DialTimeout)
- Prevent goroutines from waiting longer than necessary
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Nedyalko Dyakov <nedyalko.dyakov@gmail.com>
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* typed errors
* add error documentation
* backwards compatibility
* update readme, remove Is methods
* Update internal/proto/redis_errors.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update internal/proto/redis_errors.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* support error wrapping for io and context errors
* use unwrapping of errors in push for consistency
* add common error types
* fix test
* fix flaky test
* add comments in the example
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* wip
* wip, used and unusable states
* polish state machine
* correct handling OnPut
* better errors for tests, hook should work now
* fix linter
* improve reauth state management. fix tests
* Update internal/pool/conn.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update internal/pool/conn.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* better timeouts
* empty endpoint handoff case
* fix handoff state when queued for handoff
* try to detect the deadlock
* try to detect the deadlock x2
* delete should be called
* improve tests
* fix mark on uninitialized connection
* Update internal/pool/conn_state_test.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update internal/pool/conn_state_test.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update internal/pool/pool.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update internal/pool/conn_state.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update internal/pool/conn.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix error from copilot
* address copilot comment
* fix(pool): pool performance (#3565)
* perf(pool): replace hookManager RWMutex with atomic.Pointer and add predefined state slices
- Replace hookManager RWMutex with atomic.Pointer for lock-free reads in hot paths
- Add predefined state slices to avoid allocations (validFromInUse, validFromCreatedOrIdle, etc.)
- Add Clone() method to PoolHookManager for atomic updates
- Update AddPoolHook/RemovePoolHook to use copy-on-write pattern
- Update all hookManager access points to use atomic Load()
Performance improvements:
- Eliminates RWMutex contention in Get/Put/Remove hot paths
- Reduces allocations by reusing predefined state slices
- Lock-free reads allow better CPU cache utilization
* perf(pool): eliminate mutex overhead in state machine hot path
The state machine was calling notifyWaiters() on EVERY Get/Put operation,
which acquired a mutex even when no waiters were present (the common case).
Fix: Use atomic waiterCount to check for waiters BEFORE acquiring mutex.
This eliminates mutex contention in the hot path (Get/Put operations).
Implementation:
- Added atomic.Int32 waiterCount field to ConnStateMachine
- Increment when adding waiter, decrement when removing
- Check waiterCount atomically before acquiring mutex in notifyWaiters()
Performance impact:
- Before: mutex lock/unlock on every Get/Put (even with no waiters)
- After: lock-free atomic check, only acquire mutex if waiters exist
- Expected improvement: ~30-50% for Get/Put operations
* perf(pool): use predefined state slices to eliminate allocations in hot path
The pool was creating new slice literals on EVERY Get/Put operation:
- popIdle(): []ConnState{StateCreated, StateIdle}
- putConn(): []ConnState{StateInUse}
- CompareAndSwapUsed(): []ConnState{StateIdle} and []ConnState{StateInUse}
- MarkUnusableForHandoff(): []ConnState{StateInUse, StateIdle, StateCreated}
These allocations were happening millions of times per second in the hot path.
Fix: Use predefined global slices defined in conn_state.go:
- validFromInUse
- validFromCreatedOrIdle
- validFromCreatedInUseOrIdle
Performance impact:
- Before: 4 slice allocations per Get/Put cycle
- After: 0 allocations (use predefined slices)
- Expected improvement: ~30-40% reduction in allocations and GC pressure
* perf(pool): optimize TryTransition to reduce atomic operations
Further optimize the hot path by:
1. Remove redundant GetState() call in the loop
2. Only check waiterCount after successful CAS (not before loop)
3. Inline the waiterCount check to avoid notifyWaiters() call overhead
This reduces atomic operations from 4-5 per Get/Put to 2-3:
- Before: GetState() + CAS + waiterCount.Load() + notifyWaiters mutex check
- After: CAS + waiterCount.Load() (only if CAS succeeds)
Performance impact:
- Eliminates 1-2 atomic operations per Get/Put
- Expected improvement: ~10-15% for Get/Put operations
* perf(pool): add fast path for Get/Put to match master performance
Introduced TryTransitionFast() for the hot path (Get/Put operations):
- Single CAS operation (same as master's atomic bool)
- No waiter notification overhead
- No loop through valid states
- No error allocation
Hot path flow:
1. popIdle(): Try IDLE → IN_USE (fast), fallback to CREATED → IN_USE
2. putConn(): Try IN_USE → IDLE (fast)
This matches master's performance while preserving state machine for:
- Background operations (handoff/reauth use UNUSABLE state)
- State validation (TryTransition still available)
- Waiter notification (AwaitAndTransition for blocking)
Performance comparison per Get/Put cycle:
- Master: 2 atomic CAS operations
- State machine (before): 5 atomic operations (2.5x slower)
- State machine (after): 2 atomic CAS operations (same as master!)
Expected improvement: Restore to baseline ~11,373 ops/sec
* combine cas
* fix linter
* try faster approach
* fast semaphore
* better inlining for hot path
* fix linter issues
* use new semaphore in auth as well
* linter should be happy now
* add comments
* Update internal/pool/conn_state.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* address comment
* slight reordering
* try to cache time if for non-critical calculation
* fix wrong benchmark
* add concurrent test
* fix benchmark report
* add additional expect to check output
* comment and variable rename
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* initConn sets IDLE state
- Handle unexpected conn state changes
* fix precision of time cache and usedAt
* allow e2e tests to run longer
* Fix broken initialization of idle connections
* optimize push notif
* 100ms -> 50ms
* use correct timer for last health check
* verify pass auth on conn creation
* fix assertion
* fix unsafe test
* fix benchmark test
* improve remove conn
* re doesn't support requirepass
* wait more in e2e test
* flaky test
* add missed method in interface
* fix test assertions
* silence logs and faster hooks manager
* address linter comment
* fix flaky test
* use read instad of control
* use pool size for semsize
* CAS instead of reading the state
* preallocate errors and states
* preallocate state slices
* fix flaky test
* fix fast semaphore that could have been starved
* try to fix the semaphore
* should properly notify the waiters
- this way a waiter that timesout at the same time
a releaser is releasing, won't throw token. the releaser
will fail to notify and will pick another waiter.
this hybrid approach should be faster than channels and maintains FIFO
* waiter may double-release (if closed/times out)
* priority of operations
* use simple approach of fifo waiters
* use simple channel based semaphores
* address linter and tests
* remove unused benchs
* change log message
* address pr comments
* address pr comments
* fix data race
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Added hybrid search command
* fixed lint, fixed some tests
* lint fix
* Add support for XReadGroup CLAIM argument (#3578)
* Add support for XReadGroup CLAIM argument
* modify tutorial tests
---------
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* feat(acl): add acl support and test (#3576)
* feat: add acl support and command test
* validate client name before kill it
---------
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* feat(cmd): Add support for MSetEX command (#3580)
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* fix(sentinel): handle empty address (#3577)
* improvements
* linter fixes
* prevention on unnecessary allocations in case of bad configuration
* Test/Benchmark, old code with safety harness preventing panic
---------
Co-authored-by: manish <manish.sharma@manifestit.io>
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* feat: support for latency command (#3584)
* support for latency command
* add NonRedisEnterprise label for latency test
* feat: Add support for certain slowlog commands (#3585)
* Add support for certain slowlog commands
* add NonRedisEnterprise label for slow reset test
---------
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* feat(cmd): Add CAS/CAD commands (#3583)
* add cas/cad commands
* feat(command): Add SetIFDEQ, SetIFDNE and *Get cmds
Decided to move the *Get argument as a separate methods, since the
response will be always the previous value, but in the case where
the previous value is `OK` there result may be ambiguous.
* fix tests
* matchValue to be interface{}
* Only Args approach for DelEx
* use uint64 for digest, add example
* test only for 8.4
* updated ft hybrid, marked as experimental
* updated fthybrid and its tests
* removed debugging prints
* fixed lint, addressed comment
* fixed issues
* fixed lint
* Ensure that the args are prefixed only if theres no prefix already
* Removed automatic args prefixing
---------
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com>
Co-authored-by: destinyoooo <57470814+destinyoooo@users.noreply.github.com>
Co-authored-by: manish <bhardwaz007@yahoo.com>
Co-authored-by: manish <manish.sharma@manifestit.io>
* add cas/cad commands
* feat(command): Add SetIFDEQ, SetIFDNE and *Get cmds
Decided to move the *Get argument as a separate methods, since the
response will be always the previous value, but in the case where
the previous value is `OK` there result may be ambiguous.
* fix tests
* matchValue to be interface{}
* Only Args approach for DelEx
* use uint64 for digest, add example
* test only for 8.4
* Add support for certain slowlog commands
* add NonRedisEnterprise label for slow reset test
---------
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* improvements
* linter fixes
* prevention on unnecessary allocations in case of bad configuration
* Test/Benchmark, old code with safety harness preventing panic
---------
Co-authored-by: manish <manish.sharma@manifestit.io>
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* feat: add acl support and command test
* validate client name before kill it
---------
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* async create conn
* update default values and testcase
* fix comments
* fix data race
* remove context.WithoutCancel, which is a function introduced in Go 1.21
* fix TestDialerRetryConfiguration/DefaultDialerRetries, because tryDial are likely done in async flow
* change to share failed to delivery connection to other waiting
* remove chinese comment
* fix: optimize WantConnQueue benchmarks to prevent memory exhaustion
- Fix BenchmarkWantConnQueue_Dequeue timeout issue by limiting pre-population
- Use object pooling in BenchmarkWantConnQueue_Enqueue to reduce allocations
- Optimize BenchmarkWantConnQueue_EnqueueDequeue with reusable wantConn pool
- Prevent GitHub Actions benchmark failures due to excessive memory usage
Before: BenchmarkWantConnQueue_Dequeue ran for 11+ minutes and was killed
After: All benchmarks complete in ~8 seconds with consistent performance
* format
* fix turn leaks
---------
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
Co-authored-by: Hristo Temelski <hristo.temelski@redis.com>
* internal/proto/peek_push_notification_test : Refactor test helpers to use fmt.Fprintf for buffers
Replaced buf.WriteString(fmt.Sprintf(...)) with fmt.Fprintf or fmt.Fprint in test helper functions for improved clarity and efficiency. This change affects push notification and RESP3 test utilities.
* peek_push_notification_test: revert prev formatting
* all: replace buf.WriteString with fmt.FprintF for consistency
---------
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* all: Refactor tests for idiomatic Go and minor improvements
Replaced redundant 'for key, _' with 'for key' in map iterations for clarity in doctests/cmds_hash_test.go. Updated time measurement from time.Now().Sub to time.Since in hset_benchmark_test.go for idiomatic Go usage. Simplified variadic argument types from interface{} to any and removed unused min function in maintnotifications/e2e/utils_test.go.
* maintnotifications/e2e/utils_test: Update variadic args type in printLog function
Changed the variadic argument type in printLog from 'any' to 'interface{}' for compatibility and consistency with standard Go practices.
