Introduced TryTransitionFast() for the hot path (Get/Put operations):
- Single CAS operation (same as master's atomic bool)
- No waiter notification overhead
- No loop through valid states
- No error allocation
Hot path flow:
1. popIdle(): Try IDLE → IN_USE (fast), fallback to CREATED → IN_USE
2. putConn(): Try IN_USE → IDLE (fast)
This matches master's performance while preserving state machine for:
- Background operations (handoff/reauth use UNUSABLE state)
- State validation (TryTransition still available)
- Waiter notification (AwaitAndTransition for blocking)
Performance comparison per Get/Put cycle:
- Master: 2 atomic CAS operations
- State machine (before): 5 atomic operations (2.5x slower)
- State machine (after): 2 atomic CAS operations (same as master!)
Expected improvement: Restore to baseline ~11,373 ops/sec
Further optimize the hot path by:
1. Remove redundant GetState() call in the loop
2. Only check waiterCount after successful CAS (not before loop)
3. Inline the waiterCount check to avoid notifyWaiters() call overhead
This reduces atomic operations from 4-5 per Get/Put to 2-3:
- Before: GetState() + CAS + waiterCount.Load() + notifyWaiters mutex check
- After: CAS + waiterCount.Load() (only if CAS succeeds)
Performance impact:
- Eliminates 1-2 atomic operations per Get/Put
- Expected improvement: ~10-15% for Get/Put operations
The pool was creating new slice literals on EVERY Get/Put operation:
- popIdle(): []ConnState{StateCreated, StateIdle}
- putConn(): []ConnState{StateInUse}
- CompareAndSwapUsed(): []ConnState{StateIdle} and []ConnState{StateInUse}
- MarkUnusableForHandoff(): []ConnState{StateInUse, StateIdle, StateCreated}
These allocations were happening millions of times per second in the hot path.
Fix: Use predefined global slices defined in conn_state.go:
- validFromInUse
- validFromCreatedOrIdle
- validFromCreatedInUseOrIdle
Performance impact:
- Before: 4 slice allocations per Get/Put cycle
- After: 0 allocations (use predefined slices)
- Expected improvement: ~30-40% reduction in allocations and GC pressure
The state machine was calling notifyWaiters() on EVERY Get/Put operation,
which acquired a mutex even when no waiters were present (the common case).
Fix: Use atomic waiterCount to check for waiters BEFORE acquiring mutex.
This eliminates mutex contention in the hot path (Get/Put operations).
Implementation:
- Added atomic.Int32 waiterCount field to ConnStateMachine
- Increment when adding waiter, decrement when removing
- Check waiterCount atomically before acquiring mutex in notifyWaiters()
Performance impact:
- Before: mutex lock/unlock on every Get/Put (even with no waiters)
- After: lock-free atomic check, only acquire mutex if waiters exist
- Expected improvement: ~30-50% for Get/Put operations
- Replace hookManager RWMutex with atomic.Pointer for lock-free reads in hot paths
- Add predefined state slices to avoid allocations (validFromInUse, validFromCreatedOrIdle, etc.)
- Add Clone() method to PoolHookManager for atomic updates
- Update AddPoolHook/RemovePoolHook to use copy-on-write pattern
- Update all hookManager access points to use atomic Load()
Performance improvements:
- Eliminates RWMutex contention in Get/Put/Remove hot paths
- Reduces allocations by reusing predefined state slices
- Lock-free reads allow better CPU cache utilization
* all: Refactor tests for idiomatic Go and minor improvements
Replaced redundant 'for key, _' with 'for key' in map iterations for clarity in doctests/cmds_hash_test.go. Updated time measurement from time.Now().Sub to time.Since in hset_benchmark_test.go for idiomatic Go usage. Simplified variadic argument types from interface{} to any and removed unused min function in maintnotifications/e2e/utils_test.go.
* maintnotifications/e2e/utils_test: Update variadic args type in printLog function
Changed the variadic argument type in printLog from 'any' to 'interface{}' for compatibility and consistency with standard Go practices.
* feat(tracing): add process pipeline and dial filtering options for tracing
* refactor(tracing): implement default command filter and process pipeline options
* refactor(tracing): rename defaultCommandFilter to DefaultCommandFilter
* refactor(tracing): add BasicCommandFilter as a deprecated alias for DefaultCommandFilter
---------
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* fix(pool): wip, pool reauth should not interfere with handoff
* fix credListeners map
* fix race in tests
* better conn usable timeout
* add design decision comment
* few small improvements
* update marked as queued
* add Used to clarify the state of the conn
* rename test
* fix(test): fix flaky test
* lock inside the listeners collection
* address pr comments
* Update internal/auth/cred_listeners.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update internal/pool/buffer_size_test.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* wip refactor entraid
* fix maintnotif pool hook
* fix mocks
* fix nil listener
* sync and async reauth based on conn lifecycle
* be able to reject connection OnGet
* pass hooks so the tests can observe reauth
* give some time for the background to execute commands
* fix tests
* only async reauth
* Update internal/pool/pool.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update internal/auth/streaming/pool_hook.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update internal/pool/conn.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* chore(redisotel): use metric.WithAttributeSet to avoid copy (#3552)
In order to improve performance replace `WithAttributes` with `WithAttributeSet`.
This avoids the slice allocation and copy that is done in `WithAttributes`.
For more information see https://github.com/open-telemetry/opentelemetry-go/blob/v1.38.0/metric/instrument.go#L357-L376
* chore(docs): explain why MaxRetries is disabled for ClusterClient (#3551)
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
* exponential backoff
* address pr comments
* address pr comments
* remove rlock
* add some comments
* add comments
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Warnar Boekkooi <wboekkooi@impossiblecloud.com>
Co-authored-by: Justin <justindsouza80@gmail.com>
* update e2e test, change script
* update script and tests
* fixed bdbid parsing
* disabled majority of tests, swapped event order
* change the config tag
* revert test order
* fix typo
* reenable all e2e tests
* change the clonfig flag key for all e2e tests
* improve logging for debug purposes of tests
* longer deadline for FI in CI
* increase waiting for notifications
* extend tests
* dont fail on flaky third client
* fi new params
* fix test build
* more time for migrating
* first wait for FI action, then assert notification
* fix test build
* fix tests
* fix tests
* change output
* global print logs for tests
* better output
* fix error format
* maybe the notification is already received
* second and third client fix
* print output if failed
* better second and third client checks
* output action data if notification is not received
* stop command runner
* database create / delete actions
* database create / delete actions used in tests
* fix import
* remove example
* remove unused var
* use different port than the one in env
* wait for action to get the response
* fix output
* fix create db config
* fix create db config
* use new database for client
* fix create db config
* db per scenario
* less logs, correct check
* Add CTRF to the scenario tests (#3545)
* add some json ctrf improvements
* fix -v
* attempt to separate the output
---------
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
---------
Co-authored-by: Nedyalko Dyakov <nedyalko.dyakov@gmail.com>
Co-authored-by: kiryazovi-redis <ivaylo.kiryazov@redis.com>
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
