* wip * wip, used and unusable states * polish state machine * correct handling OnPut * better errors for tests, hook should work now * fix linter * improve reauth state management. fix tests * Update internal/pool/conn.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update internal/pool/conn.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * better timeouts * empty endpoint handoff case * fix handoff state when queued for handoff * try to detect the deadlock * try to detect the deadlock x2 * delete should be called * improve tests * fix mark on uninitialized connection * Update internal/pool/conn_state_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update internal/pool/conn_state_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update internal/pool/pool.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update internal/pool/conn_state.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update internal/pool/conn.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix error from copilot * address copilot comment * fix(pool): pool performance (#3565) * perf(pool): replace hookManager RWMutex with atomic.Pointer and add predefined state slices - Replace hookManager RWMutex with atomic.Pointer for lock-free reads in hot paths - Add predefined state slices to avoid allocations (validFromInUse, validFromCreatedOrIdle, etc.) - Add Clone() method to PoolHookManager for atomic updates - Update AddPoolHook/RemovePoolHook to use copy-on-write pattern - Update all hookManager access points to use atomic Load() Performance improvements: - Eliminates RWMutex contention in Get/Put/Remove hot paths - Reduces allocations by reusing predefined state slices - Lock-free reads allow better CPU cache utilization * perf(pool): eliminate mutex overhead in state machine hot path The state machine was calling notifyWaiters() on EVERY Get/Put operation, which acquired a mutex even when no waiters were present (the common case). Fix: Use atomic waiterCount to check for waiters BEFORE acquiring mutex. This eliminates mutex contention in the hot path (Get/Put operations). Implementation: - Added atomic.Int32 waiterCount field to ConnStateMachine - Increment when adding waiter, decrement when removing - Check waiterCount atomically before acquiring mutex in notifyWaiters() Performance impact: - Before: mutex lock/unlock on every Get/Put (even with no waiters) - After: lock-free atomic check, only acquire mutex if waiters exist - Expected improvement: ~30-50% for Get/Put operations * perf(pool): use predefined state slices to eliminate allocations in hot path The pool was creating new slice literals on EVERY Get/Put operation: - popIdle(): []ConnState{StateCreated, StateIdle} - putConn(): []ConnState{StateInUse} - CompareAndSwapUsed(): []ConnState{StateIdle} and []ConnState{StateInUse} - MarkUnusableForHandoff(): []ConnState{StateInUse, StateIdle, StateCreated} These allocations were happening millions of times per second in the hot path. Fix: Use predefined global slices defined in conn_state.go: - validFromInUse - validFromCreatedOrIdle - validFromCreatedInUseOrIdle Performance impact: - Before: 4 slice allocations per Get/Put cycle - After: 0 allocations (use predefined slices) - Expected improvement: ~30-40% reduction in allocations and GC pressure * perf(pool): optimize TryTransition to reduce atomic operations Further optimize the hot path by: 1. Remove redundant GetState() call in the loop 2. Only check waiterCount after successful CAS (not before loop) 3. Inline the waiterCount check to avoid notifyWaiters() call overhead This reduces atomic operations from 4-5 per Get/Put to 2-3: - Before: GetState() + CAS + waiterCount.Load() + notifyWaiters mutex check - After: CAS + waiterCount.Load() (only if CAS succeeds) Performance impact: - Eliminates 1-2 atomic operations per Get/Put - Expected improvement: ~10-15% for Get/Put operations * perf(pool): add fast path for Get/Put to match master performance Introduced TryTransitionFast() for the hot path (Get/Put operations): - Single CAS operation (same as master's atomic bool) - No waiter notification overhead - No loop through valid states - No error allocation Hot path flow: 1. popIdle(): Try IDLE → IN_USE (fast), fallback to CREATED → IN_USE 2. putConn(): Try IN_USE → IDLE (fast) This matches master's performance while preserving state machine for: - Background operations (handoff/reauth use UNUSABLE state) - State validation (TryTransition still available) - Waiter notification (AwaitAndTransition for blocking) Performance comparison per Get/Put cycle: - Master: 2 atomic CAS operations - State machine (before): 5 atomic operations (2.5x slower) - State machine (after): 2 atomic CAS operations (same as master!) Expected improvement: Restore to baseline ~11,373 ops/sec * combine cas * fix linter * try faster approach * fast semaphore * better inlining for hot path * fix linter issues * use new semaphore in auth as well * linter should be happy now * add comments * Update internal/pool/conn_state.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * address comment * slight reordering * try to cache time if for non-critical calculation * fix wrong benchmark * add concurrent test * fix benchmark report * add additional expect to check output * comment and variable rename --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * initConn sets IDLE state - Handle unexpected conn state changes * fix precision of time cache and usedAt * allow e2e tests to run longer * Fix broken initialization of idle connections * optimize push notif * 100ms -> 50ms * use correct timer for last health check * verify pass auth on conn creation * fix assertion * fix unsafe test * fix benchmark test * improve remove conn * re doesn't support requirepass * wait more in e2e test * flaky test * add missed method in interface * fix test assertions * silence logs and faster hooks manager * address linter comment * fix flaky test * use read instad of control * use pool size for semsize * CAS instead of reading the state * preallocate errors and states * preallocate state slices * fix flaky test * fix fast semaphore that could have been starved * try to fix the semaphore * should properly notify the waiters - this way a waiter that timesout at the same time a releaser is releasing, won't throw token. the releaser will fail to notify and will pick another waiter. this hybrid approach should be faster than channels and maintains FIFO * waiter may double-release (if closed/times out) * priority of operations * use simple approach of fifo waiters * use simple channel based semaphores * address linter and tests * remove unused benchs * change log message * address pr comments * address pr comments * fix data race --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Redis client for Go
go-redis is the official Redis client library for the Go programming language. It offers a straightforward interface for interacting with Redis servers.
Supported versions
In go-redis we are aiming to support the last three releases of Redis. Currently, this means we do support:
- Redis 7.2 - using Redis Stack 7.2 for modules support
- Redis 7.4 - using Redis Stack 7.4 for modules support
- Redis 8.0 - using Redis CE 8.0 where modules are included
- Redis 8.2 - using Redis CE 8.2 where modules are included
Although the go.mod states it requires at minimum go 1.18, our CI is configured to run the tests against all three
versions of Redis and latest two versions of Go (1.23,
1.24). We observe that some modules related test may not pass with
Redis Stack 7.2 and some commands are changed with Redis CE 8.0.
Please do refer to the documentation and the tests if you experience any issues. We do plan to update the go version
in the go.mod to go 1.24 in one of the next releases.
How do I Redis?
Learn for free at Redis University
Build faster with the Redis Launchpad
Documentation
Resources
Ecosystem
This client also works with Kvrocks, a distributed key value NoSQL database that uses RocksDB as storage engine and is compatible with Redis protocol.
Features
- Redis commands except QUIT and SYNC.
- Automatic connection pooling.
- StreamingCredentialsProvider (e.g. entra id, oauth) (experimental)
- Pub/Sub.
- Pipelines and transactions.
- Scripting.
- Redis Sentinel.
- Redis Cluster.
- Redis Ring.
- Redis Performance Monitoring.
- Redis Probabilistic [RedisStack]
- Customizable read and write buffers size.
Installation
go-redis supports 2 last Go versions and requires a Go version with modules support. So make sure to initialize a Go module:
go mod init github.com/my/repo
Then install go-redis/v9:
go get github.com/redis/go-redis/v9
Quickstart
import (
"context"
"fmt"
"github.com/redis/go-redis/v9"
)
var ctx = context.Background()
func ExampleClient() {
rdb := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
Password: "", // no password set
DB: 0, // use default DB
})
err := rdb.Set(ctx, "key", "value", 0).Err()
if err != nil {
panic(err)
}
val, err := rdb.Get(ctx, "key").Result()
if err != nil {
panic(err)
}
fmt.Println("key", val)
val2, err := rdb.Get(ctx, "key2").Result()
if err == redis.Nil {
fmt.Println("key2 does not exist")
} else if err != nil {
panic(err)
} else {
fmt.Println("key2", val2)
}
// Output: key value
// key2 does not exist
}
Authentication
The Redis client supports multiple ways to provide authentication credentials, with a clear priority order. Here are the available options:
1. Streaming Credentials Provider (Highest Priority) - Experimental feature
The streaming credentials provider allows for dynamic credential updates during the connection lifetime. This is particularly useful for managed identity services and token-based authentication.
type StreamingCredentialsProvider interface {
Subscribe(listener CredentialsListener) (Credentials, UnsubscribeFunc, error)
}
type CredentialsListener interface {
OnNext(credentials Credentials) // Called when credentials are updated
OnError(err error) // Called when an error occurs
}
type Credentials interface {
BasicAuth() (username string, password string)
RawCredentials() string
}
Example usage:
rdb := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
StreamingCredentialsProvider: &MyCredentialsProvider{},
})
Note: The streaming credentials provider can be used with go-redis-entraid to enable Entra ID (formerly Azure AD) authentication. This allows for seamless integration with Azure's managed identity services and token-based authentication.
Example with Entra ID:
import (
"github.com/redis/go-redis/v9"
"github.com/redis/go-redis-entraid"
)
// Create an Entra ID credentials provider
provider := entraid.NewDefaultAzureIdentityProvider()
// Configure Redis client with Entra ID authentication
rdb := redis.NewClient(&redis.Options{
Addr: "your-redis-server.redis.cache.windows.net:6380",
StreamingCredentialsProvider: provider,
TLSConfig: &tls.Config{
MinVersion: tls.VersionTLS12,
},
})
2. Context-based Credentials Provider
The context-based provider allows credentials to be determined at the time of each operation, using the context.
rdb := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
CredentialsProviderContext: func(ctx context.Context) (string, string, error) {
// Return username, password, and any error
return "user", "pass", nil
},
})
3. Regular Credentials Provider
A simple function-based provider that returns static credentials.
rdb := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
CredentialsProvider: func() (string, string) {
// Return username and password
return "user", "pass"
},
})
4. Username/Password Fields (Lowest Priority)
The most basic way to provide credentials is through the Username and Password fields in the options.
rdb := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
Username: "user",
Password: "pass",
})
Priority Order
The client will use credentials in the following priority order:
- Streaming Credentials Provider (if set)
- Context-based Credentials Provider (if set)
- Regular Credentials Provider (if set)
- Username/Password fields (if set)
If none of these are set, the client will attempt to connect without authentication.
Protocol Version
The client supports both RESP2 and RESP3 protocols. You can specify the protocol version in the options:
rdb := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
Password: "", // no password set
DB: 0, // use default DB
Protocol: 3, // specify 2 for RESP 2 or 3 for RESP 3
})
Connecting via a redis url
go-redis also supports connecting via the redis uri specification. The example below demonstrates how the connection can easily be configured using a string, adhering to this specification.
import (
"github.com/redis/go-redis/v9"
)
func ExampleClient() *redis.Client {
url := "redis://user:password@localhost:6379/0?protocol=3"
opts, err := redis.ParseURL(url)
if err != nil {
panic(err)
}
return redis.NewClient(opts)
}
Instrument with OpenTelemetry
import (
"github.com/redis/go-redis/v9"
"github.com/redis/go-redis/extra/redisotel/v9"
"errors"
)
func main() {
...
rdb := redis.NewClient(&redis.Options{...})
if err := errors.Join(redisotel.InstrumentTracing(rdb), redisotel.InstrumentMetrics(rdb)); err != nil {
log.Fatal(err)
}
Buffer Size Configuration
go-redis uses 32KiB read and write buffers by default for optimal performance. For high-throughput applications or large pipelines, you can customize buffer sizes:
rdb := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
ReadBufferSize: 1024 * 1024, // 1MiB read buffer
WriteBufferSize: 1024 * 1024, // 1MiB write buffer
})
Advanced Configuration
go-redis supports extending the client identification phase to allow projects to send their own custom client identification.
Default Client Identification
By default, go-redis automatically sends the client library name and version during the connection process. This feature is available in redis-server as of version 7.2. As a result, the command is "fire and forget", meaning it should fail silently, in the case that the redis server does not support this feature.
Disabling Identity Verification
When connection identity verification is not required or needs to be explicitly disabled, a DisableIdentity configuration option exists.
Initially there was a typo and the option was named DisableIndentity instead of DisableIdentity. The misspelled option is marked as Deprecated and will be removed in V10 of this library.
Although both options will work at the moment, the correct option is DisableIdentity. The deprecated option will be removed in V10 of this library, so please use the correct option name to avoid any issues.
To disable verification, set the DisableIdentity option to true in the Redis client options:
rdb := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
Password: "",
DB: 0,
DisableIdentity: true, // Disable set-info on connect
})
Unstable RESP3 Structures for RediSearch Commands
When integrating Redis with application functionalities using RESP3, it's important to note that some response structures aren't final yet. This is especially true for more complex structures like search and query results. We recommend using RESP2 when using the search and query capabilities, but we plan to stabilize the RESP3-based API-s in the coming versions. You can find more guidance in the upcoming release notes.
To enable unstable RESP3, set the option in your client configuration:
redis.NewClient(&redis.Options{
UnstableResp3: true,
})
Note: When UnstableResp3 mode is enabled, it's necessary to use RawResult() and RawVal() to retrieve a raw data. Since, raw response is the only option for unstable search commands Val() and Result() calls wouldn't have any affect on them:
res1, err := client.FTSearchWithArgs(ctx, "txt", "foo bar", &redis.FTSearchOptions{}).RawResult()
val1 := client.FTSearchWithArgs(ctx, "txt", "foo bar", &redis.FTSearchOptions{}).RawVal()
Redis-Search Default Dialect
In the Redis-Search module, the default dialect is 2. If needed, you can explicitly specify a different dialect using the appropriate configuration in your queries.
Important: Be aware that the query dialect may impact the results returned. If needed, you can revert to a different dialect version by passing the desired dialect in the arguments of the command you want to execute. For example:
res2, err := rdb.FTSearchWithArgs(ctx,
"idx:bicycle",
"@pickup_zone:[CONTAINS $bike]",
&redis.FTSearchOptions{
Params: map[string]interface{}{
"bike": "POINT(-0.1278 51.5074)",
},
DialectVersion: 3,
},
).Result()
You can find further details in the query dialect documentation.
Custom buffer sizes
Prior to v9.12, the buffer size was the default go value of 4096 bytes. Starting from v9.12, go-redis uses 32KiB read and write buffers by default for optimal performance. For high-throughput applications or large pipelines, you can customize buffer sizes:
rdb := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
ReadBufferSize: 1024 * 1024, // 1MiB read buffer
WriteBufferSize: 1024 * 1024, // 1MiB write buffer
})
Important: If you experience any issues with the default buffer sizes, please try setting them to the go default of 4096 bytes.
Contributing
We welcome contributions to the go-redis library! If you have a bug fix, feature request, or improvement, please open an issue or pull request on GitHub. We appreciate your help in making go-redis better for everyone. If you are interested in contributing to the go-redis library, please check out our contributing guidelines for more information on how to get started.
Look and feel
Some corner cases:
// SET key value EX 10 NX
set, err := rdb.SetNX(ctx, "key", "value", 10*time.Second).Result()
// SET key value keepttl NX
set, err := rdb.SetNX(ctx, "key", "value", redis.KeepTTL).Result()
// SORT list LIMIT 0 2 ASC
vals, err := rdb.Sort(ctx, "list", &redis.Sort{Offset: 0, Count: 2, Order: "ASC"}).Result()
// ZRANGEBYSCORE zset -inf +inf WITHSCORES LIMIT 0 2
vals, err := rdb.ZRangeByScoreWithScores(ctx, "zset", &redis.ZRangeBy{
Min: "-inf",
Max: "+inf",
Offset: 0,
Count: 2,
}).Result()
// ZINTERSTORE out 2 zset1 zset2 WEIGHTS 2 3 AGGREGATE SUM
vals, err := rdb.ZInterStore(ctx, "out", &redis.ZStore{
Keys: []string{"zset1", "zset2"},
Weights: []int64{2, 3}
}).Result()
// EVAL "return {KEYS[1],ARGV[1]}" 1 "key" "hello"
vals, err := rdb.Eval(ctx, "return {KEYS[1],ARGV[1]}", []string{"key"}, "hello").Result()
// custom command
res, err := rdb.Do(ctx, "set", "key", "value").Result()
Run the test
Recommended to use Docker, just need to run:
make test
See also
- Golang ORM for PostgreSQL, MySQL, MSSQL, and SQLite
- Golang PostgreSQL
- Golang HTTP router
- Golang ClickHouse ORM
Contributors
The go-redis project was originally initiated by ⭐ uptrace/uptrace. Uptrace is an open-source APM tool that supports distributed tracing, metrics, and logs. You can use it to monitor applications and set up automatic alerts to receive notifications via email, Slack, Telegram, and others.
See OpenTelemetry example which demonstrates how you can use Uptrace to monitor go-redis.
Thanks to all the people who already contributed!