redis/go-redis
1
0
Fork 0
mirror of https://github.com/redis/go-redis.git synced 2025-11-30 18:01:23 +03:00
Code Activity
2,791 Commits 75 Branches 463 Tags
f711eb0f621bbca00c887fb6159dcc536252eb5c
Commit Graph

39 Commits

Author SHA1 Message Date
ofekshenawa
f711eb0f62 feat(cluster): Implement Request and Response Policy Based Routing in Cluster Mode (#3422) 
* Add search module builders and tests (#1)

* Add search module builders and tests

* Add tests

* Use builders and Actions in more clean way

* Update search_builders.go

Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>

* Update search_builders.go

Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>

* feat(routing): add internal request/response policy enums

* feat: load the policy table in cluster client (#4)

* feat: load the policy table in cluster client

* Remove comments

* modify Tips and command pplicy in commandInfo (#5)

* centralize cluster command routing in osscluster_router.go and refactor osscluster.go (#6)

* centralize cluster command routing in osscluster_router.go and refactor osscluster.go

* enalbe ci on all branches

* Add debug prints

* Add debug prints

* FIX: deal with nil policy

* FIX: fixing clusterClient process

* chore(osscluster): simplify switch case

* wip(command): ai generated clone method for commands

* feat: implement response aggregator for Redis cluster commands

* feat: implement response aggregator for Redis cluster commands

* fix: solve concurrency errors

* fix: solve concurrency errors

* return MaxRedirects settings

* remove locks from getCommandPolicy

* Handle MOVED errors more robustly, remove cluster reloading at exectutions, ennsure better routing

* Fix: supports Process hook test

* Fix: remove response aggregation for single shard commands

* Add more preformant type conversion for Cmd type

* Add router logic into processPipeline

---------

Co-authored-by: Nedyalko Dyakov <nedyalko.dyakov@gmail.com>

* remove thread debugging code

* remove thread debugging code && reject commands with policy that cannot be used in pipeline

* refactor processPipline and cmdType enum

* remove FDescribe from cluster tests

* Add tests

* fix aggregation test

* fix mget test

* fix mget test

* remove aggregateKeyedResponses

* added scaffolding for the req-resp manager

* added default policies for the search commands

* split command map into module->command

* cleanup, added logic to refresh the cache

* added reactive cache refresh

* revert cluster refresh

* fixed lint

* addresed first batch of comments

* rewrote aggregator implementations with atomic for native or nearnative primitives

* addressed more comments, fixed lint

* added batch aggregator operations

* fixed lint

* updated batch aggregator, fixed extractcommandvalue

* fixed lint

* added batching to aggregateResponses

* fixed deadlocks

* changed aggregator logic, added error params

* added preemptive return to the aggregators

* more work on the aggregators

* updated and and or aggregators

* fixed lint

* added configurable policy resolvers

* slight refactor

* removed the interface, slight refactor

* change func signature from cmdName to cmder

* added nil safety assertions

* few small refactors

* added read only policies

* removed leftover prints

* Rebased to master, resolved comnflicts

* fixed lint

* updated gha

* fixed tests, minor consistency refactor

* preallocated simple errors

* changed numeric aggregators to use float64

* speculative test fix

* Update command.go

Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>

* Update main_test.go

Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>

* Add static shard picker

* Fix nil value handling in command aggregation

* Modify the Clone method to return a shallow copy

* Add clone method to digest command

* Optimize keyless command routing to respect ShardPicker policy

* Remove MGET references

* Fix MGET aggregation to map individual values to keys across shards

* Add clone method to hybrid search commands

* Undo changes in route keyless test

* remove comments

* Add test for DisableRoutingPolicies option

* Add Routing Policies Comprehensive Test Suite and Fix multi keyed aggregation for different step

---------

Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
Co-authored-by: Nedyalko Dyakov <nedyalko.dyakov@gmail.com>
Co-authored-by: Hristo Temelski <hristo.temelski@redis.com>
 2025-11-28 11:46:23 +02:00
Justin
14a8814540 chore(docs): explain why MaxRetries is disabled for ClusterClient (#3551) 
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
 2025-10-21 15:03:58 +03:00
Nedyalko Dyakov
75ddeb3d5a feat(e2e-testing): maintnotifications e2e and refactor (#3526) 
* e2e wip

* cleanup

* remove unused fault injector mock

* errChan in test

* remove log messages tests

* cleanup log messages

* s/hitless/maintnotifications/

* fix moving when none

* better logs

* test with second client after action has started

* Fixes

Signed-off-by: Elena Kolevska <elena@kolevska.com>

* Test fix

Signed-off-by: Elena Kolevska <elena@kolevska.com>

* feat(e2e-test): Extended e2e tests

* imroved e2e test resiliency

---------

Signed-off-by: Elena Kolevska <elena@kolevska.com>
Co-authored-by: Elena Kolevska <elena@kolevska.com>
Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>
Co-authored-by: Hristo Temelski <hristo.temelski@redis.com>
 2025-09-26 19:17:09 +03:00
Nedyalko Dyakov
0ef6d0727d feat: RESP3 notifications support & Hitless notifications handling [CAE-1088] & [CAE-1072] (#3418) 
- Adds support for handling push notifications with RESP3. 
- Using this support adds handlers for hitless upgrades.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Hristo Temelski <hristo.temelski@redis.com>
 2025-09-10 22:18:01 +03:00
Nedyalko Dyakov
a264ffb8a4 fix: SetErr on Cmd if the command cannot be queued correctly in multi/exec (#3509) 
* set error if queued fails

* try fix for cluster

* add errors to cmds in pipeline if about to be returned
 2025-09-09 18:45:37 +03:00
LINKIWI
10121e9e1c feat(osscluster): Support subscriptions against cluster slave nodes (#3480) 2025-09-02 16:03:00 +03:00
Jonathan Suever
6f41b600c5 fix(client): Do not assume that all non-IP hosts are loopbacks (#3085) 
* Do not assume that all non-IP hosts are loopbacks

* handle localhost and Docker internal hostnames

---------

Co-authored-by: Nedyalko Dyakov <nedyalko.dyakov@gmail.com>
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
Co-authored-by: ofekshenawa <ofek.shenawa@redis.com>
Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com>
 2025-09-02 15:58:50 +03:00
cxljs
e07f55bed1 chore(buffers): Set the default read/write buffer size of Redis connection to 32KiB (#3483) 
* update README.md

Signed-off-by: Xiaolong Chen <fukua95@gmail.com>

* typo: 0.5MiB -> 256KiB

Signed-off-by: Xiaolong Chen <fukua95@gmail.com>

* Set the default read/write buffer size of Redis connection to 32KiB

Signed-off-by: Xiaolong Chen <fukua95@gmail.com>

---------

Signed-off-by: Xiaolong Chen <fukua95@gmail.com>
 2025-08-18 20:04:55 +03:00
ofekshenawa
436979f5c7 feat(options): Clean failing timeout implementation (#3472) 
* Fix hard code of failing timeout

1. if not set failing time limit, default is 15 seconds.

* feat: Complete configurable FailingTimeoutSeconds implementation

---------

Co-authored-by: Shino Wu <shino_wu@trendmicro.com>
 2025-08-12 14:08:28 +03:00
Nedyalko Dyakov
94cfffa417 fix(options): Add buffer sizes to failover. Update README (#3468) 
* fix(options): Add buffer sizes to failover. Update README

* fix(spellcheck): add KiB in wordlist

* fix(comment): fix defaul value in comment

* fixes #3465
 2025-08-11 16:01:24 +03:00
cxljs
375fa5d083 chore(doc): improve code readability (#3446) 
- replace two similar functions `appendUniqueNode` and `appendIfNotExists` with a generic function.

- simplify the implementation of the `get` method in `clusterNodes`

- keep the member name `_generation` of `clusterNodes` consistent with other types.

- rename a data member `_masterAddr` to `masterAddr`.

Signed-off-by: Xiaolong Chen <fukua95@gmail.com>
 2025-08-04 17:22:16 +03:00
ofekshenawa
1eed165f9d feat(proto): add configurable buffer sizes for Redis connections (#3453) 
* add configurable buffer sizes for Redis connections

* add MiB to wordlist

* Add description for buffer size parameter
 2025-08-04 09:16:54 +03:00
Nedyalko Dyakov
05f42e2327 fix(txpipeline): keyless commands should take the slot of the keyed (#3411) 
* fix(txpipeline): keyless commands should take the slot of the keyed commands

* fix(txpipeline): extract only keyed cmds from all cmds

* chore(test): Add tests for keyless cmds and txpipeline

* fix(cmdSlot): Add preferred random slot

* fix(cmdSlot): Add shortlist of keyless cmds

* chore(test): Fix ring test

* fix(keylessCommands): Add list of keyless commands

Add list of keyless Commands based on the Commands output
for redis 8

* chore(txPipeline): refactor slottedCommands impl

* fix(osscluster): typo
 2025-06-24 10:34:23 +03:00
Nedyalko Dyakov
f4358acffc [CAE-1046] fix(loading): cache the loaded flag for slave nodes (#3410) 
* fix(loading): cache the loaded flag for slave nodes

* fix(lint): make linter happy
 2025-06-18 15:18:51 +03:00
Nedyalko Dyakov
4c635cc563 fix(txpipeline): should return error on multi/exec on multiple slots [CAE-1028] (#3408) 
* fix(txpipeline): should return error on multi/exec on multiple slots

* fix(txpipeline): test normal tx pipeline behaviour

* chore(err): Extract crossslot err and add test

* fix(txpipeline): short curcuit the tx if there are no commands

* chore(tests): validate keys are in different slots
 2025-06-18 14:18:00 +03:00
cxljs
82b00cc520 chore: remove a redundant method (#3401) 
Signed-off-by: fukua95 <fukua95@gmail.com>
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
 2025-06-16 16:55:23 +03:00
Nedyalko Dyakov
86d418f940 feat: Introducing StreamingCredentialsProvider for token based authentication (#3320) 
* wip

* update documentation

* add streamingcredentialsprovider in options

* fix: put back option in pool creation

* add package level comment

* Initial re authentication implementation

Introduces the StreamingCredentialsProvider as the CredentialsProvider
with the highest priority.

TODO: needs to be tested

* Change function type name

Change CancelProviderFunc to UnsubscribeFunc

* add tests

* fix race in tests

* fix example tests

* wip, hooks refactor

* fix build

* update README.md

* update wordlist

* update README.md

* refactor(auth): early returns in cred listener

* fix(doctest): simulate some delay

* feat(conn): add close hook on conn

* fix(tests): simulate start/stop in mock credentials provider

* fix(auth): don't double close the conn

* docs(README): mark streaming credentials provider as experimental

* fix(auth): streamline auth err proccess

* fix(auth): check err on close conn

* chore(entraid): use the repo under redis org
 2025-05-27 16:25:20 +03:00
ofekshenawa
d7ba255394 fix: prevent routing reads to loading slave nodes (#3370) 
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
 2025-05-21 13:58:21 +03:00
fukua95
03c2c0b088 chore: remove unused param (#3384) 
Signed-off-by: fukua95 <fukua95@gmail.com>
 2025-05-20 14:11:53 +03:00
fukua95
80c373be46 chore: remove unused param (#3382) 
* chore: remove unused param

Signed-off-by: fukua95 <fukua95@gmail.com>

* chore: rename a unused param to `_`

Signed-off-by: fukua95 <fukua95@gmail.com>

---------

Signed-off-by: fukua95 <fukua95@gmail.com>
 2025-05-19 19:22:51 +03:00
Nedyalko Dyakov
d54e848055 feat(options): panic when options are nil (#3363) 
Client creation should panic when options are nil.
 2025-04-30 09:33:40 +03:00
ofekshenawa
2f0a9b720a migrate golangci-lint config to v2 format (#3354) 
* migrate golangci-lint config to v2 format

* chore: skip CI on migration [skip ci]

* Bump golangci version

* Address several golangci-lint/staticcheck warnings

* change staticchecks settings
 2025-04-29 12:53:06 +03:00
Bulat Khasanov
e2149b06f7 Use correct slot for COUNTKEYSINSLOT command (#3327) 2025-04-16 18:32:57 +03:00
Nedyalko Dyakov
d236865b0c fix: handle network error on SETINFO (#3295) (CVE-2025-29923) 
* fix: handle network error on SETINFO

This fix addresses potential out of order responses as described in `CVE-2025-29923`

* fix: deprecate DisableIndentity and introduce DisableIdentity

Both options will work before V10. In v10 DisableIndentity will be dropped. The preferred flag to use is `DisableIdentity`.
 2025-03-19 19:02:36 +02:00
Bhargav Dodla
162a15432b fix: Fixed issue with context cancelled error leading to connection spikes on Primary instances (#3190) 
* fix: Fixed issue with context cancelled error leading to connection spikes on Master

* fix: Added tests

* fix: Updated tests

---------

Co-authored-by: Bhargav Dodla <bdodla@expediagroup.com>
Co-authored-by: Nedyalko Dyakov <nedyalko.dyakov@gmail.com>
 2025-03-05 22:08:27 +02:00
Nedyalko Dyakov
d0f921357d fix: add unstableresp3 to cluster client (#3266) 
* fix: add unstableresp3 to cluster client

* propagate unstableresp3

* proper test that will ignore error, but fail if client panics

* add separate test for clusterclient constructor
 2025-02-07 12:09:49 +02:00
Shawn Wang
d0fb810b13 Fix race condition in clusterNodes.Addrs() (#3219) 
Resolve a race condition in the clusterNodes.Addrs() method.
Previously, the method returned a reference to a string slice, creating
the potential for concurrent reads by the caller while the slice was
being modified by the garbage collection process.

Co-authored-by: Nedyalko Dyakov <nedyalko.dyakov@gmail.com>
 2025-02-03 16:15:00 +02:00
Justin
f1ffb55c9a Only check latencies once every 10 seconds with routeByLatency (#2795) 
* Only check latencies once every 10 seconds with `routeByLatency`

`routeByLatency` currently checks latencies any time a server returns
a MOVED or READONLY reply. When a shard is down, the ClusterClient
chooses to issue the request to a random server, which returns a MOVED
reply. This causes a state refresh and a latency update on all servers.
This can lead to significant ping load to clusters with a large number
of clients.

This introduces logic to ping only once every 10 seconds, only
performing a latency update on a node during the `GC` function if the
latency was set later than 10 seconds ago.

Fixes https://github.com/redis/go-redis/issues/2782

* use UnixNano instead of Unix for better precision

---------

Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com>
 2024-11-20 14:36:39 +02:00
LINKIWI
5756b05219 Avoid unnecessary retry delay following MOVED and ASK redirection (#3048) 2024-07-13 14:55:12 +08:00
Srikar Jilugu
9c1f4f0642 fix node routing in slotClosestNode (#3043) 
* fix node routing when all nodes are failing

* fix minlatency zero value
 2024-07-10 11:26:27 +08:00
Monkey
2d8fa02ac2 fix: fix #2681 (#2998) 
Signed-off-by: monkey92t <golang@88.com>
 2024-05-29 10:55:28 +08:00
高木同学
f7f34feddf Optimize docs useless imports and typo (#2970) 2024-04-14 22:59:09 +08:00
kindknow
6960bcc08d chore: fix some comments (#2967) 
Signed-off-by: kindknow <iturf@sina.com>
Co-authored-by: Monkey <golang@88.com>
 2024-04-14 01:15:23 +08:00
singular-seal
f5496d14dd Fix for issues #2959 and #2960 (#2961) 
Co-authored-by: Hu Ying <ying.hu@shopee.com>
 2024-04-12 23:10:12 +08:00
John Du Hart
516abe867d add CredentialsProvider to ClusterOptions (#2791) 2024-02-14 23:03:18 +02:00
ofekshenawa
a32be3d93d Add Suffix support to default client set info (#2852) 
* Add Suffix support to defualt client set info

* Change ClientNameSuffix to IdentitySuffix

* add tests
 2024-01-04 14:40:14 +02:00
ofekshenawa
86c68be278 Execute "COMMAND" command only when readonly (#2815) 
* remove command command from oss cluster

* remove command command from oss cluster

* remove cmdInfo from ring

---------

Co-authored-by: Chayim <chayim@users.noreply.github.com>
 2023-12-17 15:21:01 +02:00
chenjie199234
fd13da4fea fix missing fields in different Options (#2757) 
* fix missing fields in different Options

* fix missing fields in different Options

---------

Co-authored-by: chenjie199234 <chenjie199234@gmail.com>
Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com>
 2023-10-30 14:35:51 +02:00
Chayim
6199a2af2c Making command structs digestable (#2716) 
* intial move

* adding stringcmdable

* moving module commands to align with other changes

---------

Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com>
Co-authored-by: ofekshenawa <ofek.shenawa@redis.com>
 2023-09-20 16:08:24 +03:00