Hitless upgrades (#3021)

* feat(errors): Add specialized timeout error types for maintenance scenarios

- Added `SocketTimeoutDuringMaintananceError`, a subclass of `TimeoutError`, to handle socket timeouts during maintenance.
- Added `CommandTimeoutDuringMaintenanceError`, another subclass of `TimeoutError`, to address command write timeouts during maintenance.

* feat(linked-list): Add EmptyAwareSinglyLinkedList and enhance DoublyLinkedList functionality

- Introduced `EmptyAwareSinglyLinkedList`, a subclass of `SinglyLinkedList` that emits an `empty` event when the list becomes empty due to `reset`, `shift`, or `remove` operations.
- Added `nodes()` iterator method to `DoublyLinkedList` for iterating over nodes directly.
- Enhanced unit tests for `DoublyLinkedList` and `SinglyLinkedList` to cover edge cases and new functionality.
- Added comprehensive tests for `EmptyAwareSinglyLinkedList` to validate `empty` event emission under various scenarios.
- Improved code formatting and consistency.

* refactor(commands-queue): Improve push notification handling

- Replaced `setInvalidateCallback` with a more flexible `addPushHandler` method, allowing multiple handlers for push notifications.
- Introduced the `PushHandler` type to standardize push notification processing.
- Refactored `RedisCommandsQueue` to use a `#pushHandlers` array, enabling dynamic and modular handling of push notifications.
- Updated `RedisClient` to leverage the new handler mechanism for `invalidate` push notifications, simplifying and decoupling logic.

* feat(commands-queue): Add method to wait for in-flight commands to complete

- Introduced `waitForInflightCommandsToComplete` method to asynchronously wait for all in-flight commands to finish processing.
- Utilized the `empty` event from `#waitingForReply` to signal when all commands have been completed.

* feat(commands-queue): Introduce maintenance mode support for commands-queue

- Added `#maintenanceCommandTimeout` and `setMaintenanceCommandTimeout` method to dynamically adjust command timeouts during maintenance

* refator(client): Extract socket event listener setup into helper method

* refactor(socket): Add maintenance mode support and dynamic timeout handling

- Added `#maintenanceTimeout` and `setMaintenanceTimeout` method to dynamically adjust socket timeouts during maintenance.

* feat(client): Add Redis Enterprise maintenance configuration options

- Added `maintPushNotifications` option to control how the client handles Redis Enterprise maintenance push notifications (`disabled`, `enabled`, `au
to`).
- Added `maintMovingEndpointType` option to specify the endpoint type for reconnecting during a MOVING notification (`auto`, `internal-ip`, `external-ip`,
etc.).
- Added `maintRelaxedCommandTimeout` option to define a relaxed timeout for commands during maintenance.
- Added `maintRelaxedSocketTimeout` option to define a relaxed timeout for the socket during maintenance.
- Enforced RESP3 requirement for maintenance-related features (`maintPushNotifications`).

* feat(client): Add socket helpers and pause mechanism

- Introduced `#paused` flag with corresponding `_pause` and `_unpause` methods to
temporarily halt writing commands to the socket during maintenance windows.
- Updated `#write` method to respect the `#paused` flag, preventing new commands from being written during maintenance.
- Added `_ejectSocket` method to safely detach from and return the current socket
- Added `_insertSocket` method to receive and start using a new socket

* feat(client): Add Redis Enterprise maintenance handling capabilities

- Introduced `EnterpriseMaintenanceManager` to manage Redis Enterprise maintenance events and push notifications.
- Integrated `EnterpriseMaintenanceManager` into `RedisClient` to handle maintenance push notifications and manage socket transitions.
- Implemented graceful handling of MOVING, MIGRATING, and FAILOVER push notifications, including socket replacement and timeout adjustments.


* test: add E2E test infrastructure for Redis maintenance scenarios

* test: add E2E tests for Redis Enterprise maintenance timeout handling (#3)

* test: add connection handoff test


---------

Co-authored-by: Pavel Pashov <pavel.pashov@redis.com>
Co-authored-by: Pavel Pashov <60297174+PavelPashov@users.noreply.github.com>

packages/client/lib/tests/test-scenario/connection-handoff.e2e.ts

@@ -0,0 +1,126 @@
+import diagnostics_channel from "node:diagnostics_channel";
+import { FaultInjectorClient } from "./fault-injector-client";
+import {
+  getDatabaseConfig,
+  getDatabaseConfigFromEnv,
+  getEnvConfig,
+  RedisConnectionConfig,
+} from "./test-scenario.util";
+import { createClient } from "../../..";
+import { DiagnosticsEvent } from "../../client/enterprise-maintenance-manager";
+import { before } from "mocha";
+import { spy } from "sinon";
+import assert from "node:assert";
+import { TestCommandRunner } from "./test-command-runner";
+import net from "node:net";
+
+describe("Connection Handoff", () => {
+  const diagnosticsLog: DiagnosticsEvent[] = [];
+
+  const onMessageHandler = (message: unknown) => {
+    diagnosticsLog.push(message as DiagnosticsEvent);
+  };
+
+  let clientConfig: RedisConnectionConfig;
+  let client: ReturnType<typeof createClient<any, any, any, 3>>;
+  let faultInjectorClient: FaultInjectorClient;
+  let connectSpy = spy(net, "createConnection");
+
+  before(() => {
+    const envConfig = getEnvConfig();
+    const redisConfig = getDatabaseConfigFromEnv(
+      envConfig.redisEndpointsConfigPath,
+    );
+
+    faultInjectorClient = new FaultInjectorClient(envConfig.faultInjectorUrl);
+    clientConfig = getDatabaseConfig(redisConfig);
+  });
+
+  beforeEach(async () => {
+    diagnosticsLog.length = 0;
+    diagnostics_channel.subscribe("redis.maintenance", onMessageHandler);
+
+    connectSpy.resetHistory();
+
+    client = createClient({
+      socket: {
+        host: clientConfig.host,
+        port: clientConfig.port,
+        ...(clientConfig.tls === true ? { tls: true } : {}),
+      },
+      password: clientConfig.password,
+      username: clientConfig.username,
+      RESP: 3,
+      maintPushNotifications: "auto",
+      maintMovingEndpointType: "external-ip",
+      maintRelaxedCommandTimeout: 10000,
+      maintRelaxedSocketTimeout: 10000,
+    });
+
+    client.on("error", (err: Error) => {
+      throw new Error(`Client error: ${err.message}`);
+    });
+
+    await client.connect();
+    await client.flushAll();
+  });
+
+  afterEach(() => {
+    diagnostics_channel.unsubscribe("redis.maintenance", onMessageHandler);
+    client.destroy();
+  });
+
+  describe("New Connection Establishment", () => {
+    it("should establish new connection", async () => {
+      assert.equal(connectSpy.callCount, 1);
+
+      const { action_id: lowTimeoutBindAndMigrateActionId } =
+        await faultInjectorClient.migrateAndBindAction({
+          bdbId: clientConfig.bdbId,
+          clusterIndex: 0,
+        });
+
+      const lowTimeoutWaitPromise = faultInjectorClient.waitForAction(
+        lowTimeoutBindAndMigrateActionId,
+      );
+
+      await lowTimeoutWaitPromise;
+      assert.equal(connectSpy.callCount, 2);
+    });
+  });
+
+  describe("TLS Connection Handoff", () => {
+    it("TODO receiveMessagesWithTLSEnabledTest", async () => {
+      //
+    });
+    it("TODO connectionHandoffWithStaticInternalNameTest", async () => {
+      //
+    });
+    it("TODO connectionHandoffWithStaticExternalNameTest", async () => {
+      //
+    });
+  });
+
+  describe("Traffic Resumption", () => {
+    it("Traffic resumed after handoff", async () => {
+      const { action_id } = await faultInjectorClient.migrateAndBindAction({
+        bdbId: clientConfig.bdbId,
+        clusterIndex: 0,
+      });
+
+      const workloadPromise = faultInjectorClient.waitForAction(action_id);
+
+      const commandPromises =
+        await TestCommandRunner.fireCommandsUntilStopSignal(
+          client,
+          workloadPromise,
+        );
+
+      const rejected = (
+        await Promise.all(commandPromises.commandPromises)
+      ).filter((result) => result.status === "rejected");
+
+      assert.ok(rejected.length === 0);
+    });
+  });
+});