1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-11-03 17:13:17 +03:00
Commit Graph

14 Commits

Author SHA1 Message Date
mariadb-AlanMologorsky
c86586c228 feat(cmapi,failover): MCOL-6006 Disable failover when shared storage not detected
- Add SharedStorageMonitor thread to periodically verify shared storage:
  * Writes a temp file to the shared location and validates MD5 from all nodes.
  * Skips nodes with unstable recent heartbeats; retries once; defers decision if any node is unreachable.
  * Updates a cluster-wide stateful flag (shared_storage_on) only on conclusive checks.
- New CMAPI endpoints:
  * PUT /cmapi/{ver}/cluster/check-shared-storage — orchestrates cross-node checks.
  * GET /cmapi/{ver}/node/check-shared-file — validates a given file’s MD5 on a node.
  * PUT /cmapi/{ver}/node/stateful-config — fast path to distribute stateful config updates.
- Introduce in-memory stateful config (AppStatefulConfig) with versioned flags (term/seq) and shared_storage_on flag:
  * Broadcast via helpers.broadcast_stateful_config and enhanced broadcast_new_config.
  * Config PUT is now validated with Pydantic models; supports stateful-only updates and set_mode requests.
- Failover behavior:
  * NodeMonitor keeps failover inactive when shared_storage_on is false or cluster size < 3.
  * Rebalancing DBRoots becomes a no-op when shared storage is OFF (safety guard).
- mcl status improvements: per-node 'state' (online/offline), better timeouts and error reporting.
- Routing/wiring: add dispatcher routes for new endpoints; add ClusterModeEnum.
- Tests: cover shared-storage monitor (unreachable nodes, HB-based skipping), node manipulation with shared storage ON/OFF, and server/config flows.
- Dependencies: add pydantic; minor cleanups and logging.
2025-10-01 21:10:34 +04:00
mariadb-AlanMologorsky
0753aac849 fix(cmapi): Fix error in docker cluster: "is not currently part of the cluster" 2025-09-26 21:55:10 +04:00
Alexander Presnyakov
bc4c7e27d9 MCOL-5806: added ability to start node in read-only mode
* feat(cmapi): add read_only param for API add node endpoint
* style(cmapi): fixes for string length and quotes

Add dbroots of other nodes to the read-only node

On every node change adjust dbroots in the read-only nodes

Fix logging (trace level) in tests

Remove ExeMgr from constants

Fix tests

Manually remove read-only node from ReadOnlyNodes on node removal (because nodes are only deactivated)

Review fixes (mostly switching to StrEnum analog before py3.11, also changes in ruff config)

Read-only nodes are now called read replica consistently

Don't write hostname into IP fields of the config like PMSx/IPAddr, pmx_WriteEngineServer/IPAddr

We calculate ReadReplicas by finding PMs without WriteEngineServer

In _replace_localhost, replace local IP addrs with resolved IP addrs and local hostnames -- with the resolved hostnames.
ModuleHostName/ModuleIPAddr is kept intact.

Keep only IPv4 in ActiveNodes/DesiredNodes/InactiveNodes

feat: add mock DNS resolution builder for testing hostname/IP mappings

* Fix _add_node_to_PMS: if node is already in PMS, save it to existing items to not miss it during the reconstruction of the list
* Make tests independent from CWD

Fixed for _add_Module_entries

Fixed node removal and tests

Fixes for node manipulation tests
2025-09-26 21:55:10 +04:00
Alexander Presnyakov
a357566db7 Use is_only_loopback_hostname instead of LOCALHOSTS 2025-09-18 01:10:49 +00:00
Alexander Presnyakov
85690a0de1 refactor: standardize error handling with new context managers for CMAPI exceptions
We do it for the API to return error details.
2025-09-17 17:58:51 +00:00
Alexander Presnyakov
4a1b3b9355 feat: add forward/reverse DNS validation when adding nodes by hostname 2025-09-17 17:58:51 +00:00
Alexander Presnyakov
ade880fe78 Return original default value of deactivate_only (False) 2025-09-11 19:49:31 +04:00
Alexander Presnyakov
9b98c5c20a Created a separate package for tracing-related stuff
Added mirroring of spans into Sentry
Tracer is a facade that redirects actions to tracing backends
2025-09-03 20:32:03 +04:00
Alexander Presnyakov
a0b4bcd1ce Basic request tracer
Tracing requests

Custom log factory adds all trace values as one log record parameter (it will be empty if trace values are empty, like in MainThread where there are no incoming requests)
2025-09-03 20:32:03 +04:00
mariadb-AlanMologorsky
b6a5c1d71f fix(cmapi): MCOL-5913 cmapi write local address 127.0.1.1 instead real ip address
* feat(cmapi): NetworkManager class for some ip hostname opoerations.
 * fix(cmapi): Use NetworkManager class to resolve ip and hostname in node_manipulation.add_node function
 * fix(cmapi): Minor docstring and formatting fixes
2025-07-08 15:57:28 +03:00
mariadb-AlanMologorsky
d17fc7d9be Review fixes. 2025-04-08 14:56:06 +03:00
mariadb-AlanMologorsky
27e3b8d808 feat(cmapi): MCOL-5133: Stage3 stand alone cli tool.
MAJOR: Some logic inside node remove changed significantly using active
    nodes list from Columnstore.xml to broadcast config after remove.

 [fix] TransactionManager passsing  extra, remove and optional nodes arguments to start_transaction function
 [fix] commit and rollback methods of TransactionManager adding nodes argument
 [fix] TransactionManager using success_txn_nodes inside
 [fix] remove node logic to use Transaction manager
 [fix] cluster set api key call using totp on a top level cli call
 [add] missed docstrings
 [fix] cluster shutdown timeout for next release
2025-04-01 18:03:07 +04:00
Alexander Presnyakov
de7a4a2d7e feat(mcs): add -v option to enable logging to console
refactor(ci): switch S3 uploading to AWS CLI for logging S3 link

docs: update README with to reflect changes in Drone pipeline

feat(env): add activate script for portable Python

fix(logging): log RequestException response body
2025-03-31 16:54:46 +00:00
mariadb-AlanMologorsky
a079a2c944 MCOL-5496: Merge CMAPI code to engine repo.
[add] cmapi code to engine
2023-06-07 10:00:16 +03:00