1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-04-18 21:44:02 +03:00

45 Commits

Author SHA1 Message Date
mariadb-AlanMologorsky
6e64d3a38d fix(mcs): MCOL-5618: change list option to --list and refactor.
fix(mcs):  list option to -li/--list for mcs backup, restore and dbrm_restore commands
fix(mcs): add missed option "aro"
fix(mcs): resort options for backup and dbrm_backup to keep same ordering as in original scrypt
fix(mcs docs): updated README.md + mcs.1
2025-03-20 15:34:26 +03:00
mariadb-AlanMologorsky
6a57537cb1 fix(cmapi): MCOL-5133 remove has_active_nodes handler for cluster shutdown because TransactionManager has advance variant of this check 2025-03-20 16:30:25 +04:00
mariadb-AlanMologorsky
a60b278ee6 fix(cmapi tests): MCOL-5133 test mode set without nodes now returns adequate message 2025-03-20 16:17:58 +04:00
mariadb-AlanMologorsky
7c751ea53e fix(cmapi): MCOL-5133 maintainance flag typo. 2025-03-20 12:42:51 +03:00
Allen Herrera
d198ff77f2 adjusted list flag to --list in mcs_backup_manager.sh 2025-03-14 23:02:22 +03:00
mariadb-AlanMologorsky
75ed571f09 feat(cmapi): MCOL-5133: Stage3 stand alone cli tool.
MAJOR: Some logic inside node remove changed significantly using active
    nodes list from Columnstore.xml to broadcast config after remove.

 [fix] TransactionManager passsing  extra, remove and optional nodes arguments to start_transaction function
 [fix] commit and rollback methods of TransactionManager adding nodes argument
 [fix] TransactionManager using success_txn_nodes inside
 [fix] remove node logic to use Transaction manager
 [fix] cluster set api key call using totp on a top level cli call
 [add] missed docstrings
 [fix] cluster shutdown timeout for next release
2025-03-14 14:44:16 +03:00
Allen Herrera
001367e68a
feat(backup): added timestamps to localstorage backups major sections, new flag backup --apply-retention-only , more verbose logging in poll_check_no_active_sql_writes(), fixes to multinode race conditions to apply apply_backup_retention_policy() (#3422) 2025-03-14 11:36:01 +00:00
mariadb-AlanMologorsky
d132c1174a MCOL-5618: Sync with PR3410.
[add] list option but in argument notation "list" for backup, restore, dbrm_restore
[fix] help messages for --config-file
[fix] -p/--path > -bl/--backup-location option for dbrm_backup and dbrm_restore
[fix] -d/--directory > '-l', '--load' option for dbrm_restore
2025-03-13 07:35:27 +03:00
mariadb-AlanMologorsky
af10cb1526 feat(cmapi): MCOL-5133: Stage2 stand alone cli tool.
[fix] client class methods
[fix] Transaction management at endpoint handling level
[fix] add_node and set_mode methods to use TransactionManager
2025-03-13 07:20:29 +03:00
Alan Mologorsky
dafe35ef49
feat(cmapi): MCOL-5133: Stage1 to stand alone cli tool. (#3378)
[add] cluster api client class
[fix] cli cluster_app using cluster api client
[add] ClusterAction Enum
[add] toggle_cluster_state function to reduce code duplication
2025-03-12 11:21:32 +00:00
Timofey Turenko
36a412962d Recognize lsb_release 'RedHatEnterprise' as rhel 2025-03-12 14:32:12 +04:00
mariadb-AlanMologorsky
8e50884928 MCOL-5526: CLI rename "mcs CLUSTER" commands to just mcs.
[add] now all 'cluster' subcommand commands working without using 'cluster' subcommand itself
[fix] 'cluster' command hidding its help message
[add] explicit names for commands
[add] help-all command to list all nested help messages in man page view
[add] rich colored tabled help
[add] table panels for cluster commands
[upd] typer dependency version and some extra indirect dependencies
[add] README.md for mcs cli tool with all command reference
[add] mcs.1 man page
[add] README_DEV.md for mcs cli tool
[fix] cmapi/CMakeLists.txt to install mcs.1 man page
2025-03-07 01:51:31 +03:00
Allen Herrera
cd1939ee07 update backuo to v3.11, backup list and restore list + fixes 2025-02-21 22:29:58 +04:00
mariadb-AlanMologorsky
f6c9d3b89d MCOL-5695: Add FoundationDB package as a dependency for CMAPI. 2024-09-02 13:10:48 +04:00
Allen Herrera
0d78f2f6c6
feat(cs-package-manager): added multinode install via --nodes flag & dev upgrade along with some help texts (#3279)
Co-authored-by: Allen Herrera <allen@AllensBlackMBP.attlocal.net>
2024-08-28 20:17:40 +03:00
Allen Herrera
a02d15ad11 MCOL-5771 fixed -spoll for newer versions and config-file defaults 2024-08-23 13:30:26 +03:00
Alan Mologorsky
687aa463af
fix!(cmapi): MCOL-5454: Self-signed certificate autorenew. (#3213)
[add] managers/certificate.py with CertificateManger class
[mv] creating self-signed certificate logic into CertificateManger class
[add] renew and days_before_expire methods to CertificateManger class
[mv] several certificate dependent constants to managers/certificate.py
[add] CherryPy BackgroundTask to invoke certificate check hourly (3600 secs)
[fix] tests
[fix] bug with txn timer clean (clean_txn_by_timeout, worker and invoking of it)
2024-07-21 05:25:32 +01:00
mariadb-AlanMologorsky
1964f4243c MCOL-5618: Fix action names in wrapper.
[fix] backup-dbrm -> dbrm_backup + restore-dbrm -> dbrm_restore
2024-07-19 12:06:12 +03:00
mariadb-AlanMologorsky
7b0a04270f MCOL-5618: Fix after testing
[add] -P to restore help in mcs_backup_restore.sh
[fix] -f argument default value in both backup and restore wrapper commands
[fix] -i argument behaviour and type + default value
[fix] -P argument position and help for both backup and restore wrapper commands
[fix] disable some pylint warning for both backup and restore wrapper files
2024-06-26 20:57:10 +03:00
mariadb-AlanMologorsky
d2742b5a1b MCOL-5618: Fix packaging.
[mv] extra/cs_package_manager.sh -> cmapi/scripts/cs_package_manager.sh to pack and deliver the script with cmapi package
[add] cs_package_manager.sh to cmapi package
[fix] download paths for mcs_backup_manager.sh and cs_package_manager.sh
2024-06-21 05:47:13 +03:00
mariadb-AlanMologorsky
3934819ce1 Intermediate commit to move changes from old filepath to the new one. 2024-06-18 14:02:53 +03:00
mariadb-AlanMologorsky
f42411b6d1 MCOL-5618: Minor fixes to sync after PR #1398.
[add] -ssm argument for dbrm backup
[add] -sdbk -ssm arguments for dbrm restore
[add] silenece annoying pylint warning unused argument for bachup_commands.py and restore_commnads.py
2024-06-14 15:12:21 +03:00
mariadb-AlanMologorsky
e5722b0f02 MCOL-5618: Minor fixes to sync after #3147
[add] -r option for backup command
2024-06-07 17:53:01 +03:00
mariadb-AlanMologorsky
52f7d07766 MCOL-5618: Minor improvements to sync with PR 3144.
[add] -ns option for dbrm_restore.
2024-06-07 17:51:05 +03:00
Allen Herrera
a69d4c3a6c add comment to auto_select_most_recent_backup_for_incremental in mcs_backup_manager.sh 2024-06-07 09:06:14 +03:00
Allen Herrera
815b29b2ed resolved PR comments for cs_package_manager.sh & mcs_backup_manager.sh 2024-06-07 09:06:14 +03:00
Allen Herrera
331d578824 Added backup retention flag -r and incremental auto_most_recent option to mcs_backup_manager.sh 2024-06-07 09:06:14 +03:00
Allen Herrera
e235759dae added dbrm_restore for LocalStorage & S3 2024-06-07 09:06:14 +03:00
Allen Herrera
2f17c819a2 resolved PR comments for cs_package_manager.sh & mcs_backup_manager.sh 2024-06-07 07:20:25 +03:00
Allen Herrera
f005c1d99a added dbrm_restore for LocalStorage & S3 2024-06-07 07:20:25 +03:00
mariadb-AlanMologorsky
c70d206afe MCOL-5618: Testing fix + script minor fixes.
- [fix] wrong cooking of .sh invoke string
- [fix] restore_dbrm -> dbrm_restore, backup_dbrm -> dbrm_backup to use same naming in both python wrapper and mcs_backup_manager.sh
- [fix] mathod of checking cmapi online or not in mcs_backup_manager.sh
2024-06-07 06:44:49 +03:00
Timofey Turenko
460826748f Add dependency on libxcrypt for Rocky/RHEL9 2024-05-28 15:47:29 +04:00
Roman Nozdrin
e43472aedc fix(cmapi): handle an exception seen in CMAPI at RHEL 8 2024-05-21 16:17:03 +04:00
Alan Mologorsky
fd612d2578
fix(cmapi): MCOL-5594: Fix after testing.. (#3150)
- [fix] wrong invoke already imported constants and function from signal lib
2024-04-19 12:29:16 +03:00
Alan Mologorsky
6844923b9a
feat(cmapi,cli): MCOL-5618: Add backup, restore, backup-dbrm, restore-dbrm commands for mcs cli tool. (#3130)
* MCOL-5618: Add backup, restore, backup-dbrm, restore-dbrm commands for mcs cli tool.

[add] mcs_cluster_tool/helpers.py file with cook_sh_arg function inside
[add] MCS_BACKUP_MANAGER_SH to mcs_cluster_tool/constants.py
[add] backup and restore commands to "mcs" Typer APP

* MCOL-5618: Move mcs_backup_manager.sh to cmapi/scripts.

* MCOL-5618: Install mcs_backup_manager.sh with CMAPI package.
2024-04-01 16:51:38 +03:00
Alan Mologorsky
dec8350f0e
MCOL-5594: Interactive "mcs cluster stop" command for CMAPI. (#3024)
* MCOL-5594: Interactive "mcs cluster stop" command for CMAPI.

[add] NodeProcessController class to handle Node operations
[add] two endpoints: stop_dmlproc (PUT) and is_process_running (GET)
[add] NodeProcessController.put_stop_dmlproc method to separately stop DMLProc on primary Node
[add] NodeProcessController.get_process_running method to check if specified process running or not
[add] build_url function to helpers.py. It needed to build urls with query_params
[add] MCSProcessManager.gracefully_stop_dmlproc method
[add] MCSProcessManager.is_service_running method as a top level wrapper to the same method in dispatcher
[fix] MCSProcessManager.stop by using new gracefully_stop_dmlproc
[add] interactive option and mode to mcs cluster stop command
[fix] requirements.txt with typer version to 0.9.0 where supports various of features including "Annotated"
[fix] requirements.txt click version (8.1.3 -> 8.1.7) and typing-extensions (4.3.0 -> 4.8.0). This is dependencies for typer package.
[fix] multiple minor formatting, docstrings and comments

* MCOL-5594: Add new CMAPI transaction manager.

- [add] TransactionManager ContextDecorator to manage transactions in less code and in one place
- [add] TransactionManager to cli cluster stop command and to API cluster shutdown command
- [fix] id -> txn_id in ClusterHandler class
- [fix] ClusterHandler.shutdown class to use inside existing transaction
- [add] docstrings in multiple places

* MCOL-5594: Review fixes.
2024-02-23 21:40:50 +03:00
mariadb-AlanMologorsky
d226c85fc5 MCOL-5630: fix multi node MTR. Typer dependency bump version.
- [fix] requirements.txt to bump typer and indirect dependencies versions
2024-01-02 12:14:41 +03:00
mariadb-AlanMologorsky
49abbb4d7e MCOL-5630: fix multi node MTR. Add "cmapi is-ready" cli command.
This task is a next logical part of MCOL-5470 that implements AppManager and API endpoint
to check if CMAPI ready or not.

- [add] cmapi_app.py with is-ready command implementation
- [fix] add cmapi is-ready command to main typer app
2023-12-28 15:17:15 +03:00
Alan Mologorsky
e5385fdde7
MCOL-5610: Aliases on internally dependent tools. (#3026)
[add] mcs_aws and mcs_gsutil template files
[fix] cmapi CMakeLists to add mcs_aws and mcs_gsutil tools to usr/bin folder
2023-12-26 15:29:40 +04:00
mariadb-AlanMologorsky
7aa6e4c107 MCOL-5595: Wrong metadata in CMAPI packages.
[fix] cmapi packages metadata version format for debian systems and version + release in RHEL systems
2023-10-27 04:07:05 +03:00
mariadb-AlanMologorsky
d2aaf857f7 MCOL-5548: Fix cmapi package filenaming.
[add] dynamic year to cmapi CmakeLists
[add] package description to cmapi CmakeLists
[add] use MDB ES repo cmake dir
[fix] cmapi package filenaming
[fix] .drone.jsonnet to do cmapi build step depending on clonning mdb step
[add] cmapi_misc.cmake file
[add] lsb-release package to build deps in drone.jsonnet file
[fix] wrong debian and rpm requires curl placement
[fix] SET(CPACK_PACKAGE_FILE_NAME
[add] redhat-lsb-core package for RHEL systems
[add] SERVER_DIR argument for cmapi CmakeLists.txt
[add] get_linux_lsb_release_information() for both deb and rpm builds
2023-09-08 16:33:12 +03:00
Alan Mologorsky
b2a573cf82
MCOL-5521: Use Columnstore VERSION file for CMAPI versioning. (#2885)
[fix] columnstore_version.cmake setting extra variables for CMAPI versioning
[fix] CMAPI CmakeLists.txt using columnstore_version.cmake file
[fix] CMAPI VERSION file to template file due to using it inside CMAPI applicatio
n
2023-06-29 15:47:55 +03:00
Leonid Fedorov
d2d0e08690 Fixed stdout and stderr forking porcs handling 2023-06-16 04:38:09 +03:00
mariadb-RomanNavrotskiy
1eca477485 MCOL-5496: Fix CI adding CMAPI steps.
[fix] deb packages deps commands
[add] several additional local variables
[fix] packages url
[fix] smoke step
[fix] mtr step
[fix] regression step
[add] cmapipython, cmapibuild, cmapitest and cmapilog steps
[fix] dockerfile step
[fix] build step to include cmapi package on repodata creating
[fix] multi_node_mtr step
[fix] pkg step
[add] cmapi steps to pipelines
[fix] cmapi/CMakeLists.txt to prevent cmake byte-compile .py files for rpm packages
[add] setup-repo.sh file
[fix] now use packages from the repos in tests steps
2023-06-07 10:00:16 +03:00
mariadb-AlanMologorsky
a079a2c944 MCOL-5496: Merge CMAPI code to engine repo.
[add] cmapi code to engine
2023-06-07 10:00:16 +03:00